RightEdge - The Ultimate Backtesting and Trading System Development Platform

Importing Data

Use Import Data tool to import data from a text file into a Watchlist folder.  The Import Data tool examines the text file that you are importing and helps you ensure that the data is imported in the way that you want.  You can import multiple files with the same import settings at one time.

To begin the process, select Import Bar and Tick Data... from the Tools menu.
 

Files Tab
Fields Tab
Formats Tab
Watchlist Tab

Files Tab

Use the Files tab to select one or more files to import.

Click the Add... button to display a file dialog that will allow for multiple selection.

Note: Hold the Ctrl in the file dialog to select more than one file.

ImportFiles

The Remove button is used to remove a file that currently exists in the list.

At this point, the fields for the file that will be imported may or may not contain column definitions.  If a heading exists in the imported file, RightEdge will try its best to determine what column maps to what.  RightEdge also attempts to auto-detect the delimiter (tab, comma, etc).

Import Settings

Clicking the Load... button will allow for retrieval of previously saved field settings.  The Save button allows the currently set fields to be saved to disk for later use.  This is particularly useful if the same file is parsed on a regular basis.

 

Autodetect Click this button to force RightEdge to auto-detect the columns within the file.

 

Symbol Name Specify the name of the symbol as it will appear in the Watchlist.

 

Details Allows additional information to be set for this symbol.  For example, if this is a futures contract, set the expiration date in the symbol details screen.

 

Bars or Ticks Specifies whether this data file contains bar or tick data.

 

Bar Frequency Specifies the frequency of the imported data.

 

Tick Types Select this item to map tick types to the data file.  RightEdge supports bid, ask and last tick types.

 

Fields Tab

If the field definitions are not retrieved from disk, it will be necessary to

specify them manually.  These can be specified regardless of the tab selected by simply clicking the header of the sample data shown in the output window.  In this particular case, it was necessary to select the proper delimiter.  This file is comma delimited, therefore, we have select the Comma checkbox.

importFields

Additional Fields

Additional fields are used to define fields that are not actually contained within a line of data.  In other words, these are data fields that are not seen within the output window.  A sample of this might be a symbol name or an exchange.

ImportFieldEdit

For example, if the exchange is not listed within the data file, the exchange can become a user defined value that is set with each row.  Another source that may be used is the filename itself.  Some data streams may put information about the data in the filename.

Click the Add... button to define a new import field.  Click the Edit... button to modify the settings of an existing user defined import field.  Click the Delete... button to remove the user defined import field from the list.

Select the way the month is formatted in the data file.  M represents a 1 or 2 digit month value.  MM represents a month value that is always 2 digits.

Delimiter

Comma and/or Tab and /or Semicolon are common delimiters.  If the file is not comma, semicolon or tab separated, uncheck all of the delimiter checkboxes and type the delimiter into the Other field.

Characters to Ignore

Each character entered into this field will be stripped before being placed into the data store.  Enter one or more characters here.

Note: Enter each character without a delimiter as each character is treated independently.

Header Rows

Many data files contain one or more rows at the beginning of the file to give information about the row contents.  It is likely necessary that these lines will need to be skipped for the importer.  Specify the number of lines to skip at the beginning of the file.

Formats Tab

The formats tab is used to define the date and time elements and how they're

comprised.

ImportFieldFormats

Date Format

This dropdown contains a list of common date formats.

Time Format

This dropdown contains a list of common time formats.

Date Separator

Dash and/or Slash are common delimiters.  If the date is not slash or dash separated, uncheck both Dash and Slash and type the delimiter into the Other field.

Decimal Separator

Specify the character that is used to indicate a decimal value.

Thousands Separator

Specify the character that is used to indicate a thousands separator.

Timestamp at end of bar

Indicates that the time value encountered is when the bar is ending.  By default, RightEdge assumes that the timestamp in the imported file indicates the beginning of the bar.

Dates in descending order

By default, RightEdge expects the dates of each bar to appear in ascending order.  Select this option to reverse this.

 

Advanced

Click this button to specify the date and time elements independently.

ImportFieldDateTimeFormat

Date Order

Specifies the order in which the date elements appear.

Date Separators

Enter one or more characters used to split the date elements.  This is used for dates only.

Date/Time Separators

Enter one or more characters used to split the date and time elements.  This is used when the date and time is specified in the same column.

The time and date formats are shown below.

Date format notation

yyyy = 2007   yy = 07  yyy = 107

M = 1   MM = 01  MMM = Jan  MMMM=January

d = 1   dd = 01

 

Time format notation

h = hour    m = minute  s = second  t = am or pm

h = 12 hour

H = 24 hour

hh, mm, ss = leading zero

h, m, s = no leading zero

Time Separators

Enter one or more characters used to split the time elements.  This is used for time only.

Test Input

Enter in a test line of data here to test the current selections in the date and time drop downs.  The output will be displayed in the Test Output field.

Format Sample

The importer will generate a sample line based on the current drop down selections.  This sample is displayed here.

Watchlist Tab

RightEdge now has enough information to begin mapping the data within each

column.

ImportWatchlist

The importer will allow the symbols to be created within an existing symbol folder within the watchlist or allow for a new folder to be created.

Add imported symbols to watchlist

Check this box to import the symbols and the associated data directly into the watchlist.

Add to existing folder

Specifies that the symbol and associated data will be imported to an existing folder within the watchlist.  Select the desired name from the dropdown.

Create new folder

Specifies that the symbol and associated data will be imported to a new folder name specified.

See Also

Bar Data Cleanup