RightEdge - The Ultimate Backtesting and Trading System Development Platform

What's New In This Edition?

This topic will introduce the new features in this edition of RightEdge.

 

Breaking Changes And Other Important Information

 

License files are NO LONGER COMPATIBLE between this edition and previous editions of RightEdge.  You will need to obtain a new license file for this edition.  If you have a subscription in force, are a lease subscriber or have purchased RightEdge in the last 90 days, your customer area will contain an updated license file.  Others will need to purchase an upgrade.

 

Project files from this edition of RightEdge cannot be opened in previous versions of RightEdge.  When you open a project file from an older version of RightEdge, a copy of that project file will be saved with the previous version number added to the filename so that you can still open it up in the older version of RightEdge.  For example, if the project filename is MySystem.rep, a copy named MySystem.2008.1.0.0.rep will be saved.

 

If you use a custom data storage plugin, you will not have a data storage plugin set up in this edition.  You will need to go to the options dialog and choose your plugin there.

 

RList class
No longer has AsIList() method or InnerList property.
Count now refers to the number of available items.  This may be less than the total number of bars if MaxLookBack is not zero.  The TotalBars property will get the total number of bars.

 

RightEdge is now built on version 3.5 of the .NET Framework.  This allows trading system developers to take advantage of language enhancements such as extension methods and Linq.

 

System results plugins no longer have direct access to bar history.  Some of the built-in system results are not calculated (they should display as 0 or n/a).

 

The SystemData.TradeInsideBars property has been removed.  There is now a CreateTicksFromBars property (which is set to true by default) and an EnableTradeOnClose (which is set to false by default).

 

The BarOpened event has been removed.

 

 

Multiple Frequencies

RightEdge now has support for multiple frequencies.  The Frequency class represents a given frequency for a given symbol.  It provides a list of bars for the frequency, as well as events for NewBar and NewTick for that frequency.  There are GetFrequency methods on the symbol script and SystemData classes which allow you to retrieve a Frequency object for the desired frequency and symbol.  RightEdge will only start building bars for the frequency after you have requested it by calling GetFrequency, so generally you will want to call it in your system startup code.  To set an indicator to a different frequency, call SystemData.IndicatorManager.SetFrequency().  If the indicator needs a series as input, and you want to use a bar element series for that input, be sure to use the bar element series from the corresponding Frequency class.

 

The code below shows how to subscribe to bars in a different frequency and how to create an indicator using that frequency.

 

public class SampleSymbolScript : MySymbolScriptBase

{

     Frequency HourlyFreq;

     SMA sma25;

     SMA sma5Hourly;

   

     public override void Startup()

     {

           //    Create indicator using system frequency

           sma25 = new SMA(25, Close);

 

           //    Get hourly frequency and subscribe to new bars

           HourlyFreq = GetFrequency(BarFrequency.SixtyMinute);

           HourlyFreq.NewBar += NewHourlyBar;

         

           //    Create indicator using hourly frequency

           sma5Hourly = new SMA(5, HourlyFreq.Close);

           SystemData.IndicatorManager.SetFrequency(sma5Hourly, HourlyFreq);

     }

   

     public override void NewBar()

     {

           OutputMessage("New system bar: " + Bars.Current.PriceDateTime.ToString());

     }

   

     public void NewHourlyBar(object sender, SingleBarEventArgs args)

     {

           OutputMessage("New hourly bar: " + args.Bar.PriceDateTime.ToString());

     }

}

 

Large Data Set Support

Most systems only need a certain amount of bar history for their calculations, and there is no need to keep bar data available past that amount.  When working with large amounts of high frequency data, it may not be possible to store all of the data in memory at once.  To allow systems to be run with large amounts of data, RightEdge now includes a MaxLookBack property in the RList class.  If its value is greater than zero, RightEdge will discard any items past the maximum lookback.  If you try to access any of the discarded items, an exception will be thrown.  You can set the value in your symbol script startup method, like this:

 

   Bars.MaxLookBack = 1000;

 

If you are using multiple frequencies, you will need to set the MaxLookBack on the bar list for each additional frequency, also.

 

Depending on your system, using a maximum lookback for the bar lists may or may not have a significant effect on your memory consumption.  The system statistics, indicators, and trade history, as well as possibly your system code itself, may all use a lot of memory.  If you need to decrease memory usage, you can selectively disable elements of your system and measure the effect on your system's memory usage.  You can use the following code in your system class to help with this:

 

     long _initialMemory;

   

     public override void Startup()

     {

           _initialMemory = GC.GetTotalMemory(true);

     }

 

     public override void Shutdown()

     {

           long finalMemory = GC.GetTotalMemory(true);

           long diff = finalMemory - _initialMemory;

 

           string msg = string.Format("Memory used: {0:n}", diff);

         

           SystemData.Output.Add(OutputSeverityLevel.Informational,

                 msg, null, "", SystemOutputLocation.OutputWindowAndStatusBar);

     }

 

There are four sets of system statistics stored (long, short, all, and buy and hold).  They save a snapshot is saved for each system bar.  If the system is only running with a single symbol, the system statistics memory usage will be significantly larger than the memory used by the symbol's bar data.  However, the system statistics memory usage doesn't increase with the number of symbols.  So if you are using many symbols the system statistics may not affect your memory usage too much.  You can disable some or all of the system statistics by setting the Enabled property on them to false in your system's startup method:

 

     SystemData.SystemHistory.LongStatistics.Enabled = false;

     SystemData.SystemHistory.ShortStatistics.Enabled = false;

     SystemData.SystemHistory.SystemStatistics.Enabled = false;

     SystemData.SystemHistory.BuyAndHoldStatistics.Enabled = false;

 

However, with all of them disabled, the system results summary won't show much useful information.  Another way to reduce the system statistics memory usage is to reduce the main system frequency.  If your system operates on one minute bars, you could set the system frequency to daily, and use the multiple frequency support to run most or all of your system logic on the one minute frequency.

 

Indicators may also use a significant amount of memory.  They don't currently support a MaxLookBack setting, so they will keep all the data for all the bars of your system.  If you need to reduce memory consumption you may have to re-implement the calculations outside of a RightEdge indicator.  Each trade that your system makes also consumes some memory.  If your system trades at a very high frequency then this could lead to high memory usage.

 

Bar/Tick Generation

The frequency of your system is no longer tied to the frequency of the input data.  The main system frequency is specified in the trading system options.  This controls the frequency for many things such as when your NewBar methods will be called, the default frequency for your indicators, the frequency that system statistics will be saved, etc.  Right now you are limited to a set of predefined frequencies or "Tick" for the system frequency.  If set to "Tick", then the frequency of the symbols you have selected in the watchlist when you run the system will be used for the system frequency.  Otherwise, the input data will be aggregated to the specified frequency and passed to your system.

 

The input data bars for your system are sent to a tick generator.  If the CreateTicksFromBars property in the SystemData class is set to true, the tick generator breaks the bars down into ticks.  Currently the tick generator just generates 4 ticks for each bar, for the open, low, high, and close.  We will probably allow you to supply your own tick generator, which would allow you to determine whether the low or high tick comes first, or to randomly generate more ticks in between these values.

 

The ticks from the tick generator are sent to the frequency manager for the system.  This aggregates the ticks into the system frequency, and raises the NewTick and NewBar events.  The paper broker fills orders based on ticks, which means you can use higher frequency data on a lower frequency system for a more accurate simulation.  For example, it is possible that both the profit target and stop loss for a position could be triggered within the range of a daily bar.  If you use higher frequency data (say 15 minute bar data), then it is more likely that the simulation will reflect which price would have been hit first.

 

New Plugin System

This edition of RightEdge introduces a new plugin system.  One of the advantages of the new system is that it is easier for plugin writers to define custom settings for plugins.  A property grid is used to display the plugin settings, and XML serialization is used to save them.  This means that to add a custom setting to your plugin, you simply need to define a public property on your plugin class.  The property grid and XML serialization are standard features of .NET, and offer a high degree of flexibility in how the settings are displayed in the property grid or saved to XML.  The code below shows how a plugin can define a basic custom property.

 

[DisplayName("Time-based Bar Frequency")]

[Serializable]

public sealed class TimeFrequency : FrequencyPlugin

{

     [Description("Indicates the timespan or period for each bar.")]

     [DisplayName("Bar Length")]

     public TimeSpan BarLength { get; set; }

 

     public TimeFrequency()

     {

           BarLength = TimeSpan.FromMinutes(1);

     }

 

     // FrequencyPlugin methods go here

}

 

For more advanced functionality pertaining to property grid customization or XML serialization, please refer to the .NET documentation or other .NET resources.

 

Currently, only data storage plugins and custom frequency plugins use the new plugin system.  The new interface for data storage plugins to use is IDataStore.  Existing plugins which use the old IBarDataStorage interface will continue to work.

 

 

Custom Frequency Plugins

RightEdge uses frequency plugins to handle bar generation.  It includes plugins for time based, tick based, and volume based bars.  Additional plugins can be created to create other bar types.  To create a frequency plugin, derive a class from the abstract FrequencyPlugin class.  This class should use public properties to define the settings for the frequency.  The actual logic for generating bars should go in a class which implements IFrequencyGenerator and created and returned by the CreateFrequencyGenerator method of the FrequencyPlugin.

 

To use a custom frequency in a system, create an instance of the frequency plugin class and pass it to the GetFrequency method to get a Frequency object.  If GetFrequency is called multiple times for the same settings for the frequency plugin and the same symbol, it will return the same Frequency object.  For this to work even if different instances of the frequency plugin with the same values for the settings were used, it needs to be able to compare them to see if they are equivalent.  This is why the FrequencyPlugin class includes abstract Equals and GetHashCode methods which you must implement.

 

Currently, frequency plugins besides the time-based frequency can only be used from within a system.  The main system frequency, charts, and bar data storage all only use time based frequencies.

 

Miscellaneous

Partial bars are now stored in the PartialItem property of the RList<BarData> class.  This means that the Current property will always contain a completed bar.  There is also a HasPartialItem property which indicates whether there is a partial bar or not.

A bar data cleaner tool has been added.  It is accessible from the watchlist, and will find and fix problems with bars in the data store.  It will delete extra bars where there is more than one bar for a given date/time, and if the time for a bar is not rounded correctly (based on the bar frequency) it will round it.  For daily bars this means it will set the time to 12:00 midnight, for hourly bars it would round to the nearest hour, etc.

 

There are no longer separate SymbolBars and SystemBars.  Use the Synchronize Bars property of the trading system to control whether the bar lists should be synchronized.