Hi,

I'm with an emerging CTA trading US futures contracts.  I'm new to RightEdge, and we're considering whether it will work for us.  Here are a couple of use cases...  I wonder if someone can comment on some of my questions?  I know this is a big long post...  feel free to respond to as much or as little as you wish.  Any comments are appreciated!

UC1) Let's say we are running 20 intraday automated trading systems on a single symbol.  Let's call each one an "instance".  Each instance has its own entry and exit rules, and positions from one instance will frequently overlap with positions held by others. Furthermore, each instance may be using a different bar interval.  Maybe 3 instances run on 15 minute bars.  Maybe 5 instances run on 100-contract volume-bars.  Maybe 4 instances run on $1 range bars. Some instances use limit orders, others use market orders, others use stops, some use bracket orders to exit, others use trailing stops.  You get the idea.  So all 20 instances are on a single futures contract, and each instance needs to keep track of its own position, indicators, exit targets, etc.  And it's all on intraday or tick data, and all on the same futures symbol.  My questions here are: 

Q0) Can we code and backtest each instance independently, and at run-time decide which instances to turn on or turn off?  Does RE give us the UI and object model for this, or do we have to work this out ourselves?

Q1) Will we be tripping over ourselves trying to keep track of the positions and targets for each instance?  Or can RightEdge report to each instance only the market positions that belong to that instance?  

Q2) In a fast-moving market, will RightEdge bog down, and introduce significant software latency?  We have experienced this "bogging down" problem  with another platform, on fewer instances than I describe here, so this is a very important question.  I have seen multi-second latency introduced by our platform when we are in several simultaneous positions on tick-based strategies.  This is a major reason for wanting to find a new platform!

Q3) What is the best way to collect an ongoing record of tick data (bid/ask/trade) for our contracts of interest?  These ticks will be used to warm up indicators, to backtest new strategies, and to compare actual fills with backtests.  Our datafeed keeps tick data one-day back, so we could potentially do a once-per-day batch download and merge that into our ongoing record.  But I'm not sure if it keeps bid/ask quotes, so these may need to be collected in real-time.

Q4) From the posts I've read, it appears the native binary datastore is the fastest (compared to SQL Server for example).  Is this impression correct?  Are there other datastores that people have actually used with RE which are faster (and reasonably priced)?  I'm only asking about actual experiences here...  I'm aware that there are many database products in the marketplace.

UC2) Let's say we are running a spread strategy, based on an intra-market calendar spread between two futures contracts.  We want to compute indicators, place limit orders, place market or stop orders, on the spread.  

a) Let's say we do this by using the two futures contracts, and computing the spread ourselves (be it a difference, a log-ratio, etc).  And we might weight them, buying 5 of leg 1 for every 3 of leg 2 that we sell, for example.  

b) Let's say we do this by using the broker feed's spread symbols.  These are difference spreads in a 1:1 ratio.  So buying 1 lot means we (buy 1 contract of leg 1 and sell 1 contract of leg 2), and the price is quoted as (price of leg 1 - price of leg 2).  The difference spread can range over positive and negative values, depending on whether the market is in contango or backwardation.  So you might want to set a limit order to buy 20 contracts of a spread at a negative limit price, for example.  

Q5) In (a), are there tools in RE to implement "legging into" the spread, making sure the right quantities on both legs are filled, and perhaps simulating the function of a limit order?  This is the kind of functionality in futures spreading tools made by people like CQG and TT, for example.  If not (and I'm guessing this is the case) can anyone see a reason why I couldn't write a modular execution algo that could be called by my spread system instances to implement such spread-legging?  (As before, we might be running multiple instances at the same time, sometimes on the same symbols.)

Q6) In (b), are there any problems with backtesting or live-trading an instrument (e.g. a difference spread symbol provided by the broker's market datafeed) that takes on negative prices?  I've seen this problem with a different platform, in which negative prices cause exceptions when you try to place an order.

UC3) Let's say we are running a "market scanning" spread strategy, in which we scan hundreds of spreads (inter-market, intra-market, etc.) and rank & filter them by our own criteria.  We find a few trades that we want to enter, but only if our available margin is enough, and only if the liquidity is good enough.  

Q7) Can this type of spread portfolio strategy be automated in RightEdge?

About me...  I have a background in engineering, a decent amount of programming experience, and I'm reasonably comfortable in C#.  I've used a number of platforms over the years, including some very expensive institutional ones.  But we are an emerging CTA without huge AUM or huge budget (yet!) and I would be delighted to learn that RightEdge can do all or most of what we need.

Thanks in advance for any help you can provide

Overall, RightEdge is pretty flexible and extensible if you have the programming skills to do so.  So it should be able to do most or all of what you're looking for.

OK, here we go!

UC1 - RightEdge doesn't have built-in support for multiple systems.  So you will just have one system from RightEdge's point of view.  But you have full control over the logic of that system, so underneath it you can be running all your "instances".  The Multiple Systems sample shows how you can do this.  It runs all of its systems on the same frequency.  There is also a Multiple Frequency sample which shows how to run the same logic at different frequencies, so you would probably want to combine techniques from the two.

Q0 - Since RightEdge doesn't have a concept of multiple systems it doesn't have a UI for this.  You could create a system parameter to enable or disable each instance and then you would be able to control this in the system properties.  Or you could put all your code in a shared library which was referenced by a bunch of RightEdge systems which only contained code to delegate to the shared library and to control which instance was enabled.  That way you could have different RightEdge projects for each individual instance for backtesting purposes.  You could also pop up a dialog in the system's startup or use the /CustomStringForSystem command line parameter to control which systems are enabled.

Q1 - You will need to keep track of which position corresponds to each instance, but you won't have to trip over yourselves to do it.  The Multiple Systems sample shows how to do this.  RightEdge can track multiple positions open at the same time for one symbol, and each position has its own profit target and stop loss.  So you just need to track which instance each position belongs to.

Q2 - From RightEdge's point of view there's just a single system so I think it's very unlikely that you would run into these kind of performance problems when running with just a single symbol.

Q3 - RightEdge can save the live tick data it recieves.  But if you miss any tick data (because RightEdge isn't started, or crashes, or you have network issues, etc), it doesn't have a way to fill in what you missed.  So probably you want to save the live tick data you get (either via RightEdge or separately), and have the ability to fill in any data you miss from the datafeed.  If the datafeed does keep bid/ask quotes, then it might be simpler to just do a regular batch download.

Q4 - The binary data store is so simple that it will probably always be faster than SQL Server.  But I think the SQL Server plugin should probably be more than fast enough for your needs, and it is more flexible.  It used to have some performance issues with large datasets where an index wasn't being used correctly, but that should now be fixed.  You can always try both and compare. :-)

Q5 - No, there's nothing built in to manage spreads for multiple symbols.  You could put your spread execution in the broker plugin, and then in the system you would be able to treat each spread as a separate symbol.  To the system this would look like the broker offered the spread symbols you wanted, and the broker plugin would translate the spread orders into separate orders for its components.

You could also put your spread execution in your trading system.  In that case you would probably create your own object model for the trading code to use instead of RightEdge's Position and Order APIs.

Q6 - There are some places in the RightEdge code where we implicitly assumed that negative or zero prices were not valid.  You can check the "Ignore Bar Data Errors" option for a symbol to allow RightEdge to accept zero or negative prices.  However, there may be other places in RightEdge where we implicitly assumed that negative or zero prices were not valid.  One that I'm aware of is that you can't set a trailing stop price less than or equal to zero.  So unfortunately I think you'll have to try it and see what issues you run into.  If you do find any, please let us know and we may be able to fix them.

Q7 - Yes, I think you should be able to do that.  There's a symbol ranking sample which shows how to do something similar, although much simpler.

Thanks,
Daniel

Thank you Daniel for the lightening-fast (and detailed) response!

One of your answers was:  

Does this mean running multiple symbols is more likely to bog it down?  If not, what *would* bog it down?  I'm trying to get a feeling for how we should plan to use (and not misuse) the platform.  One thing to note is that we don't use IB (which I think delivers only 5 quotes per second) and instead use a feed that delivers all the ticks.

One thing I think I've observed in our current platform is that slowdowns happen when there are multiple live broker positions being tracked, especially in times of high tick volume.  I'm guessing there are some sleep() commands or something to synchronize various threads, and each strat instance runs in its own thread.  But I don't know the internals so this is just speculation.

As for the rest of your answers, they all sound reasonable...  So my homework is to look into the Multiple Systems & Multiple Frequency & Symbol Ranking samples you mention and see how this would work.  And in the time since my last post, a couple more questions occurred to me... 

UC4) Let's say we are running multiple systems on multiple symbols, and we have a dozen or so positions, some long and some short.  We decide to do a planned shutdown and restart which will take the platform down for a few minutes.  (Or worse, let's say there is a hardware/software or network outage.)  On restart, we need to tell each instance what its position is (e.g. System 1 is long 50 contracts of ESH15, at a price of $2050.5, System 2 is short 20 contracts of ESH15 at $2051, System 3 is short 10 contracts of CLG15 at 55.45).  Then the appropriate position objects will need to be initialized with this data, so each instance can manage its exits.  

Q8) Does RE have a way of saving state before shutdown, and reloading state after restart, to streamline this process?  If not, is there anything preventing us from doing this in our system code?   For example, are there any built-in classes that we are unable to modify, and thus unable to write state to them at restart?  I'm thinking in terms of another platform right now... on restart, we have to manually key in the positions and prices for each strategy that we are turning on, even if the position is flat.  If there are samples or best practices for this kind of thing, it would be interesting to see.

UC5) We are trading futures, and there are many standard contract expiration and roll issues that come with this.  Some platforms have "concrete" contracts for each expiration month, then stitch them into continuous contracts at runtime for backtesting based on some rule set in a "contract manager" tool.  Some platforms allow a strategy to dynamically subscribe and unsubscribe from market data events for each monthly contract based on whatever logic the strat developer wants to code.  And other platforms handle this further upstream in the datastore, creating continuous contracts and assigning them specific symbols (and duplicating the data).  

Q9) I'm getting that RE is very flexible, so it probably supports multiple ways of handling contract rolls.  Is there a standard way that has been envisioned by the RE developers, or a typical way that users of RE tend to favor?

Q10) Is there a standard/clean way supported by RE to inform the strategy when a continuous contract is about to roll, so in a backtest it can exit its position before the new contract's data begins?  Or should I use my own solution (like a metadata table for each continuous contract, for example).

Thanks again for your help.  I'm pretty excited about RightEdge, and while it looks like there will be some work involved, it may help us solve a lot of our problems, and enable us to do things that our current platform simply can't.  

Happy New Year  

Hi,

In the past RightEdge had some performance issues when running with large numbers (1000+ or so) of symbols, but that was fixed.  There's always going to be some limit on how fast it can handle how much data, but with one symbol I don't think you'll come anywhere close to that.  Even with many symbols you probably won't have issues.

Q8 - RightEdge can save and restore position and order state when you shut down and restart a live system.  Your system may have its own data that you will need to recreate or save and reload.  There is a CustomString property on the position and order objects that you can use to save data, or you can just use your own data file.

Q9 - You can represent continuous contracts in RightEdge by using an expiration date that is far in the future (ie 1/1/2100 or something).  RightEdge doesn't have any functionality built in to create continuous contracts from the data for the fixed expiration contracts, so you'd need to do that yourself.  Overall, it's up to you whether you want to backtest with continuous contracts or have logic in the trading system to decide which expiration to trade.  RightEdge doesn't support adding or removing symbols while the system is running, so if you wanted to backtest over a long period without continuous contracts, you would have to set up a separate symbol for each expiration within the period and run the simulation with all those symbols.

Q10 - You'd need to come up with your own method of storing and reading this data in your system.

Thanks,
Daniel

Thanks Daniel,  I appreciate the info very much.  I think we'll give it a try.  

If any other users out there have some experiences they'd like to share, your input would be appreciated.  In particular, if anyone uses RightEdge to trade futures, I'd love to hear from you...