Profile Picture

Synchronizing bar timing between RE and data source

Posted By Steve2008 8 Years Ago
Message
Posted Friday July 17 2009
I am working on a plugin for RE Edition2 for TD Ameritrade streaming data.

I have it working using the TDA ActiveX interface, but with 100 symbols RE chews up so much CPU that it becomes unresponsive, and I want to use a lot more than 100 symbols. I don't know whether the issue is with the TDA interface, RE Edition 2, or both. (It could be my code, but I don't think so.) There is a third-party TDA native C# interface that is conceivably more efficient, but months ago I wrote a RE plugin for it and found a bug that makes it unusable for multiple symbols. A new version is in the works, but it is not clear when it will be available.

I only need 1 minute bars, which are much less CPU-intensive than streaming data and will therefore probably alleviate the CPU bottleneck. TDA can deliver 1 minute bars, and I know from previous discussions here that I can feed these into RE by generating 4 ticks in my plugin: one each for OHLC of the bar. I have not yet programmed this, but it seems straightforward.

My question regards time synchronization. I would like RE to complete its 1 minute bar immediately after delivery of the ticks, and immediately invoke trading strategy logic. I want to avoid a situation where the RE timing loop for bar creation/strategy invocation is not synchronized with that of my data source, causing a time lag.

What, if anything, can I do to ensure time synchronization?

Thanks.


Edited: Friday July 17 2009 by Steve2008
Posted Friday July 17 2009
RE only processes when it receives a tick, so you could send a last trade tick (which contains the same value as the 'real' last trade) to force a new bar on the first second of the new minute.

Steve2008 (7/17/2009)
I am working on a plugin for RE Edition2 for TD Ameritrade streaming data.

I have it working using the TDA ActiveX interface, but with 100 symbols RE chews up so much CPU that it becomes unresponsive, andI want to use a lot more than 100 symbols. I don't know whether the issue is with the TDA interface, RE Edition 2, or both. (It could be my code, but I don't think so.) There is a third-party TDA native C#interface that is conceivably more efficient, but months ago I wrote a RE plugin for it and found a bug that makes it unusable for multiple symbols. A new version is in the works, but it is not clear when it will be available.

I only need 1 minute bars, which are much less CPU-intensive than streaming data and will therefore probably alleviate the CPU bottleneck.TDA can deliver 1 minute bars, and I know from previous discussions here that I can feed these into RE by generating 4 ticks in my plugin: one each for OHLC of the bar. I have not yet programmed this, but it seems straightforward.

My question regards time synchronization. I would like RE to complete its1 minute bar immediately after delivery of the ticks, and immediately invoke trading strategy logic. I want to avoid a situation where the RE timing loop for bar creation/strategy invocation is not synchronized with that of my data source, causing a time lag.

What, if anything, can I do to ensure time synchronization?

Thanks.
Posted Friday July 17 2009
Hi Bill,

Please bear with me as I try to clarify how this works.

I will be receiving TDA's 1 minute bars whenever they send them, presumably every minute regardless of whether there were trades during that minute; if not, I can simulate this within my plugin, so we can assume that it will effectively happen. The record contains fields for OHLC, and my plugin would send to RE either 4 trade ticks, one for each value, or only 1 trade tick, the C value, depending on what the particular trading strategy looks at. My plugin may or may not send any non-trade ticks; let's assume that it will not.

The time I get a TDA 1 minute bar is not necessarily at a minute boundary per my computer's clock.

If I send RE one trade tick per minute, will it immediately generate a 1 minute bar with identical values for OHLC, and pass control to my trading strategy? This is what I wish to happen. Or is RE's bar creation on its own schedule, perhaps tied to my computer's clock, causing a lag of conceivably up to 59+ seconds between sending the tick to RE and RE generating the bar and invoking the trading logic?

The next question is whether there is a way I can I send 4 trade ticks in rapid succession (OHLC) each minute, and only after all 4 ticks have been sent, have RE immediately generate a 1 minute bar using those ticks, and pass control to my trading strategy.

BTW, your response seems to imply that if I send RE no ticks for a symbol within a 1 minute period, no bar will get generated for that minute for that symbol. Is this the case?


Edited: Friday July 17 2009 by Steve2008
Posted Friday July 17 2009
RightEdge bases it's time off of the ticks that come in from the data plugin, not off of your computer's internal clock. (Some data plugins such as the TWS plugin don't get time updates from the broker with every tick so the time is partly based on the computer's clock, but this is up to the plugin to decide.)

RightEdge won't generate a new bar for your system until it receives a tick that would be part of the next bar. There is now a TickType of CurrentTime that you can use. So to send a 1M bar that starts at 12:00, you would send the OHLC ticks with timestamps between 12:00:00 and 12:00:59, and then send a CurrentTime tick with a timestamp of 12:01:00 which would force the bar to be sent.

Thanks,
Daniel
Posted Friday July 17 2009
Sounds like what I need. Thanks.
Posted Wednesday July 22 2009
I am getting an orange error banner at the bottom of RE saying "Data from [begin datetime] to [end datetime] was dropped because it is out of order."

I sending 5 ticks every one minute bar as suggested (4 trade, OHLC, and 1 current time at beg of next minute) on a symbol list of 7 symbols, freshly created, and purged/reentered to ensure no residual previous data.

One symbol is updating properly in RE every minute. The others are not displaying any data, apparently because RE is dropping the ticks I send. All symbols are processed identically.

Diagnostic code shows that I am properly receiving data from TDA, and am putting correct values into the tick's time and symbol fields before calling tickListener. I am sending timestamps of 1, 2, 3 and 4 seconds into the even minute for the trade ticks and on the even next minute for the current time tick, and am working off my computer's clock, eliminating the possibility that the TDA timestamps are the issue.

Assistance would be appreciated. Thanks.


Edited: Wednesday July 22 2009 by Steve2008
Posted Wednesday July 22 2009
UPDATE: I compiled this same plugin for Edition 1, and it is working correctly there. I changed the tick type for the new-minute tick from current time to previous close (for my purposes, meaningless and harmless), because current time is an ED2 innovation.

Wondering whether the bar type in ED2 is the issue, I changed it there, too, to previous close. That does not help.

So this is an ED2 issue.


Edited: Wednesday July 22 2009 by Steve2008
Posted Wednesday July 22 2009
I am able to replicate the problem with a lightly modified version of the supplied random data realtime plugin. It works properly when compiled for and installed in ED1, and produces the same type of problems noted above in ED2.

Let me know if you would like the code. I have had problems posting stuff here -- quota errors -- so I can e-mail it. I can send either the C# file or the entire VCS Express 2008 projects, zipped.


Edited: Wednesday July 22 2009 by Steve2008
Posted Wednesday July 22 2009
You can email it to resupport at rightedgesystems.
Posted Wednesday July 22 2009
UPDATE: I just sent it, and then a corrected version to fix a bug.

Edited: Wednesday July 22 2009 by Steve2008


Similar Topics


Reading This Topic


2005-2017 © RightEdge Systems