Profile Picture

Out of order tick dropping in live data plugin

Posted By shawn_liu 5 Years Ago
Message
Posted Thursday April 26 2012
A lot of live tick data was dropped in my live data plugin. I printed the live data information in the debug envirentment. Lines which started with 'Out of order tick diff' are the output of RE itself. Others are the tick data from my plugin.

The data server provides live data for four Exchanges. So every frame of tick data has its own timestamp from its Exchange. Those data from a specified Exchange are in order. But the time between different Exchanges may be different.

It seems that there is a timestamp cache mechanism inside RightEdge. RE will remember the timestamp of the most recently tick data, and compare it with the newly comming tick data, if the newly comming timestamp is smaller, the tick data is dropped.

The possible solution is : RE not only caches the timestamp but also caches the name of this symbol. When a newly tick frame is comming, RE compares both the timestamp and the symbol name. Only the 'out of order' tick with the same symbol name should be dropped.

The following is the debug information:



cu1207 : Time = 14:30:00.500, LasPrice = 57620
ru1209 : Time = 14:30:00.500, LasPrice = 26790
zn1207 : Time = 14:30:00.500, LasPrice = 15510
IF1205 : Time = 14:30:02.500, LasPrice = 2631.6
m1209 : Time = 14:30:01.000, LasPrice = 3394
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
m1209 : Time = 14:30:01.000, LasPrice = 3394
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
cu1207 : Time = 14:30:01.000, LasPrice = 57620
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
ru1209 : Time = 14:30:01.000, LasPrice = 26795
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
IF1205 : Time = 14:30:03.000, LasPrice = 2631.6
cu1207 : Time = 14:30:01.500, LasPrice = 57620
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
ru1209 : Time = 14:30:01.500, LasPrice = 26795
zn1207 : Time = 14:30:01.500, LasPrice = 15515
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
IF1205 : Time = 14:30:03.500, LasPrice = 2631.4
m1209 : Time = 14:30:02.000, LasPrice = 3394
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
zn1207 : Time = 14:30:02.000, LasPrice = 15515
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
IF1205 : Time = 14:30:04.000, LasPrice = 2631.8
m1209 : Time = 14:30:03.000, LasPrice = 3394
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
zn1207 : Time = 14:30:02.500, LasPrice = 15515
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
IF1205 : Time = 14:30:04.500, LasPrice = 2631.8
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
Out of order tick diff: 00:00:02
m1209 : Time = 14:30:03.000, LasPrice = 3394
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
ru1209 : Time = 14:30:03.000, LasPrice = 26795
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
zn1207 : Time = 14:30:03.000, LasPrice = 15515
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Out of order tick diff: 00:00:01
Posted Thursday April 26 2012
When running a system, the ticks need to all be in order, even for different symbols. A tick for one symbol updates the time for all the symbols, and hence may send out a new bar for other symbols. If a tick comes in with an earlier time it can't be added back to the bar that was already sent out.

If the difference in time between the ticks is less than 0.5 seconds, RightEdge will modify the time of the "out of order tick" so it is the same timestamp as the latest tick received. That way it can still be processed by the system without causing errors due to being out of order. To avoid dropping ticks, you can do the same thing in your plugin but with a larger time difference tolerance.

Thanks,
Daniel
Posted Tuesday June 17 2014
Where is the time difference set in TWSPlugin and IQFeed? I'm using both of these feeds along with the Random feed and it is quite common that my live data shows all zeros. Currently, I have to restart the live feed several times in order for data to appear.

Thanks, Duane
Posted Sunday June 22 2014
For the TWS plugin, the offset is set when IB sends the CurrentTime message, which the TWS plugin requests when connecting. See the client_CurrentTime method.

For IQFeed, there's no automatic offset. The data coming from the IQFeed APIs appears to already be in the computer's local time zone.

Thanks,
Daniel
Posted Wednesday June 25 2014
I see the local variable ‘accountTimeDiff’ set in the TWS client_CurrentTime method but where can I set a larger time difference tolerance per your post on 4/26 in order to prevent the error, for example, with a time difference of 1 sec? Where is the 0.5 sec set?

Thanks, Duane
Posted Friday June 27 2014
I have tracked down the issue to the random tick generator when used with either IQFeed or TWS plugins. There are no issues when using just IQFeed and TWS together. My assumption is that my clock, even though it is synced to a stratum 1, is slightly off compared to the times that are being returned on the other two.

Also, I’m not sure the 0.5 seconds gives enough differential when using multiple live data providers.

What are your thoughts to fix? I need to fix asap because I’m running live systems of it. Note, I technically do not need the random tick data (just the prior day closing price).

Thanks, Duane
Posted Monday June 30 2014
With the way RightEdge handles processing incoming events, it can't handle data that comes in out of order. So the workaround when the data is less than 0.5 seconds out of order is to change the timestamp of the out-of-order data so it matches the most recent timestamp received previously.

Would it help if you were able to increase the margin of error to something bigger than 0.5 seconds? Is it a problem that the timestamps on the incoming data would be changed?

Thanks,
Daniel
Posted Monday June 30 2014
Daniel,

   I’m not sure changing the timestamps will work. The question then becomes, what do I change it to? If I add/subtract x sec to the timestamp set in the live random plugin, it is very likely that the problem will still be present except that the ticks from the random plugin will be ahead/behind of the other two plugins.

Yes, I believe changing the margin to greater than 0.5 sec should work. Where is this set in the live random plugin?

Thanks, Duane
Posted Thursday July 10 2014
Daniel,

Where is this set in the live random plugin?

Thanks, Duane
Posted Friday July 11 2014
Hi Duane,

There are two types of adjustments going on. The first is in the live data plugin. Currently only the TWS plugin does this. TWS doesn't send a timestamp with each tick, so the plugin uses the current system time adjusted by the offset stored in the accountTimeDiff field.

The other adjustment is in RightEdge's live data processing. If it detects out of order ticks, it will drop them if they are more than 0.5 seconds out of order, otherwise it will adjust the timestamp of the tick.

If you are just using one data plugin, then the plugin should just send ticks in the correct order and there won't be any out of order issues. However when you have multiple plugins they tick timestamps may be set differently. With the IQFeed plugin the timestamp is sent by the server for each tick, with TWS the local time plus the offset is used, and with the random data plugin, just the local time is used. So it's quite possible to end up with out of order data.

I'm not sure there's a great solution to this. I could make the 0.5 second margin of correction larger or a user-controlled setting, but then the tick timestamps will be adjusted more which doesn't seem great. If your trading system doesn't depend on reacting quickly to market data, it would be possible to buffer the ticks for longer (for example for 15 seconds) and sort any ticks that come in within the buffer by the timestamp. That would be kind of complicated to write however.

What do you think?

Thanks,
Daniel


Similar Topics


Reading This Topic


2005-2017 © RightEdge Systems