Profile Picture

BarClosing in Live system with daily system frequency

Posted By kotl 6 Years Ago
Message
Posted Thursday December 29 2011
BarClosing happens too late, when exchange is already closed !

When working with NYSE and having option enabled for live data coming only during trading hours as well as option for IB Client to limit orders and data only to trading hours, BarClosing happens one minute before 24 hours pass since system was actually started.

Details:
- System frequency is daily.
- Live system started 11:00pm, so that it will work for the next trading day.
- System starts showing data in live data window at 9:30am next day.
- Live data stops at 4:00pm.
- BarClosing is called 10:59:pm next day.

The problem is - to get BarClosing to work properly and get called before exchange closes, live system needs to be started exactly w00t at 3:59pm of previous day, which is really wrong. This is happening with build 36.

Also, if this bug itself is fixed, will it possible to configure how many minutes in advance before exchange closes BarClosing is called? 5 minutes would be better than one.

Edited: Thursday December 29 2011 by kotl
Posted Thursday December 29 2011
BarClosing currently isn't meant to work for live mode.

For live mode, I'd recommend using the FilteredDailyFrequency. You can specify a start time of 9:30 am and an end time of 3:59 pm (or whatever you prefer). The bar from the filtered daily frequency will only include data in that time range, and NewBar will be raised just after the end time you specify (as long as you are still receiving live data).

Thanks,
Daniel
Posted Thursday December 29 2011
FilteredDailyFrequency does not work, especially when Live Data Start Date is specified, this exception is thrown always (otherwise it seems random on start):

RightEdge.Common.RightEdgeError: Partial bar should have been generated after processing time update tick.
Posted Friday December 30 2011
You need to update to the latest build of RightEdge for FilteredDailyFrequency to work.

Thanks,
Daniel
Posted Tuesday January 03 2012
Still does not work, throws exception if Live Data start date is specified:

Out of order time: 28/12/2011 3:59:59 PM Current time: 28/12/2011 4:00:00 PM
at RightEdge.Common.Internal.SystemRunner.xd47e553b3f44395c(DateTime xccf8b068badcb542)
at RightEdge.Common.Internal.SystemRunner.ProcessBarEvents(IEnumerable`1 eventList)
at RightEdge.Common.FrequencyManager.SendPendingBars()
at RightEdge.Common.FrequencyManager.UpdateTime(DateTime dateTime)
at RightEdge.Common.FrequencyManager.ProcessTick(Symbol symbol, TickData tick)
at RightEdge.Common.Internal.SystemRunner._tickGenerator_NewTick(Object sender, NewTickEventArgs e)
at RightEdge.Common.TickGenerator.ProcessBar(NewBarEventArgs args)
at RightEdge.Common.Internal.SystemRunner.ProcessBar(NewBarEventArgs newBars)
at RightEdge.Shared.SystemWrapper.RunSystem(SystemData systemData, SharedSystemRunData runData, ServiceFactory brokerFactory)
at RightEdge.Shared.SystemWrapper.RunSystem(String filename, ServiceFactory brokerFactory, PluginSettings dataStoreSettings)
at RightEdge.Shared.SystemWrapper.RunSystem(String filename, ServiceFactory brokerFactory, PluginSettings dataStoreSettings)
at RightEdge.Shared.TradingModuleWrapper.Run(String filename)
at RightEdge.Shared.TradingModuleWrapper.RunSystem(SharedSystemRunData systemRunData)
at RightEdge.SystemProgress.InitAndRunSystem()

Edited: Tuesday January 03 2012 by kotl
Posted Tuesday January 03 2012
Also, build 38 seems to be unable to work with multiple frequencies in Live mode, tried to get Hourly frequency and this is what I get:

Partial bar null for tick SPY DailyVolume 03/01/2012 7:40:45 PM
at RightEdge.Common.Frequency.SendTick(TickData tick)
at RightEdge.Common.FrequencyManager.ProcessTick(Symbol symbol, TickData tick)
at RightEdge.Common.Internal.SystemRunner._tickGenerator_NewTick(Object sender, NewTickEventArgs e)
at RightEdge.Common.TickGenerator.ProcessTick(NewTickEventArgs args)
at RightEdge.Common.Internal.SystemRunner.ProcessTick(Symbol symbol, TickData tick)
at RightEdge.Shared.SystemWrapper.ProcessTicks(Boolean moreTicksPending)
at RightEdge.Shared.SystemWrapper.ProcessTicks(Boolean moreTicksPending)
at RightEdge.Shared.TradingModuleWrapper.ProcessTicks(List`1 ticks, Boolean moreTicksPending)
at RightEdge.LiveSystemwindow.x8c3b46858f2d1101()
Posted Friday January 06 2012
We're investigating, please hang tight.
Posted Friday January 06 2012
For the exception when the live data start is specified, go to the service settings for the IQFeed plugin, click on "Other Settings...", and enable the "Ignore last historical bar" option. This will most likely fix the problem.

What is happening is IQFeed includes partial bars (ie bars that haven't completed yet) in its historical data. So if the time is 3:59:30 PM, the historical data will include the bar that startes at 3:59 and ends at 4:00. When you specify the live data start date, RightEdge loads the historical bars and sends them to the system before starting to send the live data. So the system ends up thinking the current time is 4:00, but a live tick for earlier than that comes in, which is an error. The "Ignore last historical bar" option in IQFeed will cause the last bar to be ignored when downloading data from IQFeed, which should fix the issue.

I wasn't able to reproduce the issue with using an Hourly frequency. Does the error occur when you have an empty system that just creates an hourly frequency? What is the system/data frequency set to and what symbols are you using?

Thanks,
Daniel

Posted Friday January 06 2012
I use IB and Yahoo for historical data for now. I have changed this setting for IB, completely purged data and downloaded again and it did not help: Once I cleaned data, Out of order time exception no longer shows up, however partial bur null is showing in two situations ( When Live Start date is set or if Live Start date is not set and Data is not active). It also seems that partial bar null problem has no correlation to hourly frequency, it was probably hidden before by bad data.

Attached is sample system that does not work, please change system frequency to Daily.

Partial bar null problem ONLY reproduces itself on start if 'Live Data' is not active. So make sure you stop it using menu Data->Stop and then starting Live system using menu System->Run Live. Once you Run live and see this exception, Data window stays active, so if you Run Live again -> it is not going to show this exception anymore.

Also, I have let it run with IB account when Data window was active before starting and got this later:

Partial bar null for tick SPY PreviousClose 04/01/2012 11:54:48 PM
at RightEdge.Common.Frequency.SendTick(TickData tick)
at RightEdge.Common.FrequencyManager.ProcessTick(Symbol symbol, TickData tick)
at RightEdge.Common.Internal.SystemRunner._tickGenerator_NewTick(Object sender, NewTickEventArgs e)
at RightEdge.Common.TickGenerator.ProcessTick(NewTickEventArgs args)
at RightEdge.Common.Internal.SystemRunner.ProcessTick(Symbol symbol, TickData tick)
at RightEdge.Shared.SystemWrapper.ProcessTicks(Boolean moreTicksPending)
at RightEdge.Shared.SystemWrapper.ProcessTicks(Boolean moreTicksPending)
at RightEdge.Shared.TradingModuleWrapper.ProcessTicks(List`1 ticks, Boolean moreTicksPending)
at RightEdge.LiveSystemwindow.x8c3b46858f2d1101()

Attachments
BarClosingTest.zip (201 views, 4.00 KB)

Edited: Friday January 06 2012 by kotl
Posted Sunday January 08 2012
In general, the out of order time exception happens when you have live data coming in with a timestamp that is earlier than the historical data you have. Usually this happens because the IQFeed plugin includes the partial bars in the historical download, or because there is a timezone mismatch between historical and live data. Hopefully that will help you if you run into this issue again.

With your system I was able to reproduce the "Partial bar null" error. It turns out this is a bug in how RightEdge works with the FilteredDailyFrequency. It will be fixed in the next build.

Thanks,
Daniel


Similar Topics


Reading This Topic


2005-2017 © RightEdge Systems