Profile Picture

SetAccountState

Posted By Freolad 9 Years Ago
Message
Posted Thursday December 18 2008
I'm working on a broker plugin and looking to get order/fills/position syncing working via SetAccountState. I'm attempting to reconcile the state of any orders + fills at the broker with the state in RE such that at the end of the syncing, RE matches the broker state. What I've implemented so far is:

1. Obtain orders + fills + positions from broker.
2. Go through all RE orders and update them based upon the corresponding state at the broker.
3. If there are additional orders / fills at the broker, add them to the PendingOrders collection in BrokerAccountState.
4. Set BrokerAccountState.BrokerOverride = true

I've stepped through the code in the debugger and all seems to be going well in SetAccountState. Except, after returning from the call to the broker plugin, one of either two things occur:

1. An exception is thrown by RE and the live strategy is stopped:

An exception of type System.Collections.Generic.KeyNotFoundException was thrown.
The given key was not present in the dictionary.
at System.ThrowHelper.ThrowKeyNotFoundException()
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at RightEdge.Common.SystemStatistics.UpdateStats(BarStatistic stats, DateTime dateTime, Dictionary`2 prices, IEnumerable`1 openPositions, IAccountInfo accountInfo, Dictionary`2 positionValues)
at RightEdge.Common.SystemStatistics.UpdateStats(BarStatistic stats, DateTime dateTime, Dictionary`2 prices, IEnumerable`1 openPositions, IAccountInfo accountInfo)
at RightEdge.Common.SystemStatistics.NewBar(NewBarInfo info, IEnumerable`1 openPositions, IAccountInfo accountInfo)
at RightEdge.Common.BaseSystemHistory.SimNewBar(NewBarInfo info)
at RightEdge.Common.SystemData.x56e784a497b3cb60(NewBarInfo x8d3f74e5f925679c)
at RightEdge.Common.Internal.SystemRunner.NewBar(Dictionary`2 bars)
at RightEdge.Shared.SystemWrapper.NewBar(Dictionary`2 bars)
at RightEdge.Shared.SystemWrapper.NewBar(Dictionary`2 bars)
at RightEdge.Shared.TradingModuleWrapper.NewBar(Dictionary`2 bars)
at RightEdge.LiveSystemwindow.x2e01f099ee7feedb(Dictionary`2 xd3a52115666f4605)


2. Or alternately, if I try to start the Live strategy again, it does start up successfully but no pending orders or open positions are displayed in the Live System tab.

Stopping and starting the Live strategy toggles between the two states above. Could someone offer some suggestions / tips? Is it something I'm doing wrong?


As an additional curiosity, while debugging the above, I commented out all my code in SetAccountState and attempted to restart a Live Strategy that previously had submitted some orders. I elected to select all the previously open positions. When I did so, RE threw this exception:

Could not convert from NZD to USD for date 1/01/0001 12:00:00 AM.

at RightEdge.Common.ReturnCode.ThrowFailure()
at RightEdge.Common.PositionInfo.GetCloseTrade(Double price, DateTime time, IAccountInfo accountInfo, PositionStats stats)
at RightEdge.Common.PositionInfo.GetUnrealizedProfit(PositionStats stats, Double currentPrice, IAccountInfo accountInfo)
at RightEdge.Common.SystemStatistics.UpdateStats(BarStatistic stats, DateTime dateTime, Dictionary`2 prices, IEnumerable`1 openPositions, IAccountInfo accountInfo, Dictionary`2 positionValues)
at RightEdge.Common.SystemStatistics.UpdateStats(BarStatistic stats, DateTime dateTime, Dictionary`2 prices, IEnumerable`1 openPositions, IAccountInfo accountInfo)
at RightEdge.Common.SystemStatistics.SetExistingPositions(IEnumerable`1 openPositions, IAccountInfo accountInfo)
at RightEdge.Common.SystemData.xb3c7e5b465e8f0f3(PortfolioXml x839850fad88ff58d)
at RightEdge.Common.SystemData..ctor(SystemDataCreationSettings settings)
at RightEdge.Shared.SystemWrapper.CreateNewSystem(SystemRunData data)
at RightEdge.Shared.SystemWrapper.RunSystem(DateTime start, String filename, ServiceFactory brokerFactory)
at RightEdge.Shared.SystemWrapper.RunSystem(DateTime start, String filename, ServiceFactory brokerFactory)
at RightEdge.Shared.TradingModuleWrapper.RunSystem(SystemDataCreationSettings settings, Dictionary`2 historicalData, SystemRunInfo dndInfo, ServiceAppDomainFactory brokerFactoryFactory)
at RightEdge.SystemProgress.InitAndRunSystem()


Any attempt to restart the Live Strategy and accept the previously submitted orders resulted in this exception and the strategy being stopped. The only way to start it up was to uncheck them all. But then I can't sync then via the broker SetAccountState either. So at the moment, if a live strategy stops, it won't pick up where it left off. But worse, I'll have potentially open positions hanging around that my strategy won't know about.

On a whim, I took a look at the LiveOpenPositions.xml document in my strategy folder. There's a lot of info in there but I notice that all the ExpirationDates for the positions are set to 0001-01-01. Is this a coincidence? This is the same date as was throw in the exception above. I'm not sure where this date is coming from.

Can anyone help with any of the above? Happy to provide more info. Thanks.
Posted Thursday December 18 2008
Hi Freolad,

After reading through your post I think I may have more questions than suggestions for you, but maybe some of this will be helpful anyway...

One observation, both of the exceptions you're getting appear to happen somewhere inside RightEdge.UpdateStats().

From the API docs on SetAccountState() regarding BrokerOverride

Note that if the broker sets this option to true, the positions created for your system will not have all of the information that the saved positions would have had. Multiple positions per symbol will be combined into a single position. There will be no profit target or stop loss on the positions (although the orders that were submitted as profit targets or stop losses may still be pending as UserSubmitted orders). The orders and positions will have no timeout / bar count exit value. Also, the open positions will only contain one trade in their history instead of whatever history of fills may have occured.

Could some of these limitations be affecting what you are trying to accomplish?

Could not convert from NZD to USD for date 1/01/0001 12:00:00 AM

Are you trading a symbol where the Assest (quote) currency is NZD?
Is your RightEdge Account Currency USD?
Do you have a watchlist entry for NZD/USD?
Are you calculating interest on Forex positions?

On a whim, I took a look at the LiveOpenPositions.xml document in my strategy folder. There's a lot of info in there but I notice that all the ExpirationDates for the positions are set to 0001-01-01. Is this a coincidence?

I think this is a coincidence, 1/1/0001 00:00 is the DateTime.MinValue. The expiration date in the live open positions file is referring to a futures/options expiration.

If you are creating new BrokerOrder records, are you setting the SubmittedDate value?
If you are creating new BrokerPosition records, are you setting the EntryDate value?
If you are creating new Fill records, are you setting the FillDateTime value?

3. If there are additional orders / fills at the broker, add them to the PendingOrders collection in BrokerAccountState.

How are you adding additional orders? When adding a fill to open a limit order, it seems that you would want move the order out of PendingOrders and instead create an entry in Positions instead. If you're filling a target/stop orders I think you just want to remove the orders from PendingOrders as well as the corresponding position from Positions.

HTH,
Mark
Posted Thursday December 18 2008
Are you creating new Symbol instances in your SetAccountState?  The KeyNotFoundException looks like it may be because you are doing this and the ones you are creating don't match the ones selected from the watchlist to run your system with.

The error converting from NZD to USD sounds like you need to add USD/NZD as a forex symbol to your watchlist, and you need to have some data for it.

Thanks,
Daniel

Posted Thursday December 18 2008
dplaisted (12/18/2008)
Are you creating new Symbol instances in your SetAccountState? The KeyNotFoundException looks like it may be because you are doing this and the ones you are creating don't match the ones selected from the watchlist to run your system with.

Hmm. Indeed I am creating new Symbol instances in SetAccountState but I thought I was supplying all the relevant info to match the ones on the Watchlist. Will investigate further. Thanks. Do I need to configure BaseCurrency and CurrencyType or will these be done automatically? Alternatively, is there anyway I can query the current symbols in the Watchlist and select the one I'm interested in?

The error converting from NZD to USD sounds like you need to add USD/NZD as a forex symbol to your watchlist, and you need to have some data for it.

Sorry it was late at night when I made my posting and I forgot to mention that I did have the NZD/USD symbol selected in the watchlist and I do indeed have plenty of data for it.

However, re-reading your comment, I noticed that you recommended the symbol USD/NZD. The NZD to USD exchange rate is _very_ rarely quoted as USD/NZD. Its usually quoted as NZD/USD. Could this be the problem though? RE is expecting a USD/NZD symbol and failing to find it?

Thanks.
Posted Thursday December 18 2008
Hi Mark,

Thanks for your thoughtful comments.

From the API docs on SetAccountState() regarding BrokerOverride

Note that if the broker sets this option to true, the positions created for your system will not have all of the information that the saved positions would have had. Multiple positions per symbol will be combined into a single position. There will be no profit target or stop loss on the positions (although the orders that were submitted as profit targets or stop losses may still be pending as UserSubmitted orders). The orders and positions will have no timeout / bar count exit value. Also, the open positions will only contain one trade in their history instead of whatever history of fills may have occured.

Could some of these limitations be affecting what you are trying to accomplish?

It certainly is possible. I don't really know enough about how RE handles this internally to comment. I had assumed that if I simply update the existing orders/positions then they wouldn't lose all their history, targets, timeouts, etc. I guess I assumed that the above would only apply to new orders / fills / positions that I had submitted. But perhaps this is not the case.

It's a difficult call to make. On the one hand I want to sync up RE's state with that of the Broker so that the trading strategy can accurately open/close positions. But it seems like the synced up orders may not be managed properly within RE (no automatic profit/stop loss orders created for them, no automatic cancellations after bar count). And the strategy may be unaware of this leaving itself potentially exposed.

But on the other hand, if I don't do this, there's the risk that the broker state won't match the RE state when the two re-connect. Thus the strategy may be trading inappropriately or worse, may be unaware of an open position which won't have an associated stop or profit target.

Could not convert from NZD to USD for date 1/01/0001 12:00:00 AM

Are you trading a symbol where the Assest (quote) currency is NZD?
Is your RightEdge Account Currency USD?
Do you have a watchlist entry for NZD/USD?
Are you calculating interest on Forex positions?

Sorry I forgot to mention in my original post that my Account Currency is USD, and I do have NZD/USD in the watchlist. I do have Apply interest to open Forex positions selected. I don't really know what the latter does in a Live situation. I assumed it was only used for backtesting.

If you are creating new BrokerOrder records, are you setting the SubmittedDate value?
If you are creating new BrokerPosition records, are you setting the EntryDate value?
If you are creating new Fill records, are you setting the FillDateTime value?

Happily I can answer yes to all of these. Smile

3. If there are additional orders / fills at the broker, add them to the PendingOrders collection in BrokerAccountState.

How are you adding additional orders? When adding a fill to open a limit order, it seems that you would want move the order out of PendingOrders and instead create an entry in Positions instead. If you're filling a target/stop orders I think you just want to remove the orders from PendingOrders as well as the corresponding position from Positions.

Hmm. Ok. I guess I assumed that I could update the status of any orders in the PendingOrders collection (to Filled or Cancelled) and when it was returned to RE that it would sort through them, update its own orders appropriately and then manage any orders that were in a Submitted state. I was attempting to retain as much of the original RE order info/state as possible to ameliorate the BrokerOverride restrictions. A pity it doesn't work like this. Thanks for the recommendation on how to implement SetAccountState. I'll look at modifying my algorithm.

Thanks.
Posted Sunday January 25 2009
Hi Daniel,

Many thanks for your earlier assistance.

dplaisted (12/18/2008)
The error converting from NZD to USD sounds like you need to add USD/NZD as a forex symbol to your watchlist, and you need to have some data for it.


To recap, I do have the symbol NZD/USD in my symbol watchlist and I do have some data for it however I still get the error. Is RightEdge expecting to use the symbol USD/NZD to calculate profit/loss when starting up/re-connecting? ie, the inverse pair to NZD/USD.

In normal trading RightEdge seems able to use the NZD/USD pair to calculate profit/loss. So if it does need the inverse pair when starting is it possible to modify the startup/reconnect code to use the same logic as during normal trading and hence use the regularly quoted pair?

Cheers.

Posted Tuesday January 27 2009
I've figured out what the problem is and a new build with the bug fixed should be out soon.

Thanks,
Daniel

Posted Wednesday January 28 2009
Build 382, which should fix the currency conversion issue with saved positions, is now available.

Thanks,
Daniel

Posted Friday September 17 2010
I'm also updating Symbol properties on Startup of the SymbolScriptBase and I'm also getting the key not in dictionary issue.  I was attempting to update Symbol Information because IQ Feed can't accept the date format on the back of the Futures contract so I have to input these as Stock's but then my account values get all screwed up because it believes I'm trading stocks and not futures.  Is there anyway around this without switching to a different data provider?  Any thoughts?

Thanks in advance.

-Eric

Posted Friday September 17 2010
The source code to the IQFeed plugin is included with RightEdge, so you could modify it to send the correct symbol name to IQFeed. The logic is located in the FormatSymbol method in IQFeed.cs.

We are also working on a way to solve this problem in a better way.

Thanks,
Daniel


Similar Topics


Reading This Topic


2005-2017 © RightEdge Systems