How to backfeed a data gap ?

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
KBochez
Contributing Member
Contributing Member
Posts: 5
Joined: Tue May 25, 2004 10:21 pm
Location: JAPAN

How to backfeed a data gap ?

Post by KBochez »

Have the following question:
In real execution or papertrading, let's assume that you unplugged by accident your network cable; this leads to a gap in your datafeed. Basically you have to restart your system and during the initialization, while getting the new real-time data, you need to fill the gap in your past data... or you won't be able to properly calculate indicators based on history: for example if you are using a 1 hour moving average and missed 10 minutes of data, you either fill the 10 minute gap and restart trading soon after or wait more than 1 hour so that the gap in your moving average disappears (which is not a pratical solution because your position can very well collapse during that 1 hour !)
Hence the question is:
How do you implement in the code the backfeeding process ? You will have to catch the new real-time data while doing the backfeeding and I can't get a clear idea of how things should be done at the code level.
Can anyone help ?
Thank you very much in advance for your help
Karl
Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt »

If you are actually dealing with a live streaming feed you need redundancy to solve this problem.

Many of the feeds I've seen do support a query response for historical data which can be used to fill in missing pieces in the event of a problem.

The answer to the question is so dependent on the particular data feed you use, its API and communications protocol, that it's almost impossible to provide an intelligent answer without specific knowledge of the data feed you have.

There are also products like MarketStream which handle this sort of thing themselves (see: www.mktstream.com).

- Forum Mgmnt
batuco9
Roundtable Fellow
Roundtable Fellow
Posts: 53
Joined: Thu May 29, 2003 12:28 am
Location: Chile

Veritrader 2 and past order generation.

Post by batuco9 »

Which reminds me of another issue, though more applicable, perhaps, to daily data. If for the same reason, or other, one misses a couple of days, would it be possible for Veritrader to eventually be able to step through past days generating the corresponding orders? I know one can get them via the log files, but I think such an option would make it much easier to review the orders that would have been made.
Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt »

In 2.0 you will be able to step through each day's trading logic to review each day's orders, which one's get fills, positions before and after the day, etc.

We are putting this functionality into our debugger so that traders (including us for our own trading) can test their logic thoroughly to make sure their system code is doing exactly what they expect.

It will also likely reduce our support requirements since we get a lot of questions or bug reports about missing trades that are due to things like insufficient capital or volume eliminating a trade.

In 2.0 you will be able to easily see the reasons why an order was not taken.

- Forum Mgmnt
Post Reply