Traders' Roundtable

I am a new customer with CSI. Recently, on more than one occasion, I have noticed lots of data corrections in many contracts after I perform a daily update. Strangely, none of these corrections are to the Close and Unadjusted Close prices. I'm comparing data from the same delivery month, so no spread/backadjusting issues there.

This ofcourse changes a lot of my backtesting results. Is anyone else here facing these issues? I emailed CSI Tech Support but no response so far. Any help would be greatly appreciated.

Below are few sample data lines for 'AD' contract before and after my last update:


Date Open High Low Close Vol OpenInt DeliveryMonth UnadjustedClose
19960201, 0.6495, 0.6538, 0.6493, 0.6531, 4697, 7534, 199603, 0.7463
19960202, 0.6611, 0.6638, 0.6596, 0.6637, 5765, 9416, 199603, 0.7569
19960205, 0.6600, 0.6612, 0.6591, 0.6606, 3689, 10977, 199603, 0.7538
19960206, 0.6588, 0.6602, 0.6576, 0.6592, 1511, 11359, 199603, 0.7524
19960207, 0.6603, 0.6610, 0.6560, 0.6566, 2844, 11322, 199603, 0.7498
19960208, 0.6638, 0.6643, 0.6620, 0.6625, 864, 11390, 199603, 0.7557
19960209, 0.6608, 0.6613, 0.6594, 0.6601, 366, 11341, 199603, 0.7533
19960212, 0.6613, 0.6623, 0.6591, 0.6595, 320, 11337, 199603, 0.7527
19960213, 0.6589, 0.6608, 0.6578, 0.6607, 130, 11360, 199603, 0.7539
19960214, 0.6599, 0.6630, 0.6594, 0.6603, 425, 11184, 199603, 0.7535
19960215, 0.6614, 0.6618, 0.6602, 0.6605, 365, 11253, 199603, 0.7537


Date Open High Low Close Vol OpenInt DeliveryMonth UnadjustedClose
19960201, 0.6507, 0.6538, 0.6493, 0.6531, 4718, 7534, 199603, 0.7463
19960202, 0.6539, 0.6638, 0.6534, 0.6637, 5786, 9416, 199603, 0.7569
19960205, 0.6615, 0.6650, 0.6580, 0.6606, 3703, 10977, 199603, 0.7538
19960206, 0.6588, 0.6602, 0.6576, 0.6592, 1511, 11359, 199603, 0.7524
19960207, 0.6603, 0.6610, 0.6560, 0.6566, 2844, 11322, 199603, 0.7498
19960208, 0.6594, 0.6643, 0.6594, 0.6625, 883, 11390, 199603, 0.7557
19960209, 0.6646, 0.6656, 0.6594, 0.6601, 387, 11341, 199603, 0.7533
19960212, 0.6613, 0.6623, 0.6591, 0.6595, 320, 11337, 199603, 0.7527
19960213, 0.6593, 0.6609, 0.6578, 0.6607, 134, 11360, 199603, 0.7539
19960214, 0.6598, 0.6630, 0.6594, 0.6603, 429, 11184, 199603, 0.7535
19960215, 0.6594, 0.6618, 0.6594, 0.6605, 365, 11253, 199603, 0.7537

Thanks!

I've been a CSI customer for years (I have no other affiliation). I've occasionally found missing or incorrect data in my historical testing. Communication with CSI has always led to corrections in the historical data. Every daily update has data corrections, some are historical (such as the ones I - and others - found during historical testing) and some are recent.

I feel that CSI is really committed to achieving accurate data, but they also face multiple original data sources, conflicts between data sources, and deadlines in their daily updates. For me, I feel they do a very good job in balancing the tradeoff between timeliness of the updates and accurate data. To their credit they don't stop analyzing the daily data once they post the daily update. When they find discrepancies later - even years later, they issue corrections via the next daily update.

Please let us know what kind of response you get from CSI on your email. If you don't hear from them in a few days, I suggest you send them another email.

Mowgli, I have often seen the same thing - data corrections from 3 or 5 or 7 or 10 (!!!) years ago. Such data corrections make little sense and in no way support the gushing claim that CSI are " really committed to achieving accurate data" as PaulZ asserts in his apparently non-affiliated support post.

If, in my employment, I claimed to be really committed to accurate data, I certainly wouldn't then take 10 years to find my mistakes.

Actually now I think about it, the situation seems so absurd on the surface that there simply must be more to the story than we are seeing. For example, perhaps the IMM pit at the CME have identified an error and today are correcting the previously reported open high and low from trading in Feb of '96? I think not.

And Mowgli - good luck with obtaining satisfactory customer support.

(disclaimer or something like it: I freely admit to having a bias against CSI. They are accepted as good by default.... only because there is nothing better. We eat that which we are given).

Paul and other CSI " veteran" users can you please suggest what "standard options" you check when collecting backadjusted data for commodities? For istance, do you leave checked "detrend/invert"? I was playing with it and found that I got different trading results, its confusing. Whats the difference between SF0_I0B and SF20_I0B ? whats better and how it influences the backtesting?

thanks in advance

The following is my personal experience of how CSI is "committed" to provide accurate and clean data:
In early this month I found that the CSI commodity number 119 (HSI) has several MONTHS of missing contracts between Mar 1989 to Jan 1991. I sent an email to CSI for investigation. Their answer was their database department found "no records of these contracts to have ever traded".
I looked up the Hong Kong Exchange web site myself and found that the exchange themselves sell the whole database of HSI for only HK$600. I emailed CSI again and direct them to the HKEX website for the data. After several days still no action was taken. I sent another email to CSI to ask them what happened. Their answer was they will not add the missing data, without any further explanation.
If you look up the CSI data for HSI, you could see a whole lot of missing data and large gaps within this period.
The conclusion: CSI no longer provides clean data. Period.

ANyone get bad data for the Euro BOBL tonight for the date of 06-06-2007, the decimal point of off by a factor of 10. Instead of 106.57 that day shows 1065.7. All the rest of the days are normal. Weird.

I can't remember the exact date but it was immediately after the alteration by the exchange to settling the instrument in half ticks. My broker made the same mistake resulting in my short position showing an alarming large loss running into many millions of dollars.

Turbowagon wrote:ANyone get bad data for the Euro BOBL tonight for the date of 06-06-2007, the decimal point of off by a factor of 10. Instead of 106.57 that day shows 1065.7. All the rest of the days are normal. Weird.

Yes I got that too. Twice. Once on my "UA 2.9.2" computer and again on my "UA 2.9.1" computer. Thus I don't believe the problem is in the software code; I think the downloaded data packets from the Boca Raton mothership were the source of the problem.

As with much else in the USA, the squeaky wheel gets the grease. The more customers who send complaints to CSI, the sooner problems will get fixed. So, email to database@csidata.com before posting here. An angry cabal of guerrilla DJ EuroSTOXX traders raised holy hell on 17 May 2007 after (this) UA hiccup, and it got fixed right away

they promise that it will be fixed with tonight's update

Wed 6/20/2007 13:01 Eastern time, performed "Download replacement Fact files" followed by "Get Data", problem has gotten worse instead of better. Now all days before 6/6/2007 are wrong. See image.

the reply i got from Rudi at CSI which worked is as follows:
Download replacement fact files with "download new stock mkt specs" UNCHECKED. Then Refresh Price history on the 2 bobl cts. nºs 996 and 553. Atferwards rebuild files.
This worked for me.

I gotta say, seeing these sorts of issues that come up with CSI data now and then, does not encourage me to switch over to them.

As a Pinnacle Data user, I seem to struggle along each day just fine - even though I'm limited to only about 40 markets, including some nice International ones. Data corrections (which don't happen often) are automatically corrected in the next day's update.

Now that TB shows the last price in the Generated Orders print screen, the motivation to catch curent active month detail is less also (thanks Tim!).

Guess I'm still confused why CSI is worth 3 to 4 times the price of Pinnacle.

equa wrote:Download replacement fact files with "download new stock mkt specs" UNCHECKED. Then Refresh Price history on the 2 bobl cts. nºs 996 and 553. Atferwards rebuild files.
This worked for me.

It worked for me too, thanks! Actually it is not NECESSARY to un-check "download new stock mkt specs", this was only recommended because it saves lots of time. (The stock market specs are a huge file). What makes this procedure succeed is (A) download new commodity facts; followed by (B) Refresh price history on #553 and #996. Image below.

LeapFrog wrote:cut

Guess I'm still confused why CSI is worth 3 to 4 times the price of Pinnacle.

see graph attached

Note interestingly, Dial Data was once somehow affiliated with TradeStation.

I like it!

And the independent research organization that produced this table was...?

LeapFrog wrote:I like it!

And the independent research organization that produced this table was...?

Futures Magazine

http://www.csidata.com/csi/reviews/reprintsep99.html

Note that this study was done in 1999. The data used was from 1994 to 1998.

Yes. We should all drop a note to the editor suggesting an updated review of that accuracy study.

Here's a link to the contact info. I myself will drop a note tomorrow with that request / suggestion. Also a longer period of testing seems in order. Even from 97-07 would be most interesting.

I'm all for keeping csi on its toes... Seems validating the accuracy would [at best] not be a simple task...

http://www.futuresmag.com/cms/futures/w ... al+contact

EDIT: My Letter:

Dear Editors of Futures Magazine,

Your article, - Futures Magazine 09/99
How Clean is Your End of Day Data?
Has come up in a discussion at Traders Roundtable Forum.

I feel that it would be appropriate for Futures to run an updated study on data accuracy. CSI data, claims to have the best database based upon the results of your 1999 article. Being that almost ten years have elapsed, I would like to request or suggest, that a new scientific study be completed and published.

This would be a wonderful service to your readership and of great importance to the trading community at large.

Cordially,

Whoops, another data problem in CSI Unfair Advantage. This time on the currency cross-rate futures at FINEX. CSI Commodity #623, Euro over Australian Dollar futures.

Looks like a digit problem. Found with Futures Data Investigator.

Whatever the problems with CSI (and they can indeed be exasperating) where else can you get what you want? Unless like Mark you write your own software.

Take stock data: can anyone point me to a reliable source for the European markets where they backadjust for dividends? Unfortunately CSI's coverage is limited to very partial coverage of the LSE.