ODD Data in DX

[comewish]

I found some weird data in CSI #263 US Dollar Index.
Does it affect backtesting result?

[sluggo]

Yes that is a data error. The error is in the very last field of data, the "Unajusted Close" field. It is the ninth piece of data on each line.

You can use CSI UA to inspect the individual contracts themselves. In this case you would want to inspect the June 2006 contract of US Dollar Index. (The eighth piece of data in your screen image, "200706", tells you the contract year and month).

When you perform this inspection yourself, you will find that the individual contracts did not close in the vicinity of "8". They closed in the vicinity of "80". So the unadjusted close field in the continuous contract is wrong. It is wrong for the June 2006 contract, and for all contracts prior to that. Quite an extensive error!

This data error will affect backtesting (and real time trading!) if, and only if, your trading systems actually USE the Unadjusted Close price. In my experience, very few trading systems use the Unadjusted Close price. For example, none of the systems shipped with TradingBlox software use the Unadjusted Close price.

This is only a guess, a wild speculation: I suspect the reason why this error exists, and has existed for three years (!), is because no CSI customer has noticed it and complained to CSI. Usually when customers point out data errors, CSI fixes them reasonably quickly. I speculate that no customer has reported this error to CSI, because (A) relatively few people trade and follow the DX contract; and (B) of those few people who do trade or follow DX, none of them use or look at the Unadjusted Close field. If a tree falls in the forest and nobody is there to hear it fall, does it make a noise? http://en.wikipedia.org/wiki/If_a_tree_ ... n_a_forest

[comewish]

Thank you sluggo, you are helpful.
I have reported this to CSI.
So I think "count on contract rolls" in TB is not real value because it not use unadjust close.

[sluggo]

Blox uses changes in the DeliveryMonth (the eighth field in your screenshot) to detect rollovers. See the Trading Blox User's Guide, section 7.2.

[bobsyd]

If a man speaks in a forest and his wife isn't there to hear him, is he still wrong?

[babelproofreader]

If a man speaks in a forest and his wife isn't there to hear him, is he still wrong?

No. He is 100 % right for speaking outside of said wife's earshot.

[comewish]

This is CSI reply:

it's not really an error, it's not only un back adjusted, it's un historically adjusted
( precision changes aren't factored in )

It probably isn't right to do it this way, and it's not what would be expected.

Next version will just show un back adjusted unadjusted data only but
historical adjustments will be applied if specified in the historical adjustment option,
and not applied if it is turned off.

Best Regards,

Josh

[Eventhorizon]

I use the free CSI data made available on the TBB site. I use an R script to roll my own ratio-adjusted contracts rather than using the Panama-style contracts included in the files.

As part of the process I go through a data-validation step checking various characteristics of the raw closes, then I fix problems using another R script. I look for unusual absolute value, unusual changes at contract rolls (% change / length of roll-over > so many standard devs), unusual daily changes (using fitted extreme value distributions. see Extreme Value Dist).

You chould be aware that in addition to the particular issue you found there are 2 other biggies: The Euro series includes the transition from DMarks to Euro's, the entire NazDaq 100 series is out by a factor of 10.

I have attached my current error-resolution file. Let me know if you are interested in any clarifications.