trades changing due to continuous contracts

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
michaelt
Senior Member
Senior Member
Posts: 37
Joined: Wed Feb 03, 2010 10:53 am
Location: princeton,nj

trades changing due to continuous contracts

Post by michaelt » Wed Nov 10, 2010 4:08 pm

Has anyone seen something similar with their testing?

I can get a different history of trades with a CSI continuous contract after it rolls to the next forward.

The trading system is unchanged.
The measurements in the system are standardized -- i.e. dollar value trma (as opposed to % trma). So the absolute value of Crude shouldn't matter.

If the value in the contract goes negative due to back adjusting I add a value to each price to avoid negative values when trading.

The continuous method is back adjusted, and not proportional.

While some trades dates are the same (and at different prices reflecting the roll in the continuous contracts) - some trades are different.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Wed Nov 10, 2010 6:27 pm

That happened to me once, years and years ago, when I forgot to enable the "Generate Forward" option of my continuous contract builder software.

I was rolling when open interest crossed; however, by failing to "Generate Forward" I had unwittingly told my software to roll over the last time the open interest of the new month exceeded the open interest of the old month. Oops. My intention was to roll over, please, on the first open interest cross, but regrettably that ain't what I told the software to do. And, if open interest crossed over more than once, the roll dates moved around. Oh, ick. P.U.

Time passed and I eventually decided I preferred calendar date based rollovers rather than open interest rollovers. There are a number of pros and cons to this choice; one of the benefits is that each calendar date occurs precisely once. And it occurs in the same position whether you Generate Forward or not :)

michaelt
Senior Member
Senior Member
Posts: 37
Joined: Wed Feb 03, 2010 10:53 am
Location: princeton,nj

Post by michaelt » Fri Nov 12, 2010 7:20 am

Thanks Sluggo.

I have found what I think are errors in the CSI continuous contracts. I have found some occurrences (not many) where there is a different day to day movement compared to the (underlying) individual contracts. It's happened in the T-Bond and Crude. I have found one period where there is a 3 month data gap in the continuous contract (Crude).

From time to time I have found historical differences in a set of trades. I have traced it back to the differences in the data.

My systems are by design indifferent to the nominal price level (as you might expect). So changing the price level (i.e. which is what happens when the contract is rolled and adjusted into new continuous) should not, of course, affect trades.

But when the relationship of day to day prices within the continuous contract changes from a prior continuous contract, it causes a different trade stream.

This is subtle, and can only be detected by examining the underlying and the continuous contracts day to day changes. And when the roll occurs, the differences for every day between continuous and underlying should be the same.

I am in the process of generating my own continuous contracts, so if an anomaly occurs again I can trace the problem. If an historical trades changes, I can check both portfolios: CSI and my continuous contracts.

Thanks.

michaelt
Senior Member
Senior Member
Posts: 37
Joined: Wed Feb 03, 2010 10:53 am
Location: princeton,nj

Copper missing data; CSI individual & continuous contrac

Post by michaelt » Tue Nov 16, 2010 8:13 am

UPDATE TO THE BELOW: I heard from CSI and they say it's a database refresh issue. If I hadn't visually inspected the file I wouldn't have found the error (although my trading account would be impacted nonetheless). If the refresh doesn't repair this issue I'll update this post. Update: That fixed the data.


Hi all,

My CSI HG20 continuous contract now has a gap from March 15 - July 1, 2010.

I checked the individual contracts (2010: H, K, N).
Both H & K data ends March 15. N has a gap from March 15- July 1.

Anyone else see this in their data?

Last week I found my continuous GC20 had the same problem, same dates, but that seems to have cleared up.

Below is the May 2010 contract (HG22010K). This is the end of the file. As May should be active at this point, the file ending March 15 doesn't make any sense to me.

20100310 339.90000000 344.60000000 335.05000000 336.80000000 34927 87193 39052 128203
20100311 338.00000000 339.95000000 333.85000000 337.70000000 31163 87811 33468 129116
20100312 339.00000000 340.65000000 335.85000000 338.00000000 23204 86863 24942 128169
20100315 338.30000000 339.15000000 329.00000000 331.50000000 0 86863 0 128169


This is the March 2010 contract.

20100310 341.10000000 342.95000000 334.00000000 335.50000000 320 2755 39052 128203
20100311 336.00000000 337.85000000 332.70000000 336.60000000 332 2574 33468 129116
20100312 335.60000000 338.50000000 335.60000000 336.95000000 584 2445 24942 128169
20100315 335.00000000 335.00000000 328.50000000 330.70000000 0 2445 0 128169

This is the July contract. You can see the data gap there as well:

20100311 339.30000000 340.75000000 335.75000000 339.20000000 1133 18724 33468 129116
20100312 340.60000000 342.00000000 338.20000000 339.50000000 778 18735 24942 128169
20100315 338.15000000 338.20000000 330.90000000 333.00000000 0 18735 0 128169
20100701 289.65000000 291.95000000 284.45000000 286.50000000 1728 2740 40805 130777
20100702 290.45000000 294.00000000 288.85000000 290.40000000 350 2396 25172 128495

September contract:
20100315 335.50000000 336.25000000 332.80000000 334.10000000 0 8851 0 128169
20100701 291.00000000 293.60000000 285.55000000 287.70000000 36666 79674 40805 130777
20100702 289.55000000 295.80000000 289.45000000 291.60000000 23925 77714 25172 128495

October:
20100315 334.35000000 334.35000000 334.35000000 334.35000000 0 309 0 128169
20100701 292.95000000 292.95000000 288.20000000 288.45000000 34 5989 40805 130777
20100702 292.30000000 296.50000000 291.60000000 292.40000000 16 5994 25172 128495

December:
20100315 338.25000000 338.25000000 333.55000000 334.85000000 0 5657 0 128169
20100701 295.95000000 295.95000000 288.00000000 289.90000000 1017 21297 40805 130777
20100702 292.90000000 297.50000000 291.80000000 293.75000000 561 21295 25172 128495

UPDATE: I heard from CSI and they say it's a database refresh issue. That fixed the data.

-Michael
Last edited by michaelt on Tue Nov 16, 2010 7:48 pm, edited 2 times in total.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Tue Nov 16, 2010 9:40 am

Unable to reproduce your problem here on my CSI setup. No missing price data in HG2, either on charts or in ascii output files. I suggest performing the following voodoo rituals, they can't hurt and they may help. The full list takes 15 - 30 minutes to complete.
  1. Launch UA. Click Database / Download Replacement Fact files. When complete, exit UA.
  2. Launch UA. Click Database / Refresh Database Price History. When complete, exit UA.
  3. Launch UA. Click Database / Perform Database Integrity Search. When complete, exit UA.
  4. Launch UA. Click Download Data. When complete, exit UA.
  5. Have another look at HG2 prices.

My ascii price data files don't show your gap:

Code: Select all

**** HERE IS THE MARCH 2010 CONTRACT DATA -- NOTICE FIELD#8 = 201003 -- THAT'S MARCH 2010 ****
20100310,341.10000000,342.95000000,334.00000000,335.50000000,     320,    2755,201003,335.50000000,HG2
20100311,336.00000000,337.85000000,332.70000000,336.60000000,     332,    2574,201003,336.60000000,HG2
20100312,335.60000000,338.50000000,335.60000000,336.95000000,     584,    2445,201003,336.95000000,HG2
20100315,335.00000000,335.00000000,328.50000000,330.70000000,     492,    2322,201003,330.70000000,HG2
20100316,330.90000000,337.10000000,330.90000000,335.70000000,     205,    2221,201003,335.70000000,HG2
20100317,340.40000000,342.15000000,339.30000000,341.00000000,     648,     959,201003,341.00000000,HG2
20100318,338.35000000,339.20000000,337.20000000,338.75000000,     188,    1393,201003,338.75000000,HG2
20100319,338.90000000,341.35000000,335.15000000,336.45000000,     191,     920,201003,336.45000000,HG2
20100322,337.65000000,337.75000000,330.35000000,337.25000000,     362,     544,201003,337.25000000,HG2
20100323,338.15000000,339.50000000,336.95000000,336.95000000,     227,     502,201003,336.95000000,HG2
20100324,334.80000000,334.80000000,331.75000000,333.65000000,     253,     399,201003,333.65000000,HG2
20100325,333.60000000,337.50000000,333.50000000,337.10000000,     212,     340,201003,337.10000000,HG2
20100326,333.60000000,341.00000000,333.60000000,339.30000000,     185,     323,201003,339.30000000,HG2
20100329,345.90000000,352.95000000,345.90000000,352.95000000,     361,     215,201003,352.95000000,HG2



**** HERE IS THE MAY 2010 CONTRACT DATA -- NOTICE FIELD#8 = 201005 -- THAT'S MAY 2010 ****
20100310,339.90000000,344.60000000,335.05000000,336.80000000,   34927,   87193,201005,336.80000000,HG2
20100311,338.00000000,339.95000000,333.85000000,337.70000000,   31163,   87811,201005,337.70000000,HG2
20100312,339.00000000,340.65000000,335.85000000,338.00000000,   23204,   86863,201005,338.00000000,HG2
20100315,338.30000000,339.15000000,329.00000000,331.50000000,   25628,   87223,201005,331.50000000,HG2
20100316,332.20000000,338.40000000,331.00000000,336.50000000,   25246,   87753,201005,336.50000000,HG2
20100317,337.40000000,343.60000000,337.15000000,341.80000000,   22361,   87148,201005,341.80000000,HG2
20100318,342.45000000,342.80000000,337.55000000,339.55000000,   19925,   86491,201005,339.55000000,HG2
20100319,339.45000000,342.40000000,335.65000000,337.25000000,   24594,   86416,201005,337.25000000,HG2
20100322,337.70000000,338.90000000,331.05000000,338.05000000,   30277,   85386,201005,338.05000000,HG2
20100323,338.55000000,340.95000000,335.80000000,337.90000000,   22111,   86639,201005,337.90000000,HG2
20100324,337.50000000,338.75000000,332.10000000,334.55000000,   24150,   86871,201005,334.55000000,HG2
20100325,333.00000000,338.70000000,332.05000000,338.05000000,   25096,   85815,201005,338.05000000,HG2
20100326,335.35000000,343.40000000,333.75000000,340.30000000,   29708,   85023,201005,340.30000000,HG2
20100329,341.15000000,354.25000000,341.15000000,353.55000000,   34567,   88290,201005,353.55000000,HG2



**** HERE IS THE JULY 2010 CONTRACT DATA -- NOTICE FIELD#8 = 201007 -- THAT'S JULY 2010 ****
20100310,342.00000000,345.95000000,336.75000000,338.30000000,    2271,   18654,201007,338.30000000,HG2
20100311,339.30000000,340.75000000,335.75000000,339.20000000,    1133,   18724,201007,339.20000000,HG2
20100312,340.60000000,342.00000000,338.20000000,339.50000000,     778,   18735,201007,339.50000000,HG2
20100315,338.15000000,338.20000000,330.90000000,333.00000000,     607,   18877,201007,333.00000000,HG2
20100316,333.30000000,339.55000000,332.95000000,338.05000000,    3752,   19256,201007,338.05000000,HG2
20100317,339.55000000,345.10000000,339.25000000,343.40000000,    1919,   19812,201007,343.40000000,HG2
20100318,343.45000000,343.55000000,339.45000000,341.25000000,    2189,   20983,201007,341.25000000,HG2
20100319,341.00000000,344.05000000,337.75000000,338.95000000,     938,   20739,201007,338.95000000,HG2
20100322,337.70000000,340.50000000,333.05000000,339.85000000,    2451,   22247,201007,339.85000000,HG2
20100323,340.40000000,342.55000000,337.65000000,339.70000000,    1169,   22600,201007,339.70000000,HG2
20100324,339.00000000,340.20000000,334.10000000,336.35000000,    2534,   24407,201007,336.35000000,HG2
20100325,334.50000000,340.25000000,333.90000000,339.85000000,    3250,   25691,201007,339.85000000,HG2
20100326,337.05000000,345.00000000,336.25000000,342.10000000,    2393,   26707,201007,342.10000000,HG2
20100329,343.90000000,355.95000000,343.90000000,355.35000000,    4183,   28323,201007,355.35000000,HG2




**** HERE IS THE SEPT 2010 CONTRACT DATA -- NOTICE FIELD#8 = 201009 -- THAT'S SEPT 2010 ****
20100310,345.50000000,346.80000000,338.15000000,339.25000000,     957,    8413,201009,339.25000000,HG2
20100311,340.20000000,340.20000000,340.20000000,340.20000000,     500,    8773,201009,340.20000000,HG2
20100312,339.15000000,340.55000000,339.15000000,340.55000000,     200,    8851,201009,340.55000000,HG2
20100315,335.50000000,336.25000000,332.80000000,334.10000000,     185,    8853,201009,334.10000000,HG2
20100316,336.20000000,339.15000000,335.55000000,339.15000000,     633,    8919,201009,339.15000000,HG2
20100317,344.00000000,345.00000000,343.40000000,344.50000000,     444,    8783,201009,344.50000000,HG2
20100318,343.30000000,343.30000000,340.65000000,342.40000000,     487,    8740,201009,342.40000000,HG2
20100319,344.00000000,344.00000000,339.10000000,340.10000000,     459,    8733,201009,340.10000000,HG2
20100322,336.60000000,341.65000000,334.50000000,341.00000000,     445,    8936,201009,341.00000000,HG2
20100323,340.00000000,341.90000000,339.70000000,340.85000000,      72,    8926,201009,340.85000000,HG2
20100324,336.75000000,337.90000000,335.55000000,337.50000000,     651,    9068,201009,337.50000000,HG2
20100325,337.50000000,341.45000000,337.10000000,341.00000000,     684,    8981,201009,341.00000000,HG2
20100326,337.90000000,346.05000000,337.90000000,343.30000000,     317,    8993,201009,343.30000000,HG2
20100329,348.35000000,356.90000000,348.35000000,356.50000000,     819,    9359,201009,356.50000000,HG2
Attachments
panel1.png
CSI UA charts of HG2 in the vicinity of 15 March 2010
panel1.png (35.92 KiB) Viewed 4013 times

michaelt
Senior Member
Senior Member
Posts: 37
Joined: Wed Feb 03, 2010 10:53 am
Location: princeton,nj

Post by michaelt » Tue Nov 16, 2010 7:47 pm

Thanks Sluggo,
Yes, that did it (I had done that earlier) and indeed the voodoo rituals (a great term) did the job. Thanks so much for going though the data as you did. I am going to move my comment at the top of my post - or should I delete the post? - so no one is confused. perhaps this issue could affect others and they can learn from it? As I did.

Do you refresh your UA database on a regular schedule? Since this problem can occur anytime (I assume) and affect the contracts, database refresh (it seems to me) needs to be done before contracts price are negatively impacted.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Tue Nov 16, 2010 8:00 pm

I run the "Futures Data Investigator" system in my copy of Trading Blox, twice a day. It sniffs out lots of different problems, one of which is "unreasonably long gaps in the data". (REF)

Then I go digging. Sometimes I post my preliminary findings (HERE) if I have the time and the inclination. Other times I don't.

Post Reply