How did you go about picking rollover timing triggers?

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
jklatt
Roundtable Knight
Roundtable Knight
Posts: 102
Joined: Sat Feb 17, 2007 11:51 am

How did you go about picking rollover timing triggers?

Post by jklatt »

From what I've read, a lot of the traders on this forum use CSI's futures data to backtest and trade with. I'm curious, how did _you_ determine the various rollover timing triggers that must be used in your every day trading?... and why?

A few months ago when I decided to finally make a serious effort in building a trading system I was pretty taken back by all of the instruments that could possibly be traded. I ended up buying a data subscription from CSI and started the data mining process. I really had no clue what I could and couldn't trade. Having no experience trading a lot of these contracts, I didn't know what was "liquid" enough and what wasn't. So I basically downloaded EVERYTHING and built a script to piece together pretty much the highest volume contract possible given the individual contracts. I then filtered based on volume figures from there. That process in and of itself was an eye opener. I found a LOT of data errors all over the place (some of which CSI still hasn't been able to answer and are still working on).

Having my confidence shaken a bit from all of the errors I found, I was a little concerned about having Trading Blox link up with CSI to help me build a futures dictionary. I figured, if they can screw up the data like this maybe the resulting futures dictionary might be off too and what might happen if I had a substantial error in a big point value and I took on way too much risk unknowingly? So that lead me to going through each and every instrument on my liquidity filtered list, looked it up on the exchange website and I built my own futures dictionary to cross reference against.

Now that the futures dictionary is almost finished, I'm approaching the part where I'll generate different rollover timing triggers for each of the contracts based on expiration and first notice dates and volumes and I'm really quite curious about the general process some of you traders out there took when you were brand new to all of this. My tradeable futures list is around 150 instruments long. Did you guys go through each and every instrument, look at the different times that you could roll and the pros and cons of doing so and then generate educated roll timing triggers?

In my mind, going through all of this detail feels like it's worth it. It'll definitely instill confidence down the road when it comes to wondering whether or not my backtests had errors in it... but I can't help but wonder, is this the type of level of detail a lot of you guys put in when starting out?
Chris67
Roundtable Knight
Roundtable Knight
Posts: 1052
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Post by Chris67 »

just one bit of advice - just because csi has a big point value of x and the exchange has a big point value of y - dont assume the exchange is correct - Ive seen plenty of errors on the exchange websites - then Ive seen plenty of errors on prime brokers lists of futures details
I think Sluggo summed it up when he said the ONLY way to find out the exact details is to buy and sell a contract and see what appears on your broker statement - thats what I intend to do with some new markets I have recently added
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Re: How did you go about picking rollover timing triggers?

Post by AFJ Garner »

klatt_attack wrote: Did you guys go through each and every instrument, look at the different times that you could roll and the pros and cons of doing so and then generate educated roll timing triggers?
Yes.
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

Um, what I said is that scrutinizing your statement after a quick in-and-out trade, provides cheap insurance against dictionary misunderstandings:

viewtopic.php?p=20189&highlight=insurance#20189
ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Post by ecritt »

Personally, I don't think there is any standard roll methodology that is universally applicable to all futures contracts. For certain markets, like currency futures and stock index futures nearly all the liquidity resides in only one contract at a time. For other markets like corn, nat gas, eurodollars, cattle, wheat, etc. the liquidity is dispersed across several different expiration dates; that are actively trading at the same time. In looking at the contracts individually, while they are generally correlated, it's not uncommon for them to have substantial differences in trend, volatility, contango and backwardation. For some markets the front months can even be moving in the opposite direction relative to the back months.

Also, there is the issue of storage. Yen today is the same as Yen 6 months from now. Same is true for S&P 500. But corn? Wheat? The spring crop can have completely different supply issues relative to the fall crop. Are they really the same product? So much so that they should be linked together and viewed as one investable product? Is the volatility and trend of last year's wheat crop relevant to this year's? I can't answer yes to these questions; wish I could since it would be a lot easier. I suspect that people use continuous contracts that roll on volume or open interest because it's convenient to do so. I also think they are paying a "convenience premium" in the process.
7432
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Wed Dec 31, 1969 7:00 pm
Location: sandy, ut

Post by 7432 »

here is an IB info page that dictates their customers' contract rolls.
some of your work can be simplified by finding a broker and finding out what their rules are.
Kiwi
Roundtable Knight
Roundtable Knight
Posts: 513
Joined: Wed Apr 16, 2003 1:18 am
Location: Nowhere near

Post by Kiwi »

ecritt wrote:Personally, I don't think there is any standard roll methodology that is universally applicable to all futures contracts. For certain markets, like currency futures and stock index futures nearly all the liquidity resides in only one contract at a time. For other markets like corn, nat gas, eurodollars, cattle, wheat, etc. the liquidity is dispersed across several different expiration dates; that are actively trading at the same time. In looking at the contracts individually, while they are generally correlated, it's not uncommon for them to have substantial differences in trend, volatility, contango and backwardation. For some markets the front months can even be moving in the opposite direction relative to the back months.

Also, there is the issue of storage. Yen today is the same as Yen 6 months from now. Same is true for S&P 500. But corn? Wheat? The spring crop can have completely different supply issues relative to the fall crop. Are they really the same product? So much so that they should be linked together and viewed as one investable product? Is the volatility and trend of last year's wheat crop relevant to this year's? I can't answer yes to these questions; wish I could since it would be a lot easier. I suspect that people use continuous contracts that roll on volume or open interest because it's convenient to do so. I also think they are paying a "convenience premium" in the process.
Although I agree with much of what you say I find difference on two points:
- you still need to create the continuous contracts (or do the equivalent somehow) to test your long term system so date or volume triggers are useful.
- you don't have to pay a convenience premium if you don't want to and have the time/skills to avoid it. you test your system on rollover rule x. then at actual rollover time range you have an opportunity to out-trade your rollover rule (as a simple example, say the rule is based on swap at open then you might wait and swap when prices provide a better exchange).
ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Post by ecritt »

"you still need to create the continuous contracts (or do the equivalent somehow) to test your long term system so date or volume triggers are useful."

Agreed, but I question whether the standard process of rolling on volume or OI, which considers only the most liquid contract, is the best way to do this. It is certainly the easiest. But, I suspect there is a negative relationship between what's easy and what's optimal in a zero sum game (negative sum really). I can tell you from my experience that performance results from trading a single back-adjusted natural gas contract are dramatically different when compared to those of trading the individual contracts (on a risk equalized basis, using the same trading system, accounting for realistic liquidity needs).

"you don't have to pay a convenience premium if you don't want to and have the time/skills to avoid it. you test your system on rollover rule x. then at actual rollover time range you have an opportunity to out-trade your rollover rule (as a simple example, say the rule is based on swap at open then you might wait and swap when prices provide a better exchange)."

That's not my point. The opportunity cost I'm referring to has to do with all of the liquidity that's not in the crowded front-month "most" liquid contract. For many markets it's substantial and seems to offer diversification benefits and larger risk premiums in the form of excess contango and/or backwardation. By the time you roll into one of these deferred contracts, most of these premiums have been collected and the convergence to spot is nearly complete, leaving you with essentially just the potential beta move of the underlying commodity. From what I've seen over the years, more than half the extractable (and sustainable) profits available from "trend following" come from the premium collection process, not the beta moves.
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

  • From what I've seen over the years, more than half the extractable (and sustainable) profits available from long term "trend following" come from the premium collection process, not the beta moves.
Fixed that for you.

Short term trend followers, many of whom can be found in the Newedge Short Term Traders Index HERE, tend not to collect very much premium. They just take volatility-breakouts (a la Larry Williams / Lee Gettess) and ride the short term trend for a few dozen hours.

An example VBO system appears on pp. 192-196 of Hill & Pruitt's book "The Ultimate Trading Guide". They forgot to mention that it is the same system they also sell for USD 495.00 on their companion website, under the moniker "Universal". Fixed that for them.
Kiwi
Roundtable Knight
Roundtable Knight
Posts: 513
Joined: Wed Apr 16, 2003 1:18 am
Location: Nowhere near

Post by Kiwi »

I don't disagree with you Eric.

Despite that, I do think most users will still take standard methods and the combination of standard testing and optimizing your actual trading is not unreasonable.
Emptor
Contributing Member
Contributing Member
Posts: 5
Joined: Thu Nov 19, 2009 10:44 pm

Post by Emptor »

Hi all,

feel like a shmuck posting my virgin post amongst a lineup of the veterans, but this is a topic I'm currently grappling with as a new user of TB/CSI. Most CSI contracts seem to backadjust nicely but there's a few that exhibit historical patches of ugliness that looks to be caused by rollover adjustments, as in the example below.

Tinkering with the rollover methodology seems to fix in certain cases but in others it's like whack-a-mole, and I'm not sure the idea of using different methodologies on different instruments just to make the charts look nice represents great practice.

Sorry if this post seems too newbieish but I haven't found an answer elsewhere in this forum and have been beating my head against the wall talking directly to CSI on the issue. What are people's thoughts?

Cheers,
Emptor
Attachments
bigchart.jpg
bigchart.jpg (180.59 KiB) Viewed 15786 times
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

Kiwi wrote:I do think most users will still take standard methods and the combination of standard testing and optimizing your actual trading is not unreasonable.
Those who have developed a sound trend following system, who use "standard" methods of rolling contracts and who follow their system using a reasonably diversified portfolio are likely to make money and to find that their actual trading results are accurately reflected in the theoretical back tested results produced day by day by their back testing engine.

Whether those results (back tested and actual) can be improved by researching different roll methods for agricultural and some other products is a different question and worthy of research. I seem to recall from extensive back testing of CL a few years ago that the best results I found came from rolling on the very last day when convergence was completed (not practical in practice) so it may not necessarily be the case that the roll premium in backwardated contracts is always unavailable to those trading the front months. I would need to re-visit my research to confirm this.

But for those new to trading, I think it is important not to get too caught up on these matters. Doing so could all too easily lead to the defeatist view that the whole game is far too complex and that they haven't a chance of making money.
marriot
Roundtable Knight
Roundtable Knight
Posts: 365
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot »

Hi Emptor,
it seem that the final goll is roll by date.
As AFJ said, do not bother too much, now.
You can begin trading with standard rollover method, just fix the obviusly wrong rolllovers, looking at charts.
Let your broker roll for you and fix your csidata every time your roll does not match with brokers action.
In a 3 months you will have find the best possible setting.
Rollover date do not affect too much results in long term.
ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Post by ecritt »

"Whether those results (back tested and actual) can be improved by researching different roll methods for agricultural and some other products is a different question and worthy of research. I seem to recall from extensive back testing of CL a few years ago that the best results I found came from rolling on the very last day when convergence was completed (not practical in practice) so it may not necessarily be the case that the roll premium in backwardated contracts is always unavailable to those trading the front months. I would need to re-visit my research to confirm this."

I suspect you are talking about the period of time that crude oil was persistently very contango while the beta move was persistently higher. In this case you had to pay the premium to catch the beta move. The really big trends occur when you get both working for you, like natural gas since 2008 (heavy contango and down beta move). What I'm taking issue with is the whole notion of rolling. A continuous contract is not the market. It represents the equity curve of a specific participation method on a one contract basis; that emphasis maximum liquidity over all other considerations.

Imagine if I told you I'm going to trade the software stocks sector. For argument's sake let's assume there are only 15 software stocks of consequence in the market and they are highly correlated with each other. I will build a continuous contract that starts by selecting the software stock with the highest volume. I will track that stock as long as it is the most liquid. As soon as some other software stock becomes more liquid, I will abandon the first, close the gap via some adjustment process, and begin accumulating results for the new most liquid stock, so on and so forth. You would probably point out that even the most liquid software stock might only account for 25% or 30% of the liquidity in the whole software sector. You might also point out that the one stock I select might not accurately reflect the software "sector" as a whole. You might go so far as to suggest I trade the actual individual stocks, or create an index (aka: continuous contract) methodology that's more representative. For many futures markets, the above scenario isn't that far from reality. It's true that futures expire, and you have to "move on", but you don't have to pick just one contract (the one with the highest possible liquidity) and ignore all others. In many futures markets there is liquidity and collectible premium in those "others". Liquidity might be the only thing available in the most liquid front month contract...in fact, that might become the new norm some day.

"But for those new to trading, I think it is important not to get too caught up on these matters. Doing so could all too easily lead to the defeatist view that the whole game is far too complex and that they haven't a chance of making money."

Yes, I'll concede that the topic is daunting. But I believe it's worth pursuing. Just looking for any like-minded people that want to discuss.
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

Emptor, I suggest you keep trying. There aren't that many different ways to build a continuous contract in
cocoa -- systematically work your way through them and get a feeling for which one(s) you like most /
dislike least. Attached below is my cocoa contract in CSI, covering about the same period of time as
yours above.

I myself happen to use "Date" to trigger my rollovers, which as you can see from the impassioned messages
in this thread, is not a universally well-loved decision. I use a different rollover date in each futures
market and I don't worry that this may be overfitting -- to me it is just an acknowledgment that different
markets have different first notice days, different last trading days, and different historical patterns of
liquidity. The very nice folks at CSI's competitor "Pinnacle Data" have made the same decision, and have
published a table of their date-based rollover choices: http://www.pinnacledata.com/clc.html (Scroll
down to "Starting Dates And Roll Over Days")
Attachments
csi cocoa backadjusted continuous contract
csi cocoa backadjusted continuous contract
cocoa.png (47.6 KiB) Viewed 16451 times
Moto moto
Roundtable Knight
Roundtable Knight
Posts: 427
Joined: Mon Jun 01, 2009 4:12 am
Location: once again in the UK

Post by Moto moto »

Life is funny - I was just discussing this subject with someone the other day. Ultimately just taking the standard CSI roll on opens, or volume across a broad portfolio clearly should be sufficient to prove that the system will work.

After that I do take the opinion that to get a far more accurate back test you really need to get as close to reality as possible, which does mean that you should modify the rollovers for each instrument based on what you would have actually done. Yes it is more work, but it depends on how much proof you need, how much your system relies on profits from the roll, and whether or not backwardation and contango issues add to or subtract from the PL. I would ultimately like to go sluggos route of rolling on date for each contract. Not there yet.

Also just for my own clarification (so as I am not missing something....)
there are continuous contracts, and there are backadjusted continuous contracts, and they are very very different sets of data.???
Also finally bought TB builder, and looking forward to getting the hands on the controls....
ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Post by ecritt »

Emptor, if I had to use continuous contracts I'd probably do like Sluggo. But, I'd keep in mind (for further analysis at a later date) that futures contracts change over time. Point values, contract sizes, notice dates, etc. are not static. What's valid today may not have been so ten years ago. Another "reasonable" option, in my opinion, is to roll on open interest AND set the "by days to expiration" option to number that ensures you are out by the first notice day.
LeapFrog
Roundtable Knight
Roundtable Knight
Posts: 695
Joined: Mon May 17, 2004 4:18 pm
Location: Boston, MA

Post by LeapFrog »

sluggo wrote:I myself happen to use "Date" to trigger my rollovers, which as you can see from the impassioned messages
in this thread, is not a universally well-loved decision. I use a different rollover date in each futures
market and I don't worry that this may be overfitting -- to me it is just an acknowledgment that different
markets have different first notice days, different last trading days, and different historical patterns of
liquidity. The very nice folks at CSI's competitor "Pinnacle Data" have made the same decision, and have
published a table of their date-based rollover choices: http://www.pinnacledata.com/clc.html (Scroll
down to "Starting Dates And Roll Over Days")
Sluggo, what do you do though with the many other instruments that Pinnacle do not cover?
jklatt
Roundtable Knight
Roundtable Knight
Posts: 102
Joined: Sat Feb 17, 2007 11:51 am

Post by jklatt »

Emptor,

Attached is my "best" CC2 continuous contract using CSI's individual contract data with best meaning "most volume".

I haven't written the scripts to systematically analyze when might be the best time to roll, but just from eye balling the contract for a few minutes here are my thoughts.

First Notice Date appears to be the first business day of the delivery month minus 10 business days. From the First Notice Date I have optimum roll tiggers ranging from about 2 business days prior to First Notice all the way up to about 9 business days.

Active contracts appear to be Mar, May, Jul, Sept, Dec so your roll months will be Feb, Apr, June, Aug, Nov and you should be looking to roll around mid month or a little bit sooner. Feb, Apr and Nov might have holidays with regard to the First Notice calculation and the month of February is shorter than the other months so it won't be an exact science if you have to pick one specific date and apply it to all of the roll months but if you want a good range to start with, I'd look at the dates between the 6th and the 18th during the month prior to the delivery month to roll on.

As a side note, the roll triggers back in the 70s appear to be far less consistent in terms of when the best time to roll occurs. Sometimes it's in the timeframe I mentioned, sometimes it's a whole month before the time frame I mentioned.

Anyway... here's the file. Hope it helps you eventually solve your issue.
Attachments
CC2.xls
(1.83 MiB) Downloaded 612 times
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

LeapFrog wrote:Sluggo, what do you do though with the many other instruments that Pinnacle do not cover?
I do the same thing that AFJ Garner does; it is the third posting in this thread you're reading now.
Post Reply