Very Long Term Trend Following: The Data Implications

[Angelo]

Angelo,

I don't mean to appear argumentative, and I have no kind of dogmatic attachement to backadjusted price series. I don't think you are arriving from the moon. It could be that all of us trying to give assistance are missing the point entirely. What will help immensely is if you can go through the example below and point out any difficulties you see with it. If there is a problem with the example and it somehow leaves out some unseen added costs of rolling, you can point it out and correct the example. That way those of us who are currently confused can correct the error in our thinking. If you can't find an error and you decide that this specific example extends into a general principle, then you get to learn something valuable. Either way, everyone wins.

Hi Jake,

thanks for spending your time to help me clarifying this matter.

I don't find any error in your example and - as you can imagine - I'm a little puzzled.

But you are right, and I definitely need to follow Will's advice (and Sluggo and c.f. ones... ) to take paper and pencil and check this out on real prices data.

Thanks again, I have been reading your posts for quite a while and they are always worthy and informative.

[Jake Carriker]

Angelo, you are welcome. Thanks in return for studying the example and keeping an open mind. I encourage you to follow your impulse to check out several different examples and think about your findings.

Jake

[Angelo]

Hi everybody,

Hoping not to annoy anyone, a little update about data.

After all your posts, I asked myself: “hey jerk, from where did you get the belief about the not-so-absolute reliability of continuous data?â€

[Jake Carriker]

Angelo,

Chande's results from table 3.12 of the book assume that zero rollovers occur with the continuous contract. He also defines a trade as 1 entry plus 1 exit, regardless of whether the trade is a rollover.

Thus, a system that trades a continuous contract that enters a position on January 1, and exits on December 31 shows 1 trade over that time period. The same system trading actual contract data shows that same trade PLUS however many rollovers occured during the holding period.

The difference in results in his table represents the missing frictional costs associated with the rollover trades that the continuous contract doesn't account for. His point is obfuscated by his method of treating rollover trades the same as bonafide entries and exits generated by the system math.

Imagine, the exact same system trades the exact same market, the only difference is whether we account for the cost (commission + slippage) of rollovers. Here are the results.

System A - continuous contract
# trades = 1
Avg winner = 100
Avg loser = 0
Length of Avg Trade = 260

System B - actual contract data
# trades = 10
Avg winner = 25
Avg loser = -15
Length of Avg Trade = 26

What utter bull-hockey! Those numbers represent the exact same system. The only difference is that System B rolls over the position 9 times. These two comparison summaries make this look like two very different trading systems. Wouldn't it be more reasonable to treat only genuine entry/exit pairs as a trade and simply add in the appropriate frictional cost every time a roll triggers? That might produce a summary that looks something like this:

System C - continuous contract with roll accounting
# trades = 1
Avg winner = 95
Avg loser = 0
Length of Avg Trade = 260

This looks like a much more reasonable way to represent the cost of rolling to me. It looks just like System A, only a little less profitable due to the frictional costs of rolling over several times.

Long story short: Chande assumes that if you test with continuous contracts you are not accounting for rolls. Trading Blox and other software applications DO, in fact, account for rolls if the user wishes. Chande's examples point out that if you test without accounting for the frictional cost of rollovers, you will likely experience results inferior to the backtests because you do rollover positions in actual trading. I don't find any statements in his book that reference a cost of rolling other than friction.

Jake

[Angelo]

System A - continuous contract
# trades = 1
Avg winner = 100
Avg loser = 0
Length of Avg Trade = 260

System B - actual contract data
# trades = 10
Avg winner = 25
Avg loser = -15
Length of Avg Trade = 26

What utter bull-hockey! Those numbers represent the exact same system. The only difference is that System B rolls over the position 9 times. These two comparison summaries make this look like two very different trading systems. Wouldn't it be more reasonable to treat only genuine entry/exit pairs as a trade and simply add in the appropriate frictional cost every time a roll triggers? That might produce a summary that looks something like this:

System C - continuous contract with roll accounting
# trades = 1
Avg winner = 95
Avg loser = 0
Length of Avg Trade = 260

This looks like a much more reasonable way to represent the cost of rolling to me. It looks just like System A, only a little less profitable due to the frictional costs of rolling over several times.

Long story short: Chande assumes that if you test with continuous contracts you are not accounting for rolls. Trading Blox and other software applications DO, in fact, account for rolls if the user wishes. Chande's examples point out that if you test without accounting for the frictional cost of rollovers, you will likely experience results inferior to the backtests because you do rollover positions in actual trading. I don't find any statements in his book that reference a cost of rolling other than friction.

Jake

Hi Jake,
this time I'm not following you.
First, I'm not really concerned in differences in
# trades
Avg winner
Avg loser
Length of Avg Trade.
Second, I'm aware Chande desn't reckon the cost of rollover when using continuous data

In this case, what really matters to me, are the huge differences in "net profit".
What remain unexplained to me is:

a) how "continuous contract type 56/25" has a net profit of 10,625, whereas "continuous contract 60/25" has a net profit of 39,862.
Same system, same markets.
The only things changing is the way in building the continuous data.

b) the huge "implicit" cost of rollovers in Chande's data. (439$ per lot in the example I wrote yesterday).

Maybe you think Chande's reasoning is wrong (due to the few explanations he adds on p. 89-90 we'are unable to check his calculations) but are a man kind enough not to say it loud.
But IF he'is not wrong, this is something that definitely needs futher thought (at least by me).

Greetings,

Angelo.

[Jake Carriker]

Hi Angelo,

a) how "continuous contract type 56/25" has a net profit of 10,625, whereas "continuous contract 60/25" has a net profit of 39,862.
Same system, same markets.
The only things changing is the way in building the continuous data.

Using software (CSI's Unfair Advantage is a popular one) that allows the construction of continuous contracts using user supplied rollover dates is one way to gain insight into this problem. I can test a Swiss Franc contract that rolls on shifts in open interest versus one that rolls on the 8th day of delivery month versus one that rolls on shifts in Volume, etc...

Setting up a test like this can be insightful. You might decide that your choice of roll date (and the contract delivery to trade) has some not insignificant affects on the properties of the resultant continuous price series. Notice that the roll-timing-determines-the-price-series problem does not go away when dealing with "actual" contract data. We still have to decide when to close out a trade in one expiration and open it in another.

If Chande had tested a continuous contract that rolled on the same dates as he rolled his "actual" contracts, the only difference in results would attribute to frictional cost of rollovers.

b) the huge "implicit" cost of rollovers in Chande's data. (439$ per lot in the example I wrote yesterday).

Based on my explanation above, we see that the the difference in results is not entirely attributable to rollovers but includes differences in profit that result from trading what is essentially two different data sets.

I was not a Tech Tools customer, and since Chande doesn't explain what the "60/25" and other designations are I don't know what kind of price series he is using. Continuous contracts are not restricted to only backadjusted series. They could be ratio adjusted, proportionally adjusted, Gann .... and so on. Who knows what Chande used? Maybe some other forum members were Tech Tools customers and could shed light?

Jake

[Jake Carriker]

Perhaps the fact that the actual contract data performs at the bottom of the pack in the table on page 90 of Chande's book has something to do with his choice of roll date as well as the impact of rollover costs.

The screenshot below from CSI is of a continuous backadjusted SF contract that rolls on the 21st day of the month 1 month prior to delivery, just like Chande's contract does. The solid blue line in the bottom graph window represents open interest in the current month used to produce the continuous series. I have circled the spots where open interest dips precipitously and flat lines for a while due to rolling early, and thus rolling into an inactive contract. It appears that this particular roll date is not very appropriate for SF.

Perhaps some of the continuous contracts in his test sample (especially 60/25?) were more appropriate for the market/system in question.

Jake

[PaulZ]

I found Jake's post very interesting so I charted a few different rollover dates in CSI's Unfair Advantage application. The OI dropped for every rollover date I tried. I then looked at Total OI, which should be independent of rollover date. At least for SF, the Total OI drops substantially ahead of contract delivery. If the Total OI drops ahead of the delivery date, then the observation that individual OI drops for any rollover date is reasonable - it is just reflecting the drop in Total OI.

The next thing to consider is identifying rollover dates for which the drop in individual OI is less extreme. For example, Jake observed that in Chande's data, rolling over SF on the 21st day of the month 1 month prior to delivery resulted in excessive drop in OI. How do we determine what a more appropriate rollover date would be? Empirically we can chart the OI for the continuous contract using different rollover dates and look for a date that results in a smaller drop in OI. I don't expect that to always be the same rollover date, throughout history. However, looking at the charts or at a spreadsheet analysis should give an idea of how consistent the OI drops are for different rollover dates.

Another approach is to rollover when the next contract OI, or Vol, or both OI and Vol exceeds the current contract OI, or Vol, or both. Just keep in mind that historical testing may be affected by using these choices as actual OI and Vol aren't released by the exchanges until the day after they release the OHLC data.

I'd appreciate any corrections to what I posted here or any amplification by people having more experience in these issues. I feel this issue is quite important as poor liquidity when rolling from one contract to the next could have a significant impact on slippage.

[Jake Carriker]

Hi Paul,

Excellent job digging into CSI and experimenting. I always find such expeditions fruitful.

However, I see something different on my charts concerning changes in Total Open Interest. Also, though there is certainly a dip in the individual OI around practically any roll date, the shape of that dip is something that can be controlled by choice of roll date.

The first chart below is similar to the one I posted previously (Chande roll date = 21st DOM, 1 month prior), only it also plots Total Open Interest as a red line in addition to the blue line that represents individual OI. Notice the shape of the both lines around rollovers. I see total OI rise during times when two deliveries are actively trading. During the time when all activity is in one month, total OI = individual OI.

The second chart is identical except for the choice of roll date. It rolls on the rather common roll date (for IMM currency futures) of the 8th day of the expiration month. On this chart the dip in individual open interest is abbreviated, and OI in the new front month picks up quickly. Compare and contrast the two charts and see what you think.

[PaulZ]

Thanks, Jake. I am able to replicate the charts you showed, and I agree with your interpretation.

The closer the rollover date gets to the last trade date, the less exaggerated is the drop in OI in the new contract.

Do you have any thoughts about using OI, Vol or both to determine rollover rather than a fixed date?

[AFJ Garner]

I greatly prefer fixed roll dates. Rolling on Vol or OI can have you rolling right on (or after) the FND or whatever. Even if you roll on OI you have to keep a very close watch on the date: so why not just choose fixed dates to avoid all the heartache.

[PaulZ]

Thanks, AFJ. A fixed roll date is certainly easier to manage, both in historical testing and in actual trading. Easier, that is, after going through every market to determine good fixed roll dates. I'm about to embark on that exercise.

[painless]

Since I am migrating my trading software to a Mac I am using the real price and backfilling each contract on the roll date with enough data to set the tool. I use % indicators on indices and some other tests so, as one poster noted, it is necessary. At the moment I have to keep the real price and adjusted price for each record which is memory consuming.

Adjusted data is fine for most systems. I cannot see much benefit in having a "real roll" unless you are in my situation. I agree with Sluggo when he said pay the money and get what you want.

[Chelonia]

Here's a list with very workable roll days.
DM = Delivery Month, MPDM = Month Prior to DM


STARTING DATES AND ROLL OVER DAYS

COMMODITY START DATE ROLL DAY
Australian $$, Day Session 1988 8th Day of DM
Australian $$, Composite Session 1990 8th Day of DM
British Pound, Day Session 1976 8th Day of DM
British Pound, Composite Session 1990 8th Day of DM
CAC40 Index 1999 2nd Thur of DM
Canadian $$, Day Session 1978 8th of DM
Canadian $$, Composite Session 1990 8th of DM
Canadian 10yr Bond 1994 26th of MPDM
Cocoa 1981 9th of MPDM
Coffee 1974 11th of MPDM
CRB Futures 1987 6th of DM
Copper 1989 22nd of MPDM
Corn 1970 22nd of MPDM
Cotton #2 1973 15th of MPDM
Crude Oil 1984 11th of MPDM
Dollar Index #2 1989 8th of DM
Dow Jones Index, Day Session 1997 Thur prior 2nd Fri of DM
Dow Jones Index, Composite session 1998 2nd Thur of DM
Dow Jones Mini 2002 2nd Thur of DM
Dow Jones Eurostoxx50 2002 2nd Thur of DM
Dow Jones Stoxx50 2002 2nd Thur of DM
Euro, Day Session 1999 8th of DM
Euro, Composite Session 1999 8th of DM
Euro, Bobl 1999 26th of MPDM
Euro Bond (Bund) 1989 26th of MPDM
Eurodollars 1983 22nd of MPDM
Euro Schatz 1999 26th of MPDM
Fed Funds 1989 22nd of DM
Feeder Cattle 1978 12th of DM
FTSE 100 Index 1990 13th of DM
German Dax Index 1997 2nd Thur of DM
Gilt, Long Bond 1989 26th of MPDM
Gold (Commex) 1975 22nd of MPDM
Gold (CBOT 24hr electronic) 2004 22nd of MPDM
Goldman Saks 1993 9th of DM
Hang Seng 1997 2nd Thur of DM
Heating Oil 1980 11th of MPDM
Japenese Yen, Day Session 1978 8th of DM
Japanese Yen, Composite Session 1990 8th of DM
Live Cattle 1971 27th of MPDM
Live Hogs 1970 27th of MPDM
Lumber 1974 1st of DM
Mexican Peso Ind 1995 8th of DM
Muni Bonds 1986 26th of MPDM
Nasdaq 100 1996 2nd Thur of DM
Nasdaq, Mini 1999 2nd Thur of DM
Natural Gas 1991 18th of MPDM
Nikkei Index 1991 3rd of DM
Oats 1975 22nd of MPDM
Orange Juice 1973 25nd of MPDM
Palladium 1983 22th of MPDM
Platinum 1973 22nd of MPDM
Pork Bellies 1970 27nd of MPDM
RBOB Gasoline (with HU history) 1985 11th of MPDM
Rough Rice 1987 22nd of MPDM
Russell 2000 Index 1993 2nd Thur of DM
Mini Russell Index 2002 2nd Thur of DM
S. & P. 400 1993 2nd Thur of DM
S. & P. 500, Day Session 1983 2nd Thur of DM
S. & P. 500, Composite Session 1994 2nd Thur of DM
S. & P. 500, Mini 1997 2nd Thur of DM
Silver (Commex) 1973 22nd of MPDM
Soybeans 1969 22nd of MPDM
Soybean Meal 1969 22nd of MPDM
Soybean Oil 1969 22nd of MPDM
Sterling, Short 1989 3rd of DM
Sugar, #11 1975 22nd of MPDM
Swiss Frank, Day Session 1976 8th of DM
Swiss Frank, Composite Session 1990 8th of DM
T-Bonds, Day Session 1983 26th of MPDM
T-Bonds, Composite Session 1978 26th of MPDM
T-Notes, 2yr, Day Session 1990 26th of MPDM
T-Notes, 2yr, Composite Session 1990 26th of MPDM
T-Notes, 5yr, Day Session 1988 26th of MPDM
T-Notes, 5yr, Composite Session 1989 26th of MPDM
T-Notes, 10yr, Day Session 1983 26th of MPDM
T-Notes, 10yr, Composite Session 1983 26th of MPDM

Wheat, Chicago 1969 22nd of MPDM
Wheat, K.C. 1977 22nd of MPDM
Wheat, Minn. 1981 22nd of MPDM

DM = Delivery Month, MPDM = Month Prior to DM

[sluggo]

http://www.pinnacledata.com/clc.html has a very similar list.

[Roger Rines]

Back-adjusted contracts have been my primary data type for testing and trading for a long time and I can say that the system signals and their real-time performance results vary so little that I don’t see any need to change how we trade or test.

In looking at our daily results as compared to what our systems report, the area where we have more of a difference isn’t around the roll-forward event, but instead they appear during the daily bar envelope where we are put into, or taken out of a trade, and the tick occurrence for that time of the day created a spread that can’t be detected using a daily bar record.

Another point to mention is that Bruce Babcock’s warning about percentage base calculation can be avoided by adding an additional column to the data record so that each day’s unadjusted close value is available for use with percentage calculations.

Pinnacle Data Concern:
About a month ago I had the opportunity to look at some Pinnacle data and found they didn't preserve the actual volume and open interest information for all of the contract expiration other than the active contract, so if you plan on using Volume or Open Interest as part of your roll timing approach, then you should understand this might not be possible with Pinnacle.

[Chelonia]

It's from Pinnacle. I'm testing these and other settings on the various products in the portfolio now, altough i'm already confident with the open interest trigger.

I like the simplified explanation of the roll over settings (1,2,3,4) as set out by Mr. Garner, but i'm looking into markets that go from backwardation to contango and vice versa. For example interest futures show this characheristic. I would like to find out what happens to price/roll when a product turns from backwardation to contango during month 6 for example. What formula can deal with that as formula's are tend to be used under certain constant conditions. So instead of 1,2,3,4, market goes 1,2,1,0. My concern is a large gap being calculated as a profit as positions are probably in the direction of the new main trend. Any input would be appreciated.

[Angelo]

Hi everybody,

speaking of continous data and rollovers, I wonder wether someone read Mike Chalek's piece on last Futures Truth magazine and have some comments, especially from the real trading .... (I must admit, I don't trade LTTF..... at least not yet...., so rollovers are not a real life problem for me).

Basically, he tested his best system on the Big Russel for six years and found a difference of 100000 $ in total profit between the better and the worst case.
During that period there were 25 rollovers, so every one of them accounted for 4000 $, that is 8 big points..... more than 1% of the actual level of the index which is an astonishing difference.

If this is not enough, here are some of his introductory comments (see p. 41):

- "back adjusted" series contract then to smooth (????) data.

- continuous contracts .... do not parallel real life trading and the results may be somewhat misleading.