Variability Due to Differing Rolling Algorithms

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
Chris67
Roundtable Knight
Roundtable Knight
Posts: 1046
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Variability Due to Differing Rolling Algorithms

Post by Chris67 » Sun May 30, 2004 5:29 am

MODERATOR'S NOTE: The topic title was changed from "Data set up for 53attd files" to "Variability Due to Differing Rolling Algorithms" to better indicate the subject for non-OOWDG customers who don't know what a 53A file is.

c.f. ,

You explained to me on a different thread that its probably more wise to generate orders off of 53attd rather than individual months .. that makes sense for purposes of continuity.

My question becomes this : I use techtools data so I have set up each future I trade with the requisite months I trade and the roll dates ( roll dates being before FN day , not expiry , for example so as not to be delivered on even though the contract is still active ).

Obviously backtested results will vary on which contracts I chose to omit .. for example may lean hoggs I omitt because its illiquid...

The backtested results seem to vary , as I guess one would expect , so is it not the case that in fact you are creating more system parameters by having to decide which months to trade and when to roll .. does this become yet another variable within the parameters of the system..

Also is there a generally accepted roll date for most contracts .. and are there generally accepted months to trade and months not to trade with the non- financial contracts.

Any help , as always, warmly appreciated

Chris

d-g
Roundtable Knight
Roundtable Knight
Posts: 290
Joined: Tue Apr 15, 2003 11:39 am

Post by d-g » Sun May 30, 2004 10:01 am

Chris,
I am sure c.f. will reply, but I will take a stab at this too.

That certain contracts trade only certain months, and that certain sessions have too low a volume to adequately trade, this is simply a fact.

It is critical to properly set up your data, and data itself is critical to testing, regardless of your testing platform, data and splicing is more important to testing than people give them credit for. It's not another "Variable" we are introducing into your life, it is another parameter setting in your data application that you really should be dealing with.

In your futuresinfo.txt, VeriTrader lists the trading months for each contract, you can also research them on the exchange site for the instrument.

I have a techtools convert book set up with the proper months, if you would like.

I use 5 day prior to contract expiration with OOWDG, and Open Interest with CSI primarily.

Hiramhon
Roundtable Fellow
Roundtable Fellow
Posts: 98
Joined: Fri May 09, 2003 12:45 am

Post by Hiramhon » Sun May 30, 2004 10:18 am

If you are frightened of trying out different rollover schemes, for fear of accidentally picking one that's too good to be true, you could adopt an industry standard one that hasn't changed in over eight years. It may be "good", it may be "mediocre", it may be "lousy", but at least you didn't pick it yourself so your conscience is clear: you haven't curve-fit.

One example is the rollover scheme used by Pinnacle Data: http://www.pinnacledata.com/clc.html I know several profitable position-traders who rollover using the Pinnacle schedule (even though their data vendor is oowdg/csi/genesis), and are happy as a clam.

Another example is the rollover scheme used by every robo-broker I've ever talked to: the "Roll On Open Interest Crossover" scheme that's built-in to CSI Unfair Advantage. Sure, it'll cost you some money to discard techtools and switch to CSI, but then you won't have to worry about being all alone and/or curve fitting. You'll be doing rollovers the same way that many many people do, which may be a comforting thought. And you're either not curve-fitting (since you didn't try thousands of alternatives), or else you're curve-fit the same way as everybody else.

Chris67
Roundtable Knight
Roundtable Knight
Posts: 1046
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Post by Chris67 » Sun May 30, 2004 10:54 am

Gentlemen ,

Thank you for your replies. Dan if you could send me that schedule I would appreciate it.

Surely if you roll 5 days before expiration however , on many contracts the FN day can be 30 days before expiration and you have to be out before this since you risk being deliverd on by first notice , not by expiration.

Also on the rollover dates does veritrader 1.5 assume you are in the new contract or is it dependant on the nature of the price action of the new month. I have noticed occasionally that you 'll be chugging away happily long xyz future , then on rollover the system doesnt give you an exit for the new contract even though , that too , is merrily trending away. Is that just a system error , or am i doing something wrong.

Thanks again

Chris

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Sun May 30, 2004 12:43 pm

Chris,

Rolling is indeed another variable in that it certainly affects results, sometimes considerably. I tend to think of it as independent of a system, however. I don't believe it makes sense to select different rolling algorithms on a system by system basis. Well at least not for systems of the same type, i.e. long-term trend-following, swing etc.

I tend to test using the same rolling algorithms across all my tests. I test using two or three standard algorithms and make sure that my results are reasonably similar across the different rolling methods.

The thing to keep in mind is that one or two trades that were kept because of miniscule differences in the indicators that determine the system if they end up being profitable can make a significant difference in the results.

Testing, even if done using the best controls and methods, will only approximate what you might encounter in real life trading. So the variation one sees as a result of different rolling algorithms is just one of the many ways that test results can and do vary based on seemingly minor changes.

Getting comfortable with this variability is part of the learning process in becoming an advanced trader. Little changes have surprisingly large effects on results. It shouldn't take a system from 50% returns and 1.6 MAR to 10% returns and 0.3 MAR but it might take it up to 55% or down to 45% with MAR varying slighly more from 1.8 down to 1.2.

I tend to look at all test results with a plus or minus 20% variability in the returns and plus or minus 40% in the MAR. For this reason, I don't believe you can categorically state that a system that is 10% worse or better is actually worse or better. It might be, it probably is but I tend to view results that are similar as practically the same.

Hiramhon's suggestion is not a bad one and one I tend to use myself. I know there are better ways of rolling but I also know that something standard like waiting until the open interest crosses over is at least easy to understand, repeatable in real-life trading, and not going to cause unintentional delivery or liquidity issues.

Advanced research into optimal rolling methods is fairly far down on my personal to-do research list and I won't likely get to it for at least a year.

My suggestion to those traders who are not experienced testers is to ignore this issue for now by picking a standard approach and concentrate on other aspects of your learning. For most of us, I believe there are other more important lessons to be learned.

- Forum Mgmnt

P.S. The one exception to the standard approach that I will be doing some research into is the liquidity issues for the interest rate futures like the Eurodollar which don't have their highest liquidity in the nearest months. So I may develop a slighly modified version of something like the Open Interest roll for the interest rate futures like ED.

Since ED is so liquid you still have plenty of liquidity in the less liquid near months that a standard approach would say to trade, I just want to find out if using the more liquid contracts 6 to 12 months out might be just as good or better historically.

Jake Carriker
Site Admin
Site Admin
Posts: 1493
Joined: Fri Sep 12, 2003 10:32 am
Location: Austin, Texas

RTH vs. consolidated data

Post by Jake Carriker » Sun May 30, 2004 3:30 pm

I am interested in hearing the views of the board regarding testing and trading on day session only data, rather than composite data that covers non-RTH (RTH = Regular Trading Hours) where they exist.

I have made my decision about which to test and trade based on my own testing, but I am interested in hearing what others do and how they came to the decision.

I am posting this here because, like rolling methods, this choice can materially change testing results. I feel that all of the choices that we make in regards to the data we use are best made consciously and with consideration rather than by defaulting to whatever the standard is. We may end up going with the standard method, after all things are often popular for a good reason. However, I like to make such choices only after testing confirms the most appropriate method.

I am wondering how the other systematic traders on this board treat this choice, especially in regards to how it relates to the way you place orders and trade in real time. For instance, if you have orders active outside the RTH, do you still test and generate orders with only RTH data? If so, please explain your thinking. On the other hand, if you test and trade consolidated data with orders that are active in after hours sessions, how has this worked for you?

Thanks in advance for sharing your views.

Best,
Jake

Hiramhon
Roundtable Fellow
Roundtable Fellow
Posts: 98
Joined: Fri May 09, 2003 12:45 am

Post by Hiramhon » Mon May 31, 2004 8:56 am

I've tested ((system A) plus (24-hour-data)) against ((system A) plus (day session only data)). After looking at the test results, I chose to implement the latter in my actual trading account.

But this was only a computer study. To do a really thorough job you'd need to open two accounts with the same amount of money, and trade real markets with real dollars for a while. The first account would trade a system using day session only data, and the second account would trade the same system on the same markets, using 24 hour data. You'd need to do this through a large number of different market conditions, including "fast markets" and "price spikes" (e.g. Natural Gas in Jan-May of 2003) and congestion and bull trends and bear trends. Then, compare the hard dollar results of trading day-session vs trading 24-hours and you'll be in a position to draw a definitive conclusion.

I guess Really Anal People would want you to open twenty five accounts. The first would trade day session data and place the orders before the Open of the day session. The other twenty four accounts would place orders (1) at 00:00 GMT ; (2) at 01:00 GMT ; (3) at 02:00 GMT ; (4) at 03:00 GMT ; et cetera :)

Chris67
Roundtable Knight
Roundtable Knight
Posts: 1046
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Post by Chris67 » Mon May 31, 2004 11:26 am

I think you are being a little unfair there and I think it does stem a little from the U.S idea that nothing exists outside of the U.S / trading hours ..

I have looked at this problem myself and I believe trading currencies purely on u.s pit traded hours is suicidal .. most of the big moves happen outside of the u.s pit hours .. thats why 95 % of the worlds fx business is done outside of the U.S pits altogether .. trading cash fx the best hours of the day are usually london mornings ... if you only test on u.s pit hours for fx i'm sure you will see some negative impact...

Granted on all other markets then its probably wise to stick to u.s hours

SL
Senior Member
Senior Member
Posts: 30
Joined: Tue Feb 10, 2004 7:02 pm
Location: New Zealand

Variability Due to Differing Rolling Algorithms

Post by SL » Mon May 31, 2004 4:59 pm

Chris,

The one thing here that wasn't mentioned in your reply was the trading time frame. It may well be that everybody is relating the same topic from different time perspectives or individual biases.

If you are trend following and trying to catch a breakout on the same day does it really matter if you only using the US day markets? As with most trend following systems it’s all about getting onto the trend and capturing the middle of the trend. The US session should gap open and catch say a market stop in any case; a few ticks here or there shouldn't spoil the party on a long-term system, would it?

If the system you are using were based on a much shorter time frame then it wouldn't surprise me at all from what you are saying about using the 24 markets, since you could miss a big daily move just using the US day hours.

On this subject also, could one not use daily data and still trade in the 24 market?

For example if your system said get long at 1.01 then you could place your stop in the 24 hour market in the hope the trade would be triggered when and if that point was reached. You know the approximate daily range and risk on the position, so exit stops could also be thrown in place as well. Is this a realistic possibility for application based on using a longer-term system in the FX markets?

Stephen

Post Reply