walk forward testing

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
Chris67
Roundtable Knight
Roundtable Knight
Posts: 1052
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

walk forward testing

Post by Chris67 »

Hi ,

If you design a system say on data from 1988 to 1998 and then test it on 1999- 2004 what sort of profitabilty would be defined as robust over 1999-2004 ,.. for example if i have a system that produces a cagr of 40 % with a 17 % drawdown over 88-98 .. then i test it over 1999 - 2004 and it produces 35 % with a 22 % drawdown .. is that a good robust sytem ?? how much would the sytem be expected to change over the diffeent test to the time frame it was designed on ??

Any help appreciated thanks

chris
Hiramhon
Roundtable Fellow
Roundtable Fellow
Posts: 98
Joined: Fri May 09, 2003 12:45 am

Post by Hiramhon »

First figure out what you want, then test for that.

For example, suppose that you want a (system + parameterset) whose CAGR and MaxDD over the period 1999-2004, are very close to its CAGR and MaxDD over the period 1988-1998. If this is what you want, wonderful. Congratulations on your self-knowledge. Now test for what you want!

You don't need a lot of mumbo-jumbo "Walk Forward Testing" superstition. Just define your objective, perform your experiment, collect your data, and draw your conclusions.

Let me refer you to these articles which discuss the folly of test protocols (such as "Walk Forward") that avoid exhaustive testing due to irrational dread of The Curve Fitting Monster

viewtopic.php?p=7772&highlight=protoco%2A#7772
viewtopic.php?p=7775&highlight=protoco%2A#7775
Post Reply