Hi ,
If you design a system say on data from 1988 to 1998 and then test it on 1999- 2004 what sort of profitabilty would be defined as robust over 1999-2004 ,.. for example if i have a system that produces a cagr of 40 % with a 17 % drawdown over 88-98 .. then i test it over 1999 - 2004 and it produces 35 % with a 22 % drawdown .. is that a good robust sytem ?? how much would the sytem be expected to change over the diffeent test to the time frame it was designed on ??
Any help appreciated thanks
chris
walk forward testing
First figure out what you want, then test for that.
For example, suppose that you want a (system + parameterset) whose CAGR and MaxDD over the period 1999-2004, are very close to its CAGR and MaxDD over the period 1988-1998. If this is what you want, wonderful. Congratulations on your self-knowledge. Now test for what you want!
You don't need a lot of mumbo-jumbo "Walk Forward Testing" superstition. Just define your objective, perform your experiment, collect your data, and draw your conclusions.
Let me refer you to these articles which discuss the folly of test protocols (such as "Walk Forward") that avoid exhaustive testing due to irrational dread of The Curve Fitting Monster
viewtopic.php?p=7772&highlight=protoco%2A#7772
viewtopic.php?p=7775&highlight=protoco%2A#7775
For example, suppose that you want a (system + parameterset) whose CAGR and MaxDD over the period 1999-2004, are very close to its CAGR and MaxDD over the period 1988-1998. If this is what you want, wonderful. Congratulations on your self-knowledge. Now test for what you want!
You don't need a lot of mumbo-jumbo "Walk Forward Testing" superstition. Just define your objective, perform your experiment, collect your data, and draw your conclusions.
Let me refer you to these articles which discuss the folly of test protocols (such as "Walk Forward") that avoid exhaustive testing due to irrational dread of The Curve Fitting Monster
viewtopic.php?p=7772&highlight=protoco%2A#7772
viewtopic.php?p=7775&highlight=protoco%2A#7775