How are people splitting up their historic data for testing puposes. I have approx 30 years . I built my system on 1990 - 2000 then tested it 1975 to 1990 and tried it out on 2000- feb 2004.
This all seems to work okay although I am finding the results a little worse on the 2000 - 2004 period.
It seems unusual to me that if you do your parameter altering on 1990 - 2000 then apply this mildly optimised strategy to 1975 - 1990 that the results are in fact even better , but that they should fall away in recent history .. i.e from 2000 onwards
Also what is people opinion of what should qualify as recent test ?? is it the last 3 . 5 or 10 years.
Any help would be warmly appreciated.
Best wishes
Chris C
PERFORMANCE ANALYSIS
The following is my observation, and test result. Perform your own testing and verify Hypothesis:This all seems to work okay although I am finding the results a little worse on the 2000 - 2004 period.
It seems unusual to me that if you do your parameter altering on 1990 - 2000 then apply this mildly optimised strategy to 1975 - 1990 that the results are in fact even better , but that they should fall away in recent history .. i.e from 2000 onwards
Also what is people opinion of what should qualify as recent test ?? is it the last 3 . 5 or 10 years.
From what little info you provided about your system … it is safe to generalize the time period between 1990-2000 and 1975-1990 resemble "some what" similar patterns for many markets.
For instance, the bull market in equities, some would argue, started with the bottom that occurred in 1974 and lasted until 2000. I have found many system results to be quite good between 1974-1990, there were a lot of sustained trends in many markets.
That, started to change after 1987. The bear market in equities and subsequent change in volatility that started in 1987 and lasted until the early 1990’s changed the performance of many systems- some for the better, some for the worse.
Beginning in the early to mid 1990 I started noticing a slight but gradual and sustained drift in the performance of many commodity systems. That held true until 2000. The RULES completely changed after 2000. We’re now in our 5th year bear market in equities, and we’ve had great trends in commodities.
The RULES of the current time period has very little in common with the time period on which you optimized your system. This period, we’re in now, more closely resembles the bear market that occurred between 1966-1974.
Try setting your parameter on 2000-now, then try applying it to 1966-1974; you will find the results quite interesting. I can assure you the result will start to drift ones again as the RULES continue to change and that is where retesting and reoptimization comes in.
In my humble opinion, what’s important is finding the link between the causal agent (the ever changing RULES of the market) and the systems performance through less naïve empirical observation. What you want to do is take the best of what the past has to offer without its dangers.
I realize I am in the minority on this but what matters is not the size of the sample data. If enough rules are tested over time some will definitely appear to work just by luck, even though they have no predictive value since they have nothing to say about the rules. Such methods can be misleading and systems optimized on such method, I find are inferior, since it does not account for the full set of initial rules.
P.S you have to remember in the long term (10+ years) there has not been a sustained long term bull (or bear) market, at the same time, in both the equities and commodities markets. When the rules change for one market, they appear to change for the other as well. So whichever market you are trading the above observations apply.
For further info check out the following:
viewtopic.php?t=835