Did I ever miss it…
c.f. wrote:
"The main problem with using historical testing as a means of system analysis is that the future will never be exactly like the past. To the extent that a system captures its profits from the effects of unchanging human behavioral characteristics that reflect themselves in the market, the past offers a reasonable "approximation" of the future, but never an exact one.
First, I'd like to say your article was the best post I've read on this forum; I enjoyed reading it.
But if you would allow me to disagree, historical testing of the distant past is no where near a reasonable approximation of the future.
The distribution of price change from years past is less likely to repeat than the distribution of price change from the recent past.
This is one reason I believe in utilizing moving real time historical results of the recent past to optimize my systems. If current distribution of price changes varies significantly from their norm of recent past it will be time for me to reoptimize my system-that spell big change ahead, it's my best indicator
. I optimize my systems using a look back period of about thirteen months on average. [that's all the sample size that I need]
There is an instance where historical testing is very important.
1)Identify past period of market hisotry where the market was following the same RULES as today. 2) We can then see if the system that worked in the recent past look-back period also worked back then. The idea is markets that exhibit stationarity of distribution of price changes follow the same RULES-irregardless of time period or time frame. If our stategy passes this taste it should increase our confidence in the strategy. It is a good taste for robustness.
Quantitative risk management is always best left to a computer, it is better at it. I work on the qualitative or discretionary aspects of my system to increase the probability of catching a good trend by analyzing patterns, sentiment, momentum, and volume. A simple breakout entry system cannot tell the probability of the outcome of a single trade...this might sound contradictory, nonetheless, as many have said before process matters much more than result, but single trade results still matter some what. Let me explain... I generally augment my semi-discretioaniary entries with statistical stationarity analysis to better judge the probabilities. Drawdowns are huge in most mechanical systems for this simple reason- low probability entry methods. Some trades are not worth the risk and should never be taken even if a breakout occurs. If one can design a system that has a low average loss and a high winning percentage with high payoff[it is very difficult but yes it is possible] we are looking at a system that should be very drawdown resistant. However, in spite of what some portfolio strategists would have us believe low percentage systems actually increase the likelihood of a major drawdown exponentially rather than decreasing it. If by careful design our high percentage system is also capable of performing at an optimal level under low win rate conditions then it could be said we've got optimal synergy between risk and reward and all the parts of our system that constitute our model....much easier said than done
It has always bewildered me why it is important to build and optimize systems with tones of historical data while giving little thought to the time frame over which the system might be valid. In my humble opinion, it is critical to re-optimize systems according to the characteristics of recent distribution of price change. Only recent time series has any predictive value. If you are trading a system that has been optimized on long term historical data, you have no way of knowing if the current distribution of price changes matches those from the past. Optimal systems trading would require that you identify the RULES of the market NOW and base your optimization on that. The only way to tell if there has been a change in the distribution of price changes is by testing and seeing an actual shift in the distribution of price changes. The next trade might just be when the distribution changes. This element of uncertainty is what Victor, or LTCM weren’t able to deal with . Talk about seating ducks
I feel mechanical systems do their best when utilized as guides, and as flexible[only on selection and entry/exit] and desciplined[how much?] trading partners that search for the rules that govern markets now.
I do not differentiate between my long term and short term systems or trades … I simply trade those time frames that exhibit the kind of behavior I’m scouring for.
Optimization is only a problem when you fail to take stationarity into account. It is a vastly time consuming process
But well worth the effort.
Now, you can fire up VeriTrader 1.5 & your Excel spreadsheet and test away ...
Have fun and Good Trading,
MT