random time series backtests

[mpok8]

I have a statistician PhD friend and he suggested that a good way to test a system might be to get a number (the more the better) of random time series and then to optimize a system over these random time series and calculate the risk adjusted return.

On average if the time series is random the system should give results better than 0 -- because it has been optimized --, but results worse than the system optimized over the actual price series.

This is because the system should be isolating a non-random and recurring (hopefully) pattern in the price series which is by definition not present in a random time series... does anyone have any comments as to whether this is a valid test procedure?

Thanks

[sluggo]

You might want to contact David Aronson, author of (this book) . He is working on the very same ideas, in loose collaboration with Ralph Vince, John Bollinger, Philip McDonnell, and others. There is a website for readers of the book, where you can send him email. Google for webpages containing the words Aronson and evidence, his is the very first hit.

[svquant]

You may also want to do a search on bootstrapping and testing of technical trading systems for some insights into what people have done to validate the statistical significance of their trading systems vs pure luck (random data).

It is a little different than what you are proposing since one does not optimize on the "random data". One generates the random having the same statistical properties as the real data and sees if the trading algorithm detects structure in the real data vs synthetic series. The easiest way to do this is to scramble the return series of the original data - you still have the same average return and std deviation and other moments as the real series. Others go farther and use linear and non-linear generating functions - the fun and PhD thesis topics never end!

[zentrader]

...One generates the random having the same statistical properties as the real data and sees if the trading algorithm detects structure in the real data vs synthetic series. The easiest way to do this is to scramble the return series of the original data - you still have the same average return and std deviation and other moments as the real series. Others go farther and use linear and non-linear generating functions...

That's what I do with my Zen Monte Carlo Simulator "Data simulation" functions. The principle is described in an article here:
http://www.zentrader.de/mcs_article_traders2007.pdf

bye,
zentrader