The future of LTTF system testing

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

The future of LTTF system testing

Post by damian »

This is not a discussion about whether LTTF is dead. It is more so a discussion about the future of LTTF system testing. This post is also leaning in the direction of opinion and conjecture.

Two years ago many people may likely have said:

"I would never trade a LTTF system that shows a 60% drawdown in the historical testing, regardless of how large the MAR".

It is perhaps safe to assume that they would hold the same opinion today and would conduct the same tests today. The problem is that most expressions (system variations) of the principles of LTTF now show a 60% drawdown....a drawdown that has not finished yet in many systems and portfolio combinations.

In theory a large number of the traditional, historically profitable, LTTF systems are never going to be traded again [in the next 10 years or so] except by those that have

a) never worried about 60% drawdowns or
b) that post fact decide that the 60%-er was a one off.

Will the LTTF systems of the future (starting tomorrow) be systems that are constructed in such a way that this 60% drawdown did not take place in backtesting. It is reasonably easy to pick parameters, rules and portfolios that minimise the LTTF drawdown of the last year. I have tried it out myself by constructing quite a few systems that produced acceptable backtest results over the last year. I set out on an exercise of optimisation to see how easy it would be. Will there be a new breed of optimisers that must develop systems that minimise that one drawdown?

One thing is firm in my mind: for many (not all) LTTF systems this drawdown is real. It shows up in real accounts and it shows up in backtesting. It has changed the backtesting landscape, particularly in terms of result interpretation and what is deemed acceptable. What follows is likely regarded as an obvious statement:

The person that in the future understands how to manage this huge break down in "what we are used to seeing" in LTTF results (real and backtest) will be the winning long term trend trader of the future.

I used the word 'manage' and it can be expanded into far more detail but. I am steadily working at finding how I will manage/take into account the last years LTTF characteristics and steadily making no profound progress at all. One thing is for sure, the events of the last year hold a huge amount of opportunity when it comes to designing LTTF systems that actually work. I don't believe any time in the backtest window of 1985 - 2002 provided such a potentially enlightening period as we have seen in the last year. It is potential as not everyone will glean the light that is available. Time to finish this long post before my candle runs out and I am left in the dark

damian

(my co-worker suggested that my thoughts here are akin to a scientist being excited by a large meteor hitting earth as the event allows him to learn more about the evolution of life. I laughed and hoped I was not that terminal)
TC
Roundtable Knight
Roundtable Knight
Posts: 101
Joined: Thu Jul 08, 2004 2:12 pm
Location: Toronto, Canada

Post by TC »

I think a 60% DD is probably at the far end of the spectrum, I suspect most professionally-managed LTTF systems are probably in the 20-40% range. Most clients cannot tolerate DD > 40% and for this reason managed funds will move heaven and earth to contain DD within the tolerance level of the majority of its investors.

Also, any fund that is experiencing a 60% DD is likely to be trading quite aggressively and could therefore have made 150% last year and consequently just be flat now when viewed over the past 2 years.
Hiramhon
Roundtable Fellow
Roundtable Fellow
Posts: 98
Joined: Fri May 09, 2003 12:45 am

Post by Hiramhon »

Drawdowns are determined by the positionsizing method used. It's not correct to say that "system A has suffered a 60% drawdown" without also mentioning the positionsizing that led to that DD. Another person could trade system A while risking one tenth of one percent of equity on each trade, and that person would certainly not suffer a 60% drawdown.

I think perhaps you mean that everyone who traded system A, suffered a drawdown percentage 1.5X bigger than they saw in historical backtesting. Some, who use small position sizing, expected a worst case DD of 10%, but they actually got 15%. Other, more aggressive, traders expected a worst case DD of 40% but they actually got 60%.

But it is not true that all people who traded system A, got the same drawdowns.
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

Post by damian »

:shock: :oops: I cant believe I forgot to mention position sizing in all of the above. I am quite embarrassed. Agree totally with your point Hiramhon.

I am putting together more on this topic...
Roscoe
Roundtable Knight
Roundtable Knight
Posts: 250
Joined: Sat Jan 24, 2004 2:06 am
Location: Houston TX

Post by Roscoe »

Damian,

Good thread. I do a lot of testing on longer-term systems and I do notice a few relatively common characteristics regarding percent drawdowns (hereafter MDD%):
1. Individual MarketSystems in the “marginal to goodâ€
Post Reply