Recent vs. Distant Results

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
TrendMonkey
Roundtable Knight
Roundtable Knight
Posts: 154
Joined: Fri Apr 22, 2005 9:14 pm
Location: Vancouver, Canada

Recent vs. Distant Results

Post by TrendMonkey »

I'm evaluating a LTTF system that meets all my criteria, its only apparent failing is that recent performance, from late 2003 until now, is pretty poor and in most cases slightly negative.

From sniffing around this site, I understand that this period has been tough for many LTTF's, and certainly I have noticed similar and usually worse results in this period from other systems.

My question is, how do you weight recent vs. distant performance, or to put it another way, at what point do you start to worry that your system doesn't work anymore, vs. has merely hit a rough patch?
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

An acquaintance is running one of the Veritrader 1.6 systems at a robo-broker. Here is a simulation of his system + portfolio, which begins on 1/1/1997 using $200K of initial equity.

As you say, calendar 2003 was unpleasant, but not unbearably so. The drawdown was overcome and it has been a steady uninterrupted march to the heavens since then. Marked-to-market today the system has achieved a CAGR of 82.2%, max Drawdown of -38.1%, longest drawdown = 9.7 months. $200K to $32 million in 7.5 years.

Good systems are out there.
Attachments
eq_200k.png
eq_200k.png (19.46 KiB) Viewed 12699 times
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

Post by damian »

sluggo wrote: The drawdown was overcome and it has been a steady uninterrupted march to the heavens since then.
An understatement. Nice skyrocket.

Trend Monkey - I have a few great model systems that look dreadful from Q1 2003 there after. Of course it depends on the portfolio composition.

I also have a few that do not look so bad, but they are pretty different to the 'bad ones' in terms length of holding period and also 'width' of entry. ie, the market has to trend a lot before an entry is made, then it holds that trend with much wider stops.

When I was 'helping' beta test TBB I gave myself an exercise of designing a system and testing it over 12/2002 - present. It had to perform well on 50 markets. Perform well defined as >20 CAGR and MAR >1. I did it ok. I then tested the last 10 years. Very nice. I then tested the last 15 years to discover that 1991 - 1993/4 was as appalling. Way worse than the much hated 2003-2005 period. Plug one hole and another one springs.

I do not know what people are doing to perhaps filter the bad trades that results in the oft-seen 2003-2005 DD. Try: buy at HH50, sell LL50, stop at MA50 for a simple 'system' that on my portfolio demonstrates the 2003-2005 DD very clearly. Then try and design something that doesn't behave like this system (ie, don't buy on really fast run ups, only buy after a long sustained up trend.... think MA's rather than break outs). Others will 100% disagree with that statement. Also try weekly data but remember that is not the same as daily in more than the obvious way, eg, 1 ATR on daily vs 1 ATR on weekly compression.

ps - There will be many other people getting very nice (hypothetical) equity curves using daily data breakouts. I don't know what they are doing right, but they are better designers than I am.
TK
Roundtable Knight
Roundtable Knight
Posts: 167
Joined: Tue Apr 15, 2003 5:45 pm

Post by TK »

damian wrote:I don't know what they are doing right, but they are better designers than I am.
Or they might be better portfolio selectors. :mrgreen:
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

Post by damian »

:D

My tongue in cheek cynical attitude is that portfolio selection is at worst blind luck and at best turns pure system trading into discretionary trading.

I am not unhappy. My LTTF system produces a hypothetical monthly EC shows a MAR of 3.3, 45 markets, start capital $150,000, with Max DD of 10% over the last 18 months of LTTF 'horror'. To do this I had to risk 3% per trade FF and hypothetically sustain 2 trades that lost 5% and 3 that lost 4% over a 10 year period. The higher MAR is a result of lower DD and these lower DD's bring a potential negative aspect: long periods (approx 1 year is common) of sideways equity growth followed by short spurts of positive equity growth. Depending on definition, MAR can be improved. In fact, using (CAGR/avg DD) I get a MAR of 4.5. Hypothetically.
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

Portfolio "selection" seems a very dangerous tool. I prefer the attitude that you should trade as many different futures as your account and your tolerance for portfolio heat will take. I am spending much time looking at individual markets and it seems as futile to predict when a market will trend as to predict future prices. Of course, there are markets which seem to have trended well and persistently for many years, while there are others such as palladium which have had long periods of nothing and then a huge spike.

I rather doubt that Sluggo's chum is sitting happily on his $32m lets put it that way. For a start Veritrader 1.6 only came out about a year ago.
TK
Roundtable Knight
Roundtable Knight
Posts: 167
Joined: Tue Apr 15, 2003 5:45 pm

Post by TK »

Recently, a couple of people have posted examples of systems with excellent equity curves in the recent period. What I don't like about them is that they are extremely long-term with 1-2 trades per year per market. My worry is that with ca. 1000 trades in 15 years, it's difficult to say whether a system has a real edge or it's been simply lucky. This is not to say that I don't like long term systems, only that it's harder to evaluate their robustness or edge. That said, I remember Jerry Parker and David Druz explaining that going long term is the right thing to do. (By the way, I really liked Druz's paper, especially the The robustness of a trading system is proportional to its volatility part.)
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

Where can I find the Druz paper? On his website? Yes, re Jerry Parker, his views at that futures conference a couple of years ago were most interesting. Re 1 to 2 trades a year, I think one can take comfort from adding many more markets to the test and taking the total portfolio above 100 assuming a huge account size.

What I have found most illuminating is to look at how well these very simple very long term systems trade at the individual contract level and compare that to the scrappy results you get on many futures from something like Aberration. Of course, this gets even better in a portfolio context.

Re putting the matter to the test in real life, it is also instructive to look at www.mtlucas.com - you don't get a much or longer term simpler system than their index is based on. And while I have seen no results for individual accounts, they claim to track the index for clients.

OK, so results have not been appetizing over the last few years and their portfolio is smalll compared to what we have been talking of (around 20 I seem to recall).
TK
Roundtable Knight
Roundtable Knight
Posts: 167
Joined: Tue Apr 15, 2003 5:45 pm

Post by TK »

AFJ Garner wrote:Where can I find the Druz paper? On his website?
Click on his name in my previous post. :D
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

Sorry, I had not noticed the link! Thank you.
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

Post by damian »

I am happy with 200-300 trades over 10 years and 50 markets. Each market trading an average of 5 times over ten years seems like a good way to make money to me, particularly when a winning trade lasts for years.

In fact, with wider stops I am happy with 150-200 trades over the same period and portfolio.

Mind you, I am also happy to make $5 million out of $200,000 over 10 years.

Strange how a 60% real life DD changes your perspective ;)

How long ago would you liked to have entered a long in CL? Since year 2000? What about ED? Mid to late 90's perhaps? Short grains since when? I wrote about 'riding whales' on this forum a few years ago and have been researching them for a while. Gunna do me some riding (my best Texan accent).

Trading does not always require many transactions. And what is the difference between trading and strategic investing? Not much in my mind. Some of the richest traders I know made most of their money from their ten best transactions. Some have only made 50 trades in the last 10 years. They however are discretionary operators.

Sorry for getting off-topic.
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

It may be enjoyable to look at Market Wizards (Copyright 1990) page 110:
The secret is being as short term or long term as you can stand, depending on your trading style. It is the intermediate term that picks up the vast majority of trend followers. The best strategy is to avoid the middle like the plague.
Those who are nervous about the long-term portion of this advice, might wish to consider adding a short-term component. Besides your one-trade-every-24-months yawner system, you can also daytrade using 1 minute bars. Successful trader John Lynch does exactly that, makes a comfortable return, and sleeps soundly. Perhaps it would relieve some of your discomfort too.
TK
Roundtable Knight
Roundtable Knight
Posts: 167
Joined: Tue Apr 15, 2003 5:45 pm

Post by TK »

It may be reasonable to look at the advice posted almost 4 years ago on the Trader Club forum:
Number of trades to prove a system is statistically righteous: about 1500, prefer 4000 or more
Successful trader Leonardo wrote and does exactly that. :wink:
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

Post by damian »

sluggo wrote: Besides your one-trade-every-24-months yawner system, you can also daytrade using 1 minute bars.
That is exactly what I am doing, plus one other time frame.
TrendMonkey
Roundtable Knight
Roundtable Knight
Posts: 154
Joined: Fri Apr 22, 2005 9:14 pm
Location: Vancouver, Canada

Post by TrendMonkey »

Damian, by that do you mean you are trading both a LTTF and a 1-minute day trading system? Kind of an extreme time diversification idea?

BTW, thanks for all the comments, everyone.
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

Post by damian »

Yes, I suppose that it is an extreme time frame difference. But that does not imply bad. After all, winning trader Mr John Lynch trades likewise as erstwhile Sluggo referenced ;)

In answer to your question, 'what do I mean by LTTF and 1 minute day trading system':

system 1: LTTF on 45 markets using weekly bars. Winning trades last > 1 year, often 2-3 years.

system 2: day trade system on 1equity index using 1 minute bars. Winning trades are in the hours.

System 3: hourly bars, traded on 4 equity indexes, winning trades 1 - 3 days.

system 4: hourly bars, same 4 equity indexes, winning trades 1 - 3 days. Almost zero equity curve correlation with System 3 and system 2.

There are other systems and aspects involved but I won't bother to discuss those details.
TrendMonkey
Roundtable Knight
Roundtable Knight
Posts: 154
Joined: Fri Apr 22, 2005 9:14 pm
Location: Vancouver, Canada

Post by TrendMonkey »

I think that would be ideal for me, but whereas I feel I have somewhat of a handle on LTTF systems, I really don't know where to start to think about a day trading system. A project for later in the summer, perhaps.
Timek
Roundtable Knight
Roundtable Knight
Posts: 104
Joined: Thu Sep 09, 2004 8:06 am
Location: London, UK

Recent vs. Distant Results

Post by Timek »

Going back to the original question (namely when do you start to worry that your system doesn't work anymore, vs. has merely hit a rough patch), this reminds me of a comment in Alexander Elder's book "Come into my trading room" - there he states that only experts can trade using automated systems as only they will know when their systems don't work any more.
This suggests that only by understanding the price movements intimately (obtained by hard graft poring over charts) can one tell whether the behaviour has changed. One would not necessarily know this, if one was just systematically trading a set of rules that one had backtested, however thoroughly.

Obviously if you have examined many backtests with different systems and parameters then you would have a better idea with what is a normal drawdown. But is this enough by itself or is there still no substitute for market knowledge?

Additionally trading more then one system (particularly if they were based on different principles) would protect you as they would be unlikely to all fail at once.
Post Reply