Recent Trend-following system performance

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
Christian Smart
Roundtable Fellow
Roundtable Fellow
Posts: 50
Joined: Fri Apr 18, 2003 8:53 pm
Location: Huntsville, AL
Contact:

Recent Trend-following system performance

Post by Christian Smart » Sun Feb 19, 2006 11:49 pm

[MODERATOR'S NOTE: This posting and the two that follow were split from another topic on the "Top 10 Trading Systems of All Time"]

>I won't be too harsh judging mechanical systems performance from the >last few years, especially 2005. From the 2005 public funds return
>(futures mag, same issue), a lot of them are in the red. The last few >years (except 2002) lacklustre performance for commodities systems.

I agree - traditional trend-following performance has been dismal the last three years. This would be more acceptable if commodities had not experienced such a large uptrend over the last three years. I did a backtest this weekend in Trading Recipes three of the "Top Ten": Aberration, Trendchannel, and Golden SX. For grins, I also tested a "Turtle-Type" Donchian 20/10 channel breakout and Mark Johnson's PGO. I used a diversified 18-market portfolio: British pound, coffee, cotton, crude oil, U.S. dollar index, euro currency, heating oil, Japanese yen, lumber, natural gas, Nikkei index, orange juice, palladium, soybeans, sugar, ten-year notes, unleaded gasoline, and U.S. bonds.
Attachments
comparison.jpg
comparison.jpg (26.08 KiB) Viewed 9232 times

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Mon Feb 20, 2006 4:19 am

Christian's helpful table above may help to answer a question which always tends to puzzle those newer to this game. Which goes something as follows: how come when my tests show that you can turn $50,000 into $10zillion in 20 years, the records of all the CTAs I have looked at suck?

With the aid of TBB we can all produce an ultra simple system which would have done jolly well and have ridden copper, sugar, the energies and so forth to a triumphant and ever increasing stream of profits over the past few years.

We might also note however, by looking at different time frames, that in earlier years the results of such systems may not have been so spectacular. So ten years ago we would probably have chosen to trade very different systems to those which would have triumphed over the past three years.

Some would deny that markets change. Well perhaps they do not, when looked at in a certain very broad perspective - the truths are still there but such truths may express themselves in different ways or timeframes. But what worked so well 10, 20 or 30 years ago may not be such a great idea today.

So the CTA like the long term individual trader will find himself having to adapt over the years: to different contracts to trade (who is trading the Spanish Bond? do you really want the illiquidity of the Muni Bond?), to maturing markets, to emerging markets and so on.

As I have often said before, the disclosure documents of the longer surviving CTAs will often be found to contain brief notes about systems dropped or altered to meet changing conditions of one sort or another.

Mind you, if you want to cherry pick your portfolio, you could easily come up with a combination which the above systems would have profited out of very nicely over the past three years. If you choose a more normal 50 future portfolio such as that traded by Mulvaney, Abraham, Dreiss etc etc..........each system would have sucked.

Perhaps after all, there is validity in running different horses on different courses. Run a shorter term system on such mature markets which still seem to work for such a system and add in some of the less mature markets. On a more "traditional" portfolio, play longer term.

But of course, we won't know the "right" answer until 3, 5, 10 or 20 years from now.

Angelo
Roundtable Fellow
Roundtable Fellow
Posts: 90
Joined: Fri Apr 29, 2005 4:31 am
Location: Italy

Post by Angelo » Mon Feb 20, 2006 7:03 am

AFJ Garner wrote:Christian's helpful table above may help to answer a question which always tends to puzzle those newer to this game. Which goes something as follows: how come when my tests show that you can turn $50,000 into $10zillion in 20 years, the records of all the CTAs I have looked at suck?
I hadn't a definitive answer, and I have not today. Any more thoughts would be welcomed.

I asked the very same question on the TR list on “April 2005â€

mike168
Roundtable Knight
Roundtable Knight
Posts: 142
Joined: Mon Jan 23, 2006 10:24 pm
Location: Australia

Post by mike168 » Mon Feb 20, 2006 5:05 pm

Wow! This has suddenly become very interesting after your sharing of the thoughts from you experienced guru. I would like to share my thoughts as follows (I have to stress again that I am not an experienced system developer or great trader):
1. As we may suspect, system performance changes with time. This may be because of either:
1.1 The price patterns are not stationary in time. They fluctuate about a mean. When we design systems, we may be basing our assumptions on a price pattern/relationship that is going to repeat in the future. Whereas in actual life, because of this fluctuation with respect to time, systems seem to "break down". In fact, it is not that the systems actually broke down, just the price patterns have deviate from the mean at this time.
1.2 The price patterns actually change. People found that it is the intermediate term systems that got affected the most. Very short term systems (and price patterns) and very long term systems (and price patterns) are relatively stable.
Perhaps Sluggo could enlighten us with your analysis on these?
2. If we gather together a sample of 10 million people who buy lottery. There will always be someone who win the jackpot. He could then write a book to teach others how to use his "method" to win. Similarly, no matter how good your system is, there is a probability that you will be broke if you trade long enough - perhaps many hundred years? The question then is not who won the most recently, but who could win the most consistently. In this respect, although I would like them to be pure mechanical system traders, I doubted this very much. I think they would be people who based their investment (not trading) on fundamentals and methodology. If we are to compare our trading results with the performance of Warren Buffet, he could easily beat most of us hands down.
3. Ralph Vince pioneered the theory of modern portfolio theory (I think). In his books he stressed that money management is more important that system parameter optimization and other systems aspects like entry and exit. Using money management and portfolio composition, a bunch of mediocre systems with positive expectancy could be turned into money spinners. So the stress should be based on portfolio composition and system money management. In his book, he even detailed how to calculate optimum % composition for systems. In TBB, up to now, the system % has to be entered in the global, and not steppable whereas the entries and exits, other parameters testing are quite versatile. In this respect, the emphasis is misplaced.
Last edited by mike168 on Tue Feb 21, 2006 10:23 am, edited 1 time in total.

mike168
Roundtable Knight
Roundtable Knight
Posts: 142
Joined: Mon Jan 23, 2006 10:24 pm
Location: Australia

Post by mike168 » Mon Feb 20, 2006 10:58 pm

Further to my post above, I would like to add that if the price pattern relationship with time is indeed as in (1.1) above, then there is serious consequences in terms of our trading practice.
Consider the following oversimplified scenario: if we have a 100 year or more trading period and we divide this into time frames of say 10 years each. Within each 10 year period, the price pattern is very much constant with respect to time. This makes certain trading systems exploiting these patterns very profitable. As we progress into other 10 year periods, these time-price pattern changes, making other systems more profitable. But since the time-price fluctuation is around a mean, a group of systems which happens to be designed around the mean would emerge as the winners after this 100 years. Does it mean we should stick to this group of systems throughout the entire 100 years? Absolutely not. As traders, we are more concerned with short (few years) time frame performance and ensure that we will not bankrupt in the immediate future.
So in this scenario what should we do? I think in contrast with the popular belief that we should not curve-fit our systems and over-optimize, we should in this case! What I mean is not we should select parameters near the peak of the 3D graphs - we should continue to select those in plateaus and valleys. What I mean is perhaps we should get a collection of say 20 systems (and their corresponding instruments). Out of this collection select the best 3-4 systems (in terms of recent,say, 2 years performance), trade them for 3 months - then continue the selection process from the start. The reason? Although over-optimized systems, cherry picked portfolios etc may break in future, who cares as long as they hold up in the immediate few months (since the price pattern changes with time anyway). This may seem radical initally but indeed it gives us an edge. As owners of TBB, we are much easier than others to do this type of optimization and system selection process. The modern PC technology also greatly facilitates this type of process as compared to,say, ten years ago.
Any thought on this? :wink:

mike168
Roundtable Knight
Roundtable Knight
Posts: 142
Joined: Mon Jan 23, 2006 10:24 pm
Location: Australia

Post by mike168 » Tue Feb 21, 2006 10:46 pm

Just received Attain Newsletter - R Mesa 5 drawdown currently at 94% and Attain is using drawdowns of over 100% to evaluate the stop-trade levels. Potential investors in Attain's systems beware - you should fund your account properly and prepare to lose at least the developers' recommended account sizes and more.

edward kim
Roundtable Knight
Roundtable Knight
Posts: 344
Joined: Sun Apr 20, 2003 2:42 pm
Location: Silicon Valley / San Jose, CA USA
Contact:

Post by edward kim » Wed Feb 22, 2006 2:39 am

AFJ Garner wrote:Christian's helpful table above may help to answer a question which always tends to puzzle those newer to this game. Which goes something as follows: how come when my tests show that you can turn $50,000 into $10zillion in 20 years, the records of all the CTAs I have looked at suck?

We might also note however, by looking at different time frames, that in earlier years the results of such systems may not have been so spectacular. So ten years ago we would probably have chosen to trade very different systems to those which would have triumphed over the past three years.

Some would deny that markets change. Well perhaps they do not, when looked at in a certain very broad perspective - the truths are still there but such truths may express themselves in different ways or timeframes. But what worked so well 10, 20 or 30 years ago may not be such a great idea today.

As I have often said before, the disclosure documents of the longer surviving CTAs will often be found to contain brief notes about systems dropped or altered to meet changing conditions of one sort or another.

Perhaps after all, there is validity in running different horses on different courses. Run a shorter term system on such mature markets which still seem to work for such a system and add in some of the less mature markets. On a more "traditional" portfolio, play longer term.
the records of CTAs "suck" for several reasons: start date, slippage, trading mistakes (not following their system), and they report returns net of fees.

a system sometimes is only as good as the data set available at that time. system parameters might change as more instruments and data become available. the kicker is that a robust system has parameters that don't change very much (regardless of the test period, a wide range of starting account sizes, and the instruments included), and also trades where no one else is trading.

LD
Full Member
Full Member
Posts: 17
Joined: Thu Mar 24, 2005 4:57 am
Location: Singapore

Post by LD » Wed Feb 22, 2006 4:11 am

This is a really good thread. I was going to start a new one with a subject like "What makes a good trend-following system that is realistically followable", but what I was going to say would fit very well in this thread.

After endless runs on Trading Blox it became obvious that the theoretical best risk-adjusted returns came from very active trading (still not day trading since TBlox only captures each day's data and doesn't generate more than one order per day) with extremely tight stop losses.

I fine tuned this backtesting over the last 10 years, and results were surprisingly stable where the equity line is almost a straight line. I would cut/paste the .html output from Trading Blox like many others in this forum do if I knew how to do it, but the results are as follows (from memory as I'm writing this from work and TBlox is set up on my home PC):

CAGR: over 420% p.a.
max total equity drawdown: around 20%

We'd all be richer than Bill Gates if we could trade this system, too bad in practice it doesn't work. Just adding slippage by 10%, which is already optimistic in real life, dunks the CAGR from 420% to 30%, and increases the max drawdown from 20% to around 40%!

RedRock
Roundtable Knight
Roundtable Knight
Posts: 939
Joined: Fri Jan 30, 2004 3:54 pm
Location: Chicago

Post by RedRock » Wed Feb 22, 2006 1:33 pm

Very close stops can not be tested accurately on daily data as many times on entry day the tight stop would have been hit intraday. Yet the bar may close in the direction of the position and the software has no way to know what happened intraday.


LD wrote:
After endless runs on Trading Blox it became obvious that the theoretical best risk-adjusted returns came from very active trading (still not day trading since TBlox only captures each day's data and doesn't generate more than one order per day) with extremely tight stop losses.

!

TK
Roundtable Knight
Roundtable Knight
Posts: 167
Joined: Tue Apr 15, 2003 5:45 pm

Post by TK » Wed Feb 22, 2006 2:05 pm

RedRock wrote:Very close stops can not be tested accurately on daily data as many times on entry day the tight stop would have been hit intraday. Yet the bar may close in the direction of the position and the software has no way to know what happened intraday.
One simple solution that allows one to test systems with tight stops (1) using daily data AND (2) without compromising the
I test what I trade and I trade what I test principle is to add a trading rule: Do not use stops on the entry date and see what the test shows.

RedRock
Roundtable Knight
Roundtable Knight
Posts: 939
Joined: Fri Jan 30, 2004 3:54 pm
Location: Chicago

Post by RedRock » Wed Feb 22, 2006 3:23 pm

TK wrote:
RedRock wrote:Very close stops can not be tested accurately on daily data as many times on entry day the tight stop would have been hit intraday. Yet the bar may close in the direction of the position and the software has no way to know what happened intraday.
One simple solution that allows one to test systems with tight stops (1) using daily data AND (2) without compromising the
I test what I trade and I trade what I test principle is to add a trading rule: Do not use stops on the entry date and see what the test shows.
Yes. Good Point!

RedRock
Roundtable Knight
Roundtable Knight
Posts: 939
Joined: Fri Jan 30, 2004 3:54 pm
Location: Chicago

October JW Henry Trend Following Commentary. Not so good

Post by RedRock » Wed Nov 15, 2006 7:36 pm

October JW Henry Trend Following Commentary

http://www.jwh.com/Documents/marketcommentary.pdf :arrow:
Attachments
marketcommentary.pdf
(199.97 KiB) Downloaded 792 times

Post Reply