Flat equity curve in recent history...

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
LeviF
Roundtable Knight
Roundtable Knight
Posts: 1430
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Flat equity curve in recent history...

Post by LeviF » Sun Mar 09, 2008 2:56 pm

I put together a system that has a relatively good MAR & CAGR, but the equity curve for the last 4 years has essentially been flat. What conclusions would a person draw from that fact? On one hand, I would like to develop something that shows a more upward trending line for recent history, on the other hand, it may be a good time to trade it because it may be due for some good performance?

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Sun Mar 09, 2008 3:42 pm

Without further details of your system, meaningful comment is going to be rather difficult. But it certainly does not sound very encouraging.

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1430
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Sun Mar 09, 2008 5:59 pm

It seems that most of the systems I put together have the worst equity curves and drawdowns in the last few years of the system. Would this be due to changing markets?

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Mon Mar 10, 2008 2:45 am

Using a simple 20/10 channel breakout system on a large and widely diversified basket of futures (IE Turtle without the frills), the performance is singularly unimpressive from the 90’s on. Now add a dual moving average filter of 375/50 so that the system takes only trades in tune with the much longer term trend. Even this filter is unable to make the system look impressive for this decade. Volumes in the futures markets have greatly increased throughout this period and some would say that the resulting noise has made it difficult to follow shorter term trends. Down to each of us to draw our own conclusions as to what has happened and how things will progress from here.

Turtle40
Roundtable Knight
Roundtable Knight
Posts: 201
Joined: Wed Oct 19, 2005 1:53 pm
Location: Guernsey, Channel Islands

Post by Turtle40 » Mon Mar 10, 2008 4:24 am

You may have to review your testing procedure. Did you test in a way that has over-optimised the parameters to the data?

Have a read of Bob Pardo's books on testing and optimising trading strategies. He suggests using a portion of data for what he calls "Walk-Forward Analysis" of WFA. He cautions that a robust system should exhibit a walk-forward efficiency of 50-60% or better when tested using out of sample data. In other words if you have 10 years of data, use 6 or 7 years to design the system and then test it on the remaining 2 or 3 years to see how it behaves.

It is also good practise to have a large enough number of trades to be statistically valid, such that very long term systems which generate infrequent trades may need a larger testing window. He also makes the case for removing any exceptional (large) winning trades that occur infrequently to make the simulation more realistic

This prevents over-fitting of the parameters and provides a better indication of how a system may perform in real life. He also recommends testing data which includes the many different market conditions he describes such as bull, bear, cycling and congested.

Personally I would be a bit wary of a flattening equity curve as it suggests the system is not robust enough.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Mon Mar 10, 2008 7:33 am

One popular but very dangerous method of overfitting a system is to cherry-pick the "best" markets, the ones that give the largest backtest results. Doing so can improve the visual qualities of the equity curve quite remarkably, and can be used to derive just about any fatuous conclusion you wish, such as "the markets have changed" or "the markets have NOT changed" or "trendfollowing is dead" or "trendfollowing is alive and well".

For example, just now I ran the stripped down dual channel breakout system mentioned by AFJ Garner, on a universe of 86 highly liquid futures markets. The resulting equity curve might suggest that this system with these markets, is a lot less profitable recently than back in the good old days pre-1998.

But if you perform the vile and sinful experiment of only running the "best" half of the markets, or (even naughtier) the "best" 25% of the markets, the resulting equity curves look quite different and may lead to quite different conclusions about pre-1998 and post-1998 performance.

Personally, I prefer to test and trade with large portfolios such as the 86 markets in the first test. So I would simply conclude: "This dual channel breakout system with parameters=(20,10) looks unappealing on the 86 market portfolio. Therefore I will look elsewhere. I will continue to use this large portfolio, but try other systems and/or other parameter values, to see whether I can find a more appealing equity curve."
Attachments
topquarter.png
topquarter.png (25.26 KiB) Viewed 7369 times
tophalf.png
tophalf.png (24.83 KiB) Viewed 7367 times
everyone.png
everyone.png (25.98 KiB) Viewed 7369 times

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1430
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Mon Mar 10, 2008 8:05 am

That top curve is basically what all my runs, no matter the system or parameters, look like. What have developers been doing to improve their performance over more recent history?

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Mon Mar 10, 2008 8:35 am

levijean wrote:What have developers been doing to improve their performance over more recent history?
I think Sluggo has given you the answer:

"try other systems and/or other parameter values, to see whether [you] can find a more appealing equity curve."

Have you tried the simple CBO? Take a look to see what happens if you alter the parameters on a portfolio of 86 futures. You may be pleasantly surprised.

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1430
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Mon Mar 10, 2008 10:12 am

I haven't run anything on 86 futures. How much capital does one need for a LTTF system with 86 futures?

LeapFrog
Roundtable Knight
Roundtable Knight
Posts: 695
Joined: Mon May 17, 2004 4:18 pm
Location: Boston, MA

Post by LeapFrog » Mon Mar 10, 2008 10:59 am

I too wonder how you test on 86 markets over 30 years? Doesn't CSI cap you out at around 69 markets and up to 25 years of history?

I sent an email to CSI a week ago enquiring (as I do annually) about their offerings, but no response.
Last edited by LeapFrog on Mon Mar 10, 2008 11:03 am, edited 1 time in total.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Mon Mar 10, 2008 11:01 am

levijean wrote:How much capital does one need for a LTTF system with 86 futures?
Another question is, "What simulations and tests and computer runs might I carry out, to help me figure out how much capital I would need for an LTTF system with 86 futures?"

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1430
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Mon Mar 10, 2008 11:21 am

sluggo wrote:
levijean wrote:How much capital does one need for a LTTF system with 86 futures?
Another question is, "What simulations and tests and computer runs might I carry out, to help me figure out how much capital I would need for an LTTF system with 86 futures?"
I know its a lot. I'm trying to develop something that i can trade with my available risk capital. Not what I theoretically could trade if I had a million dollars.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Mon Mar 10, 2008 12:25 pm

viewtopic.php?p=28483&highlight=#28483

I've seen it done. Seems to work. A trader friend 50 miles up the road from me is using it to trade 90 markets in a sub-200K account, and seems quite pleased with the results. There is a vendor-sold system called "Relativity" that uses similar ideas, and if I recall correctly it is claimed to be applicable to sub-100K accounts trading 90 markets. (Caveat emptor). Haven't tested Relativity myself or met anyone who has.

However, their appetite for risk may be greater than, or less than, yours.

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1430
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Mon Mar 10, 2008 12:32 pm

What broker(s) does one even use to have availability to so many markets?

DMFord
Roundtable Fellow
Roundtable Fellow
Posts: 88
Joined: Wed Jun 27, 2007 8:11 am
Location: NB, Canada

More than 69

Post by DMFord » Wed Mar 12, 2008 8:52 am

LeapFrog, I just upgraded my CSI stream from 69 to 100 markets.

You just pay 1$ per market per month extra on your subscription.
See their pricing page.

You might have trouble over 100 - they said this was the limit for private investors & you'd have to go onto the corporate rates.

Dave

Post Reply