By What Measure? - How do You Know if a System is Good?

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
TK
Roundtable Knight
Roundtable Knight
Posts: 167
Joined: Tue Apr 15, 2003 5:45 pm

Post by TK »

Two well-established (track records of ca. 9-10 years and more than $1 billion under management) and system-based CTAs that can boast what I consider exceptionally good and consistent performance are Quadriga and Transtrend.

If you look for consistent returns of about 20% p.a. and MAR above 1 then have a look at Quadriga AG (closed fund):

http://www.superfund.de/en/popupFacts.asp?mask=1

If you prefer consistent returns of about 20% p.a. and MAR above 2 then consider Transtrend-Diversified Program, Enhanced Risk Profile:

http://www.iasg.com/SnapshotPT.asp?ID=302

If you go for absolute returns then check the results of Quadriga Superfund Cayman (closed fund):

http://www.superfund.de/en/popupFacts.asp?Mask=256
http://www.iasg.com/SnapshotPT.asp?ID=420

This fund has only more than 3 years of history and is volatile but the CAGR of 54% and MaxDD of 35.40% are impressive and please note that this is after extremely high incentive fees of 35%.

Check their websites for insights into their methodologies. They are both trend-followers, although Transtrend is not a LTTF fund. They describe their style as Global Diversified Medium Term Trendfollowing.

And guess what–both companies are European :o.
SL
Senior Member
Senior Member
Posts: 30
Joined: Tue Feb 10, 2004 7:02 pm
Location: New Zealand

Post by SL »

I think everyone else has had a hack at this topic so perhaps I should through my 2 cents in as well. I am not speaking as a CTA or about CTA's necessarily just about system performance which is where we started with this I think (way back someplace).

From my observations obtaining a MAR ratio above 2 on a LTTF system for an extended period of time could be a challange for a number of reasons.

The MAR ratio is made up (I believe) of two main components.

1.The CAGR
2.MAX DD.

When these two parameters diverge the MAR increases (CAGR goes up and DD goes down). When they converge its bad news.

My observations here assume a constant CAGR of greater than 70% on balance and this should remain fairly static for the entire period under test.

Because CAGR has a time component in it, this plays into the reading you will get. The shorter the period of obeservations the less likely one is to hit a major problem in testing. Also if the trading account remains flat the shorter the time period under observation the better the MAR. On the flip side if the account does nothing for a while time will eat away at the MAR.

From my testing on relatively small sums of money when the account starts out the DD's are fairly low becuase the sums traded are low. If the system has a fairly high return this will play into the MAR as will the short time period calculated into the MAR and MAR figures above 3 are quite common. As the average level of DD rises upwards as the cash traded increases the MAR steadily erodes backwards and if the system is any good finds a happy medium until....

However since MAR looks at the MAX DD, sooner or later its going to hit a rock in the road, that will pull the overall MAR down. If you test over 20 years or longer that day will come. If one were to take the average DD figure instead of the MAX DD then the situation would be quite different because the big DD spike(s) would get swamped by a much larger volume of relatively lower DD figures. (How to apply makeup to the MAR)

If the CAGR is above say 70% and the MAX DD's are under 40% you should get a MAR of >2 . The problem is keeping the MAXDD below 40% with these levels of returns is difficult over 20 years (from my observations).

From my backtesting of a portfolio of around 30 futures markets over 20 years based on a LTTF system it is highly likely you will hit a spike that will deflate the MAR to below 2, a single event will do the job, thats all it takes.

If you test from 1994 onwards you may find the markets more favourable as someone mentioned. Of course it depends on what markets are traded and how many, but on balance things don't look to bad, which works against what people say about the 80's. One possible reason if you are seeing this, is that most markets currently trading or being tested over this period have all been on line for this period of time. Prior to this period some markets didn't exist. A lack of tradeble markets will not help a LTTF system at all, if the bet stratergy is fairly aggressive.

The other possible reason is that the system has to start someplace and slowly grab opportunities as the account grows. If the system takes a flying start and trades if the signal is present you will start with more markets traded and hopefully more uncorrelated markets which will help smooth volitility and improve the overall system perfomance. So again this might explain why taking observations since 1994 might look more favourable in some cases. Lastly there is a look back period where the system starts but nothing happens because you need so much data to feed into the system before the moving averages or breakouts etc. can come online, that in itself will erode MAR hanging around doing nothing for a year perhaps. These are more test related issues than CTA type issues.

So in conclusion the two components I see that seem to work against MAR are:

1. MAX DD
2. Time.

So to fudge figures, pick a short period of time with no nasty MAX DD spikes and you have a system that could easily beat a MAR of 2.

Merry xmas and best wishes for the coming year :)

Stephen
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

Thank you TK for those most interesting references. While I suspect over a 20 year view things will look different it is certainly most interesting to see such a relatively good performace over the past couple of years.
Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt »

PruBear wrote:Using VT to test, I have had a hard time getting my MAR ratio near 2.0 on any single system that has decent returns. To achieve a MAR ratio > 2.0, you're going to have to find a way of really limiting your max DD. I haven't been able to design any stand-alone system that can cut max DD w/o dramatically lowering returns.
One of the interesting things about MAR is that it runs in cycles.

When you are in a drawdown, or have just exited a fairly flat period for trend following, the average returns for all systems go down. This affects the MAR ratio for all systems, since CAGR% is the numerator for the equation.

Conversely, when you have just had a great run, your average returns will be higher. This too affects the MAR for systems.

So in a drawdown, you might see MAR values drop 0.2 to 0.4 from their peak at the top of the equity curve.

Given recent market performance, individual systems in the 1.6 to 1.7 range are difficult to surpass for long-term trend following diversified baskets over 20 years or more of data. At the time I started this thread in April 2003, the markets were still in the midst of pretty good years for trend followers. MAR ratios above 2.0 were not that difficult to achieve.

Also, some of the strategies that exhibited excellent MAR ratios, especially channel breakout strategies, have recently had their worst historical drawdowns. This too affects the potential MAR ratios that one can achieve since these systems now have to contend with a much higher historical drawdown.

- Forum Mgmnt
Chelonia
Roundtable Knight
Roundtable Knight
Posts: 525
Joined: Mon Apr 30, 2007 3:37 pm

Re:

Post by Chelonia »

Tim Arnold wrote: Wed Nov 03, 2004 2:25 pm Hi George --

MAR stands for Minimum Acceptable Return.

As for how it's calculated, there are a couple of threads that are useful.

http://www.tradingblox.com/forum/viewtopic.php?t=36

Basically the % CAGR divided by the % maximum drawdown over your test period. Nice clean measure of reward vs. pain.

Hope that helps,

Tim
Going through (very) old posts and found this. MAR does not stand for Minimum Acceptable Return :-)

MAR ratio is a measurement of returns adjusted for risk that can be used to compare the performance of commodity trading advisors, hedge funds, and trading strategies. The MAR ratio is calculated by dividing the compound annual growth rate (CAGR) of a fund or strategy since its inception by its most significant drawdown. The higher the ratio, the better the risk-adjusted returns. The MAR ratio gets its name from the Managed Accounts Report newsletter, introduced in 1978 by Leon Rose (1925-2013), a publisher of various financial newsletters who developed this metric.
Post Reply