New goodness function

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

New goodness function

Post by LeviF » Fri Apr 09, 2010 9:25 am

I often find myself putting too much weight on the MAR and worst drawdown statistics when eyeballing test results. I don't like the fact these are so sensitive to worst DD - a single point on the equity curve.

Yes, we now have R-cubed, but I think this is overly sensitive to drawdown length, which I think is already implicitly accounted for in RAR. For a while, I had been using RAR / avg 5 max DD, which i like better, but it is still sensitive to the max DD. Often, my test results have 4 nearly equal DDs and one relatively larger DD. I think the largest DD is something that needs to be prepared for, and not try to fool ourselves by optimizing it away. Therefore, I have come up with this new function called the Levi Ratio:

Code: Select all

L Ratio = test.rar / ((test.averagemaxdrawdown * 5 - test.maxopendrawdown) / 4)
This function removes the max drawdown entirely. And I like that.

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Fri Apr 09, 2010 1:30 pm

> I think the largest DD is something that needs to be prepared for, and not try to fool ourselves by optimizing it away....

Would you like to tell me more on this?
Are you saying that worst DD is not given by % risk ?
thank you

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Fri Apr 09, 2010 2:18 pm

Isn't it fun that over a hundred goodness functions have been invented and disclosed, yet people still keep inventing new ones?

"What is good?" (more precisely, "what is good in my opinion?") seems to have many different correct answers.

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Fri Apr 09, 2010 2:38 pm

Levi, Sluggo,
are you saying that the old K.I.S.S methods are died?
Chris, would you tell us something about that?

Roger Rines
Roundtable Knight
Roundtable Knight
Posts: 1946
Joined: Wed Oct 06, 2004 10:52 am
Location: San Jose, CA

Post by Roger Rines » Fri Apr 09, 2010 2:52 pm

sluggo wrote:"What is good?" (more precisely, "what is good in my opinion?") seems to have many different correct answers.
This is really hitting the "nail" on its proverbial head.

If there is anything funny about how the pursuit seems to chase a new algorithm, some of the humor must be in how little attention is paid to whether any single end of test number is stable, or even a likely representative of how that test central tendency had performed during the test ending and during various market conditions, period lengths, and, or market mixes.

While we are falling in love with a number, or a result, it might be of some value to consider the likelihood those results are going to be reasonably close when we need to depend upon them.

nodoodahs
Roundtable Knight
Roundtable Knight
Posts: 218
Joined: Wed Aug 09, 2006 4:01 pm

Post by nodoodahs » Fri Apr 09, 2010 2:59 pm

I think a large part of the issue is that most practitioners find that “goodnessâ€

Paul King
Roundtable Knight
Roundtable Knight
Posts: 207
Joined: Mon Feb 23, 2004 9:13 am
Location: Vermont, USA
Contact:

KS

Post by Paul King » Fri Apr 09, 2010 4:02 pm

Here's one I use:

If

R = net P&L / Initial Risk

System Value = (Average(R)/STDEV(losing R)) * Trades Per Year

Then the King Score (KS), a.k.a. Kitchen Sink:

KS = System Value * ((MC Confidence Return/MC Confidence DD)*10)/(MC Confidence DD Length /12))

Paul

td80
Roundtable Fellow
Roundtable Fellow
Posts: 60
Joined: Thu Mar 08, 2007 8:38 pm
Location: Irvine, CA

Post by td80 » Fri Apr 09, 2010 7:03 pm

A wise observation:
sluggo wrote: "What is good?" (more precisely, "what is good in my opinion?") seems to have many different correct answers.
Is MAR the best? I'm not so sure. Best would be absolute return for most spectators. I say "Which would you rather have, $100K at the end of 5 years or $1,000,000?". A spectator will say "$1,000,000". Then I say OK go trade it and become a speculator, and watch most of them blow up on some crazy drawdown that lasts 3.5 years.

I would take it a step further and propose to you that "good" is going to translate into "trade-able for you". Can you stick to the plan. If you stick to high MAR's (and for sake of argument let's say this behavior continues and is not curve-fitted, etc), in my opinion you will find those systems "easier" to trade than measures that place less emphasis on MaxDD.

In big MAR systems, you are being compensated for however wild the drawdowns are (i.e. big drawdowns equal BIG annualized returns). Even then I would still look at that MaxDD number and assume it is going higher than your empirical or even live results suggest. Adjust risk/leverage accordingly...

All that said I view any single system with a MAR > 2 as highly suspect, and probably wouldn't trust anything with a MAR > 1 if it has a lot of trade logic or less than a couple thousand trades.

ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Post by ecritt » Sat Apr 10, 2010 12:42 am

For me it's compounded return relative to risk taken. Max drawdown is only risk realized. It is rarely the same as risk taken.

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Sat Apr 10, 2010 2:55 am

Cagr, Max DD, Longest DD.
These are The numbers, mixing them can't change results.
Max DD is the only number that can really kill the account.
Never forget Forrest Gump.


*******************
Bear with me, I'm still learning.

DeanoT
Roundtable Fellow
Roundtable Fellow
Posts: 52
Joined: Sat Apr 15, 2006 7:37 am
Location: Melbourne, Australia

Post by DeanoT » Sun Apr 11, 2010 4:04 am

I find the attached analysis from David Harding of Winton Capital to be a good summary of the limitations of using drawdown as a statistical measure of the quality of an investment or trading system.
Attachments
Pros and Cons of Drawdown as a Statistical Measure.pdf
(40.07 KiB) Downloaded 289 times

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Tue Apr 27, 2010 10:50 am

What is the name of the goodness function that give more points at "MAR when DD length" is shorter ?
Thank you

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Tue Apr 27, 2010 12:03 pm

The word "length" appears 15 times in the Trading Blox User's Guide. Three of those 15 appearances are in the description of the goodness function you seek.

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Tue Apr 27, 2010 12:15 pm

f***me, i search internet in 3 languages and all the forum, it seem is time to read again help guide.
Thank you

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Tue Apr 27, 2010 12:40 pm

And the guide is very well written.
Anyway, i cannot find a goodness function that combine MAR with longest DD.
Maybe i am looking in the wrong direction?

Post Reply