New goodness function

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
LeviF
Roundtable Knight
Roundtable Knight
Posts: 1436
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

New goodness function

Post by LeviF »

I often find myself putting too much weight on the MAR and worst drawdown statistics when eyeballing test results. I don't like the fact these are so sensitive to worst DD - a single point on the equity curve.

Yes, we now have R-cubed, but I think this is overly sensitive to drawdown length, which I think is already implicitly accounted for in RAR. For a while, I had been using RAR / avg 5 max DD, which i like better, but it is still sensitive to the max DD. Often, my test results have 4 nearly equal DDs and one relatively larger DD. I think the largest DD is something that needs to be prepared for, and not try to fool ourselves by optimizing it away. Therefore, I have come up with this new function called the Levi Ratio:

Code: Select all

L Ratio = test.rar / ((test.averagemaxdrawdown * 5 - test.maxopendrawdown) / 4)
This function removes the max drawdown entirely. And I like that.
marriot
Roundtable Knight
Roundtable Knight
Posts: 365
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot »

> I think the largest DD is something that needs to be prepared for, and not try to fool ourselves by optimizing it away....

Would you like to tell me more on this?
Are you saying that worst DD is not given by % risk ?
thank you
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

Isn't it fun that over a hundred goodness functions have been invented and disclosed, yet people still keep inventing new ones?

"What is good?" (more precisely, "what is good in my opinion?") seems to have many different correct answers.
marriot
Roundtable Knight
Roundtable Knight
Posts: 365
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot »

Levi, Sluggo,
are you saying that the old K.I.S.S methods are died?
Chris, would you tell us something about that?
Roger Rines
Roundtable Knight
Roundtable Knight
Posts: 2038
Joined: Wed Oct 06, 2004 10:52 am
Location: San Marcos, CA

Post by Roger Rines »

sluggo wrote:"What is good?" (more precisely, "what is good in my opinion?") seems to have many different correct answers.
This is really hitting the "nail" on its proverbial head.

If there is anything funny about how the pursuit seems to chase a new algorithm, some of the humor must be in how little attention is paid to whether any single end of test number is stable, or even a likely representative of how that test central tendency had performed during the test ending and during various market conditions, period lengths, and, or market mixes.

While we are falling in love with a number, or a result, it might be of some value to consider the likelihood those results are going to be reasonably close when we need to depend upon them.
nodoodahs
Roundtable Knight
Roundtable Knight
Posts: 218
Joined: Wed Aug 09, 2006 4:01 pm

Post by nodoodahs »

I think a large part of the issue is that most practitioners find that “goodnessâ€
Paul King
Roundtable Knight
Roundtable Knight
Posts: 207
Joined: Mon Feb 23, 2004 9:13 am
Location: Vermont, USA
Contact:

KS

Post by Paul King »

Here's one I use:

If

R = net P&L / Initial Risk

System Value = (Average(R)/STDEV(losing R)) * Trades Per Year

Then the King Score (KS), a.k.a. Kitchen Sink:

KS = System Value * ((MC Confidence Return/MC Confidence DD)*10)/(MC Confidence DD Length /12))

Paul
td80
Roundtable Fellow
Roundtable Fellow
Posts: 60
Joined: Thu Mar 08, 2007 8:38 pm
Location: Irvine, CA

Post by td80 »

A wise observation:
sluggo wrote: "What is good?" (more precisely, "what is good in my opinion?") seems to have many different correct answers.
Is MAR the best? I'm not so sure. Best would be absolute return for most spectators. I say "Which would you rather have, $100K at the end of 5 years or $1,000,000?". A spectator will say "$1,000,000". Then I say OK go trade it and become a speculator, and watch most of them blow up on some crazy drawdown that lasts 3.5 years.

I would take it a step further and propose to you that "good" is going to translate into "trade-able for you". Can you stick to the plan. If you stick to high MAR's (and for sake of argument let's say this behavior continues and is not curve-fitted, etc), in my opinion you will find those systems "easier" to trade than measures that place less emphasis on MaxDD.

In big MAR systems, you are being compensated for however wild the drawdowns are (i.e. big drawdowns equal BIG annualized returns). Even then I would still look at that MaxDD number and assume it is going higher than your empirical or even live results suggest. Adjust risk/leverage accordingly...

All that said I view any single system with a MAR > 2 as highly suspect, and probably wouldn't trust anything with a MAR > 1 if it has a lot of trade logic or less than a couple thousand trades.
ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Post by ecritt »

For me it's compounded return relative to risk taken. Max drawdown is only risk realized. It is rarely the same as risk taken.
marriot
Roundtable Knight
Roundtable Knight
Posts: 365
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot »

Cagr, Max DD, Longest DD.
These are The numbers, mixing them can't change results.
Max DD is the only number that can really kill the account.
Never forget Forrest Gump.


*******************
Bear with me, I'm still learning.
DeanoT
Roundtable Fellow
Roundtable Fellow
Posts: 52
Joined: Sat Apr 15, 2006 7:37 am
Location: Melbourne, Australia

Post by DeanoT »

I find the attached analysis from David Harding of Winton Capital to be a good summary of the limitations of using drawdown as a statistical measure of the quality of an investment or trading system.
Attachments
Pros and Cons of Drawdown as a Statistical Measure.pdf
(40.07 KiB) Downloaded 444 times
marriot
Roundtable Knight
Roundtable Knight
Posts: 365
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot »

What is the name of the goodness function that give more points at "MAR when DD length" is shorter ?
Thank you
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

The word "length" appears 15 times in the Trading Blox User's Guide. Three of those 15 appearances are in the description of the goodness function you seek.
marriot
Roundtable Knight
Roundtable Knight
Posts: 365
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot »

f***me, i search internet in 3 languages and all the forum, it seem is time to read again help guide.
Thank you
marriot
Roundtable Knight
Roundtable Knight
Posts: 365
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot »

And the guide is very well written.
Anyway, i cannot find a goodness function that combine MAR with longest DD.
Maybe i am looking in the wrong direction?
Post Reply