Musings on (worthless) Past Performance

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Chuck B
Roundtable Knight
Roundtable Knight
Posts: 469
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B » Mon Feb 06, 2012 9:21 am

AFJ Garner wrote: Trading is a dangerous game and none of us are immune to its perils. My accountant told me recently of a copper trader who built up his capital to £160m and lost 40% last year.
Reminds me of the story the late Ray Kelly told me years ago about an options trader on the CBOE he knew from the beginning who had never had a losing month but in the space of two days in Oct 87 lost more than he had ever made in the prior 10+ years...taking the exact signals from his system he had always traded. Obviously not a trend follower nor a prudent trader since he clearly was short open-ended exposure I assume in order to lose so much so fast, but try telling that to almost anyone including him in Sept 1987 (sort of like LTCM I suppose).

BuyHigh SellLow
Roundtable Fellow
Roundtable Fellow
Posts: 50
Joined: Wed Apr 27, 2011 12:46 pm
Location: U.S.

Post by BuyHigh SellLow » Mon Feb 06, 2012 11:30 am

AFJ Garner wrote:
marriot wrote:No complain Anthony,
you know i love you : )

> Your biggest draw down is always in the future.

Sluggo does not agree with you
Maybe not but that does not influence me or change my mind. Take a look at the new lows reached by Tactical and Chesapeake. As for a low starting capital, take a look at Sluggo's oft posited and interesting idea to bet large to begin with (because you have to with limited capital) and reducing leverage as equity increases.

Trading is a dangerous game and none of us are immune to its perils. My accountant told me recently of a copper trader who built up his capital to £160m and lost 40% last year.
When you see the Dunns or JWHs or Chesapeakes or whoever hitting new maxDD's, does it make you (personally) more or less interested in investing with them?

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Mon Feb 06, 2012 11:40 am

The subject of this thread is the often fruitless perusal of past performance figures, hypothetical or otherwise. New maximum DDs in either sphere do not inspire me with confidence.

DPH
Roundtable Knight
Roundtable Knight
Posts: 113
Joined: Tue Nov 17, 2009 1:07 am
Location: Central Pennsylvania
Contact:

Post by DPH » Mon Feb 06, 2012 11:51 am

rabidric wrote:
DPH wrote:Ahh yes, the beautiful art that can painted with the perspective of past performance…..Makes one wonder what the likes of a Picasso, Van Gogh or Dali could have created with such tools~~~ For those feeling enchanted by these museum worthy pieces START HERE ……..If it’s too late and you’ve already fallen to their irresistibly hypnotic spell GO HERE

Consider this blurb I wrote 9 years ago as a system vendor (prior to managing money as a CTA) : Trading Systems That Work
whilst ravijm's graph may display a rate of return that would be unsustainable beyond a certain moderate capacity, I do believe he actually trades it for real( maybe 2008/9 is the "since live" bit). Futhermore, while the graph may show a run using a specific static set of rules on past data, that may not remain stable into the future; it does not mean that it is not possible to achieve that kind of graph in live trading if you evolve your system(s) as required.

It would seem to me that an easy trap to fall into, that catches many here, is that because it is difficult to achieve something very good within their paradigm, it must therefore be impossible under all paradigms. Far easier to trot out the usual received wisdom and sneer. :roll:
OHHHHHHHHHH PUUULEZE, are you serious? LOL........I'm sorry, but I personally draw the line here.

There is nothing wrong with thinking outside of one’s current paradigm, but please, one should also have some reasonable basis for their opinions (else they look like (and likely are) Pollyanna idiots)…

Just because I think you are all small thinkers stuck in your limited paradigms of what is possible does not mean that if I jump out of a window that I am going to be able to fly just because I “think it soâ€

rabidric
Roundtable Knight
Roundtable Knight
Posts: 243
Joined: Mon Jan 09, 2006 7:45 am

Post by rabidric » Mon Feb 06, 2012 12:03 pm

your paradigm seems to be LTTF.
this is the trading blox forum. not the "LTTF or STFU!" forum.
So kindly don't be an ass to me for endorsing another trader's efforts.

just because lttf yields so-so performance figures , which incidentally i do not dispute with you, doesn't mean that people can't do much better with other trading paradigms.
And no I am not talking about some kind of option-writing-esque payoff synthesis with massive hidden downside exposure.

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Mon Feb 06, 2012 12:04 pm

:lol: Audible chortle: back testing can fuel many a dream.

DPH
Roundtable Knight
Roundtable Knight
Posts: 113
Joined: Tue Nov 17, 2009 1:07 am
Location: Central Pennsylvania
Contact:

Post by DPH » Mon Feb 06, 2012 12:38 pm

rabidric wrote:your paradigm seems to be LTTF.
this is the trading blox forum. not the "LTTF or STFU!" forum.
So kindly don't be an ass to me for endorsing another trader's efforts.

just because lttf yields so-so performance figures , which incidentally i do not dispute with you, doesn't mean that people can't do much better with other trading paradigms.
And no I am not talking about some kind of option-writing-esque payoff synthesis with massive hidden downside exposure.
I apologize if I came off as an ass, it was not intentional......Maybe it stems from the fact that I spoke to a retired woman last week who was lured into investing most of her life savings based in large part on a bogus hypothetical track record (she lost about 500k out of her 600k nest egg) .

I think it is possible that what we take for granted as being "silly" hypothetical's can be taken quite literally by those without the experience we have.

For this reason I suppose I have a bit of "hot button" that gets pushed when I see this kind of thing.

However, I agree this is a forum where everybody's ideas should flow freely, so please accept my apologies if my words were not in that spirit...

DPH
Roundtable Knight
Roundtable Knight
Posts: 113
Joined: Tue Nov 17, 2009 1:07 am
Location: Central Pennsylvania
Contact:

Post by DPH » Mon Feb 06, 2012 5:15 pm

For what it's worth, here is a link to the (sort-able) 10 year performances of all CTAs reporting to Barclay Hedge who also have a minimum of 10 years of performance history. I also used only USA based CTAs here for the purposes of speed and uniformity (I am suspicious of some non-USA based CTAs that don't answer to the same regulatory regime, DESPITE MF Global!)

As you will see the highest MAR ratio was a 2.01 with the average being a 0.40

CTA Performance

My point is to show the great disparity between what can be achieved in backtesting verses what those "backtestors" were able to achieve in real life...

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Mon Feb 06, 2012 6:37 pm

Re: " your biggest drawdown is always in the future "

Here is a fragment from a post on this Forum in 2006; if you're really interested, search for it:
The key assumption behind this oft-repeated falsehood is: you will live and trade forever.

Let us consider Mr. Warren Buffet of Omaha, Nebraska, USA. Sometime in the next fifty years, he will die. After he's gone, we will plot the equity curve of his lifetime of trading. And we'll see that his largest drawdown was in the middle of his career. Not at the end. For every single day of the rest of his life, after that drawdown, Warren Buffet could proudly proclaim: MY Largest drawdown is always in the past!

stopsareforwimps
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Sun Oct 10, 2010 1:47 am
Location: Melbourne Australia

Post by stopsareforwimps » Tue Feb 07, 2012 2:21 am

sluggo wrote:Re: " your biggest drawdown is always in the future "

Here is a fragment from a post on this Forum in 2006; if you're really interested, search for it:
The key assumption behind this oft-repeated falsehood is: you will live and trade forever.

Let us consider Mr. Warren Buffet of Omaha, Nebraska, USA. Sometime in the next fifty years, he will die. After he's gone, we will plot the equity curve of his lifetime of trading. And we'll see that his largest drawdown was in the middle of his career....
I only have good data on BRKA back to 1990 which I have discounted using CPI to make them 'real'. From 1990 his longest draw down was about 1400 trading days long, from mid 1998. He is currently in a draw down that started in October '07 and currently running at 1047 trading days. So he could beat his record yet. He will need gains of about 33% from here to get out of his draw down.

Unless he is up 33% in the next two years, or dies first, he will then be in his biggest draw down since at least 1990.

I also have annualized figures for BRKA performance back to 1965, which suggest his longest draw down before 1990 was about 4 years (1973: -11.3%, 1974: -43.7%, 1975: -5%, 1976 +147.3%). So the 1998-2004 draw down appears to be the longest so far.

As Solon said to Croesus (allegedly), you cannot truly assess a man's life until it is over.

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Tue Feb 07, 2012 12:33 pm

DPH wrote: As you will see the highest MAR ratio was a 2.01 with the average being a 0.40
So, the average MAR ratio over the long term for these guys is 0.4 (Return / DD).

Thus for a 10% return over the long term they have suffered on average a max 25% DD.

A 15% return implies a max DD of 37.5% on this basis.

A 20% return implies a max DD of 50%.

Sobering. And that of course takes no account of the survivorship bias.

DPH
Roundtable Knight
Roundtable Knight
Posts: 113
Joined: Tue Nov 17, 2009 1:07 am
Location: Central Pennsylvania
Contact:

Post by DPH » Tue Feb 07, 2012 2:15 pm

DPH wrote: Sobering. And that of course takes no account of the survivorship bias.


Correct...........I will post the figures WITHOUT survivor-ship bias soon (Barclay Hedge sells the "no longer reporting" data too)...

Keep in mind this not only includes programs that blew up and are are out of business, BUT it also includes many current CTAs who have discontinued programs (including me).

Also, since reporting is voluntary many CTAs choose to report only on those programs that have done well, further skewing the averages and indices. Think about it, it you were a small CTA who had done lousy the first few years would you voluntarily report those results?

Yep, pretty much a statistical minefield out there...You need some pretty tall bullshit boots...

BuyHigh SellLow
Roundtable Fellow
Roundtable Fellow
Posts: 50
Joined: Wed Apr 27, 2011 12:46 pm
Location: U.S.

Post by BuyHigh SellLow » Tue Feb 07, 2012 3:04 pm

DPH wrote:
DPH wrote: Sobering. And that of course takes no account of the survivorship bias.


Correct...........I will post the figures WITHOUT survivor-ship bias soon (Barclay Hedge sells the "no longer reporting" data too)...

Keep in mind this not only includes programs that blew up and are are out of business, BUT it also includes many current CTAs who have discontinued programs (including me).

Also, since reporting is voluntary many CTAs choose to report only on those programs that have done well, further skewing the averages and indices. Think about it, it you were a small CTA who had done lousy the first few years would you voluntarily report those results?

Yep, pretty much a statistical minefield out there...You need some pretty tall bullshit boots...
luckily for them... stocks are worse, and yields are in the toilet.

rabidric
Roundtable Knight
Roundtable Knight
Posts: 243
Joined: Mon Jan 09, 2006 7:45 am

Post by rabidric » Wed Feb 08, 2012 5:10 am

BuyHigh SellLow wrote:
luckily for them... stocks are worse, and yields are in the toilet.
yields, stock market returns and alternative asset management returns are all probably a lot more correlated than most investors give credit for.

If the whole economic system is somehwat stagnant, then the increasing system entropy is difficult for anyone to circumvent forever. Sure, you can do better than the majority, but compared to one's own performance in the period when the system entropy was held in check(sustained global boom times) everyone is sure to do worse.

I think this explains some part of the deterioraiting performance stats of the entire set of asset managers to some extent, when looking at the last decade or two.

you might see a less bleak picture if you were doing the same thing in 1960, looking back 30 years. or in 1999 looking back 30 years. A rising tide lifts all boats, and a falling one leaves many beached etc

Robwynge
Senior Member
Senior Member
Posts: 42
Joined: Wed May 25, 2011 10:33 pm

Re: Musings on (worthless) Past Performance

Post by Robwynge » Wed Feb 08, 2012 8:41 am

DPH wrote:
I used a VERY simple strategy of buying the top 5 CTAs in 2007 as measured by their Sharpe Ratio.

I then compared that to my results five years later at the end of 2011 to see how I did. (below)..

In a nutshell, it failed miserably. Performance fell apart, the out of sample Sharpe Ratios fell an average of 70%. Returns fell by roughly 60% and maximum drawdowns increased by roughly 30%.

I had hoped that maybe at least their peer group rankings remained somewhat constant (I.E. everybody fell apart equally) but nope, their peer group rankings went out the window.
I wonder if it would be fair to look at this with a somewhat longer lens. Ok, past performance among a group that was "the best," on at least some measure, is not predictive of results over the intermediate term (5 years in this case). But I wonder if we had the ability to look over the longer term, say the next 10 years, whether the very fact that these CTA's were able to achieve top tier status is anyway predictive of longer run results.

After all, think of the CTA who NEVER make top tier, but always achieve middle-of-pack (or worse) results. I wonder if we looked out 10 years (or pick your definition of "long term"), if some performance distinction would emerge between the class of CTAs who achieved top tier status for at least some period as compared to those who never did? If not, then results would be truly random and I guess we're back to perfectly (almost) efficient markets. If so, we might as well treat our futures trading journey as one long trip to Las Vegas, without all the tacky glitz.

tetranomad22435
Contributing Member
Contributing Member
Posts: 6
Joined: Thu Jan 06, 2011 8:39 am
Location: Brisbane, Australia

Post by tetranomad22435 » Wed Feb 08, 2012 8:23 pm

Chris67 wrote:was I the only one who used to really fancy Jamie Summers - the bionic woman !!! she was cute
Hmm, wasn't sure whether I remembered the bionic woman, so I googled "Jamie Summers" and received some rather NSFW hits. :shock: Appears the bionic woman was Jaime Sommers...

rajivm
Roundtable Knight
Roundtable Knight
Posts: 129
Joined: Fri Jul 25, 2003 12:50 am
Location: New Delhi, India

Post by rajivm » Fri Feb 10, 2012 12:18 pm

DPH wrote:Ahh yes, the beautiful art that can painted with the perspective of past performance…..Makes one wonder what the likes of a Picasso, Van Gogh or Dali could have created with such tools~~~ For those feeling enchanted by these museum worthy pieces START HERE ……..If it’s too late and you’ve already fallen to their irresistibly hypnotic spell GO HERE

Consider this blurb I wrote 9 years ago as a system vendor (prior to managing money as a CTA) : Trading Systems That Work
The responses to my posting the results of short term system reminds me of this famous story..
http://en.wikipedia.org/wiki/The_Country_of_the_Blind
Anyway, This is a system I use...And never had a negative year..so I am happy...
I just feel I wasted lot of my trying to find LTTF systems that could generate cash flow.
I hope new people here who want to make a living as traders, understand the fact that for generating cash flow out of markets it is important to go short term
companies report earnings every quarter..So if you are in business of trading..why should you not report earnings every quarter..::))
---

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Fri Feb 10, 2012 12:36 pm

So tell us about it...

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Fri Feb 10, 2012 12:43 pm

rajivm wrote: Anyway, This is a system I use...And never had a negative year..so I am happy...

Let us be entirely clear: you posted above a simulation created using Trading Blox for a system back test which commenced in 2003 and since that date has recorded a theoretical CAGR of almost 300% with a theoretical maximum drawdown of under just 25%. What you did not do was to post your actual results in trading this actual system from that actual date.

So, since 2003 have you continuously traded this identical system and have you achieved an CAGR of 300% with a maximum peak to valley drawdown of just under 25%? If you have not , then with respect your reference to the Country of the Blind is as irrelevant as your back test is misleading.

If by contrast you have, in real trading, achieved this outstanding performance using a significant sum of real money then I congratulate you in all sincerity and wish with equal sincerity that I had your very considerable trading skills. You are my dear sir, a true Wizard.

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Fri Feb 10, 2012 1:53 pm

Well, Ray is trading stocks with some kind of fundamental filter.
This sound very different from what we usually discuss here.
Then he said: "How can you work 3000 trades every year?
I try to keep down numbers of trades at a max of 500 per year."
So, he his using Stop and Limist orders.
The simulation he has show us is not different from the one i have put on here.
This go back some years, but this does'nt not mean that i have trade it from the starting point.
Overall i have to say that his simulation is stimulant.

Post Reply