System blow up

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

System blow up

Post by LeviF » Mon Jul 26, 2010 11:01 am

Has anyone here experienced a system blow up? To what do you attribute the cause and what did you learn from the experience?

Timek
Roundtable Knight
Roundtable Knight
Posts: 104
Joined: Thu Sep 09, 2004 8:06 am
Location: London, UK

Post by Timek » Mon Jul 26, 2010 3:34 pm

A related interesting question would be, what makes you think that your system has "blown up"? Has it left without enough money to take the next trade, exceeded your personal max drawdown, or simply left you unable to trust it's trading style?

This is further complicated by trading multiple systems which, I think, is a preferable way to go.

Regards,
Timek

td80
Roundtable Fellow
Roundtable Fellow
Posts: 60
Joined: Thu Mar 08, 2007 8:38 pm
Location: Irvine, CA

Post by td80 » Wed Jul 28, 2010 7:20 pm

If you mean blow up as in violate my expectations, then yes. Looking back does it still "work", yes (new all time high simulated). What is the culprit? Some Curve fitting which led to unrealistic expectations (and statistics) which then led to big red lights and klaxons when shortly into production (~3 months) it started behaving erratically.

When I say curve fit I'm not talking about amateur night either. I'm talking about what happens when you haven't experienced a real, serious, liquidity crisis on a global scale and tested accordingly. Since we haven't really had one in modern market history besides a hint of it with LTCM, it bit me in the rear.

This is why I am adamant that you should plan for unknown unknowns by sticking to the basics and not optimizing risk (leverage) and other parameters.

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Thu Jul 29, 2010 1:40 am

td80 wrote: This is why I am adamant that you should plan for unknown unknowns by sticking to the basics and not optimizing risk (leverage) and other parameters.
The disclosure documents of long standing CTAs are littered with brief details of systems which "blew up".

One of my ultra favorite CTAs has run the same basic system for many years but reduced leverage after exceeding the max DD parameters and closing those public funds he ran at the time. The system recovered from this over leveraged DD (as witnessed by the CTA's continued propriety trading) but it was too late for investors. The 50%+ DD is still evident from the published track record of the manager/system.

Caution does seem to be the best option.

td80
Roundtable Fellow
Roundtable Fellow
Posts: 60
Joined: Thu Mar 08, 2007 8:38 pm
Location: Irvine, CA

Post by td80 » Thu Jul 29, 2010 3:30 pm

AFJ Garner wrote:Caution does seem to be the best option.
This is so true. It is like picking apples from a mutant, super elevated apple tree with a fixed ladder. How far out do you want to lean? Do you want to get out on that rather thin looking branch to get the most apples possible?

The problem is as traders we naturally want to optimize to get good results, but the lure is to optimize to get perfect results (without falling down the apple tree to our severe injury/death).

Somehow we need to find that happy medium between generating alpha and reaching too far. This is where some degree of "art" comes into this business in addition to manual executions and decisions about which systems to deploy for purely mechanical traders.

If I just asked a computer what to do it would of course choose the most "optimal" path which could be at the edge safety in the tree! This is why I argue with some people here who claim MAR is fairly irrelevant and the only thing that matters to them is CAGR (or Total Return) when evaluating a system.

Post Reply