When To Stop Trading

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Griffy
Contributing Member
Contributing Member
Posts: 6
Joined: Sat Aug 12, 2006 2:21 pm
Location: California

When To Stop Trading

Post by Griffy » Sun Oct 01, 2006 2:04 am

I have tried searching through the forums for this particular topic but have failed to find any relevant information. If I missed something please let me know.

My questions is, when would someone stop trading a mechanical system? After Max DD has been breached? Max consecutive losses is greater than historically tested? I know its probably different for many traders but I just thought that it might be a good topic to go over.

Also, in order to avoid ruin, is it typcial for a LTTF trader to employ multiple systems? Say 3>? I would be interested if anyone who actively trades might share how many systems they feel comfortable trading.

I bring this up because I recently went back over the turtle rules and saw how it mentioned that many turtles stopped following the rules and/or changed the rules provided by Richard Dennis because they thought the markets had changed or something else was happening.

Anyway, all thoughts are much appreciated.

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Sun Oct 01, 2006 5:23 am

To my shame and regret, I have only just started playing around with random entries and Monte Carlo analysis.

Such testing may give you more robust answers than running simpler tests for any given system and timeframe.

BARLI
Roundtable Knight
Roundtable Knight
Posts: 650
Joined: Sat Jan 17, 2004 6:01 pm
Location: USA

Post by BARLI » Sun Oct 01, 2006 5:35 am

I look at biggest loss of my trading systems, also for every single market there's biggest dollar loss I can accept, once it's reached I'm out of a position. So going over 20 years of data I can see worst/best dollar losses/profits per trade per contract and hence I make a decision what should be done

TrendMonkey
Roundtable Knight
Roundtable Knight
Posts: 154
Joined: Fri Apr 22, 2005 9:14 pm
Location: Vancouver, Canada

Post by TrendMonkey » Mon Oct 02, 2006 1:35 am

Since virtually every word of trading wisdom implores one again and again to maintain discipline and stick with the system, this seems to me to be just about the most important question of all. Unfortunately I don't have a better answer than Barli's.

Dean Hoffman
Roundtable Fellow
Roundtable Fellow
Posts: 87
Joined: Wed Jul 02, 2003 3:04 pm

Post by Dean Hoffman » Tue Oct 03, 2006 4:14 am

I recently created a presentation that touches on this issue of when to stop trading, and how to try and predict maximum future drawdown. (The first part of the preseation talks about a typical investor blunder)

You can open it here:

http://www.traderstech.net/Flash50/Flash50.html

(this is a big file, it may need a few minutes to buffer before playing)

Dean Hoffman
Last edited by Dean Hoffman on Wed Oct 11, 2006 3:17 am, edited 2 times in total.

Old European
Roundtable Knight
Roundtable Knight
Posts: 144
Joined: Mon Apr 19, 2004 3:04 pm
Location: Old Europe

Post by Old European » Tue Oct 03, 2006 9:44 am

1. I think that Dean Hoffman's contribution to this thread is a very interesting one

2. I guess that c.f. and Tim are sufficiently open-minded and won't have any problem with a forum member praising a competing product

3. I'm personally also convinced that the Trading Blox Monte Carlo methodology is far superior, because it takes inter-market and auto-correlations into account

Old European

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Tue Oct 03, 2006 1:13 pm

The older I get the more I understand and try to practice patience and tolerance. Not always as successfully as I would like but it is vital to make the effort.

I have appreciated both Ted's posts and Dean's in the past. I do not feel that there is any danger whatsoever that this forum will go the way of Le Beau's site.

c.f. and Tim do an excellent job of censoring rude and innapropriate posts.

I feel that Ted made a valid point but can equally understand that Dean feels Mechanica does a better job for his personal circumstances.

So, tolerance and patience.

For me, TB is the perfect tool.

Tim Arnold
Site Admin
Site Admin
Posts: 8540
Joined: Tue Apr 06, 2004 1:41 pm
Location: Boston, MA
Contact:

Post by Tim Arnold » Tue Oct 03, 2006 1:23 pm

Thanks AFJ, well said.

So I think we have cleared the air with regard to the Mechanica portion of the presentation. There are many products out there and we encourage all to find the right tools for their particular journey.

Now we can get back to the question at hand...

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Tue Oct 03, 2006 2:51 pm

I thoroughly enjoyed Dean’s most useful presentation and thought I would raise a few points which occurred to me. No criticism or offence intended – I just wish to open up a discussion.

1. Monte Carlo simulation is not an appropriate tool for selecting when to enter a CTA run investment unless that CTA is willing to divulge his exact system and let the potential investor run MC simulations of the system itself. CTA performance is reported monthly and is not going to carry a great deal of statistical significance in an MC simulation.

2. I would not feel comfortable in relying on the results of an MC simulation for a non disclosed system. Especially when the test contains multiple systems. No disrespect intended to Dean.

3. In the testing I have done so far I have enjoyed concentrating on exits and using “almost randomâ€

Turtle40
Roundtable Knight
Roundtable Knight
Posts: 201
Joined: Wed Oct 19, 2005 1:53 pm
Location: Guernsey, Channel Islands

Post by Turtle40 » Tue Oct 03, 2006 5:53 pm

Dean,

I thoroughly enjoyed your presentation, and learnt some new things.

Thanks!

-Turtle40

SBG
Full Member
Full Member
Posts: 17
Joined: Thu Apr 13, 2006 11:39 am
Location: North Carolina

Post by SBG » Wed Oct 04, 2006 9:51 am

AFJ,

I had some of the same thoughts.

Would a CTA be willing to give up enough info regarding the system for one to run an effective analysis via MC?

However, it could prove useful to an individual trader trading multiple systems. Assuming the trader had all info regarding the systems he/she is trading (one would hope so) than the trader could use the MC analysis presented by Mr. Hoffman in the decision process of how to allocate funds across systems.

LeapFrog
Roundtable Knight
Roundtable Knight
Posts: 695
Joined: Mon May 17, 2004 4:18 pm
Location: Boston, MA

Trade outcomes versus equity percentage changes.

Post by LeapFrog » Thu Oct 05, 2006 12:14 pm

One of the several good points Dean makes is doing MC runs on percentage changes in equity (not the trade sequencing of outcomes of wins versus losses).

Not yet being a proud owner of TB (soon I hope) can Ted, Tim or any other user tell me if TB does this? Seems fairly important, no?

Thanks in advance.

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Fri Oct 06, 2006 7:28 pm

Yes, Trading Blox uses daily percentage changes in equity for the Monte Carlo analysis. This is important when looking at drawdowns because when the ^$%&$ hits the fan all the positions tend to go against you at once, even those that aren't normally correlated. Using daily equity preserves this nasty tendency of the markets as a group.

Further, as mentioned above, you can decide at what level of granularity you wish to do the resampling i.e. daily, 5 day chunks, etc. This allows you to preserve most of the autocorrelation of equity movement present in actual trading. This is also important when looking at drawdowns because when bad periods come, they are not simply one bad day but generally a period of mostly bad days, i.e. the real problem is that while equity curves are generally not autocorrelated they tend to be much more so after a prolonged series of trends (like we just had in Oil, Gold, Silver, Sugar, Copper, etc.). Mark Johnson suggested this technique and I find it very useful.

My testing indicates that these two approaches when combined give one a much more accurate picture of the likely drawdowns. I had not used Monte Carlo analysis much previously because it tended to seriously understate drawdown levels. The ability to preserve the autocorrelation in the resampling makes the Monte Carlo features in Trading Blox much more realistic, IMHO.

- Forum Mgmnt

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Sat Oct 07, 2006 8:18 pm

First, I want to thank Griffy for asking a question that really gets at the heart of the difficulty in developing, evaluating, and trading mechanical systems. Indeed, it is a very hard question to answer completely in anything less than a 100 or so pages of text.

Fortunately, while a complete answer is difficult, the short answer to the question is simple: NEVER

If you have a good basis for trading a system you will have developed one that relies on the continued existence of certain human traits that will not go away. If you have done proper testing over a sufficient amount of historical data, if you have not built a curve-fit system, you should continue to trade it. Sooner or later human traders will exhibit the characteristics upon which your system's profitability or "edge" is built, and you will return to profitable trading.

These are big ifs, especially if you are new to trading and to the development and testing of trading systems. It is not easy to fulfill the assumptions required by my simple answer.

So I strongly suggest that you become an expert at system development if you intend to trade so that you will know that you have a good system that will continue to work in the future.

There is no substitute, however, for actual experience. Which means that I suggest you start trading and if (more likely when) your trading suffers from a drawdown worse than you imagined, you can use that as an opportunity to hone your investigative skills to find out where your assumptions and/or expectations are flawed.

Some of the most likely candidates:
  • A Lack of Understanding of the Optimization Paradox
  • Testing Using Too Little Data
  • Testing Using Too Few Markets
  • A Naïve Faith that History Will Repeat Itself Exactly
  • Poor Testing Software and Methods
Dean Hoffman wrote:I recently created a presentation that touches on this issue of when to stop trading, and how to try and predict maximum future drawdown.
I finally had some time to review Dean's presentation, so I can now add my own opinion to those expressed here earlier.

First with regard to the presentation, I believe Dean makes some good points: Buying the hot manager is not a good idea; likewise switching out of a good manager just because of a drawdown is not a good idea. Yet as Jack Schwager pointed out this is precisely what many investors do when evaluating fund managers for alternative investments.

As a corollary to this, I assert that this is what most system traders do. They start trading what worked best in the most recent past. They run tests and trade those systems that have the best recent performance and they optimize to find parameter sets that worked best over the last several years. In this they often do themselves a disservice, as except for with very short-term systems, the recent past is not a good predictor of the near-term future.

I think Monte Carlo analysis is useful when done properly (this implies the ability to preserve autocorrelation AND intermarket correlation using the methods outlined above in my earlier post which I believe Mechanica cannot do  ). I find it useful because it helps make test results fuzzier.

This is a good thing.

All too often I think beginners especially treat test results like a precise analysis. They think one system is better than another because it has 1.8 MAR versus 1.85 for another. Reality just isn’t so. Historical tests are at best fuzzy estimates of likely future performance. Looking at Monte Carlo outputs helps you start to think of tests in that way.

Nevertheless, while I like Monte Carlo analysis, I find Dean’s approach to determining the likely maximum drawdown for a system to be overly facile. Monte Carlo is a useful tool but I do not believe that it offers anywhere near the level of predictive accuracy that Dean attributes to it.

A Monte Carlo simulation is simply a random reordering of a fixed set of data that represents the past. If the historical test does not contain a realistic representation of what one is likely to encounter in the future then a random reordering of that data will also be of dubious value. If the results used as the basis for the simulation come from a system that has been curve-fit and therefore overestimates the likely future performance of that system; any resulting Monte Carlo analysis will also understate the likely drawdown, perhaps severely so. As with many types of analysis, you have the Garbage In, Garbage Out problem.

AFJ Garner alludes to this issue in his post when he states that: "I would not feel comfortable in relying on the results of an MC simulation for a non disclosed system. Especially when the test contains multiple systems.â€

Dean Hoffman
Roundtable Fellow
Roundtable Fellow
Posts: 87
Joined: Wed Jul 02, 2003 3:04 pm

Post by Dean Hoffman » Sun Oct 08, 2006 4:58 am

Forum Mgmnt,

Quick couple comments..

I will agree that the idea at the end of my presentation about determining a possible account size was very aggressive. Furthermore, I made that rather clear. However, I think it is you who has crossed the line to the absurd to compare a thought out statistically derived aggressive strategy based on an a pre-defined and accepted level of risk to a seemingly stupid act of bravado from guys who were outright gambling at level's way beyond any predetermined accepted levels of risk (Amaranth). A simple Monte Carlo test on their part would have shown them how likely and inevitable their disaster was.

At some point you have to determine by what risk measures your going to determine your leverage. If you're going to leverage yourself based on the absolute worst case Monte Carlo simulation then your ONLY logical leverage choice would be zero!

Think about how most of us determine position size. If I have $100,000 and have decided I will risk 2% or $2000 then that allows me to determine my leverage. In other words, if a logical stop point (ATR or whatever) computes a $500 risk on a trade then 4 contracts are what I will trade (I would be under-trading to buy only one contract in this case). Therefore, my stop determines my leverage (read & remember that sentence again). This does not guarantee I have chosen the right stop. It is only saying that based on my research (tolerance, etc.) that I feel it is where my optimal (or acceptable) risk to reward edge lies.

To me it's very similar when determining leverage with a CTA. If I have enough funds to only trade one CTA and I project based on his previous performance or Monte Carlo simulations etc. that he will likely earn 30% and have drawdowns no greater than 20% then I can use position sizing logic to determine my trade (nominal account) size. In other words, assume I have decided I want to risk $50,000 on this advisor in the worst case. A simple calculation would be to say $50,000 is a 20% drawdown (my worst projection) of a $250,000 account. I would be rather disappointed if my projections turned out right and only traded at a $100,000 nominal level and only earned $30,000 instead of the $75,000 I could have earned by trading at the risk level I statistically derived made sense.

I understand this is very aggressive because maybe my Monte Carlo simulation is off. Maybe in spite of my predicted worse case 20% drawdown it goes to 30%, instead, etc. However, for the highly aggressive investor who can (or only wants) to invest in one CTA and DESIRES to do it aggressively (fully accepting the risks), then I know of no better way to determine the maximum nominal account size. If the advisors previous max DD was 10% and the most severe (properly done robust) Monte Carlo simulation predicts at a 99% confidence level no drawdown greater than 20% then I see nothing wrong with determining a nominal account size based on those computed probabilities (this all assumes you believe that Monte Carlo simulations have some validity. I do).

Once again, this is not conservative; I agree it’s highly aggressive (and made that clear). However, it is a bet based on smart due diligence in order to maximize returns. This is far different than a bet based on a Godlike complex of infallibility (IE Amaranth). If you wanted to tone it down a bit then simply add another 50% to the worst projected MC sim Drawdown and leverage yourself that way etc.

I agree most investors would be much better off approaching more conservatively. However, I do think it’s an interesting way to back into a maximum nominal account size if you’re trying to LOGICALLY figure out a good way to put the odds in your favor with an aggressive bet. (there are some that worry about under trading too).

On a separate note, I am staying away from the Trading Blox vs. Mechanica thing! That was NEVER my intention. I think they are both fantastic products and I am forever grateful to both Bob Spear and c.f. for making these available to us. I highly recommend them both and I have no vested interest in either of them. However, I will add one point; I think one should consider the comparison comments only from those who own both. I’ve seen a lot of comparison comments from people that don’t. As somebody that has both I can tell you unequivocally that there would be no way for me to draw a meaningful comparison if I only had one, OR if I somehow had a vested interest in (or against) one of them. No offence to Bob Spear or Forum Mgmnt, but to think either of you can comment without bias is rather silly. It’s no different if I (or somebody that has a vested interest in or against me) is talking about one of “myâ€
Last edited by Dean Hoffman on Sun Oct 08, 2006 5:10 pm, edited 2 times in total.

Old European
Roundtable Knight
Roundtable Knight
Posts: 144
Joined: Mon Apr 19, 2004 3:04 pm
Location: Old Europe

Post by Old European » Sun Oct 08, 2006 11:06 am

Dean and Forum Mgmnt, thanks for your excellent contributions to this interesting thread.

Dean, your message was good food for thought and opened the discussion. It's a pity that you were attacked immediately. Don't take it personally. I'm sure many forum members really appreciated your post.

Forum Mgmnt, I agree 100% with your views. The main problem with mechanical trading systems is their instability. Stable systems simply don't exist. 'There is no holy grail'. Look for example at the most problematic parameter of all systems, namely the exact composition of the trading universe. Adding just one single market to an already broad portfolio of markets can dramatically alter the test results. I have been struggling for a long time with this issue, until I found a solution I can live with (see earlier thread). But it certainly isn't a miracle solution.

It's better not to fool oneself when testing. The only thing one can do is using long time series, generating lots of trades (by investing in many markets and/or having not too long average trade durations) and working with a limited number of parameters in order to develop acceptable minimally stable systems. But the statistical significance of the model test results remains always questionable. Monte Carlo analysis can certainly help a bit (at least when auto- and intermarket correlations are properly taken into account). I personally simply assume that future drawdowns will be 50% (of course an arbitrary number) larger than historic drawdowns.

Dean suggested a way to time the investment in a mechanical trading system. It sounds (and it is) very logical. But it is indeed very dangerous not to build in any safety margin (see above). This reminds me of the famous Kelly formula for position sizing: a mathematically correct formula that however will lead to guaranteed disaster when apllied literally. I personally use a Martingale-anti-Martingale formula for position sizing that increases (decreases) bet sizes with increasing (decreasing) system drawdown, but that de-leverages very quickly when a pre-determined threshold is reached.

Let's keep the discussion going!

Cheers,

Old European

Nussgipfel
Roundtable Knight
Roundtable Knight
Posts: 115
Joined: Sun Nov 20, 2005 12:02 pm
Location: Westport

Post by Nussgipfel » Sun Oct 08, 2006 11:30 am

Thanks all for these great comments/presentation. For somebody trying to broaden and widen his horizon in mechanical trading I found all of this very helpful indeed. Hope some day to be able to meaningfully add to these kinds of discussions as well.

MarkH
Roundtable Fellow
Roundtable Fellow
Posts: 53
Joined: Fri Dec 05, 2003 10:24 am
Location: Corpus Christi, Texas

Post by MarkH » Sun Oct 08, 2006 12:47 pm

I agree with everything Old European just posted and was writing the following before I saw it. Anyway, I thought I would go ahead and post what I drafted to add to the discussion.

MarkH


The way I see it, monte carlo analysis of a multi-strategy/multi-market mechanical trading program (where you are scrambling historical daily returns) to generate many alternative equity curves is a useful, if limited exercise. As c.f. implies, it is absolutely useless, though, if the daily returns you are scrambling have been generated from a trading program consisting of systems (and/or market portfolios) that have been overly optimized or curve-fit. However, assuming you can get past this fundamental problem (the most fundamental problem in this business), the alternative equity curves generated by the monte carlo runs help you understand (and plan for) one of c.f.’ other points: history won’t repeat itself exactly. Because many of the alternative equity curves will visually show greater than historical drawdowns, it will better prepare you for another truism: larger drawdowns will occur in the future.

So, I agree with most of what c.f. said; however, I don’t think Dean “crossed over from the land of the aggressive well into the land of absurd.â€

MarkH
Roundtable Fellow
Roundtable Fellow
Posts: 53
Joined: Fri Dec 05, 2003 10:24 am
Location: Corpus Christi, Texas

Post by MarkH » Sun Oct 08, 2006 4:49 pm

I was thinking some more about c.f.’ initial point in his post above. c.f. said if you have a properly designed trading program the answer to this thread’s question “When to stop tradingâ€
Attachments
zzz russel 2000 imaster equity curve.xls
equity curve image
(29.5 KiB) Downloaded 573 times

Dean Hoffman
Roundtable Fellow
Roundtable Fellow
Posts: 87
Joined: Wed Jul 02, 2003 3:04 pm

Post by Dean Hoffman » Mon Oct 09, 2006 11:44 pm

Tushar S. Chande has also created a formula that attempts to predict maximum future drawdowns (1). It is primarily derived from average daily volatility. Personally I think the formula is too conservative, however, I think the real value here is the understanding of how critical daily volatility is to potential losses and overall risk.

On the surface, if two managers both made 30% returns and had maximum drawdowns of 10% and had similar statistics in many other regards (margin percentage usage, recovery time, average DD’s, portfolio size and robustness, etc.) The one stat that I would be vitally interested in would be the average daily change percentage. Larger daily volatilities (percentage changes) foretell of bigger potential risks in my opinion.

I think this metric is one of the most important and simultaneously underused. I don’t care how identical things appear, a manager who can achieve the same returns as another with significantly lower daily fluctuations has a vastly superior edge in my humble opinion.

This is also part of the reason that I think CTA analysis is very difficult given the current standards for reporting. I’m specifically referring to the standard of reporting on monthly intervals. Personally, I would NEVER invest in a CTA without seeing his DAILY performance history. A good example is an option writing CTA I recently heard about that shows very low month-to-month volatility yet has relatively high intra-month volatility. (I don’t have the exact figures in front of me but I believe them to be true as they were from a source I consider reliable).

For those interested in the Tushar S. Chande formula I have attached one part of it as an Excel file. You can get the entire set of formulas and his whole book here:

http://www.amazon.com/Beyond-Technical- ... F8&s=books

Regards,
Dean Hoffman

(1) Beyond Technical Analysis, Second Edition -- Wiley 2001
Attachments
Tushar S. Chande DD Formula.xls
(15.5 KiB) Downloaded 464 times

Post Reply