Things you might do, even though they HURT performance

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Things you might do, even though they HURT performance

Post by sluggo » Fri Feb 18, 2011 12:24 pm

System developers talk a lot about optimization: hunting for just the right parameter settings that maximize performance. They do things like (a) run gigantic numbers of parameter stepping backtests; (b) use genetic algorithms to adaptively search for the best parameter set; (c) argue endlessly about which measure-of-goodness is best at measuring which equity curve is best; and so forth.

On the other hand, you might intentionally decide to modify a trading system in a way that hurts performance -- if it improves something else. I think I've read that Jerry Parker (Chesapeake Capital) admits to doing this; I believe he said he takes profits prematurely -- partially scales out of profitable positions before the trade exit signal is received -- because this makes his customers happier. Even though he himself doesn't like the idea. However he has decided that happy customers are more important than happy Jerry Parker.

Another thing a system trader might do is, deliberately reduce the number of instruments in the portfolio (or the number of systems in the Suite), even though this reduces the benefits of diversification and hurts performance. You might do this because it lets you trade a smaller account, or because fewer markets/systems/orders means fewer opportunities for mistakes, or simply because it reduces the amount of time you spend sitting at a screen.

One more thing you might do is, sacrifice performance a little bit, if doing so greatly reduces your risk of "ruin". If you're trading OPM, "ruin" might mean "suffering a big nasty drawdown that scares all your investors away".

If you're truly scared of a big drawdown, you might decide to implement the Before and After idea below. Before is a garden variety mechanical system with standard fixed fractional position sizing (grey inset at upper left). After is the same system, but using more conservative / paranoid position sizing (grey inset).

The After system sizes positions based on the "Closed Equity" (which is equal to mark-to-market total account equity, MINUS the open trade equity of all currently open positions). In effect it says "I dare not pretend that Open Trade Equity is really mine; it could disappear in a heartbeat. Shoot, Open Trade Equity is just a euphemism for Paper Profits!")

The After system also uses the Dradwown Reduction idea from the book Way of the Turtle p.258. For each 5% of drawdown, the system reduces positionsizes by 20%. When in a drawdown, the system begins to place significantly smaller bets. And if the drawdown gets worse, the bets get smaller still. This hurts performance but protects against disaster.

As you can see, the After system has got lower CAGR, MAR, and Sharpe. Nevertheless, you may feel that the After system is SAFER for the long term health of your CTA business, than trading the no-safety-equipment Before system.

That's just one example. I'm curious to know, what tradeoffs have YOU made, what stunts / tricks / speed governors have you installed, that hurt your performance but you did it anyway?
Attachments
Aftersys.png
AFTER
Aftersys.png (31.63 KiB) Viewed 7674 times
Beforesys.png
BEFORE
Beforesys.png (31.91 KiB) Viewed 7674 times
Last edited by sluggo on Sun Feb 20, 2011 11:01 pm, edited 1 time in total.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Fri Feb 18, 2011 12:39 pm

Some classic examples of deliberately sacrificing performance
  • Use only End-of-Day data rather than intraday data, to calculate signals and place orders. (simpler)
  • Only trade systems that use Market-On-Open and Market-On-Close orders. Don't trade systems that use Stop orders or Limit orders. (simpler)
  • Roll the Nymex energy contracts less often than 12X per year. Those bastards will find a way to screw you, even on a rollover. (fear of pain)
  • Only trade the markets that your current broker offers. Don't consider changing brokers / adding brokers to get access to additional markets. (lethargy)
  • Only trade the systems that come presupplied with your trading software. :D Don't consider writing new systems or hiring programmers to write them for you (lethargy)

rhc
Roundtable Knight
Roundtable Knight
Posts: 464
Joined: Tue Nov 18, 2008 8:46 pm
Location: Oz

Post by rhc » Fri Feb 18, 2011 8:59 pm

Sluggo says;
I'm curious to know, what tradeoffs have YOU made, what stunts / tricks / speed governors have you installed, that hurt your performance but you did it anyway?
I like to use the Weekly timeframe for trading both stocks and futures.
(download data over the weekend, review orders over the weekend. It suits my lifestyle . . . I like it!)
When I get an exit signal I will exit on Monday’s OPEN.
Since this is a weekly system you generally know that you will be exiting on Monday’s OPEN by late Friday before the CLOSE.
(for example: a Moving Average(MA) exit requires a penetration of that MA and by late Friday before the market has closed you will know where price is relative to the MA and you know whether or not you will be exiting on Monday’s OPEN. . . . .in most cases based on my review of daily data)
You know all this before the CLOSE on Friday.

Some time ago I did some comparison tests using the same system and exiting on the CLOSE of Friday rather than the OPEN of Monday and the results showed that the Friday CLOSE exit tactic was superior
Of course sometimes the Monday exit will be more advantageous than the Friday exit but on a whole, for this particular system the Friday exit results in improved performance.

So why not exit at the CLOSE of Friday if performance is improved?

Answer:
Living in the Southern hemisphere, by the time Fridays CLOSE comes around in the US markets I’m still in the land of Zzzzzzzzzzzzzzz.
I don’t really fancy waking up at 4:00am to check the position of price w.r.t. the MA . . . . . I’ll just wait for my weekend review and exit (if I have to) on Monday . . . . to my detriment!! (on average)

*****************

(I suppose the same might occur when trading the daily timeframe . . . . .intraday data might allow you to exit on the close of the day rather than MOO the next day. Results might be improved by doing this. But of course it requires extra effort)

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Sat Feb 19, 2011 2:22 am

I have decided to not over optimize where is possible.
As an expample on a Triple moving average i optimize by 10 Long average, by 5 medium and short.
Using a the same very large portfolio on all your systems seem to hurt performance, untill you understand that it is the first and easiest way to reduce curvefitting.

When i bougth TB, i have ask Darryl Tremmeling to convert my Best systems. (thank you Sluggo to point me there)
Today i am using only one of them and it is not so good.
All the others came from TB or this forum.
I am running 10 systems all MOO, but really i miss open on all 24 hr Futures.
Darryl has write an "Order Generate" blox which semplified a lot daily stuff. Anyway check positions against Broker Statement, take 2 hours every morning
Lethargy ? First year was a 24 hour running test every day.
Now i have reduced a little bit.
Today i can say that simulations seem to be about close to reality.
Still working on compute real slippage.
On the other side, being a very small player i can trade tiny markets like KFX, PSI 20, Milk and even some Currencies at Finex. And that is does not hurt.

kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti » Sat Feb 19, 2011 4:02 am

sluggo wrote:Some classic examples of deliberately sacrificing performance
  • Use only End-of-Day
  • Only trade systems that use Market-On-Open and Market-On-Close orders.
  • Roll the Nymex energy contracts less often
  • Only trade the markets that your current broker offers
  • Only trade the systems that come presupplied with your trading software
One more way to sacrificing performance, maybe :D :lol: :D :
I designed a system not using last bar prices; in that way I could trade 24hr markets and have signals well in advance.
rhc&marriot could think about it: you place orders in advance and have all the time to go to the local milonga for Tango :lol: :D :lol:
With so many breakout traders maybe it could help you in not being broken out :?: :?: :?: :?:


Best regards, as ever

kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Re: Things you might do, even though they HURT performance

Post by kianti » Sat Feb 19, 2011 4:12 am

sluggo wrote:Another thing a system trader might do is, deliberately reduce the number of instruments in the portfolio
Would you be kind enough to give us more clues on diversification?
Is trading 5 US grains better than trading 1 US grain with 5 different systems?

Best regards, as ever

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Sat Feb 19, 2011 4:13 am

Hi Kianti, can you make an example?
thank you

kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti » Sat Feb 19, 2011 4:30 am

marriot wrote:Hi Kianti, can you make an example?
thank you

Code: Select all

if Close[-1]>Close[-2] then Long

rajivm
Roundtable Knight
Roundtable Knight
Posts: 129
Joined: Fri Jul 25, 2003 12:50 am
Location: New Delhi, India

My Performance Hurting Tradeoff

Post by rajivm » Sat Feb 19, 2011 10:18 am

I trade stock markets -
1. Never been able to garner enough guts to pick all signals if there are 4-5 signals in one day.
2. Also if stoploss is closeby the position Size will be large..So I often deliberately reduce the position Size and then I cry 2 days later for my foolishness
Raj

7432
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Wed Dec 31, 1969 7:00 pm
Location: sandy, ut

Re: Things you might do, even though they HURT performance

Post by 7432 » Mon Feb 21, 2011 5:14 pm

sluggo wrote: The After system sizes positions based on the "Closed Equity" (which is equal to mark-to-market total account equity, MINUS the open trade equity of all currently open positions).
for those that use stops and want to try closed equity,

if I am holding an open position and roll from march to june, TB figures I've closed the trade and counts it toward closed equity even though my stop might not change as far as risk is concerned.
so if holding positions longer than one roll over, it might be better to crunch the closed equity as total equity minus total risk(distance to stops).
this way if your account is at 1 million, risk is 25% and you've just rolled all positions from march to june, and you want to use closed equity, your next position will be sized based on 750k of equity rather than 1,000k.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Sat Mar 05, 2011 3:16 pm

  • Trade a huge kitchen-sink portfolio which includes numerous poor performers (in past history)

rgd
Roundtable Knight
Roundtable Knight
Posts: 396
Joined: Thu Aug 10, 2006 5:21 pm

Post by rgd » Mon Mar 07, 2011 11:06 am

trade models which have horrible equity curve, and detract from performance, most of the time

rabidric
Roundtable Knight
Roundtable Knight
Posts: 243
Joined: Mon Jan 09, 2006 7:45 am

Post by rabidric » Mon Mar 07, 2011 11:35 am

-Throw out all the fancy heat limiters and position reduction...hurts the smoothness, but being 10 times better off at the end of a decade makes up for a lot of jagged pixels.

-Trade multiple entries types/systems, even though they are worse than "the single best entry". Got more bases covered that way, in real unpredictable trading.

Jens Albrecht
Senior Member
Senior Member
Posts: 40
Joined: Thu Feb 08, 2007 3:19 pm
Location: Dortmund, Germany

Post by Jens Albrecht » Mon Mar 28, 2011 11:01 am

This did for me:

1. Using rather simple but robust Donchian system instead fancy many-parameter systems.

2. Reduce portfolio to atm 16 markets. I use the systematic method I once have showed here: viewtopic.php?t=4185&highlight=

But note there is a flaw in the method presented which I found later. I hadn't ever again the time to add on this. One can't expect to limit R-Squared of an LTTF by - say- 99.9%. But it is possible to limit R-Squared reasonably with a given portfolio in a statistical sense, e.g. R-Sqaured > 98% holds with 99% probability.
Practically this worked for me since 2007 and in all backward / forward tests. One remarkable point (if you dare): The method suggests trading different instruments with different weights. This means 2 of my 16 markets are the same.

3. EOD data anyway. I have better things to do rest of the day :D

4. Timesavings that hurt performance.
I coded a script to transfer orders from TB to Interactive Brokers. After I got confidence that it worked I don't check orders anymore. I check positons and get a comparison of simulated and real equity curve from Excel. I debug if I observe differences. This brings me down to about 5 mins a day for the order run. Knowingly it could hurt performance but did not yet seriously.

You see my ideal is Ed Seykota. At least I rather spent my time reading books than sitting in front of the order screen. :lol:

Regards, Jens

drm7
Roundtable Fellow
Roundtable Fellow
Posts: 96
Joined: Sun Apr 20, 2003 9:02 pm
Location: Richmond, VA

Post by drm7 » Mon Mar 28, 2011 3:34 pm

>Jens

Looks like your "blunt instrument" (to quote Eckhardt) seems to work for you. How has the actual performance and R-squared of the equity curve matched your backtest?

On a similar note: your 2007 test set commissions at zero - does commissions and slippage noticably hurt your "live" performance?

You said you trade different instruments with different "weights." Does that mean you trade 2x of the "base" contract for corn, but only 1x of T-notes? How did you simulate that? I am concerned that optimizing the position sizing (above and beyond standard risk per trade) may lead to sub-par future results, as instrument performance and correlations between instruments can change over time. (Gold and silver were horrible for many years, but recently are incredible.)

Post Reply