Correlation: Between Markets or MarketSystems?

[Roscoe]

This question keeps nagging at me and I would appreciate the input of some of the fine minds on this forum: the primary purpose of determining correlation would seem (to me anyway) to be in order to buffer drawdowns that could otherwise result from having all your trades go the one way at the one time for whatever reason. Traditionally markets are assigned to logical Sectors (Energies, Softs, Grains, and so on) on the assumption that all "like" markets in each Sector will be correlated, and we try to avoid taking too many positions in any one Sector on the basis that Sectors are likely to be non-correlated to each other, so when trades in one Sector are winning that would offset losses in another Sector and thus buffer our portfolio drawdowns (well, that is the theory anyway).

That all said, is it preferable to test for correlation between Markets or between MarketSystems?

The more that I think about this the more I think that testing for correlation between MarketSystems is the right way to go because that looks at what really matters, namely the individual equity curves of each MarketSystem. For example: it is entirely likely that different systems in correlated Markets could have non-correlated equity curves (or vice-versa) depending on the MarketSystems in use on each Market.

What say you?

[danZman]

it is entirely likely that different systems in correlated Markets could have non-correlated equity curves

Bingo. That's exactly why we try to trade a suite of systems that are non-correlated. If you use several different styles to trade (trend following, reversal systems, volatility, etc) and even different time horizons (intraday, daily, weekly), you might just have something.

I suppose one of the better ways to trade would be to create several "styles" of systems with different timeframes and in different markets (stocks, forex,futures,options).

I recall a discusion about this before with c.f..

D

[sluggo]

Suppose you could command a phalanx of eager graduate students to perform the following experiment:

1. Measure the correlation between the price histories of Gold and Crude Oil
2. Run some trading system or another, on Gold and on Crude Oil
3. Capture the equity curves when running the trading system on Gold and Crude Oil
4. Measure the correlation between the Gold equity cuve and the Crude Oil equity curve
5. Which is bigger? The correlation in item (1) or the correlation in item (4)?

There would be numerous choices to make along the way, such as: how many months or years of price history to use; whether to measure correlation on "price" or "price changes"; how big a slice of time to use when measuring correlation; what trading system to simulate; which commodities to use (Gold and Crude Oil may not be your favourite choices); and others. These choices are embedded in any discussion of market correlation and market-system correlation.

Wouldn't it be interesting, if the equity curves from trading Gold and trading Crude Oil, showed significantly more correlation than the price histories of Gold and Crude Oil?

Conversely, wouldn't it be just as interesting, if the equity curves showed significantly less correlation than the price histories? Would that be the ever elusive "free lunch" that investors seek?

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Now suppose you have a second bevy of eager grad students, and you launch them on the following experiment:

1. Buy historical price data for >100 commodities for >20 years
2. Pick a trading system such as PGO filtered Donchian (here) or VLT Turtle (same link) or another; ideally a profitable and robust system
3. Add code to the system so the simulator will print RAR, Robust Sharpe, R-Cubed, and other performance statistics of your liking, on every test.
4. Using the random portfolio selector Block kindly provided by nickmar and Jake Carriker (found here), run that trading system on 4000 randomly chosen 20-commodity portfolios out of the 100 commodity universe
5. Read the printed statistics file ("Print Output.csv") into Excel and calculate the median values of Robust Sharpe Ratio and the other performance statistics, from the 4000 simulation runs.
6. Repeat steps 4&5 but this time, run on 4000 randomly chosen 40-commodity portfolios. Record the median Robust Sharpe Ratio
7. Repeat steps 4&5 but this time, run on 4000 randomly chosen 60-commodity portfolios. Record the median Robust Sharpe Ratio
8. Make a table of (number of commodities in portfolio) versus (median Robust Sharpe Ratio). What conclusions can you draw?

Perhaps the median Robust Sharpe Ratio gets better and better and better, as you add more and more commodities. Which may have something to say about correlation.

Or perhaps the median Robust Sharpe Ratio stays about constant, getting no better and no worse as you add more and more commodities. Which probably tells you something different about correlation.

Or .... (you get the picture)

[AFJ Garner]

Sluggo has published some very interesting research somewhere on this site (or at least I think it was on this site - difficult to tell now that the bookmarking facility has gone) showing unintuitive but compelling results: equity curves of systems trading crude and gold may be less correlated than the prices or price changes of those commodities themselves.

Yes, yes, depends on all sorts of considerations, time periods and so forth.

I also like more rather than less instruments in a portfolio.

Having said that, no amount of study gets away from the stark fact that in a crisis such as we saw in July / August this year, all your hard work will very probably go to pot: every d**ned instrument you trade will suddenly become highly correlated as will the equity curve of systems trading those instruments. Every beastly position will go against you. Well, I exaggerate of course, nevertheless………….

You can fiddle to your heart’s content but at the end of the day the only thing likely to save your trend following butt when the **** hits the fan is total risk control, tempering total portfolio heat.

Or another way to put it: be diversified but don’t imagine it will give you immunity.

Or if it did seem to give immunity in July / August 2007 (according to that bevy of back tests you have just run) don’t believe that it will necessarily oblige in the next crisis.

Don’t be a pig: if your risk is too high then you can expect a regular and very painful boot up the backside.

Well, that’s just my thinking but I really don’t think it’s too far from the truth.

[RedRock]

Don’t be a pig: if your risk is too high then you can expect a regular and very painful boot up the backside.

While I agree with this line of reason, I’ll play devil’s advocate. My former boss at the first brokerage where I worked was a successful trader and businessman. He started with a small loan from his mother and had built up a small fortune. Enough so he could dabble in ownership of major league sports teams… Well I recall a ceramic statue caricature of a pig bearing his likeness and sporting his trading badge acronym. I believe it was a gift from some “friendsâ€

[svquant]

While commanding a bevy of grad students does sound enticing, before we send them on their way with a powerful simulation platform, perhaps it is best just to review some math... Sort of reminds me of someone I once worked with who went through the control theory of how a PLL works and other circuits vs just saying - throw it into SPICE (sorry for this inside joke).

The original question posed was correlation of instruments vs systems. It is interesting and there is perhaps no right answer or at least when you ask the question you need to understand some boundary conditions that may haunt you when the "rubber hit the road"

Assume two instruments and a single trading system. We have the return series for each instrument, r1 & r2, and the tradinsg system signals for each system, s1 & s2. Note the signal s = {-1,0,1} for each period. The equty of the system for each instrument is just r1*s1 and r2*s2. The correlation of the markets is just corr(r1,r2)=cr and the correlation of the signals is corr(s1,s2)=cs. Now lets look at the four cases at hand:

• cr>0, cs=1: obviously the correlation of the instruments will be exactly the same as the correlation of the equity curve, corr(r1*s1,r2*s2)
  cr>0, 0<cs<1: the correlation of the equity curve will be less than the instruments themselves. This is the results hinted at with gold/oil earlier in this thread.
  cr>0, -1<cs<0: Even better than above condition due to negative correlation of the systems. This is why trading a trend & counter trend system on an instrument works so well - especially when they both have positive expectation. In that case cr=1 (same instrument) and cs < 0, it is all goodness.
  cr<0, cs>0: See above
  cr<0, cs<0: Need to be careful since you might be doubling down when you do not think so...
  cr=-1, cs=-1: Yikes! 2x the allocation you might think you have. Easy though experiment is to think about trading an SP500 EFT and an inverse SP500 EFT they have. Both equity curves could be positive if the system is good, cr=-1 by design (or very close) and cs=-1 by design too since it is the same system.

Well not really mathematical but wanted to show that there are regions where some of the generalizations or simulations results are absolutely correct and other places where that might not be so.

Proofs of the above left to the graduate students after they finish all those simulations