The key idea is diversification. As discussed here earlier (ref1) what we seek is diversification of the trading systems' outputs (their equity curves). One common way to quantify diversification is to measure the correlation of the trading systems' equity curves -- an approach developed by Markowitz in the 1950s. (ref2)
The first step is to realize that "the correlation of A and B" is meaningless unless A and B are stationary; unfortunately, equity curves are certainly not stationary. So we don't actually calculate the correlation between equity curves; instead, we calculate the correlation between equity curve returns series. We do this because equity curve returns ARE stationary.
Let E[j] be the total equity on day number j. Then the equity curve return on day number j, written R[j], is simply R[j] = ((E[j] / E[j-1]) - 1). As you can verify yourself by making a couple of plots, the E[j] series is not stationary but the R[j] series is stationary. So given equity curves A and B, first we calculate the returns series of A and the returns series of B, and then we calculate the correlation of (returns series A) to (returns series B).
Markowitz explained that if two equity curves Y and Z are not identical (i.e. if their returns series correlation is less than one), and if they have the same volatility, then the volatility of the combination (ECY + ECZ) is less than the volatility of (ECY + itself), and less than the volatility of (ECZ + itself). In other words, diversification reduces volatility. Since volatility is the denominator of several gain-to-pain ratios, diversification (reduced volatility) can increase gain-to-pain. But please use a bit of common sense; if ECY is a lovely nice equity curve and ECZ is a complete and total stinker, with gut-wrenching drawdowns, psychotic swings, and enormous hand over fist losses, then adding ECZ to ECY will not increase your happiness! It will reduce both pain AND gain, and you won't like the result.
Figure 1 below shows Markowitz's diversification equation (the final line). His book expresses the result in terms of Variance and Covariance; I've reworked the formulas to use Standard Deviation and Correlation instead. In my experience, mechanical systems traders are more comfortable with SD and Correlation; these are somehow more intuitive.
Figure 1: Markowitz's equation.
Let's sanity-check the formula by supposing for a moment that ECA and ECB are perfectly correlated, i.e., rho(AB) = 1.000. Then sigma(A+B) = sqrt(sa^2 + sb^2 + 2*sa*sb). The expression under the radical reduces to (sa+sb)^2 and so we see that when correlation=1.000, sigma(A+B) = sigma(A) + sigma(B). Exactly as expected. Sanity check passes. When correlation equals 1, there is no benefit from diversification. Volatility is not decreased.
Next let's suppose that A and B each have one unit of volatility, i.e., sigma(A) = sigma(B) = 1, and let's explore the effect of varying the correlation rho(AB). When we implement Markowitz's equation in Excel, we get these results (Figure 2):
As the correlation rho(AB) between equity curve return series A and B gets smaller, so does the volatility sigma(A+B) of the combination. When the correlation falls to -1 ("perfect anti-correlation"), the volatility of the combination goes to zero. If you define "pain" to mean "volatility of equity curve returns series" then pain is zero! A lot of the volatility drop-off action occurs in the rarified zone where correlation < -0.80.
You can perform similar Excel experiments on examples where the two equity curves being blended have different volatilities; you'll find that the lower the correlation, the lower the volatility of the blended result. And if correlation goes negative (becoming "anti-correlation"), results get even better.
To explore this idea a bit further, I started with a typical equity curve of a typical mechanical trading system, trading a portfolio of futures contracts for the past 21 years (Figure 3). I'll call this "equity curve A". (Sorry about the watermark. It's there so that if the image gets copied into another website or blog, viewers of that copying site will find out about tradingblox.com and possibly come visit the Roundtable Forum.)
Figure 3: Equity Curve A
Equity Curve A has decent, but not outstanding, performance: Sharpe= 1.16, MARratio= 0.86, CAGR= 19.9%/year, MaxDD= 23.1%, Longest Drawdown = 21 months, Annual Volatility = 15.0%.
I decided to search for other equity curves with varying amounts of correlation to equity curve A. From hundreds of thousands of candidates, I chose the seven equity curves shown in Figure 4: Equity curves B, C, D, E, F, G, and H. I've plotted the monthly returns of these seven curves, versus the monthly returns of equity curve A. (Trading Blox makes this easy; you use data from the file "Monthly Equity Log.csv" in the TradingBlox\Results folder.)
Figure 4: Correlation between equity curve A and equity curves B-H
The top-left panel of Figure 4 shows the correlation of equity curve A to itself. Not surprisingly, Excel calculates the correlation to be +1.000. Then as we progress from equity curve B to equity curve H, correlation to ECA steadily falls. (The best-fit regression lines are shown in red.)
Blending these equity curves with equity curve A turns out to boost performance as well. The table below shows the standalone performance statistics of equity curves A-H (top half of the table), and also the performance statistics when A is blended with B, when A is blended with C, when A is blended with D, etc. (bottom half).
Code: Select all
Ecurve Sharpe MAR CAGR MaxDD LongestDD AnnVol CorrToA ======================================================================= A 1.16 0.86 19.91% 23.1% 21.0 15.0% +1.000 B 1.14 0.64 17.59% 27.3% 22.5 15.0% +0.726 C 1.13 0.54 16.23% 29.9% 32.4 15.0% +0.314 D 1.13 0.76 17.53% 22.9% 18.0 15.0% +0.041 E 1.14 0.74 17.78% 23.9% 39.1 15.0% -0.089 F 1.13 0.69 16.62% 24.1% 24.8 15.0% -0.240 G 1.13 0.90 16.77% 18.6% 21.0 15.0% -0.399 H 1.13 0.78 18.09% 23.1% 29.2 15.0% -0.601 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - A+B 1.24 0.98 21.24% 21.7% 16.3 15.0% A+C 1.42 1.26 22.90% 18.2% 10.4 15.0% A+D 1.59 1.58 27.85% 17.7% 14.0 15.0% A+E 1.70 1.80 31.17% 17.4% 14.1 15.0% A+F 1.85 1.97 33.28% 16.9% 13.5 15.0% A+G 2.09 2.64 38.12% 14.4% 10.2 15.0% A+H 2.56 3.79 45.71% 12.1% 11.3 15.0%
This table shows that each of the equity curves B-H gives a pleasing result when combined with equity curve A. Gain-to-pain ratios increase nicely, max drawdown falls, and duration of longest drawdown falls too. Furthermore, the smaller the correlation, the better the improvement. Negative correlation ("anti-correlation") is better still.
Careful inspection of the table reveals that equity curves B-H are each "about as good" as the original equity curve A. This was a deliberate choice I made when doing the (massive) computer runs which uncovered these equity curves. When choosing equity curves B-H, I didn't want to blend A with something that was "much worse", nor did I want to blend A with something that was "much better". If B-H were not "about as good" as A, readers might become confused, possibly thinking that the performance increase came from the goodness or badness of B-H, rather than from the magnitude of the correlation between B-H and equity curve A.
I plotted the gain-to-pain ratios from the table above; the plots are in Figures 5 and 6 below. Note that equity curve H (correlation = -0.6) is at the left, and equity curve A (correlation = +1.0) is at the right.
Figure 5: Sharpe vs Correlation-to-Equity-Curve-A
Figure 6: MAR ratio vs Correlation-to-Equity-Curve-A
Suppose you've got hundreds of thousands of equity curves, and suppose that you've combined each of them with equity curve A. Suppose you know the Sharpe ratio and the MAR ratio and the correlation-to-equity-curve-A of each combination. You could plot this mountain of data as dots on a scatterplot. In fact that is exactly what I did. My scatterplot looked just like Figure 5, except there were zillions and zillions of dots BELOW the blue line. The blue line was the "efficient frontier" of Sharpe vs correlation to ECA, and the seven equity curves B-H that I chose, were dots ON (not below) the efficient frontier.
So don't be discouraged if your initial experiments give results which are below the efficient frontiers in Figure 5 and Figure 6. I too got LOTS of results which were below the efficient frontier. Stubborn perseverance and plenty of computers+licenses are needed. To get to the gold, you've got to break a lot of rocks.
In case you are wondering "what does anti-correlation look like?" have a glance at Figure 4. This shows anti-correlation on monthly returns scatterplots. "But what does it look like as an equity curve?" Take a look at Figures 7 and 8 below. These are plots of equity curve F (-0.240 correlation to equity curve A) and the equity curve of the A+F blend. When equity curve A bends down, equity curve F bends up (and vice versa). Not perfectly of course; the correlation is -0.24, not -1.00!
Figure 7: ECF (Sharpe= 1.13, MAR= 0.69, MaxDD= 24.1%, corrToA= -0.240)
Figure 8: ECA+ECF (Sharpe= 1.85, MAR= 1.97, MaxDD= 16.9%)
A few final remarks
Markowitz's result, diversification reduces volatility, also explains why trading a portfolio of numerous diverse instruments is beneficial. When trading N different instruments you are in effect creating N different equity curves: (system S trading instrument 1), (system S trading instrument 2), ..., (system S trading instrument N). Then you add all of these equity curves together by trading them simultaneously out of the same account. The equity curves are not perfectly correlated to one another: rho(JK) < 1.00 for all pairs of single instrument equity curves J and K. So, by the Markowitz equation, volatility is reduced.
Markowitz aimed his book at a late-1950's audience who sought to "buy and hold the right portfolio". Thus Markowitz never bothers to mention equity curves explicitly; there was no need, since a security's price and the equity curve of holding an unmargined Long position in that security are one and the same. However, owners of Trading Blox software and readers of this Roundtable are more likely to be futures traders or forex traders who (a) actively trade in and out -- the opposite of buy and hold; (b) take short positions; and (c) use leverage. Therefore, these traders have equity curves which are substantially different than the price series of the instrument(s) they trade. That's why I talk so much about equity curves while Markowitz never mentions them. My trading systems short Coffee, at 5X leverage. Markowitz's didn't.
Please note that I measured correlation on MONTHLY returns, and measured performance (gain-to-pain ratios, CAGR, DD, etc) on DAILY equity curves. These are the conventional choices; performance tracking services like IASG, BarclayHedge, AutumnGold all measure trading managers, hedge funds, and fund-of-funds correlations using monthly data. Trading system backtesting software such as Blox, Mechanica, PowerST all measure Sharpe, MAR ratio, CAGR, MaxDD on daily data. However, the conventional may or may not be the best. Perhaps other insights would arise from measuring correlation on daily returns (for example), or from measuring performance on weekly or monthly data. Other researchers may wish to explore this area.
Since volatility is the denominator of the Sharpe Ratio, as rho(AB) approaches -1.0, the denominator sigma(A+B) approaches zero and so the Sharpe Ratio approaches +infinity. However, after the manner of the Options Greeks, I suggest that you calculate the sensitivity d(Sharpe) / d(Correlation). You will find that the sensitivity goes to infinity as well. You might want to ponder that for a moment, in the context of In-Sample vs. Out-of-Sample correlations. (Yes, I realize this post doesn't include Out-of-Sample measurements. Please do as I say and not as I do!)
I've presented Markowitz's equation to calculate the volatility of the sum of two equity curves ECA and ECB. It's left as an exercise for the reader, to calculate the volatility of the average of two equity curves: sigma( (ECA/2) + (ECB/2) ). The result is delightfully compact when you assume the two curves are uncorrelated (rho=0), and it leads to further insight when you calculate the volatility of the average of N different uncorrelated equity curves: sigma( (EC1/N) + (EC2/N) + ... + (ECN/N) ). In fact, some people feel this latter equation provides the most intuitively satisfying explanation of "Why diversification works".
It will be amusing to see whether this thread on the Traders Roundtable spawns other articles, posts, and blog entries elsewhere (with or without attribution). "The Mathematics Of Diversification" and/or "Harry Markowitz Is Your Friend" and/or "Rho, Sigma, and All That" might soon be coming to a web page or RSS feed near you. Consider "Diversification For Dummies", it could be HUGE.
I'm sure I'll discover typos, mistakes, and other embarrassments after I post this message, so I plan to make corrections and other changes in the upcoming days. Don't be surprised if the edit count gets pretty large.