Blending noncorrelated (or anti-correlated) equity curves

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Blending noncorrelated (or anti-correlated) equity curves

Post by sluggo » Sun Feb 13, 2011 6:33 pm

Fund-of-funds operators try to get higher returns through diversification: they spread their money across several trading managers. They hope that when one manager has a drawdown, another will have a "draw-up", thereby smoothing out the combined equity curve, and increasing gain-to-pain ratios like Sharpe, MAR, etc. We can accomplish this in Trading Blox by simulating a Suite that contains two or more trading systems, and studying the resulting equity curve.

The key idea is diversification. As discussed here earlier (ref1) what we seek is diversification of the trading systems' outputs (their equity curves). One common way to quantify diversification is to measure the correlation of the trading systems' equity curves -- an approach developed by Markowitz in the 1950s. (ref2)

The first step is to realize that "the correlation of A and B" is meaningless unless A and B are stationary; unfortunately, equity curves are certainly not stationary. So we don't actually calculate the correlation between equity curves; instead, we calculate the correlation between equity curve returns series. We do this because equity curve returns ARE stationary.

Let E[j] be the total equity on day number j. Then the equity curve return on day number j, written R[j], is simply R[j] = ((E[j] / E[j-1]) - 1). As you can verify yourself by making a couple of plots, the E[j] series is not stationary but the R[j] series is stationary. So given equity curves A and B, first we calculate the returns series of A and the returns series of B, and then we calculate the correlation of (returns series A) to (returns series B).

Markowitz explained that if two equity curves Y and Z are not identical (i.e. if their returns series correlation is less than one), and if they have the same volatility, then the volatility of the combination (ECY + ECZ) is less than the volatility of (ECY + itself), and less than the volatility of (ECZ + itself). In other words, diversification reduces volatility. Since volatility is the denominator of several gain-to-pain ratios, diversification (reduced volatility) can increase gain-to-pain. But please use a bit of common sense; if ECY is a lovely nice equity curve and ECZ is a complete and total stinker, with gut-wrenching drawdowns, psychotic swings, and enormous hand over fist losses, then adding ECZ to ECY will not increase your happiness! It will reduce both pain AND gain, and you won't like the result.

Figure 1 below shows Markowitz's diversification equation (the final line). His book expresses the result in terms of Variance and Covariance; I've reworked the formulas to use Standard Deviation and Correlation instead. In my experience, mechanical systems traders are more comfortable with SD and Correlation; these are somehow more intuitive.

Image
Figure 1: Markowitz's equation.

Let's sanity-check the formula by supposing for a moment that ECA and ECB are perfectly correlated, i.e., rho(AB) = 1.000. Then sigma(A+B) = sqrt(sa^2 + sb^2 + 2*sa*sb). The expression under the radical reduces to (sa+sb)^2 and so we see that when correlation=1.000, sigma(A+B) = sigma(A) + sigma(B). Exactly as expected. Sanity check passes. When correlation equals 1, there is no benefit from diversification. Volatility is not decreased.

Next let's suppose that A and B each have one unit of volatility, i.e., sigma(A) = sigma(B) = 1, and let's explore the effect of varying the correlation rho(AB). When we implement Markowitz's equation in Excel, we get these results (Figure 2):

Image
Figure 2

As the correlation rho(AB) between equity curve return series A and B gets smaller, so does the volatility sigma(A+B) of the combination. When the correlation falls to -1 ("perfect anti-correlation"), the volatility of the combination goes to zero. If you define "pain" to mean "volatility of equity curve returns series" then pain is zero! A lot of the volatility drop-off action occurs in the rarified zone where correlation < -0.80.

You can perform similar Excel experiments on examples where the two equity curves being blended have different volatilities; you'll find that the lower the correlation, the lower the volatility of the blended result. And if correlation goes negative (becoming "anti-correlation"), results get even better.

To explore this idea a bit further, I started with a typical equity curve of a typical mechanical trading system, trading a portfolio of futures contracts for the past 21 years (Figure 3). I'll call this "equity curve A". (Sorry about the watermark. It's there so that if the image gets copied into another website or blog, viewers of that copying site will find out about tradingblox.com and possibly come visit the Roundtable Forum.)

Image
Figure 3: Equity Curve A

Equity Curve A has decent, but not outstanding, performance: Sharpe= 1.16, MARratio= 0.86, CAGR= 19.9%/year, MaxDD= 23.1%, Longest Drawdown = 21 months, Annual Volatility = 15.0%.

I decided to search for other equity curves with varying amounts of correlation to equity curve A. From hundreds of thousands of candidates, I chose the seven equity curves shown in Figure 4: Equity curves B, C, D, E, F, G, and H. I've plotted the monthly returns of these seven curves, versus the monthly returns of equity curve A. (Trading Blox makes this easy; you use data from the file "Monthly Equity Log.csv" in the TradingBlox\Results folder.)

Image
Figure 4: Correlation between equity curve A and equity curves B-H

The top-left panel of Figure 4 shows the correlation of equity curve A to itself. Not surprisingly, Excel calculates the correlation to be +1.000. Then as we progress from equity curve B to equity curve H, correlation to ECA steadily falls. (The best-fit regression lines are shown in red.)

Blending these equity curves with equity curve A turns out to boost performance as well. The table below shows the standalone performance statistics of equity curves A-H (top half of the table), and also the performance statistics when A is blended with B, when A is blended with C, when A is blended with D, etc. (bottom half).

Code: Select all

Ecurve  Sharpe   MAR      CAGR       MaxDD   LongestDD  AnnVol  CorrToA
=======================================================================
A       1.16     0.86     19.91%     23.1%     21.0     15.0%   +1.000
B       1.14     0.64     17.59%     27.3%     22.5     15.0%   +0.726
C       1.13     0.54     16.23%     29.9%     32.4     15.0%   +0.314
D       1.13     0.76     17.53%     22.9%     18.0     15.0%   +0.041
E       1.14     0.74     17.78%     23.9%     39.1     15.0%   -0.089
F       1.13     0.69     16.62%     24.1%     24.8     15.0%   -0.240
G       1.13     0.90     16.77%     18.6%     21.0     15.0%   -0.399
H       1.13     0.78     18.09%     23.1%     29.2     15.0%   -0.601
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
A+B     1.24     0.98     21.24%     21.7%     16.3     15.0%
A+C     1.42     1.26     22.90%     18.2%     10.4     15.0%
A+D     1.59     1.58     27.85%     17.7%     14.0     15.0%
A+E     1.70     1.80     31.17%     17.4%     14.1     15.0%
A+F     1.85     1.97     33.28%     16.9%     13.5     15.0%
A+G     2.09     2.64     38.12%     14.4%     10.2     15.0%
A+H     2.56     3.79     45.71%     12.1%     11.3     15.0%
The "Longest DD" column shows the Trading Blox-calculated Longest Drawdown (in months). "AnnVol" is the annual volatility of the equity curve. You can see that I adjusted the amount of leverage used (risk per trade) in each simulation, to get a value of 15.0% annual volatility in every case. All equity curves are thus normalized to a constant level of volatility. (I used the "mere factor of two slowdown" method mentioned in this thread http://bit.ly/gbNY32 to select the appropriate leverage). The final column "CorrToA" contains the Excel-calculated correlation values that were shown in Figure 4 above.

This table shows that each of the equity curves B-H gives a pleasing result when combined with equity curve A. Gain-to-pain ratios increase nicely, max drawdown falls, and duration of longest drawdown falls too. Furthermore, the smaller the correlation, the better the improvement. Negative correlation ("anti-correlation") is better still.

Careful inspection of the table reveals that equity curves B-H are each "about as good" as the original equity curve A. This was a deliberate choice I made when doing the (massive) computer runs which uncovered these equity curves. When choosing equity curves B-H, I didn't want to blend A with something that was "much worse", nor did I want to blend A with something that was "much better". If B-H were not "about as good" as A, readers might become confused, possibly thinking that the performance increase came from the goodness or badness of B-H, rather than from the magnitude of the correlation between B-H and equity curve A.

I plotted the gain-to-pain ratios from the table above; the plots are in Figures 5 and 6 below. Note that equity curve H (correlation = -0.6) is at the left, and equity curve A (correlation = +1.0) is at the right.

Image
Figure 5: Sharpe vs Correlation-to-Equity-Curve-A

Image
Figure 6: MAR ratio vs Correlation-to-Equity-Curve-A

Suppose you've got hundreds of thousands of equity curves, and suppose that you've combined each of them with equity curve A. Suppose you know the Sharpe ratio and the MAR ratio and the correlation-to-equity-curve-A of each combination. You could plot this mountain of data as dots on a scatterplot. In fact that is exactly what I did. My scatterplot looked just like Figure 5, except there were zillions and zillions of dots BELOW the blue line. The blue line was the "efficient frontier" of Sharpe vs correlation to ECA, and the seven equity curves B-H that I chose, were dots ON (not below) the efficient frontier.

So don't be discouraged if your initial experiments give results which are below the efficient frontiers in Figure 5 and Figure 6. I too got LOTS of results which were below the efficient frontier. Stubborn perseverance and plenty of computers+licenses are needed. To get to the gold, you've got to break a lot of rocks.

In case you are wondering "what does anti-correlation look like?" have a glance at Figure 4. This shows anti-correlation on monthly returns scatterplots. "But what does it look like as an equity curve?" Take a look at Figures 7 and 8 below. These are plots of equity curve F (-0.240 correlation to equity curve A) and the equity curve of the A+F blend. When equity curve A bends down, equity curve F bends up (and vice versa). Not perfectly of course; the correlation is -0.24, not -1.00!

Image
Figure 7: ECF (Sharpe= 1.13, MAR= 0.69, MaxDD= 24.1%, corrToA= -0.240)

Image
Figure 8: ECA+ECF (Sharpe= 1.85, MAR= 1.97, MaxDD= 16.9%)

A few final remarks

Markowitz's result, diversification reduces volatility, also explains why trading a portfolio of numerous diverse instruments is beneficial. When trading N different instruments you are in effect creating N different equity curves: (system S trading instrument 1), (system S trading instrument 2), ..., (system S trading instrument N). Then you add all of these equity curves together by trading them simultaneously out of the same account. The equity curves are not perfectly correlated to one another: rho(JK) < 1.00 for all pairs of single instrument equity curves J and K. So, by the Markowitz equation, volatility is reduced.

Markowitz aimed his book at a late-1950's audience who sought to "buy and hold the right portfolio". Thus Markowitz never bothers to mention equity curves explicitly; there was no need, since a security's price and the equity curve of holding an unmargined Long position in that security are one and the same. However, owners of Trading Blox software and readers of this Roundtable are more likely to be futures traders or forex traders who (a) actively trade in and out -- the opposite of buy and hold; (b) take short positions; and (c) use leverage. Therefore, these traders have equity curves which are substantially different than the price series of the instrument(s) they trade. That's why I talk so much about equity curves while Markowitz never mentions them. My trading systems short Coffee, at 5X leverage. Markowitz's didn't.

Please note that I measured correlation on MONTHLY returns, and measured performance (gain-to-pain ratios, CAGR, DD, etc) on DAILY equity curves. These are the conventional choices; performance tracking services like IASG, BarclayHedge, AutumnGold all measure trading managers, hedge funds, and fund-of-funds correlations using monthly data. Trading system backtesting software such as Blox, Mechanica, PowerST all measure Sharpe, MAR ratio, CAGR, MaxDD on daily data. However, the conventional may or may not be the best. Perhaps other insights would arise from measuring correlation on daily returns (for example), or from measuring performance on weekly or monthly data. Other researchers may wish to explore this area.

Since volatility is the denominator of the Sharpe Ratio, as rho(AB) approaches -1.0, the denominator sigma(A+B) approaches zero and so the Sharpe Ratio approaches +infinity. However, after the manner of the Options Greeks, I suggest that you calculate the sensitivity d(Sharpe) / d(Correlation). You will find that the sensitivity goes to infinity as well. You might want to ponder that for a moment, in the context of In-Sample vs. Out-of-Sample correlations. (Yes, I realize this post doesn't include Out-of-Sample measurements. Please do as I say and not as I do!)

I've presented Markowitz's equation to calculate the volatility of the sum of two equity curves ECA and ECB. It's left as an exercise for the reader, to calculate the volatility of the average of two equity curves: sigma( (ECA/2) + (ECB/2) ). The result is delightfully compact when you assume the two curves are uncorrelated (rho=0), and it leads to further insight when you calculate the volatility of the average of N different uncorrelated equity curves: sigma( (EC1/N) + (EC2/N) + ... + (ECN/N) ). In fact, some people feel this latter equation provides the most intuitively satisfying explanation of "Why diversification works".

It will be amusing to see whether this thread on the Traders Roundtable spawns other articles, posts, and blog entries elsewhere (with or without attribution). "The Mathematics Of Diversification" and/or "Harry Markowitz Is Your Friend" and/or "Rho, Sigma, and All That" might soon be coming to a web page or RSS feed near you. :) Consider "Diversification For Dummies", it could be HUGE.

I'm sure I'll discover typos, mistakes, and other embarrassments after I post this message, so I plan to make corrections and other changes in the upcoming days. Don't be surprised if the edit count gets pretty large.

+SLUGGO+
Last edited by sluggo on Sat Feb 19, 2011 9:18 am, edited 7 times in total.

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Sun Feb 13, 2011 8:44 pm

Thanks for taking the time to post this.

Turtle40
Roundtable Knight
Roundtable Knight
Posts: 201
Joined: Wed Oct 19, 2005 1:53 pm
Location: Guernsey, Channel Islands

Post by Turtle40 » Mon Feb 14, 2011 5:01 am

Thank you for your continued input! I can't honestly say I fully understand the maths, but the article is fascinating none the less.
How do you have the time to do all of this?
Please write a book-it would be a best seller I'm sure!

Thanks again.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Wed Feb 16, 2011 9:25 am

Here are the plots of ECA, ECH, and ECA+ECH.

Equity curve H is the one that's anti-correlated (rho = -0.601) to A the most. Blending H with A gave delightful results: Sharpe ratio > 2.5, MAR ratio > 3.5 (!)


Image

Image

Code: Select all

Ecurve  Sharpe   MAR      CAGR       MaxDD   LongestDD  AnnVol  CorrToA
=======================================================================
A       1.16     0.86     19.91%     23.1%     21.0     15.0%
H       1.13     0.78     18.09%     23.1%     29.2     15.0%   -0.601
A+H     2.56     3.79     45.71%     12.1%     11.3     15.0%
Image

Image

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Wed Feb 16, 2011 10:29 am

Are these real, tradeable systems, or just an academic exercise (curve fit, long/short only, no commish & slip, etc)

drm7
Roundtable Fellow
Roundtable Fellow
Posts: 96
Joined: Sun Apr 20, 2003 9:02 pm
Location: Richmond, VA

Post by drm7 » Wed Feb 16, 2011 10:35 am

This is great work Sluggo! My concern with the conclusions lies more with the practical outcome of this analysis rather than the theory. First of all, correlations aren't stationary, and second, the original systems behind the original two (or three or four) equity curves may break down anyway, rendering the correlation analysis almost a moot point (ALMOST a moot point...)

But yes, I do agree in principle that trading as many futures, stocks, systems, timeframes (with a positive expectation) as possible is a sound practice.

marriot
Roundtable Knight
Roundtable Knight
Posts: 347
Joined: Thu Nov 20, 2008 3:02 am

Post by marriot » Wed Feb 16, 2011 11:55 am

A lot of bad things can happen, but i would like to have this kind of "know how".
Almost sure it can help to find a more efficent blend of systems.

I hate Sluggo !

:lol:

LeapFrog
Roundtable Knight
Roundtable Knight
Posts: 695
Joined: Mon May 17, 2004 4:18 pm
Location: Boston, MA

Post by LeapFrog » Wed Feb 16, 2011 12:38 pm

Sluggo - well done! We on this forum are very fortunate, IMHO, to have a contributor such as yourself who is willing to share this kind of research with us. That was a lot of research and a lot of work posting it in such a succinct and concise form - even I could understand it after a couple of reads through.

I remember CF had posted a comment on this forum years ago along the lines of "combining uncorrelated systems" was the key to good system results and you have taken that idea a lot further and with real rigor.

What I really like is the path you have shown that enables others such as myself to now follow, quickly and with method, to improve our own trading suites. I particularly liked your application of AnnVol to "equalize" multiple systems, as well as an intuitive overlay of "as good as" other metric considerations - that was key for sure in getting at the non-correlation comparison - very well done!

The idea of using monthly averages instead of month end results is intriguing too and I suspect will further reinforce your main points.

Took a lot of computer power, time, and effort, to produce your posting - I for one am highly appreciative. It has inspired me to try some things in my own research that I hadn't thought of before.

You could easily find a magazine to publish your work if you were so inclined - not sure I want you to do that - keep this just amongst us girls. :D

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Wed Feb 16, 2011 7:55 pm

First, thank you for the very kind words! I'm terribly flattered.

Second, I think "curve fitting" and "academic exercise" are subjective judgments; knowledgeable and well-meaning people can disagree about whether X is or is not curve fitting; whether Y is or is not an academic exercise.

For example, is the following an academic exercise?
  1. Buy, rent, borrow, or steal several dozen computers (cores) and software licenses
  2. Run hundreds of thousands of simulations, save the equity curves & stats
  3. Filter out (discard) the ones that aren't "about as good" as Sharpe=1.16, MAR=0.86
  4. Plot the keepers as blue dots on a scatterplot of (Correlation to ECA) versus (Sharpe) like Figure 5 above
  5. Pick out seven of the dots that lie along the top (max Sharpe) frontier, approximately evenly spaced between rho=+1 and rho=-0.6
  6. Label these seven dots B through H
Is that "an academic exercise"? (Upon reading step #1, some would reply: Hell Yes).

Is that "curve fitting" ? Some would say Yes, others No. Use your own judgement.

I used my standard commission & slippage setup that I use for most production simulations; it over-estimates (compared to actual C&S from brokerage statements) frictional costs by a little bit. I prefer to be slightly pessimistic because I find it greatly reassuring to know I'm not being optimistic.

Keep in mind, this is exactly what fund-of-funds operators do. It's just that they have got far fewer than 800,000 equity curves (track records of trading managers) to work with. Oh, and they can assemble a Suite (an FoF, also known as a Pool) containing more than two equity curves, if they wish. Meaning that they need to analyze 16-dimensional scatterplots. They use Computers and Software to help with this.

bobsyd
Roundtable Knight
Roundtable Knight
Posts: 361
Joined: Wed Nov 05, 2008 2:49 pm

Post by bobsyd » Wed Feb 16, 2011 8:41 pm

Ditto very appreciated and all that.

My 2 cents is that the biggest caveat is actually embedded in one of Sluggos comments above:

"You might want to ponder that for a moment, in the context of In-Sample vs. Out-of-Sample correlations. (Yes, I realize this post doesn't include Out-of-Sample measurements. Please do as I say and not as I do!)"

which is also implicit in drm7's post.

rhc
Roundtable Knight
Roundtable Knight
Posts: 464
Joined: Tue Nov 18, 2008 8:46 pm
Location: Oz

Post by rhc » Wed Feb 16, 2011 10:21 pm

Hi Sluggo,
I finally found some spare time and read your contribution above.
Yet another thought provoking thread. This is great stuff

Thanks ever so kindly for sharing this.
In fact thanks for sharing all your posts & comments including this classic which is very much related to the above and has been very helpful for me (and may be helpful to others)
viewtopic.php?t=6880&highlight=developing+systems

. . . . . . . and now to the application of above post

kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti » Thu Feb 17, 2011 6:47 am

sluggo wrote:Is that "curve fitting" ? Some would say Yes, others No. Use your own judgement......
I would say no, I use intuitively some of the above ideas.
Thanks for the time and effort to the Pavarotti of testing and simulation.

Best regards, as ever

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Sat Feb 19, 2011 5:12 pm

It's left as an exercise for the reader, to calculate the volatility of the average of two equity curves: sigma( (ECA/2) + (ECB/2) ). The result is delightfully compact when you assume the two curves are uncorrelated (rho=0), and it leads to further insight when you calculate the volatility of the average of N different uncorrelated equity curves: sigma( (EC1/N) + (EC2/N) + ... + (ECN/N) ). In fact, some people feel this latter equation provides the most intuitively satisfying explanation of "Why diversification works".

Here's the algebra; it's nothing but straightforward plug-and-chug manipulation:

Image

And the big result is: the more uncorrelated equity curves you average together, the smaller the volatility of the result. Furthermore, volatility falls as (1 / sqrt(N)) where N is the number of equity curves you've averaged. That's why diversification works.

(assuming the equity curves are uncorrelated and have equal volatility)

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Sat Feb 19, 2011 8:25 pm

The concept is simple. Finding un/anti correlated equity curves is not.

bobsyd
Roundtable Knight
Roundtable Knight
Posts: 361
Joined: Wed Nov 05, 2008 2:49 pm

Post by bobsyd » Sun Feb 20, 2011 9:31 am

Bravo again and thank you Sluggo for all your recent posts. They are unusually valuable and prolific contributions, even in comparison with your usual high standards.

The three Topics you have recently initiated that stand out for me are this “Blending…â€
Attachments
Volatility Reduction Graph.JPG
Volatility Reduction Graph.JPG (58.65 KiB) Viewed 16967 times

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Sun Feb 20, 2011 12:15 pm

It just keeps getting better and better; I think a log log plot shows this nicely
Attachments
Capture.PNG
Capture.PNG (17 KiB) Viewed 16934 times

Jez Liberty
Roundtable Knight
Roundtable Knight
Posts: 123
Joined: Tue Nov 03, 2009 8:49 am
Location: London
Contact:

Post by Jez Liberty » Mon Feb 21, 2011 9:41 am

The equations in the posts above show that the decrease in volatility is function of both N and rho.
Simplifying for rho=0 allows to neatly reduce the equation for N equity curves. But I was curious to see how much rho plays its part in decreasing volatility when it is not assumed to be 0. As LeviF says, it might not be that easy to find uncorrelated equity curves...

Inspired by the previous posts, I did a bit of analysis and wanted to contribute it here.

Below is a chart showing the decrease in volatility as a function of N and rho:

Image

Note that the assumption used to generate this chart is an equal correlation between ALL equity curves blended together (as well as identical volatility)

It is interesting to note that while mixing 100 uncorrelated equity curves reduces volatility by 90%, mixing even 1,000 equity curves all strongly correlated to each other (rho = 0.9) will only decrease volatility by just over 5%. I do find that number being so low quite surprising (I checked my formulas and could not find any glaring mistake - let me know if you see something I cocked up in there somewhere).



For those interested in the maths used to derive the N-generic volatility decrease as a function of both correlation (rho) and number of equity cuves (N) - with the assumption of equal volatilities and correlations - here it is:

Going back to sluggo's equation (applying Lemmas 1 and 2):

Image

And assuming that ECA and ECB have the same volatility (sigmaA = sigmaB = s) but a correlation ranging from -1 to 1, the formula would become:

Image

The implication - as highlighted in sluggo's initial comments - is that the blended volatility varies between 0 (rho = -1) and s, which is the individual equity curves volatility (when rho = 1).

Turning to the N-generic formula:

Image

The formula above shows that the negative rhos can cause a problem, as they would force the sum under the root to be negative for a high-enough N (when rho < -1 / (N-1) ). This is why the chart above only draws volatility for rho between 0 and 1.

Thinking about an example though, it would be impossible to get three equity curves all displaying a correlation of -1 against each other, i.e. if A is the perfect opposite of B and C (correlation = -1), C cannot be the perfect opposite of B.
But it does not matter: A and B correlated to a coefficient of -1 means that volatility = 0. No need to include an additional third (nth) equity curve.

We could even actually calculate, for a given number of equity curves N, the required correlation between all equity curves (assuming equal volatility and correlation) to reach a blended volatility of 0: rho = -1 / (N-1)

A few examples:
N=2 -> rho = -1
N=3 -> rho = -0.5
N=5 -> rho = -0.25
N=11 -> rho = -0.1
N=26 -> rho = -0.04
N=51 -> rho = -0.02
N=101 -> rho = -0.01
...

Of course, all these results are theoretical, as the assumption that all equity curves have equal correlation is most likely never true; but they should give some idea of how correlation affects volatility decrease, in the context of diversification.

ps: I used this handy site for generating the mathematical notations/equations. Syntax and examples here.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Mon Feb 21, 2011 10:27 am

Trading Blox Builder ships with eleven presupplied systems. Simulate two thousand parameter settings of each system on a sensibly chosen portfolio, over a sensible slice of price history. Calculate the correlation between their (monthly) equity returns series - a 22000 x 22000 matrix of correlation coefficients. Be slightly clever about this, you don't want to actually store (22000 ^ 2) floating point numbers, not even on disk.

Plot your observed correlations as a frequency histogram. I think you'll be surprised at the location of the mean, median, and mode. I think you'll be surprised at the size of the standard deviation and the ratio (mean / standard deviation). I think you'll be surprised at the max and the min of your dataset.

I think you'll be surprised at the ease or difficulty of finding 1000 equity curves that are all correlated to each other with rho(i j) = +0.90 for (1 <= i,j <= 1000).

An exercise like this may inspire you to create additional systems which operate on entirely different principles than the Blox presupplied eleven systems.

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Mon Feb 21, 2011 10:41 am

There must be some other factors at play here aside from just equity curve correlation. I can come up an uncorrelated system with negative expectancy that hurts overall portfolio performance. Or what about two systems with -1 correlation (system 2 takes the opposite trade of system 1).

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Mon Feb 21, 2011 10:55 am

But please use a bit of common sense; if ECY is a lovely nice equity curve and ECZ is a complete and total stinker, with gut-wrenching drawdowns, psychotic swings, and enormous hand over fist losses, then adding ECZ to ECY will not increase your happiness! It will reduce both pain AND gain, and you won't like the result.

Quite obviously, the average of two equity curves ECY and ECZ will have a return which is somewhere between the return of ECY and the return of ECZ. The surprise from Markowitz is that when you average together ECY and ECZ, you might get a volatility which is shockingly low ... depending upon correlation. If ECY and ECZ are perfectly correlated (correlation is +1.000), the volatility of (ECY+ECZ)/2 is the average of ECY's volatility and ECZ's volatility. On the other hand, when correlation is negative, the volatility of (ECY+ECZ)/2 can be shockingly low -- possibly even zero.

So the goal is to find an ECZ which isn't a stinker, but which is massively anti correlated with ECY. This requires stubborn perseverance.

If you believe, as I do, that desirability or "goodness" can be approximated by (return / volatility), then goodness can go to infinity if volatility goes to zero.

Edit - Please note, as CF did on the Veritrader forums several years ago, that you might choose an ECZ which looks sickly on its own, but if it manages to lower the volatility of (ECY+ECZ)/2 by a LOT, but only manages to lower the returns of (ECY+ECZ)/2 by a little, then it's still a good idea.

CF's example is: ECZ is a breakeven system, its return equals zero. However ECZ is so anticorrelated to ECY that the volatility of (ECY+ECZ)/2 is reduced by a factor of 4. Thus return = (ECYreturn + 0)/2 is cut in half. But volatility is cut by a factor of 4. So the ratio (return/volatility) RISES by a factor of two. Goodness is doubled. Pleasing.
Last edited by sluggo on Mon Feb 21, 2011 11:21 am, edited 3 times in total.

Post Reply