Is Modern Portfolio Theory Dead?

Discussions about Money Management and Risk Control.
DPH
Roundtable Knight
Roundtable Knight
Posts: 115
Joined: Tue Nov 17, 2009 1:07 am
Location: Central Pennsylvania
Contact:

Is Modern Portfolio Theory Dead?

Post by DPH »

Here is an amusing thread that takes (good) aim at Modern Portfolio Theory.

http://venturepopulist.com/2009/05/mode ... o-fallacy/

The author accurately elucidates (in my opinion) about the Achilles heel in MPT, its reliance on past correlations. In a nutshell, MPT tells us to combine non-correlated assets for a smoother equity curve and better risk adjusted performance, but this can be blown apart by one basic fact…….(drum roll)………CORRELATIONS CHANGE!

The implications of this are tremendous….All the pretty equity curves that one can generate by combining non-correlated trading systems (many of which I have seen here) and “diversifiedâ€
Last edited by DPH on Thu Nov 10, 2011 5:50 pm, edited 1 time in total.
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

I suggest you post the same message to the Mechanica User's Forum, warning them in the strongest possible terms that you think it is ignorant and dangerous to use the new MVO features of that program.

Modern Portfolio Theory uses Mean-Variance Optimization (see WIKIPEDIA article) to construct the Efficient Frontier and to find the optimal portfolio weightings. It employs past correlations (calling them by the name "covariance matrix") in the optimization, and is prone to the exact same difficulties that you mention above.

Please, you owe it to your Mechanica colleagues, warn them of the danger you perceive! Especially since you have stated on the record (LINK) that you personally prefer Mechanica over Trading Blox. Mechanica users are your amigos, your neighbors, your team members, your family! Warn them.
Attachments
From mechanica sales web page
From mechanica sales web page
MVO.gif (33.19 KiB) Viewed 19468 times
mojojojo
Roundtable Knight
Roundtable Knight
Posts: 131
Joined: Mon Oct 31, 2005 2:07 pm

Post by mojojojo »

Is it dead? Nope. It's too ingrained into to institutional (Pension, Endowment, etc) manager's way of thinking. Add in the fact that the CFA is basically based off of it, although they are getting better. It will be here for a while.

I think the smart managers have known the issues for a while now and don't live by it.

I think that MVO can be updated to make it less broken but it still has limitations. Anything will have limitations though, so you can't expect a perfect theory/model.

I like the idea of a BSR, but not sure how you can get the data needed to calculate it for various managers. The only hope would be for managers to calculate it themselves and distribute it. I have no how likely that would be though. It seems to inherintly favor long only strategies though as theoretically short positions have unlimited risk.
stopsareforwimps
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Sun Oct 10, 2010 1:47 am
Location: Melbourne Australia

Re: Is Modern Portfolio Theory Dead?

Post by stopsareforwimps »

DPH wrote:Here is an amusing thread that takes (good) aim at Modern Portfolio Theory. ...
Beta, the child of MPT, explains nothing. Correlations are not stable. In particular they are not stable in the face of changing 'volatility'. Return distributions are not normal by factors of millions in the tails. And if you are computing millions of correlations from tens of thousands of data points, almost all binary digits in the correlations will be noise.

What do? You have to make some adjustments. Here are some things to consider:

1. Compute your correlations based only on days with big moves. Ignore correlations on days with small moves, or give them only a light weight.

2. Apply a Bayesian skepticism factor to the correlations and returns. For example you could apply the techniques from these or similar articles: "The Optimizer’s Curse: Skepticism and Postdecision Surprise in Decision Analysis" by James E. Smith, Robert L. Winkler, or Richard Michaud "The Markowicz Enigma: 'Is Optimized' Optimal?" Financial Analysts Journal Jan-Feb 1989 pp 31-42.

If you start with the premise that everything is 100% correlated and that all expected returns are equal and are considerably lower than "past returns", and only gradually move from that view, you are likely to do better than by plugging historical returns and correlations straight into the optimizer.

What to use for volatility? I think it was Jarrod Wilcox who suggested that future volatility is better predicted by past returns than by past volatility. You might want to consider doing that.

3. There is also the information ratio.

If you have an edge, the more independent bets you can make that exploit that edge, the more certain victory will be. The noise will fade into the background and your edge will become more and more prominent.

Let's say you have a coin that returns heads 55% of the time. The more you flip if the more likely it is that you will win compared to making random choices:

Code: Select all

Flips Probability of winning
1 0.55
3 0.57475
11 0.63312
101 0.84375
1001 0.99924
This is a different thing from diversification. It assumes that the bets are independent. More precisely that the edge in each bet is independent.

4. You can compute factor models and diversify the factors rather than the individual assets. As there are far fewer factors (usually about a dozen) that come out, there is less chance of your correlations being highly sophisticated noise.
svquant
Roundtable Knight
Roundtable Knight
Posts: 126
Joined: Mon Nov 07, 2005 3:39 am
Location: Silicon Valley, CA
Contact:

Post by svquant »

Actually MVO is worse as many people have noted in both academic and industry studies. Due to the noise in financial markets and lack of the ability to give accurate forecasts especially of future returns (mean part) MVO by many is considered an error amplification process, i.e. giGO. So people look towards minimum variance portfolios (drop the mean part) or closer to what many CTAs do - "risk parity" portfolios.
RedRock
Roundtable Knight
Roundtable Knight
Posts: 944
Joined: Fri Jan 30, 2004 3:54 pm
Location: Arizona

Post by RedRock »

the portfolio was uncorrelated until it wasn't.
the systems were uncorrelated until they weren't
the CTAs provided nice diversity against one another, until they didn't.
etc
Chris67
Roundtable Knight
Roundtable Knight
Posts: 1052
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Post by Chris67 »

SEEMS We always come back to the same problem in any investment technique which is correlation
watching London cocoa trade hand in hand with the bund/ btp spread - you know you have problems !!

Surely when one tests a large portfolio over 30 years of data then there have been many times in history when correlations have gotten extreme and the results therefore indicate this to you = also you need times of high correlations on your winning positions to make the really big returns !! but as stated elswehere it does seem things are getting more correlated and its not hard to understand why

Another way to judge how bad MPT is , is by looking at the long term returns of those who preach and use it i.e. Pension Funds - their all bankrupt so not hard to assess the true value of this model
rabidric
Roundtable Knight
Roundtable Knight
Posts: 243
Joined: Mon Jan 09, 2006 7:45 am

Post by rabidric »

MPT doesn't work becaue the core assumption is that "this time" will be the same as "last time".


In very long histories, patterns do repeat, hence the benefit of very long backtest windows relative to the walk forward period, but in a one for one comparison with "the last time", "this time" IS different, sometimes very substantially....
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

The idea of diversification is not stupid, far from it. The stupid part is where you gear up to the eyeballs in reliance on historic correlations which have fooled you into taking more risk than you ought. On a separate thread Sluggo quite rightly reminds us in a timely fashion that if you can't sleep at night take some money off the table, reduce risk.

I have long been skeptical of combining futures systems in the hope that zigs and zags will balance out in the future as they have in the past - or at least skeptical of the bit about increasing leverage in reliance on the past.

I am a firm believer in diversification but sometimes fail to take my own advice. I must try a lot harder on that score. What MFGI has rubbed in my face (and I'm sure a lot of others) is that diversification is necessary in every sense of the word. Diversification over asset classes is not a waste of time: we have to store money somewhere and better not to be all in the same place. So, back in a way to the Harvard investing method that friend Faber writes so much about. You need many different custodians, fund managers, types of property. Real property, physical property, ultra diversified index tracking ETFs (which invest in actual stock not absurd swaps or single bank- risk notes) and so on and so on.

It is not dumb at all. The dumbness comes in over-leverage. Fine, you will still get times when all these lose money at the same time but you are unlikely to get completely f*****. You will still have some lunch, even if it is not a free one.
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

This is also why I am a big believer in the significance of margin-to-equity ratios. Once again, given two track records, and all other things being equal, the approach that used less margin (probably) had less potential for risk.
I am pondering the same questions myself in relation to managing portfolio risk. Impossible to do in back - testing for reason so often discussed here re the difficulties of obtaining meaningful historical margin figures but intuitively less margin use should indeed equate in general to less risk. Unless it is all intra day trading.

I have been tinkering with risk limits using daily standard deviation of equity returns but I am talking of simple and not over-complex trend following where again a lower standard deviation should very roughly mean less risk. Cut positions when daily Std Dev reaches certain levels. Or again, use some measure of ATR on an individual position basis. As well of course as using some sort of sector risk control however ineffectual that can be from time to time.

In the end it all boils down to risk. Risk less, sleep easier, survive.

The old adage about taking care of risk has become ever more important.
White Cube
Roundtable Fellow
Roundtable Fellow
Posts: 98
Joined: Fri Oct 20, 2006 2:25 pm
Location: London, UK

Post by White Cube »

Impossible to do in back - testing for reason so often discussed here re the difficulties of obtaining meaningful historical margin figures
If we cannot get meaningful historical figures maybe we could calculate them?
  • We could calculate margin with the algorithm used by standardized portfolio analysis of risk (SPAN). But I guess this would be very challenging.
  • Margin depends on volatility. Maybe we could use an approximation: I remember reading an article by D Hoffman where he said:
    I have approximated margin based on 2 times the 5 day average true range multiplied by the point value, and then averaged that over a period of 5 years.
  • This approximation should be much easier to code in TB
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

Perhaps it would be useful to substitute "Volatility" for "Margin", based on these assumptions
  • When exchange officials lower and (especially) when they raise margin requirements, they are responding to changes in volatility.
  • Unlike Margin, Volatility can be measured simply from a price series, so you've got it going back 30+ years.
  • For each market or "Market Complex" (the beans/meal/beanoil complex, the crude/heating/gasoline complex, the hogs/corn complex, ...), you can do a data-fitting regression in Excel to see what the historical relationship has been, between Margin Requirements and Volatility. Maybe the volatility measurement that correlates best with Margin, is 10-day-ATR. Maybe it's HH(20) - LL(20), the height of the 20 day Donchian Channel.
It isn't perfect but it IS objective and it'll get you some of what you want, perhaps most of what you want, in the absence of historical data for margin requirements. Once you find the volatility measure that fits best (has the highest goodness-of-fit r2 coefficient), you automatically get a two parameter linear model (predictor) of margin requirement vs volatility. Done. Voila.

You could do all the calculations off-line (i.e. using software other than Blox) and store your calculated-from-volatility PseudoMargin numbers in files. What Blox's esteemed competitor "M" calls companion files. This way your Blox systems don't have to do all those messy calculations, so your systems will run faster. Especially useful for large Stepped Parameter simulations comprising hundreds or thousands of Tests.

Your Blox code would read the companion files (one per instrument) using the instrument.LoadExternalData() function.

It's exactly the same mechanism you would use if you did have historical data of actual margin requirements for each date of past history. You're just plugging in estimated numbers rather than actual numbers.
Attachments
oldpost_154.png
oldpost_154.png (47.02 KiB) Viewed 18359 times
Chuck B
Roundtable Knight
Roundtable Knight
Posts: 481
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B »

Instead of margin to equity, I've always monitored notional value to account equity both on an instrument basis and total basis. I've always felt it gives a "potentially" better indicator of ultimate risk exposure than volatility although I use volatility to size positions and scale out to hold volatility risk to a maximum level.

STIRs are somewhat special cases since the notional value of the contracts are generally very large in comparison to its volatility (or margin). It's "easy" to ramp up a very large notional values where "normal" volatility-based risk measures would indicate a "small" risk. Monitoring notional position value in STIRs was an eye opening experience way back when I first looked at what my system was carrying in open positions.

It's the unexpected discontinuities where stuff like volatility risk measurements can be blown out of the water. Look at most any trend system on T-bill or Eurodollar futures through Oct 1987 for example.

Another time that crops to mind is intraday trading in the S&P in October of 1998 when the Fed cut rates after the bonds were closed, and there was a fantastic discontinuity to the upside. One second ATR (or any vol measure) on 3min charts was about 2 handles and then a 50 handle whoosh happened....25ATR sweep in a "fast market". Fills out of the pit were a mess that day for some.
White Cube
Roundtable Fellow
Roundtable Fellow
Posts: 98
Joined: Fri Oct 20, 2006 2:25 pm
Location: London, UK

Post by White Cube »

Here are the limited results of comparing S&P 500 future historical vs calculated margin from 1987 to 2009 thanks to the historical data provided here: viewtopic.php?t=6692&highlight=historical+margin+data

You will find the blox I used for margin estimates in the Blox MarketPlace. I used the formula mentioned in my previous post: Margin has been approximated based on 2 times the 5 day average true range multiplied by the big point value. The result has then been averaged over a period of 5 months.

Correlation coefficient: 0.671766

(Note on the second figure that the calculated margin follow the historical more closely from November 3 1997. That is the date of the S&P split. Correlation coefficient : 0.7886)
Attachments
Historical vs calculated margin.jpg
Historical vs calculated margin.jpg (86.8 KiB) Viewed 18277 times
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

Chuck B wrote: STIRs are somewhat special cases since the notional value of the contracts are generally very large in comparison to its volatility (or margin). It's "easy" to ramp up a very large notional values where "normal" volatility-based risk measures would indicate a "small" risk.
Indeed but what is the answer here do you think?
Chuck B
Roundtable Knight
Roundtable Knight
Posts: 481
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B »

AFJ Garner wrote:
Chuck B wrote: STIRs are somewhat special cases since the notional value of the contracts are generally very large in comparison to its volatility (or margin). It's "easy" to ramp up a very large notional values where "normal" volatility-based risk measures would indicate a "small" risk.
Indeed but what is the answer here do you think?
As with many things along these lines, I don't think there is a specific answer. What I've done is simply monitor notional to equity ratios coupled with ATR to arrive at my own understanding of a STIR position. There have been a few times when the position size presented by my system was past my choke point (even though margin was very small of course), and I've used discretion to reduce size to the sleeping point.

Some of the craziest moves in STIRs have of course happened when some crazy stuff was going on that one couldn't help but notice, so using some common since discretion makes sense to me. For example, one of the biggest overnight moves in the Eurodollar happened from Monday of 87 crash into Tuesday. I recall Eckhardt saying that he had a large short ED position on, but that the market was hardly moved on Monday of the crash, so his gut said to simply get out and he did.

Another example is the Short Sterling in 1992. A few months before the BP broke out of its trading bands, there was a nasty gap that smoked out nice shorts in trend systems with a huge exit slip (chart 1 below). Next as yet again trend systems were short during the summer, some crazy up and down volatility started whacking the market around (chart 2). So action like that coupled with knowledge of your net notional exposure can help one's "gut" feel.

Two weeks into that big volatility expansion and all hell broke loose when the BP busted out (chart 3). Note the huge range bar at the low for the move was a screaming signal to get out (to me, as I use discretion at rare times on both exits and position sizing) and even to get ready to flip to a long-side bias with very tight stops (i.e. using any rising pattern to setup an entry on a long with a close exit -- it's either going to prove itself right away or not type of trade). I'll attach three charts showing this market that year.
Attachments
Large range wick at bottom, same open as close, then whoosh...
Large range wick at bottom, same open as close, then whoosh...
ss_3.gif (15.9 KiB) Viewed 18201 times
Vol expansion begins...
Vol expansion begins...
ss_2.gif (18.47 KiB) Viewed 18201 times
Spring time massive gap up...
Spring time massive gap up...
ss_1.gif (17.26 KiB) Viewed 18201 times
Aaron01
Roundtable Knight
Roundtable Knight
Posts: 113
Joined: Thu Jun 02, 2011 1:21 pm

Post by Aaron01 »

Chuck B wrote:
AFJ Garner wrote:
Chuck B wrote: STIRs are somewhat special cases since the notional value of the contracts are generally very large in comparison to its volatility (or margin). It's "easy" to ramp up a very large notional values where "normal" volatility-based risk measures would indicate a "small" risk.
Indeed but what is the answer here do you think?
As with many things along these lines, I don't think there is a specific answer. What I've done is simply monitor notional to equity ratios coupled with ATR to arrive at my own understanding of a STIR position. There have been a few times when the position size presented by my system was past my choke point (even though margin was very small of course), and I've used discretion to reduce size to the sleeping point.
I would think that some form of volatility or std dev would be best for STIRs, since a couple hundred bucks of margin can control the notional value of half a million dollars.

Perhaps something like OI or V would also be helpful, as it may be something of an indication as to when prices will break, particularly if stuck in a range.

Admittedly, I've not done any research on this, as of yet, and am just talking off the cuff :lol:
Rush
Roundtable Fellow
Roundtable Fellow
Posts: 50
Joined: Wed Dec 31, 1969 7:00 pm

Post by Rush »

Indeed but what is the answer here do you think?
You may consider exploring non-linear position sizing methods in respect of ATR.
For what matters, physicists say that for extremely low dimensions laws are different.
stopsareforwimps
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Sun Oct 10, 2010 1:47 am
Location: Melbourne Australia

Re: Is Modern Portfolio Theory Dead?

Post by stopsareforwimps »

stopsareforwimps wrote:
DPH wrote:Here is an amusing thread that takes (good) aim at Modern Portfolio Theory. ...
Correlations are not stable. In particular they are not stable in the face of changing 'volatility'.
I ran a random portfolio of futures contracts through a program to analyze the correlations as a function of the volatility. The attached animated image shows how the correlations changed as the volatility changed.

MPT would have it that they do not change.

I will post the correlations csv file if people are interested.

How to read the graphic... you can stop the animation in most browsers by pressing the Esc key.

Green means a strong negative correlation (-1 to -0.5). White means a weak negative correlation (-0.5-0). Yellow means a weak positive correlation (0-0.5). Red means a strong positive correlation (0-5-1). Note every contract is 100% correlated with itself, thus the red line from top left to bottom right.

The image is animated. It starts with least volatile days (percentiles 100-50), then the next frame is percentiles 50-25, then 25-12.5 etc. The last frame is the most volatile 0.7% of days. This frame is held for 1/2 a second, while the other frames are only held for 1/10 of a second.

The pictures do not say which contracts are which. However you can see the equity index contracts in a block on the bottom right. The green line more of less surrounding them is the VIX. Currencies are top left then energy then grains long bonds meats metals short bonds softs then stock. At the bottom left you can see some correlation between markets and currencies. About a third of the way in at the bottom are some of the bond and interest rate markets which are green as they tend to be inverse to stock markets.

The graphic is not quite symmetrical because of the way I select which days are volatile - it is based on the movement in the contract at the left not the one at the top.

You can see the volatility increase as the animation proceeds as more red and yellow appears. After about a second the animation repeats.

Here are the averages of the absolute values of the correlations by (lower bound of) the percentile range.
Low end of Percentile Range (0=most volatile days),Avg Correlation Abs Size
0.78,0.310481684782609
1.56,0.297045092155009
3.13,0.242728638941399
6.25,0.205787401937618
12.5,0.201091553638941
25,0.151293455812854
50,0.117171752126654
100,0.0963124323671498
Between the 100th and 50th percentiles - the most stable days - the correlations are only about 9.6%. On the most volatile days (the top percentiles between 0.7% and 0%) the average correlations are 31%, over three times as high. So anyone using a single correlation is going to get wildly wrong results in real life.
Attachments
correlation-animation.gif
correlation-animation.gif (197.79 KiB) Viewed 17530 times
Eventhorizon
Roundtable Knight
Roundtable Knight
Posts: 229
Joined: Thu Jul 08, 2010 2:36 pm
Location: Boulder, CO
Contact:

Post by Eventhorizon »

Nice work stopsareforwimps.

Here's something similar I did a while ago. Almost the same idea (contracts filtered and ordered by volatility on the vertical axis, dependent contracts on the horizontal). I used a different set of contracts and looked at only 3 cases: 5% most volatile days upwards, downwards and absolute value. The ordering of the contracts is essentially lowest (-ve) in the top right average correlation to highest (+ve) in bottom left.

Another issue for correlations is to [url=http://beyondtheblueeventhorizon.blogsp ... arket.html]explore the change in characteristics over time[/ur]. It seems there is some evidence to support the risk on - risk off concept and this paradigm has been gradually evolving over the last 10 years. Perhaps this sheds some light on the idea of system longevity.

Finally, I am working on understanding how to do Minimum Covariance Determinant analysis as a means of better identifying extremes (outliers) in price data. The basic idea being to find a robust way of sub-setting the data into "normal" and "extreme" rather than just picking a percentile for a given contract. I plan on posting about it at some point.

How did you do the animation? I have been trying to get animated gif's out of R without success so far.
Post Reply