I probably should have been clearer. The concept is simple. Finding un/anti correlated equity curves that improve portfolio gain/pain metrics is not.LeviF wrote:The concept is simple. Finding un/anti correlated equity curves is not.
Blending noncorrelated (or anticorrelated) equity curves

 Roundtable Knight
 Posts: 229
 Joined: Thu Jul 08, 2010 2:36 pm
 Location: Boulder, CO
 Contact:
LeviF wrote:
Consider Sluggo's proposition:
Jez,
Here's another equation builder you might like (but you need LaTeX installed in your blog code): LaTeX Editor
Great thread!
Edited for typos / clarity.
Allocation is what's missing. The excellent expositions above are using equal allocations to make the math easier and to make a particular point very clear.There must be some other factors at play here aside from just equity curve correlation. I can come up an uncorrelated system with negative expectancy that hurts overall portfolio performance. Or what about two systems with 1 correlation (system 2 takes the opposite trade of system 1).
Consider Sluggo's proposition:
Let's build an example where one equity curve has standard deviation of 0.01 and another has 0.1 (10 times larger). Let's say their correlation is 0.7. A portfolio allocation of 93.5% to the first system and 6.5% to the second results in a minimum variance portfolio with a standard deviation of 0.0007!if ECY is a lovely nice equity curve and ECZ is a complete and total stinker, with gutwrenching drawdowns, psychotic swings, and enormous hand over fist losses, then adding ECZ to ECY will not increase your happiness! It will reduce both pain AND gain, and you won't like the result.
Jez,
Here's another equation builder you might like (but you need LaTeX installed in your blog code): LaTeX Editor
Great thread!
Edited for typos / clarity.
In the special case of blending exactly two equity curves, the optimum allocation has a simple closedform solution, shown below. For Eventhorizon's example, the optimum allocation to ECA, the low volatility equity curve, is 93.043% (107/115) of the account. ECB, the high volatility equity curve, gets 6.957% (8/115).Eventhorizon wrote:Let's build an example where one equity curve has standard deviation of 0.01 and another has 0.1 (10 times larger). Let's say their correlation is 0.7. A portfolio allocation of 93.5% to the first system and 6.5% to the second results in a minimum variance portfolio with a standard deviation of 0.0007!
Edit  added later  Let's do a couple of sanity checks
 When ECB's volatility is zero, the formula says the optimum allocation equals 1 for B and zero for A. Sounds right.
 When ECB's volatility is infinity, the formula says the optimum allocation equals zero for B and 1 for A. Sounds right.
 When ECB and ECA have the same volatility, the formula says the optimum allocation equals 1/2 for B and 1/2 for A, no matter whether correlation (rho) is +1, 0, 1, or anything in between. Sounds right, yes?
Last edited by sluggo on Mon Feb 21, 2011 7:37 pm, edited 1 time in total.

 Roundtable Knight
 Posts: 229
 Joined: Thu Jul 08, 2010 2:36 pm
 Location: Boulder, CO
 Contact:
By way of explanation ... the differences in solution arise because I simulated 2 series of length 1000 with the characteristics mentioned above and jammed them through some code I use to calculate the eigendecomposition of a covariance matrix of real multiple data series!
A sledgehammer compared to Sluggo's elegent closedform solution!
Reminds me of a joke:
Several scientists were all posed the following question: "What is 2 Ã— 2 ?"
The engineer whips out his slide rule (so it's old) and shuffles it back and forth, and finally announces "3.99".
The physicist consults his technical references, sets up the problem on his computer, and announces "it lies between 3.98 and 4.02".
The mathematician cogitates for a while, then announces: "I don't know what the answer is, but I can tell you, an answer exists!".
Philosopher smiles: "But what do you mean by 2 Ã— 2?"
Logician replies: "Please define 2 Ã— 2 more precisely."
Medical Student : "4." All others looking astonished : "How did you know?" Medical Student : "I memorized it."
A sledgehammer compared to Sluggo's elegent closedform solution!
Reminds me of a joke:
Several scientists were all posed the following question: "What is 2 Ã— 2 ?"
The engineer whips out his slide rule (so it's old) and shuffles it back and forth, and finally announces "3.99".
The physicist consults his technical references, sets up the problem on his computer, and announces "it lies between 3.98 and 4.02".
The mathematician cogitates for a while, then announces: "I don't know what the answer is, but I can tell you, an answer exists!".
Philosopher smiles: "But what do you mean by 2 Ã— 2?"
Logician replies: "Please define 2 Ã— 2 more precisely."
Medical Student : "4." All others looking astonished : "How did you know?" Medical Student : "I memorized it."
i was wondering:
does the use of root mean square measures(i.e. StDev) induce any parculiar effects when doing all these variance analysis for weightings?
i.e. if we used a Rescaled MAD or AAD (median and average absolute deviations, respectively), would that be more sensible?
I understand that Stdev responds better to the wilder stuff which is most likely the bits that matter to us in terms of "pain", but AAD is more arithmetically consistent over a large dataset with frankly completely unknown underlying population distributions(returns of asset timeseries transformed by application of tradelogic).
personally it is nice to understand these principles, but i wouldn't try and engineer this stuff too much, it is rather too "top down" to my taste.
 i prefer to plant a nice mix of good seeds and accept the wood at the end, than try and engineer a perfectly even forest canopy with heavy manipulation.
does the use of root mean square measures(i.e. StDev) induce any parculiar effects when doing all these variance analysis for weightings?
i.e. if we used a Rescaled MAD or AAD (median and average absolute deviations, respectively), would that be more sensible?
I understand that Stdev responds better to the wilder stuff which is most likely the bits that matter to us in terms of "pain", but AAD is more arithmetically consistent over a large dataset with frankly completely unknown underlying population distributions(returns of asset timeseries transformed by application of tradelogic).
personally it is nice to understand these principles, but i wouldn't try and engineer this stuff too much, it is rather too "top down" to my taste.
 i prefer to plant a nice mix of good seeds and accept the wood at the end, than try and engineer a perfectly even forest canopy with heavy manipulation.
rabidric wrote:does the use of root mean square measures(i.e. StDev) induce any parculiar effects when doing all these variance analysis for weightings? i.e. if we used a Rescaled MAD or AAD (median and average absolute deviations, respectively), would that be more sensible?
Certainly, feel free to do it that way. In fact if you operated a Fund of Funds, you could emphasize this detail in your marketing presentations and literature, thereby showing how you stand apart from the herd of other FoF sellers. You'd probably want to exhibit a couple of realworld examples where Markowitz's Nobel Prize winning variancebased approach yielded noticeably inferior results compared to your Rescaled MAD / AAD based approach. Just to drive home the point that you're not merely arguing a theoretical distinction, but rather putting more riskadjusted returns in your investor's pockets, in the real world, here and now.
lol, thanks for the sarcy reply. i should have expected that i guess. I wonder if that is your equivalent to "you could take that to your trading tribe".
problem is, this is my goddamn trading tribe!!!!
ah well.
I still think the whole approach outlined in the OP , whilst theoretically valid, is rather too contrived, though i don't discount it's benefits when they are gained indirectly.
My woodland/forest canopy based analogy to this can be further expanded that i will type up in full if people would like to hear it. I think it can intuitively expose the danger of relying on this approach.
Since you are all quite enlightened anyway it seems , i'll leave it for now tho
maybe if the thread gets to another page.....
problem is, this is my goddamn trading tribe!!!!
ah well.
heh, i've read those papers too. I also ran simulations on my own "deliberately nasty" synthetic distributions. in the end it didn't make any real difference, sometimes it was better, sometimes not. couldn't be arsed to test for confidence, as that seemed to me to be squeezing bollox out of a bag of nuts at that point.sluggo wrote:You'd probably want to exhibit a couple of realworld examples where Markowitz's Nobel Prize winning variancebased approach yielded noticeably inferior results compared to your Rescaled MAD / AAD based approach.
I still think the whole approach outlined in the OP , whilst theoretically valid, is rather too contrived, though i don't discount it's benefits when they are gained indirectly.
My woodland/forest canopy based analogy to this can be further expanded that i will type up in full if people would like to hear it. I think it can intuitively expose the danger of relying on this approach.
Since you are all quite enlightened anyway it seems , i'll leave it for now tho
maybe if the thread gets to another page.....
actually screw my bullshit analogy, Jez Liberty's EXCELLENT post in this thread says it all just fine really.
We know that over the long haul correlations of even combined equity curves tend to be about as stationary as speedy gonzalez on speed, bouncing around inside the large hadron collider.
As such, if we drink too much koolaid from this particular trough, and get cocky enough to base our leverage on derived pain statistics of these carefully selected equity curves, then we are likely to choke when the percieved benefit of such alchemy displays large sensitivity to changes in "rho", and our predicted pain statistic goes out the window.
Having said that, I don't think there are many on this forum who disagree.
Enjoy the indirect benefits of diversification when they occur(less pain on a day to day basis, better compounding etc), but don't chase them too much(base leverage choices on them too heavily). seems like a good rule to live by.
Good discussion though, credit to the way the OP and others have laid out all the underlying reasoning in such clear homemade formats. Above all I think it has shown me that I need to up my game in the data manipulation and presentation stakes!
We know that over the long haul correlations of even combined equity curves tend to be about as stationary as speedy gonzalez on speed, bouncing around inside the large hadron collider.
As such, if we drink too much koolaid from this particular trough, and get cocky enough to base our leverage on derived pain statistics of these carefully selected equity curves, then we are likely to choke when the percieved benefit of such alchemy displays large sensitivity to changes in "rho", and our predicted pain statistic goes out the window.
Having said that, I don't think there are many on this forum who disagree.
Enjoy the indirect benefits of diversification when they occur(less pain on a day to day basis, better compounding etc), but don't chase them too much(base leverage choices on them too heavily). seems like a good rule to live by.
Good discussion though, credit to the way the OP and others have laid out all the underlying reasoning in such clear homemade formats. Above all I think it has shown me that I need to up my game in the data manipulation and presentation stakes!
I hope folks don't misinterpret Jez's chart. He started with the assumption that all N of the equity curves had the same correlation to each other  which simplifies the math tremendously. But please don't interpret his bottom most red line as some kind of a "Sound Barrier" which is impossible to breach. It's merely the limitation of this plot plus this assumption, nothing more.
To illustrate that it is indeed possible to do better, I've attached a set of four equity curves. They all have the same volatility but they don't all have the same correlation to one another; some correlations are positive (but small), while other correlations are negative (and large). When you blend these four equity curves together (with 25% allocation to each), the blend has a volatility of 0.13. This is marked with a black dot on the chart. It's way beyond the bottom most red line (the red line crosses N=4 at volatility=0.50). There is no sound barrier.
In fact if you think about it for a few minutes, you'll be able to invent a procedure for generating N different equity curves that blend together to give volatility (reduction) V, for any N and V you care to specify. This demonstrates that there is no "Sound Barrier"  you've got an algorithm to generate N equity curves that blend together giving any V you choose. Even zero. (Eventhorizon's post may get you started).
In the spreadsheet, the four equity curves are called "Lucy", "Ricky", "Ethel", and "Fred".
To illustrate that it is indeed possible to do better, I've attached a set of four equity curves. They all have the same volatility but they don't all have the same correlation to one another; some correlations are positive (but small), while other correlations are negative (and large). When you blend these four equity curves together (with 25% allocation to each), the blend has a volatility of 0.13. This is marked with a black dot on the chart. It's way beyond the bottom most red line (the red line crosses N=4 at volatility=0.50). There is no sound barrier.
In fact if you think about it for a few minutes, you'll be able to invent a procedure for generating N different equity curves that blend together to give volatility (reduction) V, for any N and V you care to specify. This demonstrates that there is no "Sound Barrier"  you've got an algorithm to generate N equity curves that blend together giving any V you choose. Even zero. (Eventhorizon's post may get you started).
In the spreadsheet, the four equity curves are called "Lucy", "Ricky", "Ethel", and "Fred".
 Attachments

 Ecurves_Four.xls
 Four equity curves that blend nicely
 (58.5 KiB) Downloaded 256 times

 Black dot at N=4, V=0.13 shows the volatility of this blend of 4 equity curves
 Mod_Four.png (136.42 KiB) Viewed 7922 times

 Roundtable Knight
 Posts: 123
 Joined: Tue Nov 03, 2009 8:49 am
 Location: London
 Contact:
Very true. I think I would still be working out the maths if I had not made this simplifying assumption!sluggo wrote:I hope folks don't misinterpret Jez's chart. He started with the assumption that all N of the equity curves had the same correlation to each other  which simplifies the math tremendously.
The chart/assumption definitely has some limitations, an important one of them being that only positive correlations were plotted on the chart (the reason I did not plot negative correlations is because the formula errors out after N = a small number, depending on the value of rho).sluggo wrote:But please don't interpret his bottom most red line as some kind of a "Sound Barrier" which is impossible to breach. It's merely the limitation of this plot plus this assumption, nothing more.
As I mentioned at the end of the post, by adding negatively correlated equity curves, you could do much better (than the red line). Heck, even theoretically obtain a zero volatility!
The chart expands on bobsyd and sluggo's "Vol vs N" charts (which assume rho = 0  in effect plotting the same red line in the chart). I wanted to check: "What if can not produce multiple uncorrelated equity curves? Would I benefit in a similar, albeit less favorable way, by throwing up hundreds of strongly correlated equity curves?"Jez Liberty wrote: We could even actually calculate, for a given number of equity curves N, the required correlation between all equity curves (assuming equal volatility and correlation) to reach a blended volatility of 0: rho = 1 / (N1)
A few examples:
N=2 > rho = 1
N=3 > rho = 0.5
N=5 > rho = 0.25
N=11 > rho = 0.1
N=26 > rho = 0.04
N=51 > rho = 0.02
N=101 > rho = 0.01
...
The chart seems to point to the answer being a "No: focus on finding low/negatively correlated equity curves, rather than throwing hundreds of nearidentical ones", which basically points back to the point in your initial post, sluggo.
As the negative rho example and sluggo's update clearly show, picking only a few equity curves can result in a more dramatic vol reduction.
"It's quality, not quantity!"
I have actually updated the chart to plot the negative rhos as well. Some curves go slightly "offtangent" towards the bottom, as the errors start creeping in just before 0, but it definitely shows that there is no "sound barrier".
I have also plotted sluggo's "special blend" of Lucy, Ricky, Ethel and Fred (black dot), which is close to rho=0.3, if all curves had the same correlation. They do not, but their average correlation is not far off at 0.24, if my calcs are correct. Maybe the chart/formula is not a bad estimator even in cases where curves do not display all equal correlations (the estimation error would depend on the standard deviation of the correlations I presume)
 Attachments

 ChartVolvsNrho2.png (176.18 KiB) Viewed 7895 times

 Roundtable Knight
 Posts: 2040
 Joined: Fri Apr 25, 2003 3:33 pm
 Location: London
 Contact:
Quite. Much as I applaud and appreciate the research in the above posts ....................I suspect that real life long term success (in investment or otherwise) probably also requires a healthy dose of scepticism and involves a large measure of chance and an acceptance that the most carefully laid plans are at best only a rough sketch drawn with a blunt stick in ever moving sands.rabidric wrote: As such, if we drink too much koolaid from this particular trough, and get cocky enough to base our leverage on derived pain statistics of these carefully selected equity curves....................
Or am I just feeling more than usually puzzled by a world largely governed by tribal thugs where "truth" and "reason" seem so often curiously lacking and which is emphatically not inherited by the meek.

 Roundtable Knight
 Posts: 2040
 Joined: Fri Apr 25, 2003 3:33 pm
 Location: London
 Contact:
"Determinism" is the name of a broader philosophical view that conjectures that every type of event, including human cognition (behaviour, decision, and action) is causally determined by previous events. In philosophical arguments, the concept of determinism in the domain of human action is often contrasted with free will. The argument called indeterminism(otherwise "nondeterminism") negates causality as a factor and opposes the deterministic argument. Determinists believe the universe is fully governed by causal laws resulting in only one possible state at any point in time.
Once I have found the Philosopherâ€™s Stone and achieved Gnosis I will be able to guide the assembled group further towards enlightenment and truth.As a 32yearold assistant professor of mathematics at Helicon University, Hari Seldon visits Trantor from his native Helicon to attend the Decennial Mathematics Convention. He presents a paper which indicates that one could theoretically predict the Galactic Empire's future. At first, Seldon has no idea how this could be done in practice, and he is fairly confident that no one could actually fulfill the possibility. Shortly after his presentation, he becomes a lightning rod for political forces who want to use psychohistory for their own purposes. During his flight to escape the various political factions, he discovers how psychohistory can be made a practical science.
I may be some time, so please don't hold your collective breath.
loving that hari seldon story AFJG. lmao!
also loving jez's second post as much as his first they highlight the relationships brilliantly and provide good meat for further discourse:
if anything the new chart shows better the perils I am arguing the potential adverse dV/dRho gets nutty for small portfolios with nice anticorrelations that then mutate in the future.
The "quality" , if it changes for the worse, can drop the "benefits" dramatically.
"quantity", while being a quality all of it's own(stalin?), is perhaps less susceptible , but even so, still suffers the dV/dRho problem.
Which brings us to the nub of it all: the greater the benefits of "s[m]oothing the pain" that derive from this methodology, the greater the undoing when it goes wrong. The going wrong part will depend on the stationarity of the correlations etc YMMV, But even if it only goes wrong 0.1% of the time, that could still be once every 3 years.
In the end, it is just another optimization problem to be abused:
the smaller the number of series being optimized, the lower the stability, the larger the number being optimized, the more mediocre the performance*, and no matter what you do, shit will always happen.
Leverage will decide how much you get hurt.
P.S. I don't mean to be a Jezebel proclaiming it is all useless. In reality you might be surprised at the choices i have madeI actually do prefer quality over quantity personally, and enjoy taking a swing for the fences. I just think provided it doesn't badly cripple your ability to play the game, one can never be too long on scepticism!
* see new "Subset Trading" thread.
also loving jez's second post as much as his first they highlight the relationships brilliantly and provide good meat for further discourse:
if anything the new chart shows better the perils I am arguing the potential adverse dV/dRho gets nutty for small portfolios with nice anticorrelations that then mutate in the future.
The "quality" , if it changes for the worse, can drop the "benefits" dramatically.
"quantity", while being a quality all of it's own(stalin?), is perhaps less susceptible , but even so, still suffers the dV/dRho problem.
Which brings us to the nub of it all: the greater the benefits of "s[m]oothing the pain" that derive from this methodology, the greater the undoing when it goes wrong. The going wrong part will depend on the stationarity of the correlations etc YMMV, But even if it only goes wrong 0.1% of the time, that could still be once every 3 years.
In the end, it is just another optimization problem to be abused:
the smaller the number of series being optimized, the lower the stability, the larger the number being optimized, the more mediocre the performance*, and no matter what you do, shit will always happen.
Leverage will decide how much you get hurt.
P.S. I don't mean to be a Jezebel proclaiming it is all useless. In reality you might be surprised at the choices i have madeI actually do prefer quality over quantity personally, and enjoy taking a swing for the fences. I just think provided it doesn't badly cripple your ability to play the game, one can never be too long on scepticism!
* see new "Subset Trading" thread.

 Roundtable Knight
 Posts: 2040
 Joined: Fri Apr 25, 2003 3:33 pm
 Location: London
 Contact:
Agreed. The work is illuminating and instructive. Like all properly conducted and intelligent backtesting.rabidric wrote:I don't mean to be a Jezebel proclaiming it is all useless.
You do the work, and then hope that what your have observed as having been effective in the past, is a workable method for at least some period of time going forward.
It is all you can do.
Until Hari perfects psychohistory and/or the determinists are proved right and come up with something similar.
Re: Blending noncorrelated (or anticorrelated) equity curve
sluggo wrote:Since volatility is the denominator of the Sharpe Ratio, as rho(AB) approaches 1.0, the denominator sigma(A+B) approaches zero and so the Sharpe Ratio approaches +infinity. However, after the manner of the Options Greeks, I suggest that you calculate the sensitivity d(Sharpe) / d(Correlation). You will find that the sensitivity goes to infinity as well. You might want to ponder that for a moment, in the context of InSample vs. OutofSample correlations.
yes. you wrote that.
yes. i agreed after reading it...
yes. i had to fill out many superfluous lines of text to express that!
it was worth highlighting for one small paragraph in an entire thread of well written content, it was by far the most important thing of all. It should probably be bolded in red and supersized and capitalized. but I know how you love to just leave nuggets of wisdom in plain sight but easily passed by!
yes. i agreed after reading it...
yes. i had to fill out many superfluous lines of text to express that!
it was worth highlighting for one small paragraph in an entire thread of well written content, it was by far the most important thing of all. It should probably be bolded in red and supersized and capitalized. but I know how you love to just leave nuggets of wisdom in plain sight but easily passed by!

 Roundtable Knight
 Posts: 1428
 Joined: Mon Dec 22, 2003 12:24 pm
 Location: Des Moines, IA
 Contact:
I just came across this blog post that made me think back to this discussion.
http://www.michaelcovel.com/2011/01/09/ ... thoutyou/
http://www.michaelcovel.com/2011/01/09/ ... thoutyou/

 Roundtable Knight
 Posts: 2040
 Joined: Fri Apr 25, 2003 3:33 pm
 Location: London
 Contact:
I did find this comment by Eckhardt gelled with my own thoughts concerning trading and investment:
This thread emphasizes with great clarity the benefits of diversification amongst lowly correlated instruments in a portfolio. My rather badly expressed comments are merely designed to say: "agreed but the relationships demonstrated between various instruments in the past may not hold in the future". Obvious, trite perhaps.
Comrade Lenin's comments on trading systems are just as applicable to a fund of funds. ( He does look just like my pal Vlad Ilyich, does he not?).
A colleague and I advise (inter alia) a fund of CTA funds. Our suggested constituents were naturally chosen partly on past performance figures and back testing (as well as on qualitative and more subjective considerations). Will the suggested mix still be the same in 10 years time? That seems unlikely. People will die, funds will go out of business, styles will drift.
None of this invalidates the research on this thread. It merely means that it will be an ongoing and never ending process. There is no finality, only change and adaptation.
In back testing one can devise smooth, low volatility equity curves pointing upwards and to the right. It will be harder to match those results going forward.
Thanks to TK for pointing out the interview. http://www.futuresmag.com/Issues/2011/M ... spx?page=3Aristotleâ€™s Ship of Theseus gradually had every one of its parts replaced. Was it still the Ship of Theseus? Over the decades every part of our systems has been modified. Itâ€™s our scientific approach that persists and supports the whole structure.
This thread emphasizes with great clarity the benefits of diversification amongst lowly correlated instruments in a portfolio. My rather badly expressed comments are merely designed to say: "agreed but the relationships demonstrated between various instruments in the past may not hold in the future". Obvious, trite perhaps.
Comrade Lenin's comments on trading systems are just as applicable to a fund of funds. ( He does look just like my pal Vlad Ilyich, does he not?).
A colleague and I advise (inter alia) a fund of CTA funds. Our suggested constituents were naturally chosen partly on past performance figures and back testing (as well as on qualitative and more subjective considerations). Will the suggested mix still be the same in 10 years time? That seems unlikely. People will die, funds will go out of business, styles will drift.
None of this invalidates the research on this thread. It merely means that it will be an ongoing and never ending process. There is no finality, only change and adaptation.
In back testing one can devise smooth, low volatility equity curves pointing upwards and to the right. It will be harder to match those results going forward.
â€œThe past is a foreign country, they do things differently thereâ€
I am a fan of research like this, and having run smaller variations of this for a grad school project, I can appreciate the computing power necessary to do it! Thank you sluggo.
As is mentioned by drm7, rabidric, and AFJ Garner, it is also good to take such research and keep an eye towards the market reality, which as one of my mentors has told me is that "the market will always move in the direction that will hurt the most amount of people"  another way of stating the contrarian approach to trading.
Yes, the biggest flaw is the changing correlations, which can switch on a dime for no apparent reason. Take oil and the S&P. At one point, the two were very positively correlated: economy is strong and growing economies drive more usage of oil: oil prices rise hand in hand with the stock market. Then like a light switch, oil surpasses some point where market sentiment suddenly sees that it is a hazard to future economic growth if it goes any higher. Now oil's correlation swings to the negative side: if oil is up, the stock market has pressure to the downside. Move forward past the recession, and once again, oil and equities are positively correlated: if stocks are picking up, that means the recession must be drawing to a close, so oil demand will pick up. Enter the Mideast turmoil, and oil passes that magic level that once again flips correlation the other way.
Similar to what can happen to spread traders. Ask calendarspread traders in the wheat pit from 23 years ago... You won't find them in the pits, because after 20+ years of trading the spread the same way, things changed. From the story, as I was told from a floor trader that stood in the pit, three of these gentlemen lost over a cumulative $50M+ and all "blew out" as they kept putting on larger and larger positions while they waited for a mean reversion that did not come. A small LongTerm Capital Management situation, if you will.
Kenyes: "The markets can remain irrational longer than you can remain solvent".
I also think a lot of quant firms found out in 2008 that even longterm correlations are not a guarantee of continued correlation. (CSI has a great correlation database with access for only $99, allowing you to test one asset versus another, or one against all, from 2 to 40 years. I find it very interesting and use it quite a bit.)
BUT.... until someone comes up with a better approach, then Markowitz optimization is still the way to approach the market and the way most, if not all, professionals still invest. I think the adjustment has been that more people now understand that correlations are not stationary, and that in times of crisis, they tend to all move towards 1.0; therefore as stated above by several members, do not base your maximum risk / leverage figures solely on the backtest and be cognizant of the fat tail risk. For example, if by investing in real estate and equities and corporate bonds together lowers your volatility by 50% over a single market, you probably may want to increase your exposure by somewhat less than the 100% that is implied...
As is mentioned by drm7, rabidric, and AFJ Garner, it is also good to take such research and keep an eye towards the market reality, which as one of my mentors has told me is that "the market will always move in the direction that will hurt the most amount of people"  another way of stating the contrarian approach to trading.
Yes, the biggest flaw is the changing correlations, which can switch on a dime for no apparent reason. Take oil and the S&P. At one point, the two were very positively correlated: economy is strong and growing economies drive more usage of oil: oil prices rise hand in hand with the stock market. Then like a light switch, oil surpasses some point where market sentiment suddenly sees that it is a hazard to future economic growth if it goes any higher. Now oil's correlation swings to the negative side: if oil is up, the stock market has pressure to the downside. Move forward past the recession, and once again, oil and equities are positively correlated: if stocks are picking up, that means the recession must be drawing to a close, so oil demand will pick up. Enter the Mideast turmoil, and oil passes that magic level that once again flips correlation the other way.
Similar to what can happen to spread traders. Ask calendarspread traders in the wheat pit from 23 years ago... You won't find them in the pits, because after 20+ years of trading the spread the same way, things changed. From the story, as I was told from a floor trader that stood in the pit, three of these gentlemen lost over a cumulative $50M+ and all "blew out" as they kept putting on larger and larger positions while they waited for a mean reversion that did not come. A small LongTerm Capital Management situation, if you will.
Kenyes: "The markets can remain irrational longer than you can remain solvent".
I also think a lot of quant firms found out in 2008 that even longterm correlations are not a guarantee of continued correlation. (CSI has a great correlation database with access for only $99, allowing you to test one asset versus another, or one against all, from 2 to 40 years. I find it very interesting and use it quite a bit.)
BUT.... until someone comes up with a better approach, then Markowitz optimization is still the way to approach the market and the way most, if not all, professionals still invest. I think the adjustment has been that more people now understand that correlations are not stationary, and that in times of crisis, they tend to all move towards 1.0; therefore as stated above by several members, do not base your maximum risk / leverage figures solely on the backtest and be cognizant of the fat tail risk. For example, if by investing in real estate and equities and corporate bonds together lowers your volatility by 50% over a single market, you probably may want to increase your exposure by somewhat less than the 100% that is implied...
I hope you realize that we are measuring the correlation of Equity Curves
Equity Curves are the outputs of multimarket trading systems
Price data (such as oil prices and S&P prices) are the inputs to multimarket trading systems
Because modern futures and forex trading systems do not buyandhold, and because they sometimes take short positions, and because they use leverage, these systems produce outputs which are not equal to their inputs. Therefore the correlation of their outputs (equity curves) is very different than the correlation of their inputs (price data). This is a second, underappreciated, free lunch on Wall Street.
System 1 trading portfolio P produces equity curve 1. System 2 trading the same portfolio P produces equity curve 2. The correlation between (equity curve 1 and equity curve 2) is VERY different than the correlation between (prices of the various instruments in portfolio P).
Repeat after me: A trading system is a function that transforms prices into an Equity Curve
(In my experience, the really good systems are nonlinear and nondifferentiable functions (i.e. their derivatives are discontinuous)).
Equity Curves are the outputs of multimarket trading systems
Price data (such as oil prices and S&P prices) are the inputs to multimarket trading systems
Because modern futures and forex trading systems do not buyandhold, and because they sometimes take short positions, and because they use leverage, these systems produce outputs which are not equal to their inputs. Therefore the correlation of their outputs (equity curves) is very different than the correlation of their inputs (price data). This is a second, underappreciated, free lunch on Wall Street.
System 1 trading portfolio P produces equity curve 1. System 2 trading the same portfolio P produces equity curve 2. The correlation between (equity curve 1 and equity curve 2) is VERY different than the correlation between (prices of the various instruments in portfolio P).
Repeat after me: A trading system is a function that transforms prices into an Equity Curve
(In my experience, the really good systems are nonlinear and nondifferentiable functions (i.e. their derivatives are discontinuous)).
Last edited by sluggo on Sat Feb 26, 2011 3:59 pm, edited 1 time in total.