Correlation - Inputs vs. Outputs
Correlation - Inputs vs. Outputs
I think Sluggo raises some excellent points about the correlation of markets in THIS post - whether they should be measured as inputs or system outputs but there still seems to me - a lot of problems associated with both methods
If you take each instrument in your portfolio and run it through your system individually and measure the correlated daily returns - you will obvioulsy get different answers to measuring the correlation of price - BUT and here's teh big but !!!
What happens if you notice for example 2 instruments that have a price correlation of 0.97 over 25 years only have an output correlation of 0.69 when individually run through your system - but you notice the correlation of these outputs was 0.92 for 1985-2000 and 0.5 for 2000-2007 ?
Surely you back at square one ? What do you do now ?
Similarly what is highly correlated and what isnt ?
Then imagine you trade 120 markets across 7 systems - do you measure the out put of every correlation and create 7 new futures info files - one for each system because the output correlation is totally different in each system ?
Then you ask - well whatever I do with correlation it degrades the perofrmance of my system as opposed to running it raw with no correlation boundaries or grou risks - yet if I run it raw - I may find myself with 25 long equity market positions on ? And then again although this has held up well over 25 years of back testing - what if ??? .. the real big one occured ?
Then you say - sod it - Im pouring myslef a large scotch !!! this is all too much for my brain to cope with and you really are back at square one ?
If you take each instrument in your portfolio and run it through your system individually and measure the correlated daily returns - you will obvioulsy get different answers to measuring the correlation of price - BUT and here's teh big but !!!
What happens if you notice for example 2 instruments that have a price correlation of 0.97 over 25 years only have an output correlation of 0.69 when individually run through your system - but you notice the correlation of these outputs was 0.92 for 1985-2000 and 0.5 for 2000-2007 ?
Surely you back at square one ? What do you do now ?
Similarly what is highly correlated and what isnt ?
Then imagine you trade 120 markets across 7 systems - do you measure the out put of every correlation and create 7 new futures info files - one for each system because the output correlation is totally different in each system ?
Then you ask - well whatever I do with correlation it degrades the perofrmance of my system as opposed to running it raw with no correlation boundaries or grou risks - yet if I run it raw - I may find myself with 25 long equity market positions on ? And then again although this has held up well over 25 years of back testing - what if ??? .. the real big one occured ?
Then you say - sod it - Im pouring myslef a large scotch !!! this is all too much for my brain to cope with and you really are back at square one ?
Wouldn't it be nice if there were no ambiguities? Wouldn't life be great if trading turned out to be expressible as a continuous, infinitely differentiable, unimodal objective function? Then anybody with a copy of Excel and a desire to succeed could find The Single Unique Global Optimum way to trade, and best of all, would KNOW it was the ideal & optimal approach. She could make a fortune quickly and sleep soundly too, secure in the knowledge that all other methods are inferior to hers.
On the other hand, anybody with a copy of Excel could find the same global optimum. In fact, everybody would find the same global optimum. Because there is no ambiguity, everybody who hunted for an optimum way to trade, would find the same solution. Innovation and creativity would not provide any "edge", nor would a Research staff of 100 Ph.D's (they find exactly the same Global Optimum that Elizabeth-with-Excel finds), nor would top quality software tools like Trading Blox, nor would dozens of years of experience. System Development would become a solved problem with a well-known answer printed in thousands of textbooks in dozens of languages. What a dull, tedious, colorless existence.
On the other hand, anybody with a copy of Excel could find the same global optimum. In fact, everybody would find the same global optimum. Because there is no ambiguity, everybody who hunted for an optimum way to trade, would find the same solution. Innovation and creativity would not provide any "edge", nor would a Research staff of 100 Ph.D's (they find exactly the same Global Optimum that Elizabeth-with-Excel finds), nor would top quality software tools like Trading Blox, nor would dozens of years of experience. System Development would become a solved problem with a well-known answer printed in thousands of textbooks in dozens of languages. What a dull, tedious, colorless existence.
-
- Roundtable Knight
- Posts: 427
- Joined: Mon Jun 01, 2009 4:12 am
- Location: once again in the UK
my 2 cents.....are you trading correlation or trends?
While it seems that most systems work better with more futures markets being covered, could it just be that you don't need as many and hence correlation becomes less of a problem. The equity curve may just not be as smooth.
Sometimes it seems that KISS is the obvious solution that we just dont want to accept. I am sure even if as Sluggo suggests everyone had an "edge" they would try and improve that - often to their detriment?
Or alternatively adopt the attitude that you are going to have a system that constantly modifies and optimises in some way for continually changing correlations.....
either way, pick one and stick to it. One thing I have been guilty of, and seen many others guilty of is continually changing between ideas, systems and not necessarily settling on one..... then I remind my self nothing works all the time.
While it seems that most systems work better with more futures markets being covered, could it just be that you don't need as many and hence correlation becomes less of a problem. The equity curve may just not be as smooth.
Sometimes it seems that KISS is the obvious solution that we just dont want to accept. I am sure even if as Sluggo suggests everyone had an "edge" they would try and improve that - often to their detriment?
Or alternatively adopt the attitude that you are going to have a system that constantly modifies and optimises in some way for continually changing correlations.....
either way, pick one and stick to it. One thing I have been guilty of, and seen many others guilty of is continually changing between ideas, systems and not necessarily settling on one..... then I remind my self nothing works all the time.
Sluggo - when I purchased Veritrader many years ago and it subsequently became TB I was under the illusion that for that kind of money I would definately be given the holy grail and would be a billionnaire many times over by now
Is there a refund policy ???
Of course Sluggo is absolutely right - I just raised some additional factors as I see them on the long never ending journey to answers that may well not exist .... now where is that scotch !
Is there a refund policy ???
Of course Sluggo is absolutely right - I just raised some additional factors as I see them on the long never ending journey to answers that may well not exist .... now where is that scotch !
Perhaps we can draw a parallel between trading systems and girlfriends. Is there an optimal girlfriend? Which is the best solution?Moto moto wrote:One thing I have been guilty of, and seen many others guilty of is continually changing between ideas, systems and not necessarily settling on one... then I remind my self nothing works all the time.
Perhaps. But is causation the main issue? To my mind, if we use correlation measures as a tool to control risk, for example, it doesn't matter whether a price move in one instrument causes a price move in another instrument or whether there is no cause and effect. If the prices of two instruments tend to move at similar times (for whatever reason) in the same direction (if I am long both instruments simultaneously) or in opposite directions (if I am long one instrument and short the other) then the volatility of my equity curve will be higher than if the prices of these two instruments had been less correlated. So if my aim is to reduce volatility, then looking at past correlations might be useful....albeit with no guarantee that such correlations might be repeated in future.rabidric wrote: causal links between markets come and go. you know better than a stat tool when things are linked or not.
To illustrate this, here are the equity curves and drawdowns of a simple long term trend following system trading a portfolio of 75 futures. The entry/exit block has only 1 parameter.
The first equity curve and drawdown graph are run without any risk limits, with correspondingly scary drawdowns.
The second pair or graphs shows the same system but with some of the heat reduced by taking profits more quickly. Naturally, the drawdowns are reduced but the CAGR is slashed in the process.
Finally, the third pair shows the same system but with a correlation filter applied to the profit-taking. The correlation filter limits 'unnecessary' taking of profits. This time, the heat is reduced, but without such a detrimental effect on return.
I've haven't tested this thoroughly or extended this to other systems, timeframes and markets yet. But on the surface, these results suggest to me that incorporating an element of analysis of correlation of inputs might be useful in system design. I'm going to look at the correlation of system outputs next. Meanwhile, I'm interested in whether others have seen similar potential benefits in managing correlations.....
- Attachments
-
- correlation filter - drawdowns
- withcorrlimitsDD.PNG (37.67 KiB) Viewed 6606 times
-
- correlation filter - equity curve
- withcorrlimits.PNG (31.56 KiB) Viewed 6606 times
-
- take profits - drawdown
- reduceriskDD.PNG (34.38 KiB) Viewed 6606 times
-
- take profits - equity curve
- reducerisk.PNG (31.48 KiB) Viewed 6606 times
-
- no risk limits - drawdown
- norisklimitsDD.PNG (36.72 KiB) Viewed 6606 times
-
- no risk limits - equity curve
- norisklimits.PNG (30.96 KiB) Viewed 6606 times
-
- Roundtable Knight
- Posts: 427
- Joined: Mon Jun 01, 2009 4:12 am
- Location: once again in the UK
The testing I ran factored in correlation - measured not in any scientific way, but more along the lines of metals, grains, precious metals, rates, equites, FX etc; Never really went much further than this.
It clearly reduced volatility and also returns. There was no profit taking involved just limiting of the positions. eg; long SP500, long NKY, cannot also go long NASDAQ. Very simple stuff..... the big issue we found was how to determine what to drop out....eg; some tests missed the internet bubble.
point being I wonder if the results would be improved via careful correlation analysis, and limiting overall correlated heat, yet still maintaining positions in as many instruments..... OR if the heuristic rules of thumb work just as well.
This also interests me from the point of view of running an equity portfolio whereby most stocks are highly correlated with the market.... so do you just buy an index?
(unfortunately I am not mathematical enough to do this yet.... but working on the ideas, after I work out my current project.)
And good point on the girlfriends and optimisation....unfortunately having more than less in the portfolio is too expensive.
It clearly reduced volatility and also returns. There was no profit taking involved just limiting of the positions. eg; long SP500, long NKY, cannot also go long NASDAQ. Very simple stuff..... the big issue we found was how to determine what to drop out....eg; some tests missed the internet bubble.
point being I wonder if the results would be improved via careful correlation analysis, and limiting overall correlated heat, yet still maintaining positions in as many instruments..... OR if the heuristic rules of thumb work just as well.
This also interests me from the point of view of running an equity portfolio whereby most stocks are highly correlated with the market.... so do you just buy an index?
(unfortunately I am not mathematical enough to do this yet.... but working on the ideas, after I work out my current project.)
And good point on the girlfriends and optimisation....unfortunately having more than less in the portfolio is too expensive.