My research says: the markets haven't changed in 31 years

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
Chris67
Roundtable Knight
Roundtable Knight
Posts: 1046
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

My research says: the markets haven't changed in 31 years

Post by Chris67 » Wed Jul 20, 2011 2:51 am

Im conducting a host of research at the moment to try to really look closer and deeper into market changes in the last 30 years. The research is not finished by any manner of means but its already chucking up some interesting findings.
Im specifically looking at daily volatilities of markets, daily volatility of systems on those markets i.e. P/L swings on a daily/monthly basis of consistent portfolios, correlations (input and output) , etc
So far Im failrly surprised to find that there is actually a remarkable lack of change between 1980 and 2011. In fact some research is indicating that markets are less whippy and system daily p/l's are less volatile than they were over previous periods in history. This flies in the face of some expressed opinions on this forum about the markets becoming more whippy and causing certain trend following systems to struggle.
However - and this is the big "but" (if you'll excuse the expression) - I certainly agree that certain trend followers and certain systems clearly dont work as well / att all as they did in previous times.

Maybe an early conclusion to reach here is that something has certainly changed but it isnt related to market volatility / whippiness and markets individually dispalying tendencies that were not seen - say - in the 1980's of 1990's
It could be that markets just dont trend as much anymore , do not extend as far anymore , that central banks and governements have compressed many markets into tighter ranges, there is more central bank manipulation (printing money ?, intervention, etc)
It may well be caused by another reason.
Certainly longer term TF systems have held up better and thsi conflicts with my findings that markets dont actually appear to be as whippy per se ?
As stated I need to do more work and I will update on findings here.

Id be interested to hear other peoples findings and thoughts - either on here or PM me ?

edited by Moderator -- changed thread title from "Interesting Research" to something that accurately reflects the contents of the lead post

Moto moto
Roundtable Knight
Roundtable Knight
Posts: 427
Joined: Mon Jun 01, 2009 4:12 am
Location: once again in the UK

Post by Moto moto » Wed Jul 20, 2011 4:39 am

not really sure how best to test this but as food for thought..... looking purely at a the daily volatility wont help, I would think you need to look at how mean reverting within a trend over a period of days instruments are in comparison to previous years. ie; after a buy trigger point have the instruments shown an increased tendency to mean revert first and then trend than they used to (it appears so). Maybe you need to look at prior and post trigger volatility OR look at momentum immediately after a trigger to be able to compare the times that the model worked as to the times the model now does not work.
(my two cents)

Years ago I did a test and it always freaks me out........but a very simple test to run is count the number of up days for an instrument in a clear uptrend and it will make you realise there is more about magnitude and other such factors. Often the ratio of up days v down days is similar which seems counter intuitive for a trend to occur.

Usually for ASX stocks it seems the down days outnumber the updays even in clear uptrends......maybe it is because we are at the bottom of the world :)

Eventhorizon
Roundtable Knight
Roundtable Knight
Posts: 229
Joined: Thu Jul 08, 2010 2:36 pm
Location: Boulder, CO
Contact:

Post by Eventhorizon » Wed Jul 20, 2011 5:08 pm

Chris,

I would echo Moto's point about looking at daily vol. alone, but take the thought in a slightly different direction.

I have been working on a tail risk overlay - trying to identify increasing instability in the asset price history and use this as a signal to dial back position sizes. As part of that I reviewed how to go about estimating volatility.

Textbook volatility is the standard deviation of log of the daily price change. The shortcoming of this is that we are effectively looking at one single point in time each interval to estimate vol. Consider two price series, both increase close-close at an average rate of 0.1% per day with a standard deviation of 1%. They both have the same vol measured the traditional way. What if one had an average daily hi-lo range of 1.5%, the other 3%. It's hard to argue their volatilities are identical!

There are at least 5 different methods of estimating standard deviation of prices (see Parkinson, Garman/Klass, Rogers/Satchell, Diebold, Yang/Zhang) that are based on the full range of daily price action. Some of the papers are public domain - I can upload what I have if anyone is interested.

I am wondering if you may find that a more sophisticated measure of volatility that include information on daily range (data points influenced by shorter term traders) may show more evolution!

MarkS
Roundtable Fellow
Roundtable Fellow
Posts: 70
Joined: Wed Dec 15, 2010 12:10 pm
Location: Chicago, IL

Post by MarkS » Wed Jul 20, 2011 7:35 pm

This is simply speculation on my part, watching my own "actual" results versus historical back-testing:

If you start with the assumption that "trend" = "price instability" or "prices seeking a new stable level (sideways)", then I believe that due to information, computers, the internet, and the vastly larger sums of money in the market pushing ever more managers to get better information faster (similar to the equities markets), that trends do not last as long, even if they move the same amounts. Simply, they happen faster.

Again, speculation, but say the cocoa crop in Ivory Coast got infected by some fungal disease in 1983. First, the traders of the physical on the ground in Africa might buy the futures expecting prices to rise. Then some news might travel by phone to the traders at large cocoa purchasers/processors. These in turn would buy some contracts, and their brokers would start whispering to the floor traders next to them that Nestle is a big buyer of cocoa contracts for some reason. So the floor traders might take positions. The movement in price might have triggered some research house to call some people in country, to see what was going on. And so on and so forth. But the buying came slowly and as information took time to travel and get acted on, the trend would continue to plod on. Uncertainty would be the key word, and lots of back and forth action might take place. Even when the fundamentals finally turned, it would take a while for the underlying futures to turn. That would allow your stops or exits to catch up and capture a large amount of the profit.

In the present, there are research reports everywhere you look. The internet can give you enough information to paralyze you, if you are willing to pay for all the feeds. I can probably tell you with better certainty the condition of the Brazilian sugar crop today then my own garden. When new data comes out, it appears to me that prices move with determination to a new steady point. Trends are simply over faster -- they might reach the exact same point as twenty years ago under the same conditions, but they will get there much more quickly and with less back and forth uncertainty.

I have watched very large moves occur this year, and then before my exit even starts to ratchet up, the price falls back down just as quickly as new data comes out countering the old one. That was what got this conversation started with the other partner in our fund.

It is under this hypothesis that I am now testing stops and exits that ratchet much more quickly, but my data is inconclusive so far. Tricky part is to find the start of this faster movement, as it was not a binary point, but an evolution that continues to this day. Curve fitting? Perhaps, but perhaps an adjustment to how markets have changed?

Chris67
Roundtable Knight
Roundtable Knight
Posts: 1046
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Post by Chris67 » Thu Jul 21, 2011 11:44 am

Mark S - Excellent post and thanks very much
Plenty to say on this

My own career bears out your story - when I was a currency trader many moons ago in front of the voice brokers I used to trade Dollar / Dem - I can honestly say (really honestly!!!) in 1991 I wouldnt have ever heard of tthe "SP500 Future" let along "crude" oil or U.S "30yr Bonds"
On a recent visit to a modern day currency trading room I was completely dismayed by teh fact that the average interbank (not sure that market really exists - but equivalent execution position Euro / Usd trader) is basically sitting their working out Dec 2014 Corn divided by March 2012 Cotton and multipliyng it by teh ATR in SP500 to see if he should hit the next bid in teh Euro - for a quick tick move !!!!! and there in I believe lies teh problem you have outlined so well
Its all about information / technology and weight of money !!
Everybody has teh same info - BUT !!! markets trend still
Somewhere in the midst of all that info lies the answer - it could be what your doing with ratcheting stops up quicker - or it could be doing the opposite - starting with close stops and widening them as the move grows ? It could be doing a bit of both ????
It could be purusuing another job or it could be that this is absolutely typical of teh way things are before the next big moves begin ??
Best
C

SimJimons
Senior Member
Senior Member
Posts: 31
Joined: Mon Jun 20, 2011 9:59 am

Post by SimJimons » Fri Jul 22, 2011 10:19 am

For what it's worth, my view is that markets of today experience much more short-term noise than they did, let's say, 20 years ago. This can easily be noticed by looking at a 20-day breakout system, for example, which used to perform beautifully (look at the Turtles) in the 80's, but are more or less a dead system today.

Given that the above is a correct observation, which I truly believe it is, I wouldn't recommend using tighter stops/exits, as someone suggested, but instead I would do the exact opposite. This may sound counter intuitive at first, but is really rather straightforward if you think about what noise is; random movements. What effectively happens when you tighten your stops is that you make the system more exposed to noise, which by definition is unpredictable. And if the level of noise has increased, well, it's a lose-lose situation.

But, if you believe that the level of noise has decreased, providing clearer trends at less volatility, it's probably prudent to tighten stops/exits. However, there is very little support for such a view. I mean, humans are risk averse, which means that we want to reduce risk as much as possible, and by taking profits fast (i.e. trading on shorter time-frames) we feel less pain, in the short run. Hence, it is rational to believe that markets are (and increasingly become) more efficient on shorter times-frames than on longer ones. In conclusion, I believe that empirical evidence as well as logical reasoning supports the notion that markets indeed have changed (presenting more noise), but only in the shorter-term spectrum of the markets.

Moto moto
Roundtable Knight
Roundtable Knight
Posts: 427
Joined: Mon Jun 01, 2009 4:12 am
Location: once again in the UK

Post by Moto moto » Mon Jul 25, 2011 4:56 am

hi SimJimons, given those thoughts, would you think then an asset allocation mix between say shorter term systems and longer term systems can be reweighed based on how much noise is is expected in the markets.

eg; you expect more noise so you allocate 70% of portfolio to a short term system designed to capture noise through tighter profit taking levels, 30% to LTTF, when you expect less noise you allocate 70% to LTTF and 30% to short term system.

OR do you think that its best just to widen/lessen stops and keep the one system.

Unless you systemise some noise measure, I guess this would be a discretionary allocation.
and while this is something that only time will tell if it is right, as noise is certainly noise to some but not to others, is the shorter term noise actually becoming more efficient OR more random???
I was wondering your thoughts (or others) on this.
thanks.

(alternatively keep all discussion in this thread on a similar vein
viewtopic.php?t=8789)

SimJimons
Senior Member
Senior Member
Posts: 31
Joined: Mon Jun 20, 2011 9:59 am

Post by SimJimons » Wed Jul 27, 2011 3:49 am

Hi Moto moto, that's a good question. Generally speaking I wouldn't try to change my allocation to short- vs long-term systems dynamically, but stick with the allocation I believe has the most merit. Currently I’m leaning towards the long-term end only. Obviously, if you think you are somehow able to predict the level of noise it may be interesting to try and change allocation between them on a continuous basis. Unfortunately I think this is more or less futile. In my view the most likely scenario is that noise will increase going forward and the shorter the time frame the more difficult it will be to make money using a trendfollowing approach. Hence, I don’t see the short-term as an opportunity that comes and goes, but more as a permanently gone one. But that doesn't mean one can't make money trading a short-term mean reversal system. In summary, I think trend trading is most profitable long-term while it’s probably still ok to trade contrarian short-term, since the latter actually thrive on "noise". Obviously the return profile will be very different if you start trading contrarian, which may or may not be something you are comfortable with.

I don't know if this answered your question at all? :)

PS. When I write short-term I'm referring to 10-20 day breakout type of systems, not intraday trading systems.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Wed Jul 27, 2011 9:01 am

On the other hand, the Newedge Short Term Trader's Index (link) includes managed futures programs with
  • Average trade duration less than 10 days
  • More than $100 million of customer assets under management
  • Open to new investment
  • Publicly reported returns
  • Total assets of the programs in the index is $6.9 billion and growing
These guys appear to be profitable, even after deducting management fees and profit incentive fees. Whether they trade with-the-trend or against-the-trend (or indifferent-to-the-trend), I personally don't know. I suppose you could download the prospectus of each program, read each of them, and find out.

Research report attached. (Sorry about the watermark. It's there in case the file gets copied into another website or blog; viewers of that copying site will find out about tradingblox.com and possibly come visit the Trader's Roundtable.)
  • At the same time, our ongoing conversations with managers and investors have produced
    a mountain of anecdotal evidence that short-term traders' returns exhibit low correlations --
    not only with the returns of any other investment alternative but with one another's returns
Attachments
STTI_newedge.pdf
Short Term Traders Index
(1.99 MiB) Downloaded 321 times

Moto moto
Roundtable Knight
Roundtable Knight
Posts: 427
Joined: Mon Jun 01, 2009 4:12 am
Location: once again in the UK

Post by Moto moto » Sun Jul 31, 2011 5:14 am

yes it answers the question thanks....mainly with your idea of the short term being 10-20 days :)

I find it more fascinating I guess that it seems that a lot of the more manual discretionary type traders I know, myself included seem to reach much the same conclusions as the more automated/systematic styled traders.....but each in their own ways.
For me the short term is intraday to 5 days - intermediate 10-20 days, and intermediate being what I consider the toughest area, and long term is long term - anything over this really....so I continue to manually short term trade - and yet still maintain my hair and sanity, but also long term trade on a more systemised basis.......
just interesting

thanks for that Sluggo - I will download and look.

rajivm
Roundtable Knight
Roundtable Knight
Posts: 129
Joined: Fri Jul 25, 2003 12:50 am
Location: New Delhi, India

Post by rajivm » Sun Jul 31, 2011 10:38 am

Moto moto wrote:yes it answers the question thanks....mainly with your idea of the short term being 10-20 days :)

I find it more fascinating I guess that it seems that a lot of the more manual discretionary type traders I know, myself included seem to reach much the same conclusions as the more automated/systematic styled traders.....but each in their own ways.
For me the short term is intraday to 5 days - intermediate 10-20 days, and intermediate being what I consider the toughest area, and long term is long term - anything over this really....so I continue to manually short term trade - and yet still maintain my hair and sanity, but also long term trade on a more systemised basis.......
just interesting

thanks for that Sluggo - I will download and look.
very interesting,
I trade short term systematically and longer term manually.. :)

SimJimons
Senior Member
Senior Member
Posts: 31
Joined: Mon Jun 20, 2011 9:59 am

Post by SimJimons » Mon Aug 01, 2011 8:43 am

Well, I guess 10-20 days may not be short-term in most peoples minds, but I was under the impression that we were talking about daily systems, and given that I feel that 10-20 days is in the short end of that particular frequency :)

I find it interesting that you use both systematic and discretionary methods in your trading. I actually believe that is a really good diversifier, while slightly more subject to human emotions, of course. This is something we've been discussing at our shop, since we're crazy for diversification :D I guess the biggest problem with such an approach is assessing trader skill, and keeping faith in him/her when he/she is on a losing streak. But it's probably easier if it's a one man shop :)

Chelonia
Roundtable Knight
Roundtable Knight
Posts: 497
Joined: Mon Apr 30, 2007 3:37 pm

Post by Chelonia » Mon Aug 01, 2011 9:26 am

That being the case I would ask for track record 1 being the system(s), and track record 2 being the discretionary trader + combined. Another way of measuring this so called alpha would be to come up with the tracking error between system output and real world results. Anyone?

Overall I think investors like the alpha story in a systematic trend following model but it´s a very thin line between ego/emotions and common sense and that is where the risk lies I think.

Moto moto
Roundtable Knight
Roundtable Knight
Posts: 427
Joined: Mon Jun 01, 2009 4:12 am
Location: once again in the UK

Post by Moto moto » Mon Aug 01, 2011 9:12 pm

SimJimons wrote: I find it interesting that you use both systematic and discretionary methods in your trading. I actually believe that is a really good diversifier, while slightly more subject to human emotions, of course.
there is no doubt that you can still be discretionary and still be rather systematic in the method and approach
The obvious problems can occur - holidays, divorces, sickness...but if you are rather systematic and disciplined in your method and really only listen to the market and not journalists, brokers, analysts, then taking stops, regular position sizing and portfolio management can be much the same.

However there are some times being discretionary works to avoid problems....(partially covered by Curtis Faith Trading from your Gut) and as the thread heading says to be fluid and dynamic with an ever changing market....and as stated this can only be measured by history as to if you are any good at it.....plus you need experience to be able to rely on the instinct.
As Chelonia says that thin line needs to be constantly monitored. I am trying to automate certain areas of things (tough and I seem to get 80% there an the last 20% hits a wall as its a trade off between spending more time monitoring different markets rather than monitoring and maintaining the system, maybe I want too much or am self sabotaging - or maybe I just dont have the skills here and am biting off too much) I do have a current line of thinking here (PM if interested) I am trying to get to but so far its not quite happening. I would be happy to have everything automated - close the garage door and watch the account grow, but there is reality and then there is fantasy :)

The reason I am shorter term discretionary and longer term more systemised is largely to do with the varying instruments traded....here in Oz I shorter term trade a wide variety of stocks based on the context of the market, whats happening in each stock, the fact I need an income. (this does change your positioning dynamics, and does not make comparisons between instruments and styles that relevant for me)
Longer term it can be futures and FX looking to capture the major trends - plus I can sleep at night as it a different time zone, its more scalable, I can move savings toward it and view it as part of the longer term plan. :)

The big, big difference is in whos money is at stake - some clients prefer the star manager, some prefer a system, whereas its my money so I am trying to get the best of both worlds with no questions asked at present.

It is a trade off as sometimes certain styles are not only suited to different people, but also different end goals if you like - are you raising money, are you trading for growth, for income, for diversification, for asset retention, for fun???? a never ending dark pool of questions. :)

Chelonia
Roundtable Knight
Roundtable Knight
Posts: 497
Joined: Mon Apr 30, 2007 3:37 pm

Post by Chelonia » Tue Aug 02, 2011 3:02 am

close the garage door and watch the account grow, but there is reality and then there is fantasy
Good to hear I´m not the only one working out the garage :wink:

Post Reply