Will somebody please do this research? / has anyone already?

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Chris67
Roundtable Knight
Roundtable Knight
Posts: 1046
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Will somebody please do this research? / has anyone already?

Post by Chris67 » Thu Nov 03, 2011 12:30 pm

It certainly seems at the momentliek Trend Following (TF) is hard. Most TF's, myself included, getting a nice thumping on the year and even those Holy Grail providors such as Winton, Bluetrend and Aspect are not exactly setting the Wolrd on fire (it seems Winton's new pemanenet return profile is encapsulated by the boundaries of+/- 2.5%)
Everybody has a theory as to why TF "seems" to be getting harder - these range from:
1. Markets are more volatile
2. Markets trend less
3. Trends are more shallow
4. Trends reverse quicker
5. Markets are more correlated
6. Everyone is a TF

And of course there are many of the view nothing has changed - this is a typical losing period.
Has anyone conducted their own research on teh above points ? Each one of them is quantifiable - if so what have they found ?
I ve been busy on this area held back , most probably, only my simplistic views to these projects and my "basic" level of maths
Personally the only thing on the list I see related to market correlation - the rest seems failry normal compared to years gone by - correrlations certainly appear to be headed toward 1 across the board - maybe that 100 market plus portfolio isnt what it used to be ?
xt99 seems to be doing very nicely on 48 markets
Perhaps more systems than markets ? Then again, perhaps not ?

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Thu Nov 03, 2011 2:22 pm

God knows. Whether he exists or he does not, I am not he and must therefore fully confess that I, at least, do not know. What I do know is that historically things have picked up from these awful troughs.

Will they this time?

I expect so but for the definitive answer you will need (a) to consult a friendly fortune telling gypsy currently being brutally ejected by the authorities from an unauthorized travelers' site somewhere in your neighborhood and to cross her palm with silver; or (b) seek Revelation from the almost certainly non-existent Divine; or (c) throw appropriately marked "Poohsticks" from a local bridge and see which emerges first.

Excuse my skepticism, exhaustion, irreverence and general bad behavior. It has been a shitty week in a shitty year.

Quo Vadis?

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Thu Nov 03, 2011 2:30 pm

If you uncover answers to these questions, what do you do with said information?

Toosday
Roundtable Fellow
Roundtable Fellow
Posts: 70
Joined: Fri Jun 03, 2011 11:21 am
Location: Austin, TX

Post by Toosday » Thu Nov 03, 2011 2:35 pm

I have done a bit of research in this area and I for one am not going to call trend following dead. I do believe that the entire monetary system is dysfunctional right now because we are not seeing the movement of funds that we would normally see. Zero interest rates and wide open monetary policy have disrupted exchange rates ie movement of funds and resources between countries. They have also disrupted movement of funds between asset classes ei bonds, stocks and commodities.

The way I look at it is that the govts are not implementing price controls but are trying to put a cap on volatility (hence change in price). In the end we all know how caps or price controls work. We also know that trend followers usually benefit when the price controls blow up. That is how I am trying to look at it.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Thu Nov 03, 2011 2:55 pm

When you finally do have the time to quantify and then test each of these six hypotheses, please report your research results. It's quite likely that others might be interested to know what you found out.

Chris67
Roundtable Knight
Roundtable Knight
Posts: 1046
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Post by Chris67 » Thu Nov 03, 2011 4:10 pm

Thanks for your comments
Sluggo I most certainly will
I will put up my research results along with teh way I arrived at the results if others, perhaps more mathematically inclined than myself, could then comment onteh mistakes, they think, I have made in my approach - perhaps we will all get there together !
I want to look at correlation changes through the last 25 years
Volatility changes over the same period

I think the above post has also kind of summed it up - interest rates at zero - and probably there for a while, massive, instant information reporting for all to see and central banks determined to keep markets in territories they do not belong. I also agree that although Governments can hold markets in areas / systems they shouldnt be for a long time - they invariably , if not always , blow up eventually
My personal thoughts at this stage are that CAGR over a given 5 year period wont change - but teh return profile maybe 4 years of crap followed by 1 year of 100% plus returns in 5 - its a tough World and a tough game we find ourselves in - oh let us hope inflation emerges as if its deflation I think we may have a problem !!!
I for one will probably run a forum !!!

Chuck B
Roundtable Knight
Roundtable Knight
Posts: 469
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B » Thu Nov 03, 2011 4:39 pm

I'm reminded of a seminar I was at almost 20 years ago where the whole subject of the death of trend following came up since I think at the time it was a cover story on Futures magazine. Anyway, someone in the group made the comment that "currencies are great trending markets" upon which Tom Basso spoke out to correct him by saying, "you mean, currencies have recently been great trending markets".

Since those days way back then when Trading Recipes was just out and was the only s/w available to test portfolios with risk-based position sizing, etc, the whole world of tf trading has massively evolved.

The point about correlations heading toward one at times is likely an important clue that the environment we're in now is a highly repressed, financially, one. It is artificially repressed by the actions of governments and central banks. It seems unlikely that repression will just gradually fade away, and the financial world evolve into a glorious Disneyland-like meadow of pretty flowers. More than likely my gut feel is that the financial repression will be met with a bang at some point. A discontinuity...sort of like looking at the thermodynamic properties of air through a speed of sound shock wave.

If one is loaded up with a portfolio of trend followed trades, one's hope is that you are on the "right" side of that discontinuity as it has the chance to be very large; however, consider that there is a "whole world of trend traders" who likely will have similar positions. Here the water gets real muddy.

Muddy in the sense that, evaluated over the very short term as that discontinuity explodes, the price action could be phenomenal in the opposite direction (i.e. similar but on a grander scale to what the Euro/$ did last week or perhaps the AUD/JPY this week). If we assume Eckhardt's theory that the market's job is for the most people to lose money as possible, then I would guess that move would be violent and large enough to clean out all the trend traders positions and perhaps even trigger some into the opposite direction. After that brief occurrence lasting hours to a few days, the real move, the major discontinuity appears by a massively illiquid movement back in the direction of the trend as it existed prior.

I just typed all that off the top of my head and hope it makes sense. I guess my thought is more a worry about the explosive nature of this financial repression when the time comes. I would assume, by the way, if/when this happens, data feeds will be overwhelmed, electronic trading connections will be lagged and/or unavailable, and communication with a human to take/place your orders will be non-existent (or if you get a desk on the line, their electronic system will be overwhelmed too).

In any event, that is a worst case scenario that every CTA/trader should have a thoroughly rehearsed plan of how he/she will respond.

Given the way **** hits the fan all at the same time, it's almost a wonder this didn't happen this week while the whole MF Global debacle is in limbo. :shock:

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Thu Nov 03, 2011 5:13 pm

God Chuck. Now you really have cheered me up. But I know just what you mean. There is always that hideous nagging fear that it is different this time. The world economy seems to be in meltdown and it is hard to see a rising trend developing in commodity prices (or anything else much) when demand seems likely to slacken off. But I'm certainly no economist.

At times like these civil unrest seems a possibility also, god forbid.

Quite frankly, if half of what you say turns out to be half right, the commonsense course of action is to remain in cash. So what if TF takes off again next month or the month after? Miss the boat and miss the agony for a bit.

Perhaps it's time to hunker down, chill out, cease trading, enjoy life and see how this whole economic disaster plays out over the next year. And I jest not.

drm7
Roundtable Fellow
Roundtable Fellow
Posts: 96
Joined: Sun Apr 20, 2003 9:02 pm
Location: Richmond, VA

Post by drm7 » Thu Nov 03, 2011 5:27 pm

We've had a lot of huge moves in the past few years (gold, silver, corn, cotton) and we are reaching a transition point where things get volatile. There is a great interview with Paul Tudor Jones done in 2008 that I think has a lot of applicability to the current environment.

http://www.absolutereturn-alpha.com/Art ... es-II.html

One of the many excellent quotes talks about "trading the last third of the move," where everything goes a bit crazy. He says "you just have to live it."

There is also a Seykota-esque riff on people "hampered by the need to understand and rationalize why something should go up or down."

There are no easy answers to the current situation, unfortunately.

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Thu Nov 03, 2011 5:48 pm

I know of one ex-Turtle who has been out of the market since mid September. He is a systematic trader. I know because we have an investment with him.

Who knows when he will re-enter.

techtrade89
Senior Member
Senior Member
Posts: 28
Joined: Tue Dec 30, 2008 5:36 pm

Post by techtrade89 » Thu Nov 03, 2011 10:58 pm

For what it's worth, my primary trading approach is relatively short term (a week or so), not really trend following and lately I've been having a rough time. I've been trying to adjust my systems to compensate. My working hypothesis is that: (i) markets are somewhat different from the backtest environment due to (a) extensive government involvement/influence and (b) general degrading of economies (and hence normal economic function) and (ii) selected key political events have an outsize importance and frequency due to (i). Examples of (ii) are (x) credit rating downgrades of entire countries, at semi-random times in consultation with authorities and (y) political wrangling to create continent wide bailouts ---- if the politicians agree, we soar, if they don't we crash, and markets are dominated by this dynamic leading up to the events. A really rough trading environment all around, in my view.

Chris67
Roundtable Knight
Roundtable Knight
Posts: 1046
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Post by Chris67 » Fri Nov 04, 2011 3:08 am

One thing Im struggling with is my inclinations on trading seem to be okay at present but this conflicts with what "I should" be doing - I think it goes back to the system vrs discretionary split at the moment.
Markets are certainly moving and opportuinities exist - certainly not opportunities that TF systems can catch - but when markets zipo around 10-20% in a month then there is certainly monies to be made

Early indications of what I have looked at indicate that if you look at , say 20 different TF systems, then you can definately see a marked change in performance from Oct 2008 - lest we forget that it is 2008 that really marked the whole start of what we are seeing now. Some TF systems have made a bit since Jan 2009, some have lost and the majority are flat - I think its a lottery as to which one's made and which one's lost - whether you had a 10 day ma or a 50 day or in some examples whether you had a 20 day ma or a 22 day ma ?
Anyway the point is that I think it reflects that ever since the financial collapse of 2008 markets have been held in artificial territory and I think that in itself tells you where they are going to go when they break - but when they break and what they do first is the crunch question
The real valuations on markets at present probably resembles something like
Short term interest rates (if theyt resembled what it actually costs to borrow money short term) should probably be reflecting 5-7% interest rates i.e fron month short sterling
Stocks - should probably be 30-60% lower
Commodities - a lot lower - the Government money that pumped into the system has obviously been split betwen stocks and commodities
Curencies - teh yen should probably be 1.5 , teh Euro parity at best and the swiss fran at least 25% higher - all of these markets are being held at artificial levels and as sure as old "Badger Lamont " screwed up on the Pound in teh ERM - These markets will eventually go to where they should be - but probably in a way Sterling moved in 1993 or the Yen moved on those fateful 2 days in 1998 i.e. 30 % lower in 30 hours

Best

Rush
Roundtable Fellow
Roundtable Fellow
Posts: 50
Joined: Wed Dec 31, 1969 7:00 pm

Post by Rush » Fri Nov 04, 2011 7:36 am

Dear Chris,

I have done such test.
It is possible to build precise metrics and definition and give a quantitative answer to the ever recurring question "Has the Market changed?".
It puzzles me that everybody chose to answer in qualitative terms.
I have carried my research over the last 30 years and over the most liquid Futures.

Currently I am in the process of writing an external research paper as it is only internal now. I post it here then.

Chris67
Roundtable Knight
Roundtable Knight
Posts: 1046
Joined: Tue Dec 16, 2003 2:12 pm
Location: London

Post by Chris67 » Fri Nov 04, 2011 9:15 am

Rush - thats extremely kind and thanks for your update - I for one look forward to seeing your findings - I till will post what I see thanks
C

Chuck B
Roundtable Knight
Roundtable Knight
Posts: 469
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B » Fri Nov 04, 2011 9:16 am

When you ask the question "has the market changed", a lot of qualifiers are needed to make sense of that question.

There is no question that the micro-structure of today's futures markets are profoundly different than in the past when the market essentially had massive direct human involvement. In today's electronic world, (let's put this in electrical engineering terms too) we have lost the the capacitance and inductance of the trading pits.

We have lost the capacitance and inductance of a massive trading pit full of humans.

I think that is a profound analogy, and the consequences of it were never fully foreseen ages ago by someone like Patrick Young as he wrote his book on the disintermediation of futures exchanges. I fall into that camp also to some extent and not at all in others.

Does anyone recall the "Bund fat finger" down wick from sometime in the late 90's? This was around the time the Eurex stole the Bund contract from the LIFFE, but I can't recall if the LIFFE pit was still up at that time. Anyway, someone sold a huge quantity of Bunds, by accident, on the Eurex that wicked the market down a huge amount and back again in a very short period of time. I was fascinated by that occurrence, but it immediately started me thinking about how "dangerous" electronic markets could be. These wicks can trigger off stops on long term positions, and before you can even realize what happened, the market is right back where it was prior to the event.

The capacitance of the trading pit was gone in the electronic markets -- that was my thought that day. I knew the markets were changing forever, but at the same time a part of me didn't want to believe in it too.

I was an early believer in electronic markets as I saw it from a user's point of view (and perhaps since being involved with computers and programming from a very young age starting with a DEC computer and teletype terminal in middle school in 1971). From the user's standpoint, it was a dream come true. Instant click access to place an order with instant confirmation it is in the market. Instant fill notifications, tick-by-tick position P&L, etc, etc, all putting the end user in direct control as opposed to the old days of trading enough size so you could get a pit post number to call, placing orders via phone, trying to get a fill out of the pit, waiting for someone to pick up the phone, waiting for a return call, etc.

Hence from an end user's standpoint, electronic futures markets seemed like a no-lose situation back in 1998. I have a good friend who was a LIFFE member since its very early days, and in Sept of 1998 I visited him one day, and we toured the LIFFE trading floor. We were sharing system design information and testing, etc, and after viewing a Eurex screen for the first time along with a CBOT Project-A screen, we went down to the pits for my tour.

The thing that struck me the most was the massive, empty spot were the Bund pit had recently been. After a couple of years of trying to get a hold, the Eurex sucked the volume away from the LIFFE in a few short months in early 1998. I was staring at the end of capacitance and inductance of the Bund futures market.

Here in the US at the time, Chicago laughed at the thought of ever giving up the trading pit. London and Europe laughed at Chicago for thinking a pit had a future. 12 months later, plus a bit maybe, the LIFFE floor for the rest of the markets was closed for good as LIFEE Connect came on-line fully. Now the capacitance and inductance of trading pits for the FTSE, Euribor, et al, were gone. For good.

I'm writing this now missive as I see screens nearby of the T-bond, Dax, EuroStoxx50, E-mini S&P, EuroFX, etc, responding to the 8:30am NY time NFP report. Whew. Lack of cap/ind is well represented today although the magnitude of today is nothing compared to what it can easily be. I get to sit back on an in-office vacation thanks to MFG and Jon-boy.

So, there is no question that the internal structure of the markets has dramatically changed since the pit days. Exchanges have implemented all sorts of circuit breakers of various types ranging from (stupid) percentage movement stoppages (i.e. look at JEF yesterday to see the stupidity of having fixed percentage amounts for all stocks and not something based on volatility) to a fixed number of ticks variation from a "reference price", etc. All of these impositions are attempts to add capacitance and inductance into electronic trading, but they all are a very crude and poorly (inconsistently) implemented "solution" to not having a trading pit.

The data-based concerns of this internal structure remake are far reaching, and the range of potential outcomes of future non-linear market action is vastly greater than the past. This doesn't even include the effects of HFT action in individual stocks.

One view that I partly have is that the "markets" have evolved into perhaps the most complicated and interconnected system ever devised by man. Said system is (or is becoming) highly dominated by pre-written programs of all types and sizes, but the thing they all share in common is dependence on the last tick by market and the recent past of data. If anyone has ever studied complex non-linear systems, you can get an idea of the potential range of outcomes from many thousands of programs all interacting with each other through a common set of exchange systems.

Toosday
Roundtable Fellow
Roundtable Fellow
Posts: 70
Joined: Fri Jun 03, 2011 11:21 am
Location: Austin, TX

Post by Toosday » Fri Nov 04, 2011 9:54 am

I thought I would share two things that I consider in my portfolio construction that I believe have limited observations in the data that we (and the market) use to make our decisions. I no doubt that there are many more changes but these are two that I have been thinking about. The first is that the number of positions on in a trend trading portfolio have increased over time. I do not believe that this is a paradigm change just something we have not seen before. Here is a graph showing a robust simple trend following portfolio trading 3 dozen markets with a pretty consistent CAGR over the test period and the sub periods.

The increase in number of positions is evident by looking at the trend line. I interpret this as taking more risk to make the same money. Obviously this is why portfolio risk limits are in place. However this causes you to filter out trades which implies that you assume future profits will come from the trades you chose to keep on instead of filter out. Damned if you do damned if you don't but nonetheless risk management is the only thing you can control.

The second thing I think about is the the rise of flash crashes. I believe flash crashes come from a mixture of more people trading with stops and high frequency trading. Those two processes together create a feedback loop that can work against trend traders. I have not enough work on this but I do think these are a "cost" to long term trend traders.
Attachments
Graph1.pdf
(49.79 KiB) Downloaded 256 times

adamant
Roundtable Knight
Roundtable Knight
Posts: 118
Joined: Mon Aug 25, 2008 1:20 am
Location: Norway

Post by adamant » Fri Nov 04, 2011 11:02 am

In an interview in 2000, Jim Simons stated that commodity markets used to trend nicely but don't any more. He was wrong (see gold for a good example). The day human psychology changes and people stop trading based on their emotions, we will have a problem. I don't see that happening any time soon.

Do markets become more efficient over time? Probably. Will there still exist ways to exploit the inefficiencies created by human emotion in 20 years? Probably. Is trend following my best bet with regard to consistently making money in the market? I am fairly certain of that.

daveineagan22581
Senior Member
Senior Member
Posts: 42
Joined: Mon Mar 14, 2011 9:50 pm

Keynes

Post by daveineagan22581 » Fri Nov 04, 2011 11:30 am

Chris67 wrote:...These markets will eventually go to where they should be...
Was it Keynes that said:

"Markets can remain irrational longer than you can remain solvent"

Chuck B
Roundtable Knight
Roundtable Knight
Posts: 469
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B » Fri Nov 04, 2011 11:33 am

adamant wrote:In an interview in 2000, Jim Simons stated that commodity markets used to trend nicely but don't any more. He was wrong (see gold for a good example). The day human psychology changes and people stop trading based on their emotions, we will have a problem. I don't see that happening any time soon.

Do markets become more efficient over time? Probably. Will there still exist ways to exploit the inefficiencies created by human emotion in 20 years? Probably. Is trend following my best bet with regard to consistently making money in the market? I am fairly certain of that.
I doubt there are many who disagree with that basic tenet around this forum. This issue really is how one defines a trend. What is most important about trying to trendfollow in trading is not whether or not there will be price "trends" but instead what the volatility about that mean price trend will be. If we have a long uptrend in say Natural Gas that moves from 4.000 to 9.000 over 8 months, but the volatility about that uptrend of over 100% in price is +/- 40% of the mean line drawn between those two endpoints, few will be able to "trendfollow" in the traditional sense that move.

Particularly if that volatility is of a very high frequency nature such as a losing 25% of value in 2 days only to regain all of that spike down over the ensuing 3 days....that type of thing.

The nature of electronic trading and lack of capacitance, inductance and even resistance (in EE terms) compared to vast majority of the past will easily lend itself to these highly non-linear outcomes. Hence not preparing for a potential significant shift in the quantity of future dispersion events might prove highly detrimental.

For long term TF trading, at a minimum there will likely continue to be an increasing trend in higher and higher execution costs (i.e. the wicks, illiquid rapid runs through stops). The most costly part might end up being how one's TF system can adapt (can it?) to the short but outlier-sized volatility events created by electronic systems.

The game has changed from simply testing a system on lots of data, portfolios, position sizing algorithms, etc, etc, and then putting a robust method into production trading. That was the 1990s trend.

Chuck B
Roundtable Knight
Roundtable Knight
Posts: 469
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B » Fri Nov 04, 2011 11:43 am

Toosday wrote: The second thing I think about is the the rise of flash crashes. I believe flash crashes come from a mixture of more people trading with stops and high frequency trading. Those two processes together create a feedback loop that can work against trend traders. I have not enough work on this but I do think these are a "cost" to long term trend traders.
This is exactly my thought just above. The examples are everywhere, everyday in the US stock market. HFT and other systems all inter-connected through multiple exchanges with all sorts of high-frequency system issues or lag, data dissemination, etc, etc, can result in some wild and totally non-existent market action when looking at tons and tons of past data.

One simple example is BIDU from right at 9:45am NY time yesterday morning, Nov 3rd. Follow a tick chart from 9:45:00 for the next 30 seconds or so. Look at a one-minute chart of it over the first hour yesterday.

Hence the thought of pondering the future, future trends, systems applied to those trends, etc, I really think it would be a mistake not to prepare for a significantly more dramatic environment looking forward than existed in the say the heating oil pits in the 1990s.

Post Reply