Do commodities vary in the short term?

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
blueberrycake
Roundtable Knight
Roundtable Knight
Posts: 125
Joined: Mon Apr 21, 2003 11:04 pm
Location: California

Do commodities vary in the short term?

Post by blueberrycake » Sun Jun 01, 2003 12:08 am

I'd like to follow up on a question that came up in another thread. Do different commodities exhibit different characteristics in the short run? In particular, I'd be interested to hear if anyone has been successful (in real trading) with short term systems that work well with one commodity (or group of related commodities), but break-down when applied to a broader basket? (I-Master excluded)

A related question is, has anyone improved their system's performance (in real trading, not backtesting) by applying non-symmetric rules to the long and the short sides?

-bbc

Josh M.
Senior Member
Senior Member
Posts: 38
Joined: Mon Apr 21, 2003 2:22 am
Contact:

Post by Josh M. » Sun Jun 01, 2003 5:02 pm

After years of testing my conclusion has come to this: different markets have different character.

This is similiar to what was said in the thread you reference and I feel it is true. Therefore as a system developer I had to add another facet to my evaluation of markets, that being whether or not the character of a market has changed.

I feel the markets had a character change in '96 and we could possibly be seeing another one now with more and more trading being done overnight and overnight atr levels increasing. My L/S Model, which is in the public domain, collected some 1800 points from mid '99 to now yet has not hit an equity high for 7 months. This could signify a major change, but maybe not.

I-master has held up rather well and so have some others. My sense though is I have to evaluate market characteristics such as trend and congestion in addition to market character such as likelihood of the market reversing the next morning or afternoon.

blueberrycake
Roundtable Knight
Roundtable Knight
Posts: 125
Joined: Mon Apr 21, 2003 11:04 pm
Location: California

Post by blueberrycake » Sun Jun 01, 2003 6:53 pm

I am slowly coming to the same conclusion as well. I was particularly interested though when I read Toby Crabel's disclosure document where he says that he trades all of his markets using the exact same system with the same parameters. While his recent performance hasnt been stellar, he has been very consistent for a long time now.

A fund that he recently acquired "Buethe - Crabel" (or something like that) which is based on extremely short term trading, is also described as trading the exact same system across a portfolio of twenty-one commodities.

-bbc

Josh M.
Senior Member
Senior Member
Posts: 38
Joined: Mon Apr 21, 2003 2:22 am
Contact:

Post by Josh M. » Sun Jun 01, 2003 7:41 pm

I'd say one system with the same parameters in different markets is very possible with the most simple of methods. Where is this info on crabel?

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Mon Jun 02, 2003 10:43 am

I'm a strong supporter of not using market specific parameters.

There are not enough trades (or more importantly trends) in a single market to develop a meaningful statistical basis for assessing a trading system's merits (unless you use a very short-term system).

A single market might get three or four major trends a decade, if you are lucky. It's not hard to find the parameters that work best for those six or eight trends over a 20 year test. The question is how likely is it that those six or eight trends are indicative of what the future holds.

My answer is: Not Very.

The same problem holds if you are looking at shorter timeframes using a short-term system. However, there is an even larger problem, that of identifying the market conditions prospectively. It is very easy to analyze a market historically and say: "This works great on the S&P Index" but what happens when the market changes, as it surely did in March 2000 for most traders. How do you know when the market conditions have changed.

I'm not saying there are not ways around this problem, but make sure you test the mechanism for identifying a market state itself that relies only on information that is actually available at the time you would have done the trading.

If you can come up with a way of filtering that market that is mechanical, you can reliably test that mechanical system, and see if trading differently in different market conditions is something that can be determined realtime rather than after the fact.

I think this is possible, it is just much easier and much more common to fall into traps, or as the cartographers used to say: "This way lie dragons."

Josh M.
Senior Member
Senior Member
Posts: 38
Joined: Mon Apr 21, 2003 2:22 am
Contact:

Post by Josh M. » Mon Jun 02, 2003 11:51 am

I think for long term trend following you are correct in regard to not using market specific parameters. In short term trading I think it is prudent if not necessary though.

Unless a short term system uses volatility to adapt it will most likely need different parameters for different markets imo. Really what this comes down to though is TYPE of system.

kmulford
Senior Member
Senior Member
Posts: 41
Joined: Mon May 12, 2003 5:59 pm
Location: Setauket, New York

Post by kmulford » Mon Jun 02, 2003 2:06 pm

Josh,

How does one avoid curve-fitting when determining one market's personality? Is post-dictive error a problem with systems made to fit a single commodity's (financial future's, currency's, etc.) personality? In constructing your TYPE of system (slow or fast, in grades; etc.), test your ideas not only for how well they work out (profitability in historical simulations); but also, step back and ask yourself if your ideas are not underpinned by information (about the market's personality) that you could not have known over some stretch of the tested time series.

The reason for using a single (or, a limited) set of parameters across numerous traded instruments is to ensure that the mechanical trading system employed is robust. This method of avoiding curve-fitting is independent of the type of system (slow or fast; volatility-based or not; etc.).

Another danger in constructing methods of trading single markets is the proliferation of parameters used. Each new parameter introduced to capture the subject market's personality removes degrees of freedom, hindering the robustness of your method. How many are too many? Do your results seem too good to be true? Have you seen volatilty of results from out of sample testing?

--Ken

Josh M.
Senior Member
Senior Member
Posts: 38
Joined: Mon Apr 21, 2003 2:22 am
Contact:

Post by Josh M. » Mon Jun 02, 2003 5:22 pm

kmulford wrote:Josh,


The reason for using a single (or, a limited) set of parameters across numerous traded instruments is to ensure that the mechanical trading system employed is robust. This method of avoiding curve-fitting is independent of the type of system (slow or fast; volatility-based or not; etc.).



Another danger in constructing methods of trading single markets is the proliferation of parameters used. Each new parameter introduced to capture the subject market's personality removes degrees of freedom, hindering the robustness of your method. How many are too many? Do your results seem too good to be true? Have you seen volatilty of results from out of sample testing?
- Hi Ken, all good questions and statements. If I'm not mistaken c.f. defined robust as performing the same or similar to how it did in back testing, which I agree with. By that definition some trading systems would be both unprofitable and robust on certain markets. On other markets the same system could be robust and profitable.

-If robustness is the goal then using the same parameter/system is the way to go. I believe even long term trend followers will look for the trendy markets though, this is similiar to developing a system to work in a market over vast amounts of data imo.

-I want to utilize both robust and profitable systems. To me this is utilizing a non-curve fitting approach to develop a trading system for a market in which the logic is sound and works in multiple markets. Of course all of this is just my opinion.

-More valid questions in your last paragraph and it takes time to learn how to avoid those pitfalls. Do you trade many markets? How is it you choose to avoid some and embrace others? Would you trade a system on every market it is robust in as defined above? Good trading.

-josh

Post Reply