Traders' Roundtable

Based on AFJ Garners recently touted Turtle system tune up does anybody think that as soon as you introduce profit taking / time based exit and portfolio managers that you are effectively optimising more and more
I was thinking if you look at the BBO system on its own - you effectively have 3 parameters / degrees of freedom - average close - entry and exit
Compare this to a Donchomian system that has all sorts of parameters before you start getting intoi the World of profit taking / portfolio managers / risk reducing agents such as correlated instruments / total portfolio heat, etc - surely at this point you have a big heap of optimised system ? Obvioulsy the results tell the story - the BBO system on its own with its 3 parameters tested out on a basket of 100 markets over 20 years plus - struggles to get its MAR head above 1 whereas the Donchian stystem with a few bells and whistles soon starts getting confident up around an MAR of 2 - what would you trust ?
C

There are sincere, well-meaning, honest traders who believe the Bollinger Breakout system "is" a one-parameter system. At least, it is to them, because they freeze the Entry Threshold at 2.0 and the Exit Threshold at 0.0. They don't allow themselves to tweak and twiddle and vary these values. What is their reason; why do they behave this way? Because those were the numerical values originally suggested by Mr. John Bollinger, the man who popularized Bollinger Bands.

To these traders, the Bollinger Breakout system has only got one parameter: the "Close Average (days)" parameter.

Whether or not these people are "correct", is probably a matter of opinion.

Whether or not the one-parameter version of the system is "more robust" than the three-parameter version, is probably also a matter of opinion.

I would suggest less is more. My personal opinion is simple trumps complex over the long run. This may be because I am somewhat lazy/crazy and I don't like a lot of knobs to turn/have become a bit if a tuning skeptic.

Think about it though, adding these bell's and whistles in your Turtle example over time seems like a great idea in hindsight, but when you're in the thick of things going haywire, are you really going to trust and desire even more variables?

I would prefer to distill the essence of what I want to capture and otherwise do little optimization. There are people/institutions that throw darts at a board, and there are those who are approaching it like a science experiment in a perfectly controlled lab. I suggest being somewhere in the middle

Dont forget about hidden parameters:

EMA? SMA? WMA?
Close? H+L/2? OHLC/4? HLC/3?
Total Equity? Closed Equity? Core Equity?
Smoothing factor of 1? 2? 10?
Trade on open? close? stop? next week?

I think it is better to be aware of what all the parameters do, rather than be ignorant of them.

Levi wrote:Dont forget about hidden parameters:

I've always thought that was a slippery slope. For example, isn't it true that the decision to simulate system X includes within it, the decision not to simulate systems Y, Z, A, B, C, ...? Therefore aren't those "hidden parameters"? Every system you've ever heard about, is a parameter of the system you're working with now, right? After all you have the freedom to include or exclude those other systems as you please; these are "degrees of freedom," and that makes them parameters. Is it not so?

In the case of the Bollinger Breakout, someone decided to give the indicator named Bollinger Bands a 100% weighting, the indicator named ADX a 0% weighting, the indicator named RSI a 0% weighting, the indicator named CCI a 0% weighting, the indicator named ROC a 0% weighting, and the indicator named RegressionSlope a 0% weighting. Aren't those weights "hidden parameters"? Tradestation contains 200 indicators and textbooks discuss even more. You / John Bollinger / the developer of your system had the degrees of freedom to put in, or leave out, each of these indicators. Therefore they are hidden parameters, as plain as day.

So each and every system, no matter how simple it may appear, actually contains more than a thousand hidden parameters, reflecting not only what is in the system, but also what was deliberately left out of the system by conscious choice.

Ugh.

I have had many discussions with myself RE: Sluggo's quite apt examples of "hidden parameters". It is a clever path to insanity and some of us are further along than others. There is nothing wrong with philosophizing regarding these things but it can lead to analysis paralysis or, its' close cousin, sheer terror of trading any system.

In order to preserve sanity, I suggest working under the assumption that if it isn't in your recipe, it is not a parameter. Furthermore I suggest everyone be particularly aware of what I would call their "Trading Logic" parameters. These and leverage tend to be the most slippery of slopes.

Hi Chris,

you raise a very good point.

I think it’s very easy when backtesting to introduce “what we have learn about the pastâ€

[quote="Angelo"]Hi Chris,


I think it’s very easy when backtesting to introduce “what we have learn about the pastâ€

AFJ Garner wrote:I made a choice about trading Aberration on the original parameters earlier in the decade and lo......had I known what I now know I would not have traded that system as it then was.

Have you set Blox end date to [1995] and run tests on Aberration to see what the parameter space(s) looked like at that time?

Levi wrote:

AFJ Garner wrote:I made a choice about trading Aberration on the original parameters earlier in the decade and lo......had I known what I now know I would not have traded that system as it then was.

Have you set Blox end date to [1995] and run tests on Aberration to see what the parameter space(s) looked like at that time?

I have done all those sort of experiments with many sorts of systems. On topic, how would one have felt about Turtle 20/10 in 1990? Pretty optimistic I imagine.

Actually, by 1990, it appears the Turtle returns were quite a bit lower for the 1980 to 1990 period vs 1970 to 1980. Maybe that was a sign of things to come...

I wonder what a longer term system looks like on 1980 to 1990 vs. Turtle.

AFJ Garner wrote:On topic, how would one have felt about Turtle 20/10 in 1990? Pretty optimistic I imagine.

How would one have felt about Turtle 20/10 in 1991? And in 1992? And then in 1993, 1994, 1995, 1996, etc.? If one had asked this question year by year or month by month, one would probably have discarded or modified the Turtle system long time ago. Not just now, in 2010, when its equity curve has been flat for many years. And that's where walk-forward tests can help. They allow you to test your system and your parameter selection procedure without peeking into future. In a standard backtest, a system can trade in 1990 with parameters selected on the basis of the 2000-2010 performance. How realistic is that?

But walk-forward testing, if done properly, is a dream-shattering experience. Perhaps that's why so many (if only subconsciously) choose to eschew it.

TK wrote:
But walk-forward testing, if done properly, is a dream-shattering experience. Perhaps that's why so many (if only subconsciously) choose to eschew it.

I walk fwd and bkwd in early proof of concept testing. But use different periods for before beginning trading.

I would note that other systems, such as the Donchian and Bollinger Band systems, have been showing similar performance degradation over the past decade, and may have a very difficult time in the coming years.

Algonquin wrote:I would note that other systems, such as the Donchian and Bollinger Band systems, have been showing similar performance degradation over the past decade, and may have a very difficult time in the coming years.

What is the basis for this prediction? I neither agree nor disagree, just curious.

I'm not stating a prediction. I'm stating a possibility. Run rolling backtests on these systems and you clearly see that performance is consistently worsening over time. The trend for these systems is down.

2008 WASNT A BAD YEAR FOR THESE SYSTEMS ?
Also IMHO - do your rolling back tests inferr that the decline will continue ? who knows ? they may suddenly all become the best systems to use ? thats the trouble - we just dont know whats around the corner
My 2 cents worth is that if you buy break-outs in the direction of the trend (or sell) then you are doing the right thing - therefore if the system is degrading (or if thats your conclusion) then maybe that tells us more about global financial markets than individual systems - and then again maybe it doesn't ?