Optimization methods

[Jason Czech]

I'm aware of two broad methods of optimization. One way is to run your system across all instruments (not taking position sizing & equity into account) you would potentially trade, gathering the statistics for each & averaging them out to see how the system does, on average, at each parameter value. In Wealth Lab this is called a "Raw" optimization. The other method is to optimize at the "Portfolio" level, where the statistics are calculated based on how the system would have actually performed based on starting capital and position sizing.

I'm curious to know what the forum members think is the better method of optimization, and why?

Raw optimization 'seems' like the best method in my opinion, since the results aren't dependent on the order of trade signals. On the other hand, it may be that the best signals are those that come first in the cycle of profitable trades.

My own testing of trend following systems applied to the stock market has indicated that faster signals tend to give the best performance in terms of profit/loss per trade when doing a Raw optimization, yet slower signals seem to be better in Portfolio level optimization.

-Jason

[Bondtrader]

In my opinion it's a mistake to ask about better and best, because different traders with different goals and different capitalizations and different risk preferences ("utility functions") will gravitate toward different answers. Hence there is no universal best for all traders, and about all you can do is talk about "what seems to be best for me".

So I suggest you replace better and best with preferred and favorite, to remind yourself (and those reading your message) that it's a matter of individual subjective judgement.

In my case, I prefer to only analyze the equity curves which result from full portfolio runs using full positionsizing rules. It's the way I trade so it's my favorite way to test. I don't trade single contracts and I don't test that way either. However, other traders feel differently. They have other preferences, which are no less valid than mine.

[Jason Czech]

In my opinion it's a mistake to ask about better and best, because different traders with different goals and different capitalizations and different risk preferences ("utility functions") will gravitate toward different answers. Hence there is no universal best for all traders, and about all you can do is talk about "what seems to be best for me".

Bondtrader,

First, thanks for the response. I agree with you, the question would have been better put as "I'm curious to know what the other forum members prefer to use as their method of optimization, and why?" Words like 'should' and 'best' are considered profanity among traders...

This is a topic I haven't seen discussed here, or anywhere else for that matter, yet it seems to me that it's a very important decision to be made in the process. Particularly true when the findings of one method contradict the other.

At this point, my belief (always subject to change) is that optimization on the basic buy & sell signals of the system may be better done at the raw trade level, with later position sizing optimization done at the portfolio level. This is just what my less than briliant mind thinks may be the most robust method.

-Jason

[Forum Mgmnt]

At this point, my belief (always subject to change) is that optimization on the basic buy & sell signals of the system may be better done at the raw trade level, with later position sizing optimization done at the portfolio level. This is just what my less than briliant mind thinks may be the most robust method.

I have to disagree.

If you are trading the same number of contracts per market each time, then this type of optimization works. However, I don't recommend trading that way, therefore I don't recommend optimizing this way.

In general, during optimization what you are interested in is the risk/adjusted performance of the buy and sell signals. That means the performance taking money management into account.

Consider several different sets of parameter values for an entry signal. With each of the distinct parameter sets, the system rides a trend for approximately the same distance since they use the same exit.

Suppose a given signal has a better risk-adjusted parameter set that it lets you take 1/2 the risk with roughly equivalent win/loss percentages. Further, imagine that these low risk entries occur for a high percentage of the winning trades but for a much lower percentage of the losing trades.

The method you describe as "raw optimization" won't detect this reliably, since the winner won't be any bigger than the riskier entry afforded by the other parameter sets, and the losses might not be smaller with the low risk entry method.

If you tested the same signals using a fixed-fractional betting strategy, you'd get two contracts in the one case for every one contract in the others. This would make your winners twice as profitable. This is an important difference that "raw optimization" would not detect. This single difference might account for a large percentage of the profits.

To summarize my point, money management can be the most significant aspect of system profitability. The interplay between signals and money management is extremely important. Optimizing without taking this into consideration is ignoring the most important component of the system.

I'm curious why you think your method would be more "robust"?

[Jason Czech]

If you tested the same signals using a fixed-fractional betting strategy, you'd get two contracts in the one case for every one contract in the others. This would make your winners twice as profitable. This is an important difference that "raw optimization" would not detect. This single difference might account for a large percentage of the profits.

This is an excellent point...I hadn't considered that when I wrote the original post.

I'm curious why you think your method would be more "robust"?

My rational was in the context of an ATR Breakout system applied to the stock market only. The system typically generates several hundred buy signals every year, but only trades about 5 percent of those (at 2% risk) due to being a long-term system almost always fully invested. With such a large percentage of signals ignored, I believe that there is a significant amount of luck involved in taking the 'right' trades. With this in mind, I felt that it would make the most sense to trade the parameters that perform best across all potential trades, rather than those which happen to be selected by the portfolio level test. I would imagine this is less of a problem when working with futures.

On the other hand, optimizaton results at the portfolio level consistantly contradict the findings of Raw optimization in my tests(probably due to built-in risk adjustment you mentioned)...so my logic may not be logical after all!

-Jason