Parameter Search Algorithms

[shakyamuni]

I'm experimenting with different search algorithms to investigate the parameter space.

Would any systems developers like to share some of their own experiences/insights/techniques with respect to this issue?

[ksberg]

Here are some questions that come to mind, and how I have approached similar situations.

• What characteristics do I want in an optimal solution?
  Is is nessecary to examine the entire problem space?
  Can I sample the problem space? What conclusions can I draw from the aggregate sample?

When optimizing, I look for solutions that are adjacent to near optimal performance. I find that optimal parameters vary per test condition (time segment, length of segment, market selection, etc), and so I define a robust parameter as being within a range of values that produce near-optimal performance. With this definition, it's not necessary to examine the entire problem space because I'm not interested in solutions that may offer better performance, yet rapidly diminish as one moves away from the optimial setting.

What I've just done is partition a larger problem into a set of smaller problems by introducing a constraint on the solution. Now I can use a number of techniques to look for local optima within each partition. If you think of parameter optimization as overlying a grid on a range of possible values, the first step is to open up the grid size (or parameter increment). Then, when I find values on this grid that are performant, I will recursively refine the search with a finer grid size, but only on a few selected areas (using a smaller parameter increment).

This type of approach works for parameters you can increment (like channel breakout length). It does not work for things like portfolio market selection. In that case, what I will take random samples from the result space, and examine distribution of the aggregate results (profits, draw downs, etc). The aggregate won't give me an optimal selection, but it will indicate the consistency of the sample, and I can refine my next step based on those results.

In both cases, I've introduced techniques to prune the sample space so that is becomes feasible and efficient to search for results.

Cheers,

Kevin

[shakyamuni]

Thanks for your comments.

[Ted Annemann]

I use plain ordinary SA. The problem with ASA is, nobody can get it to work except Lester Ingber

http://portal.acm.org/citation.cfm?id=2 ... coll=GUIDE

It's worth reading the Corana et al article, just to have a look at their multimodal test function. Specially designed to give deterministic algorithms apoplexy!