I have a system with about 15 parameters. When I run the desired optimization it results in about 7m plus tests which is just not feasible. How do other users handle this. Do they run a much broader stepping of values to reduce the total number of tests, identify the prospective regions and then tighten the secondary run when they work out which parameters have more of an impact.....or what?
thanks jim
Large Optimization Runs
-
blinkybill
- Roundtable Knight
- Posts: 196
- Joined: Thu Mar 27, 2008 6:44 pm
Divide and conquer.
One method that people use, is the one you suggest: start out at coarse granulatity (i.e. small numbers of biggish "steps" in the Stepped Parameter settings), find zones of goodness. Concentrate on those zones and gradually shrink the step-size as you zoom in.
Another method is to reduce the number of parameters stepped per run. For discussion, assume your system has six user variable parameters. Do a coarse granularity run where you vary all six parameters, coarsely, and find a reasonable looking "zone" to begin with. Starting from the center of that "zone", do a run where you vary parameters 1, 2, 3, 4 BUT DON'T VARY parameters 5, 6. Vary them over a wide range with rather fine-grained stepping. Whatever result you like best, choose that value of parameter 1 and (temporarily) "freeze" parameter 1 to that value. Now run again, this time varying parameters 2, 3, 4, 5 but not varying parameters 1 and 6. Whatever is the optimum, "freeze" parameter 2's value to that number. Run again varying 3,4,5,6 and freezing 1 & 2. Repeat six times until you've got frozen values for all 6 parameters. Now consider this to be the center of a zone of goodness. Explore around this center, either varying all six parameters at a time, or a subset.
Often I try to keep each individual "Simulation" (collection of tests) small enough that it runs in 4 hours or less. So I can do one in the morning, one in the afternoon, and one (or two, if awake) at night. To accomplish the six-parameter optimization above, would need one or two coarse runs varying all six parameters, six runs of four-at-a-time, and two or three explorations around the center. So, 2+6+3 total simulations, at 3 simulations per day, means 4 days from start to finish.
I speed this up by (A) purchasing more Blox licenses, and (B) running on fast computers.
aside remark: with 15 parameters, and only 3 values for each (LessThanCenter, Center, MoreThanCenter), that's 3^15 = 14,348,907 tests at extremely coarse granularity. If you only have 7,000,000 tests you must be using amazingly coarse steps.
One method that people use, is the one you suggest: start out at coarse granulatity (i.e. small numbers of biggish "steps" in the Stepped Parameter settings), find zones of goodness. Concentrate on those zones and gradually shrink the step-size as you zoom in.
Another method is to reduce the number of parameters stepped per run. For discussion, assume your system has six user variable parameters. Do a coarse granularity run where you vary all six parameters, coarsely, and find a reasonable looking "zone" to begin with. Starting from the center of that "zone", do a run where you vary parameters 1, 2, 3, 4 BUT DON'T VARY parameters 5, 6. Vary them over a wide range with rather fine-grained stepping. Whatever result you like best, choose that value of parameter 1 and (temporarily) "freeze" parameter 1 to that value. Now run again, this time varying parameters 2, 3, 4, 5 but not varying parameters 1 and 6. Whatever is the optimum, "freeze" parameter 2's value to that number. Run again varying 3,4,5,6 and freezing 1 & 2. Repeat six times until you've got frozen values for all 6 parameters. Now consider this to be the center of a zone of goodness. Explore around this center, either varying all six parameters at a time, or a subset.
Often I try to keep each individual "Simulation" (collection of tests) small enough that it runs in 4 hours or less. So I can do one in the morning, one in the afternoon, and one (or two, if awake) at night. To accomplish the six-parameter optimization above, would need one or two coarse runs varying all six parameters, six runs of four-at-a-time, and two or three explorations around the center. So, 2+6+3 total simulations, at 3 simulations per day, means 4 days from start to finish.
I speed this up by (A) purchasing more Blox licenses, and (B) running on fast computers.
aside remark: with 15 parameters, and only 3 values for each (LessThanCenter, Center, MoreThanCenter), that's 3^15 = 14,348,907 tests at extremely coarse granularity. If you only have 7,000,000 tests you must be using amazingly coarse steps.