Optimization causing spikes...

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Optimization causing spikes...

Post by LeviF » Mon Apr 14, 2008 8:58 pm

What should a person do about the performance spikes caused by optimization? When I started optimization, 3 was chosen as the parameter because it was relatively high and smooth. Now that I have run tests and changed other parameters, 3 has become a tall spike and values further to the left seem more attractive for robust trading. After all, the point of all of this is to develop something that will hold up in the future...
Attachments
chart.JPG
chart.JPG (35.49 KiB) Viewed 2765 times

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Mon Apr 14, 2008 9:53 pm

"What should a person do?"

Maybe, a person should use his accumulated wisdom plus judgement plus experience to make a difficult decision in the face of contradictory information and uncertainty?

Or maybe a person should read the first Market Wizards book, interview #6 (p.167):

Q: What is the most important advice you can give the average trader?
A: That he should find a superior trader to do his trading for him, and then go find something he really loves to do

LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

Post by LeviF » Mon Apr 14, 2008 10:11 pm

I suppose "what should a person do" is a poor choice of words. I just posted this for discussion, reflection, etc...

DMFord
Roundtable Fellow
Roundtable Fellow
Posts: 88
Joined: Wed Jun 27, 2007 8:11 am
Location: NB, Canada

Post by DMFord » Tue Apr 15, 2008 2:37 am

Yup,

I've seen similar things - if you "optimise" one or two parameters at a time, (e.g. entry conditions, then portfolio filters, then exits) the system can get quite highly tuned to selection you've already made.

Then when you revisit the earlier parameters, you find them much more sensitive.

I'm not clever enough, or experienced enough to understand the mathematical reason - but I've realised its something to beware of.

On a statistical basis, what can you do except go for the most robust parameters ? Try un-optimising some of the others & see if it gets better.

Ideally, we'd keep track of every test we made, how it contributed to the result and how it impacted other elements of the system.

Dave

Post Reply