Page 1 of 1

Suggestions on the backtesting process

Posted: Wed Oct 19, 2011 8:13 am
by Bounty
Couldn't find any posts related to this. If there are threads related to this just let me know..

It's easy to test a gazillion parameters with TB. Any suggestions on the best way to track/process the results? For instance, using a spreadsheet the shows the system, parameter settings, date range, etc.?

Posted: Wed Oct 19, 2011 11:22 am
by sluggo
Search for posts by user "bobsyd" ; as I recall, he confronted some of the same bewilderances.

Posted: Sat Oct 22, 2011 6:45 pm
by dantes
I believe this is the post Sluggo's referring to?

viewtopic.php?t=6281&highlight=

Posted: Tue Oct 25, 2011 8:32 am
by Bounty
Yes, I saw that, thank you.

Was just reading a post by AFJ about how dramatically the results can change just by changing one parameter such as earn interest from on to off. That's exactly what I've experienced. Being a nube, the the fact that there are so many parameters that can have an impact and distort the results is sobering. The result is that I have to dig into the details and spreadsheets to try to verify what is happening. Not a lot of fun for my personality type, but necessary and it insures that I understand how the system operates.

Posted: Tue Oct 25, 2011 9:06 am
by AFJ Garner
Bounty wrote:Yes, I saw that, thank you.

Was just reading a post by AFJ about how dramatically the results can change just by changing one parameter such as earn interest from on to off. That's exactly what I've experienced. Being a nube, the the fact that there are so many parameters that can have an impact and distort the results is sobering. The result is that I have to dig into the details and spreadsheets to try to verify what is happening. Not a lot of fun for my personality type, but necessary and it insures that I understand how the system operates.
I was not saying that in the case of the substitute Turtle system that it was the impact of interest/non interest which made the difference. Nor indeed the ranking in which the orders were presented to the risk manager. Nor am I trying to frighten anyone or put them off. See a post I have just made on CSI data changes. I am merely saying that it is very easy indeed to fool oneself with back-testing and that as the years go by I realise with ever greater clarity why there is such a difference between back testing and reality.

I'm not trashing the game, I'm not trashing back testing. But you know that old adage about it taking a decade to become a doctor? Well, its true. I continue to make dumb mistakes and hopefully learn from them.