Suggestions on the backtesting process

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
Bounty
Senior Member
Senior Member
Posts: 33
Joined: Tue Jun 19, 2007 8:59 am

Suggestions on the backtesting process

Post by Bounty »

Couldn't find any posts related to this. If there are threads related to this just let me know..

It's easy to test a gazillion parameters with TB. Any suggestions on the best way to track/process the results? For instance, using a spreadsheet the shows the system, parameter settings, date range, etc.?
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

Search for posts by user "bobsyd" ; as I recall, he confronted some of the same bewilderances.
dantes
Roundtable Knight
Roundtable Knight
Posts: 128
Joined: Sun May 08, 2011 1:40 pm

Post by dantes »

I believe this is the post Sluggo's referring to?

viewtopic.php?t=6281&highlight=
Bounty
Senior Member
Senior Member
Posts: 33
Joined: Tue Jun 19, 2007 8:59 am

Post by Bounty »

Yes, I saw that, thank you.

Was just reading a post by AFJ about how dramatically the results can change just by changing one parameter such as earn interest from on to off. That's exactly what I've experienced. Being a nube, the the fact that there are so many parameters that can have an impact and distort the results is sobering. The result is that I have to dig into the details and spreadsheets to try to verify what is happening. Not a lot of fun for my personality type, but necessary and it insures that I understand how the system operates.
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

Bounty wrote:Yes, I saw that, thank you.

Was just reading a post by AFJ about how dramatically the results can change just by changing one parameter such as earn interest from on to off. That's exactly what I've experienced. Being a nube, the the fact that there are so many parameters that can have an impact and distort the results is sobering. The result is that I have to dig into the details and spreadsheets to try to verify what is happening. Not a lot of fun for my personality type, but necessary and it insures that I understand how the system operates.
I was not saying that in the case of the substitute Turtle system that it was the impact of interest/non interest which made the difference. Nor indeed the ranking in which the orders were presented to the risk manager. Nor am I trying to frighten anyone or put them off. See a post I have just made on CSI data changes. I am merely saying that it is very easy indeed to fool oneself with back-testing and that as the years go by I realise with ever greater clarity why there is such a difference between back testing and reality.

I'm not trashing the game, I'm not trashing back testing. But you know that old adage about it taking a decade to become a doctor? Well, its true. I continue to make dumb mistakes and hopefully learn from them.
Post Reply