How to know if system is working?

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
LeviF
Roundtable Knight
Roundtable Knight
Posts: 1428
Joined: Mon Dec 22, 2003 12:24 pm
Location: Des Moines, IA
Contact:

How to know if system is working?

Post by LeviF » Tue May 06, 2008 6:51 am

What methods do you use to tell if a system is performing in line with historical testing? How many real trades or months should be analyzed to be significant?

DMFord
Roundtable Fellow
Roundtable Fellow
Posts: 88
Joined: Wed Jun 27, 2007 8:11 am
Location: NB, Canada

Post by DMFord » Wed May 07, 2008 3:32 am

Hi,

I started using TB in August 2007. I put together a cautious out-of-the-box system targetting 12% CAGR. I checked the entried & exits every day until Christmas (5 months) I came to see that what happened in TB represented what happened in the actual trading.

There were occasional discrepancies (to err is human) but these were the exception, rather than the norm. I also found that some errors end up making you money (although I wouldn't recommend this as a strategy!) I found 1 or 2 markets did not behave well - and I frequently had entries & stops missed - so I dropped them from the portfolio. But the only way to discover this was to try them.

After this I felt I trusted the package / system enough to be much more aggressive, and have now developed my own variations & upped my targetted returns

As such, I think a few months would be the minimum, in order to take in a variety of market conditions. Also, just beacuse you're losing - doesn't mean the system is wrong - as long as the system is accurately tracking what you see in your account, it is working.

Hope this helps a bit.

Post Reply