Hypothetical vs real time results.

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
Dutchtrader
Roundtable Fellow
Roundtable Fellow
Posts: 58
Joined: Wed Apr 30, 2003 4:35 pm
Location: Netherlands

Hypothetical vs real time results.

Post by Dutchtrader »

Hi everyone,

While I am in a stage of testing and learning, there are enough traders in this forum ( I guess ) who already trade their system(s). What is their experience in case of the hypothetical vs real time results till now? There are many aspects which may influence your results: slippage higher on average, not enough liquidity, electrical failure, psychological..etc,etc

And if your results are lower then expected( which I reasonably assume ) can you give me an idea how much ( % )? Could you give a beginner like me, give criteria what, on average, the difference is between hypothetical and real time results?


Thanks in advance,

Marc
Ted Annemann
Roundtable Knight
Roundtable Knight
Posts: 118
Joined: Tue Apr 15, 2003 7:44 pm
Location: Arizona

Post by Ted Annemann »

My trading results have generally been better than computer testing has indicated. I use the "R-multiple" and "Geometric Average Trade" statistics for comparison.

However this is no surprise since I test using -$200 per contract for slippage. The actual trades in the real markets seem not to skid this badly (all my orders execute on the Open). I will continue to use -$200 for testing because it gives a pessimistic overestimate of drawdown. I'd much prefer for actual drawdowns to be less than simulated drawdowns than vice versa.
Kiwi
Roundtable Knight
Roundtable Knight
Posts: 513
Joined: Wed Apr 16, 2003 1:18 am
Location: Nowhere near

Post by Kiwi »

For long term multicommodity trading my experience is that the classic $70 slippage and commission isnt too far off as long as you pay under $30 round trip to your broker. I spent a year evaluating every single trade (70% stops, 30% market orders, a few limit) and found that the unders and overs balanced out.

The only exceptions were in Lumber (where I discovered locked limit days) and Rice which can be a little illiquid. For these I guess a 150 would more than cover it. I also discovered that roll overs typically cost more than the system assumes so when I do an evaluation I use 150 instead of 70 to allow for rollovers during each trade and this seems to provide a good worst case.

I haven't traded intraday index systems thru brokers so can't give an opinion on that. I have found though, that counter trend systems can have negative slippage so, after evaluation for a few months, you might drop them back to 0.

My caution would be that anyone new to this game start of trading lighter (maybe 50% - put the rest of the money in a high interest account for the first year) than they think they can. Real drawdowns can exceed historical drawdowns and open trade drawdowns are likely to be twice the closed trade drawdowns. Both of these issues may cause you to make mistakes such as not taking trades just when things come right. I remember Randy Stuckey telling me that a number of people who had bought his systems had come back to him and complained that they didnt work but when he looked at their accounts he found that they skipped trades under pressure or because they thought they new better (you know --- the TBond cant go higher now!!) Start light and learn how you respond to drawdowns ... really work on this as I think its the thing that kills most system traders.

Also go and read all of the earlier developer interviews at:
http://www.futurestruth.com/articles.html
and think hard about what they say about "how to trade their systems". They have a lot of experience of people failing in system trading because its psychologically difficult to keep the faith when its all seems to be falling apart around you. An 8 month 30% drawdown is very hard to explain to your wife.


Good Luck,
John
Kiwi
Roundtable Knight
Roundtable Knight
Posts: 513
Joined: Wed Apr 16, 2003 1:18 am
Location: Nowhere near

Good Quotes from System Developers

Post by Kiwi »

I think that one of the most important things a systems trader can do (if he or she can afford it) is to trade muliple systems that have low degrees of correlation to one another. I have actually done extensive work in this area, not only on my own systems, but also on some of the other systems tracked by Futures Truth.
Have You Been Trading This Year?

Yes, I couldn't in good conscience sell a system unless I traded it. Last year was a good one for our portfolios, especially the smaller ones that most buyers trade. And this year is well ahead of pace for most of the portfolios. But every year will see equity rallies and drawdowns with xxxxx, or any other system. The key to trading is to build a plan around the performance of your system and recognize when performance, both good and bad, is within the bounds of historical. When it dramatically steps outside those bounds, the cause needs to be determined and steps taken.
Note that I personally trade a system developed by a guy who doesnt trade --- but I understand why he doesnt.
I think the most important thing I wish somebody had shared with me early on was the importance of position sizing and money management. I remember my first few years as a system developer were spent focusing almost exclusively on single contract based systems. Quite frankly, most of the testing products available really only allow for this type of testing. When I eventually did start testing on money management based platforms, I got a very rude surprise! I found out that the best single contract based systems almost invariably were not the best money management systems. This flew in the fact of much conventional wisdom that says, "Find the best single contract based system first and then apply money management to it." I learned the hard way that this does not work. I had to pretty much start from scratch rebuilding money management based systems.
Someone said fear and greed drive the commodities markets. I think this is partially true. I suspect fear is the stronger of the two. It sure seems that way as I've observed people consistently taking early profits. We've seen people take $5000 early profits on XXXX trades that eventually were closed out with over $20,000 in profits.
Once you have researched and chosen a system to follow, trade that system's rules precisely, and give it time to work. Know the system and believe in it. Make sure that you understand how the developer intended the system to be traded and try to do exactly that. If you can't execute trades the way the system requires, don't expect to duplicate the system's performance. If you are unable to follow a system exactly, many brokerage firms have a "system assist" division that will trade your account based on the rules of the system. Once you're following a system, give the trades some time to work. A system with a long term track record with good results may still have periods of months with no profits. You have to be ready to accept that and stick with the system long enough to see it work.
edward kim
Roundtable Knight
Roundtable Knight
Posts: 344
Joined: Sun Apr 20, 2003 2:42 pm
Location: Silicon Valley / San Jose, CA USA
Contact:

Post by edward kim »

I use the same $200.00 skid that Ted uses - it's also a good idea to see how each market skids differently. Coffee will skid a lot more than the E-mini S&P, and the full Yen contract will skid way way more than the mini-contract (I use a tick skid rather than a dollar skid.)

Edward
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

Post by damian »

hi ed/edward/eck :D

=> mini yen has less skid than big yen? I didn't think that the mini even traded (well, perhaps a few contracts per day).

I may be very wrong, but I also thought that the mini and full yen contract are both traded on globex.

Have I overlooked something?

I also find that coffer skids, as does cotton and lumber. So does palladium.... but that is to be expected.

I have even had split fills on 10 lot corn orders!

cheers
damian
edward kim
Roundtable Knight
Roundtable Knight
Posts: 344
Joined: Sun Apr 20, 2003 2:42 pm
Location: Silicon Valley / San Jose, CA USA
Contact:

Post by edward kim »

Hey Damian,

The spreads on certain mini contracts are much wider than the full contract, while others have better liquidity and volume. Let's say at midnight Eastern Standard Time, the full Yen (12.5 million) will look like this:

300 bid .008621, 120 ask .08624

while the mini contract will look like this:

1 bid .008598, 1 ask .08640

Let's say your stop gets hit on the ask, but you need to get out of your contracts at the bid. Your first contract will slip 42 ticks, and who knows what the rest will get out at. Even though the contract is smaller, the big contract will only slip 3 ticks, and you probably won't skidded because there is a lot of size on the bid and ask.

Edward
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

Post by damian »

Thanks ed, this is exactly how how I understood the situation to be. I mis-read your first post.

cheers
damian
damian
Roundtable Knight
Roundtable Knight
Posts: 815
Joined: Tue Apr 15, 2003 8:43 pm
Location: dusseldorf

Post by damian »

Some of you may be shocked that I don't look at this on a constant basis. Today, after a little under 2 years of real time results, I spent my time comparing historical test output (from the time that I actually started trading) to the results that are real over the same time period.

I am amazed at how similar they are (at least at the portfolio level - I am yet to drill into the individual market results). There is no reason why they shouldn't be I suppose. It is just that I always took back test data with a grain of salt. Like some other people, I tend to focus more on looking at the back test equity curve (as another topic, I am yet to quantify this method. i follow other such attempts on this forum with interest).

After 87 real time personal account trades on my system, it was an interesting excersise. Go ahead, make funny shapes with your eyebrows :)
Last edited by damian on Sun May 18, 2003 6:21 am, edited 1 time in total.
Dutchtrader
Roundtable Fellow
Roundtable Fellow
Posts: 58
Joined: Wed Apr 30, 2003 4:35 pm
Location: Netherlands

Thanks

Post by Dutchtrader »

Thanks Ted, Damian, Ed and Kiwi for your input! :)

Marc
Post Reply