Performance reporting dilemma

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
Dean Hoffman
Roundtable Fellow
Roundtable Fellow
Posts: 87
Joined: Wed Jul 02, 2003 3:04 pm

Performance reporting dilemma

Post by Dean Hoffman »

When running simulations including compounding the results can become extreme (ending equity in the billions etc.). One possible solution to this is to simply reset the equity back to the initial amount each year. However, this creates its own problems. For example, assume a 100k investor only risking 2% of equity per trade. This investor would have to skip trades with risk above $2,000. However, if this investor made a 100% return year one and reinvested the profits then they would have 200k (or so after taxes) year two and be able to risk $4,000 per trade, thus skipping fewer trades etc. If the test showed equity being reset back to 100k this would show trades being skipped in year two that in fact would NOT have been skipped had the investor reinvested the profits. Similarly, if you lost 50% of equity year one you would have to skip many more trades year two. Resetting the equity would magically give back the money you lost (don’t we wish!)

Other problems have to do with showing a simulation where contract size is above what can realistically can be traded in that market. However, your starting equity might have supported trading that market but later equity got too big to trade that market. Do you therefore show a $10,000,000 account trading at the amount that a $100,000 account would? This skews the proper relationship of performance to equity amount. You might have been able to trade the market in the beginning of the simulation but not at the end. If you limit that market to a percentage of open interest then you skew the results had you assumed you started later in the simulation. Your returns would appear smaller and smaller each year as you traded relatively less and less contracts as a percentage of equity. However, this might not have been the case had you started with 100k during the later part of the simulation and not hit the contract limitations assumed by the larger amount of equity based on compounding from an earlier starting date.

All of this creates a rather complex reporting dilemma!

I’m curious how forum members would want to see reporting to properly deal with the changing variables above?
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

One of the markets I trade (and therefore, include in my tests) is the fullsize Nasdaq. The long term average $risk per contract of ND at trade entry, i.e. (Bigpointvalue * (entryprice - stopprice)), for my systems, is about $21K. That's the average value; for some ND trades it's $17K and for others it's $25K.

In some tests using some position sizing approaches, I'll run with a 60+ market portfolio risking 0.5% of equity per trade. Since the simulations are less susceptible to roundoff errors when trading several contracts (rounding 1.3 contracts down to 1 is a bigger difference than rounding 7.3 contracts down to 7), I like to simulate with enough equity so I'm trading at least 10 contracts per trade. It makes me feel less nervous that rounding off the #contracts is distorting the simulation results.

If you think of this as a Math Word Problem and ask, "What is the minimum account equity needed to guarantee position sizes of at least 10 contracts in the worst case", you get this little equation:

(10 * $25000) = 0.005 * StartEquity

for which the solution is: StartEquity = $50 million.

With this approach you can start the account at $50M and run the simulation. Every year on January 1, sweep the excess equity (above $50M) into a cash account. Report performance in terms of Compounded Growth and plot it on a chart that has "equity multiples" on the Y axis, "date" on the X axis.

Another market with a huge $risk per contract at trade entry, is the DAX.
Dean Hoffman
Roundtable Fellow
Roundtable Fellow
Posts: 87
Joined: Wed Jul 02, 2003 3:04 pm

Post by Dean Hoffman »

Forum Mgmnt,

I’m curious to know if you have given thought to the reporting issues I mentioned above. How might you suggest they be dealt with during a Veri-Trader simulation?

Regards,
Dean Hoffman
Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt »

Dean,

When I run tests, I'm generally only interested in getting the best answer to the question: "what results am I likely to get in the future with this aproach?".

In the example you outline, a new investor who has $100K for trading is really interested in the system's performance at that level, not how much money he'll be able to make once he reaches $300K. So a simulation with a restart would offer a more realistic picture of the first year's trading.

I generally test with a fairly high starting balance and ignore the fact that the number of contracts exceeds available liquidity because I'm interested in historical returns for the first few years only.

I certainly run tests with volume constraints turned on, but for a test of any length you quickly exceed these constraints on most markets except currencies and interest rate futures as equity balloons because of compounding. So I generally turn off the volume constraints in VeriTrader and make sure my portfolios only contain liquid futures that I will be able to trade for at least several years at the sizes I am interested in.

However, I do believe that an equity reset approach as outlined by sluggo is better and a variation of that suggestion has been on the "features to implement" list for over a year.

This is especially important for smaller accounts since a test using this approach will give you results which better reflect the reduced opportunities available when single contract risk exceeds available risk equity and you can't trade a given market.

- Forum Mgmnt
Dean Hoffman
Roundtable Fellow
Roundtable Fellow
Posts: 87
Joined: Wed Jul 02, 2003 3:04 pm

Post by Dean Hoffman »

Bob Spear (of Trading Recipes) emailed me his thoughts on the subject and said I could post.

"Dean
I don't pay a lot of attention to ending equity. I Do, however, pay a lot of attention to compound annual return info. That number is not influenced by equity level. Similarly, I don't pay attention to the equity chart but Do pay attention to the logarithmic equity chart. The equity chart is interesting but ultimately useless. What I really look at are monte-carlo studies showing distribution(s) of returns and drawdowns and, of course, the various percentile charts (using a high number of bars per iteration).
I don't think it is a particularly bad idea to test trading more contracts than one could actually put on because I'm most interested in the many many 5 year (or so) chunks that the monte-carlo studies show."

Hope this helps
Bob
ksberg
Roundtable Knight
Roundtable Knight
Posts: 208
Joined: Fri Jan 23, 2004 1:39 am
Location: San Diego

Post by ksberg »

DHoffman wrote:Bob Spear (of Trading Recipes) emailed me his thoughts on the subject and said I could post.

"Dean
I don't pay a lot of attention to ending equity. I Do, however, pay a lot of attention to compound annual return info. That number is not influenced by equity level. Similarly, I don't pay attention to the equity chart but Do pay attention to the logarithmic equity chart. The equity chart is interesting but ultimately useless. What I really look at are monte-carlo studies showing distribution(s) of returns and drawdowns and, of course, the various percentile charts (using a high number of bars per iteration).
I don't think it is a particularly bad idea to test trading more contracts than one could actually put on because I'm most interested in the many many 5 year (or so) chunks that the monte-carlo studies show."

Hope this helps
Bob
Interesting. This parallels the direction I've moved myself (strength of compound growth, log equity curve, distribution of returns and drawdowns).

I think part of the dilemma are the expectations set by existing "standards".

Cheers,

Kevin
Post Reply