How to understand the conflit between performance and monte

[yoyo2000]

I have two systems and I perform them on the same market groups in the same period with the same position sizing rules.
And run the monte carlo simulations with their relating data separately.Thus I have two performance reports and two monte carlo simulation results.

the problem is there is a sharp conflit between those performance reports and mc simulations,that's,in performance reports,system A is superior than system B,but in mc simulations,system B is better than system A,how should I understand it?

and below are those two performance reports and mc simulations:

[yoyo2000]

this is a package including 4 snapshots of results of A & B's MC simulation

[Paul King]

It depends on your objectives what 'better' actually means. System A has better Net Profit, System B has better Gross Profit. System A has bigger maximum drawdown. You need to choose which variables are most important to you in order to determine what is better in your terms.

Looking at your system expectancy ((using average win * win%) - (average loss * loss%) )/ average loss, we get:

System A E=(3349.11*0.512 - 1217.62 * 0.488) / 1217.62= 0.92 (i.e. you should make 92c per unit of initial risk on average)

System B E=(3206.88 * 0.4978 - 1359.45 * 0.5022) / 1359.45 = 0.67

If we multiply E by the number of trades in each case we see that:

System A makes 0.92 * 209 = 192R (i.e. 192 times the approximate initial risk - I have assumed that your average loss is an approximation for initial risk since it is not explicitly stated in any performance reports)

System B makes 0.67 * 231 = 155R

Therefore, in terms of profit per unit of initial risk, System A is superior.

Hope this helps

Paul

[sluggo]

Your software has a bug: it reports drawdowns bigger than 100%, which isn't realistic.

Some traders and some Mechanica customers, like to focus on "CAGR" as a measure of profit, and "Max Drawdown" as a measure of risk. But not all people prefer these choices, which is why there are a vast profusion of system goodness measurements. One guy likes to create his own terminology and calls them "Bliss Functions" but don't get confused by the words. These are just mathematical expressions that try to measure what you like and don't like about a trading system.

Sharpe Ratio, Sortino Ratio, Return Retracement Ratio, Ulcer Index, Calmar Ratio, Semideviation Ratio, Seykota Lake Ratio, Kestner K-Ratio, MAR Ratio, Sterling Ratio, the list goes on and on. Because different traders have different opinions about what is important.

I, for example, would have preferred to see your software's Monte Carlo analysis of the Information Ratio (what TBB calls "Modified Sharpe Ratio") because I happen to like it better than Max Drawdown. If you had presented it for both A and B, I am guessing that I would have preferred the MC results for system B. But that is only a raw guess in the absence of data.

[yoyo2000]

Thanks Paul King,from the performance comparision,system A is superior,but from the statistical distribution of 10000 monte carlo simulation,the MDD % and Net % of A are bigger than those of B,from the theory of monte carlo simulation,should we consider that the result from performance report is a special result from the samples,but the result of 10000 MC simulation is an average result from the same sample?

sluggo,the MDD% in MC simulation should be based to the initial capital,so a MDD% more than 100% is normal.Do you agree with my point about MC simulation above?

An explaination from the theory and the view of statistics would be much appreciate.

regards.

[Paul King]

Yoyo,

Monte Carlo simulation. historical backtesting, and trading are not the same thing.

Historical backtesting is a representation of trading results that may have ocurred given a specific set of entry, exit, and position sizing rules, when applied to historical price data (which may have errors, omissions, bad ticks etc. that make the results only an approximation of real trading).

MC simulation takes a sample distribution of trades, assumes that sample is representative of all the trades that could ever be signalled by the system, and then uses statistical methods to simulate the range of possible results. If your sample (as it turns out) is not representative, then the results will not be either.

Real trading has its own set of problems that range from slippage, to technology problems, to exchange outages to psychological mistakes.

The only results that are actually 'real' are the ones in your brokerage account. All other 'testing' is only a guide to the effectiveness of a particular method or system, and a way to compare and contract different approaches.

Paul

ps. I have written a short article called 'The Main Caveats of Backtesting' that may interest you. It is available from my web site in the 'Articles' section.

[yoyo2000]

Hi,paul king,your statement is very good,of course I know there are much difference between test/simulation and reality,but the purpose of test/simulation is to build some self-confidence with my system before I have to paid for the costly cost,so I want to get a more close way.




PS:your article 'The Main Caveats of Backtesting' is not free,could you please sent me some sections to me please?

[Paul King]

I happen to believe that people actually get more out of advice/books/publications that they pay something for, and are more likely to value the content accordingly.

The article is $4.95 which is less than the cost of commission on one (discount brokerage) trade. There is a some free content on my site regarding trading systems and methods so I do not feel I am unjustified in actually charging a nominal amount for some of the more interesting and unique content that is based directly on my experiences of building my trading business.

Good luck in your trading.

Paul