Old European wrote:I first want to say that I'm convinced that c.f. (and sluggo) is doing a great job in implementing Monte Carlo in TBB. The graphs presented so far look really really nice.
I also think I understand why he is adding this Monte Carlo functionality: the system developers' marketplace asks for it and competing products allow also for MC testing.
I however have a fundamental theoretical problem with Monte Carlo and system testing.
As sluggo described very well above, it's all about looking at trading results in parallel universes. These universes can be created in two (or more) different ways. Or one randomizes the order in which trades are generated or one randomizes the daily (or ...) returns.
I am not convinced that this makes a lot of sense. While backtesting I always note that a lot of the time when I'm long USD against EUR I am also long USD against CHF ... when I'm short LC I am also short FC ... and so on. There are intermarket correlations. When one randomizes the trade order, one doesn't take these correlations into account and the newly created parallel universe has nothing to do with the real world.
The same thing applies to the randomization of daily returns. Again when backtesting I always observe that there are regularly periods of mostly positive daily returns. Why? Because there are strong trends in a number of markets and a market trends until it finally bends. When the daily returns are randomized, all of this gets lost and the newly create world is very irreal.
Ideally one should only look at parallel universes and virtual market data time series that show similar correlations and trending behavior as the real world. These are however extremely difficult to construct.
Cheers,
Old European
Is Monte Carlo Analysis Useful?

 Roundtable Knight
 Posts: 1842
 Joined: Tue Apr 15, 2003 11:02 am
 Contact:
Is Monte Carlo Analysis Useful?
[MODERATOR'S NOTE: This topic was split from one in the Trading Blox Support Area discussing the new Monte Carlo feature in Trading Blox.]

 Roundtable Knight
 Posts: 1842
 Joined: Tue Apr 15, 2003 11:02 am
 Contact:
I had the same gut reaction to Monte Carlo analysis of Trading Systems when I first examined the idea. However, I now see it and its output like any other statistic from any simulation, more information useful for some things but with limitations or shortcomings; If we examined any statistic: MAR, Sharpe, Maximum Drawdown, etc. we could make an argument for it being incomplete, innaccurate, misleading...
I see three major shortcomings with Monte Carlo analysis when performed in the traditional ways:
Indeed, we do this partially with Trading Blox' upcoming Monte Carlo feature. We allow users to group days together to preserve the effects of multiday market movements and to partially account for serial correlation (sometimes referred to as autocorrelation).
What we are left with, after our best attempts at improvement, is imperfect. It does not have the same meaning as might an analysis of failure rates for parts coming off an assembly line. A 95% confidence level does not mean that there is really a mathematical 95% chance that something will or will not happen.
Nevertheless, randomly scrambling our equity curves does show us how sensitive our outputs are to the specific ordering of events. It also, in the process, erodes our faith in the numbers we get as outputs to a specific simulation. This is a good thing.
In the beginning especially, we tend to think of test results as authoritative and weighty; e.g. this system is better than that because MAR of 1.9 is better than 1.75 , or Sharpe of 1.4 is better than 1.7 etc. This "precision" is misleading as the likely margin of error often exceeds all but the largest differences in performance testing output. So to the extent that looking at the outputs of Monte Carlo analysis erodes our "faith" in the specific outputs of a single test, this is a good thing.
Also if we can alter/lower our expectations as a result of seeing graphs showing a 20% chance of catastrophe based only on randomly changing the equity each day, that too is a good thing. It will be far easier to ride out a drawdown if we expected it because of Monte Carlo modeling than if it freaks us out because it exceeded our historical test by some specific but arbitrary margin.
 Forum Mgmnt
I see three major shortcomings with Monte Carlo analysis when performed in the traditional ways:
 No accounting for serial correlation of returns.
 No accounting for intermarket correlation.
 No accounting for the psycholigical effects of the market.
Indeed, we do this partially with Trading Blox' upcoming Monte Carlo feature. We allow users to group days together to preserve the effects of multiday market movements and to partially account for serial correlation (sometimes referred to as autocorrelation).
What we are left with, after our best attempts at improvement, is imperfect. It does not have the same meaning as might an analysis of failure rates for parts coming off an assembly line. A 95% confidence level does not mean that there is really a mathematical 95% chance that something will or will not happen.
Nevertheless, randomly scrambling our equity curves does show us how sensitive our outputs are to the specific ordering of events. It also, in the process, erodes our faith in the numbers we get as outputs to a specific simulation. This is a good thing.
In the beginning especially, we tend to think of test results as authoritative and weighty; e.g. this system is better than that because MAR of 1.9 is better than 1.75 , or Sharpe of 1.4 is better than 1.7 etc. This "precision" is misleading as the likely margin of error often exceeds all but the largest differences in performance testing output. So to the extent that looking at the outputs of Monte Carlo analysis erodes our "faith" in the specific outputs of a single test, this is a good thing.
Also if we can alter/lower our expectations as a result of seeing graphs showing a 20% chance of catastrophe based only on randomly changing the equity each day, that too is a good thing. It will be far easier to ride out a drawdown if we expected it because of Monte Carlo modeling than if it freaks us out because it exceeded our historical test by some specific but arbitrary margin.
 Forum Mgmnt

 Roundtable Knight
 Posts: 144
 Joined: Mon Apr 19, 2004 3:04 pm
 Location: Old Europe

 Roundtable Fellow
 Posts: 50
 Joined: Fri Apr 18, 2003 8:53 pm
 Location: Huntsville, AL
 Contact:
Faulting Monte Carlo by saying that it cannot account for correlation is like faulting Calculus because it is limited to single variables. That's an incorrect claim. Correlation can definitely be accounted for in Monte Carlo analysis. There are standard techniques of accounting for correlation and lots of published research, dating back at least 20 years. One type of correlation has been implemented in commerical Excel addin packages including Crystal Ball and @Risk.
And there is an even easier technique that is simpler and can be calculated on a hand calculator. The method of moments can be used to aggregate the mean and standard deviations of the returns for each market (correlation is easy to account for in this approach). Given enough markets (and a couple of other conditions that most likely will hold), the two toplevel moments can be fit to a Normal distribution. In my work, I have found that this analytic approximation is as accurate as Monte Carlo simulation.
And there is an even easier technique that is simpler and can be calculated on a hand calculator. The method of moments can be used to aggregate the mean and standard deviations of the returns for each market (correlation is easy to account for in this approach). Given enough markets (and a couple of other conditions that most likely will hold), the two toplevel moments can be fit to a Normal distribution. In my work, I have found that this analytic approximation is as accurate as Monte Carlo simulation.
MonteCarlo
Still trying to understand montecarlo analysys, i came up with these differents montecarlo lines.
One is on the center, the other is almost on the lowest.
Does it mean something?
thank you
One is on the center, the other is almost on the lowest.
Does it mean something?
thank you
 Attachments

 mc low.jpg (47.6 KiB) Viewed 3682 times

 mc medium.jpg (52.19 KiB) Viewed 3683 times

 Roundtable Knight
 Posts: 1429
 Joined: Mon Dec 22, 2003 12:24 pm
 Location: Des Moines, IA
 Contact:
Re: MonteCarlo
No.marriot wrote:Does it mean something?
Re: MonteCarlo
A prediction of the future? Prolly not. But I do infer aspects of 'health and character' from these displays. Some may find scientific value within (or not). For me, the value is more right brain.Levi wrote:No.marriot wrote:Does it mean something?