Monte Carlo analysis of trading systems
The Mistake
Can you share with us a little bit more of knowledge about 'the fatal mistake'?
- Attachments
-
- montecarlo1.jpg (142.79 KiB) Viewed 18385 times
-
- Site Admin
- Posts: 1493
- Joined: Fri Sep 12, 2003 10:32 am
- Location: Austin, Texas
Well, if the idea is to scramble the monthly returns I don't think that the "used" numbers should be placed back in the "bag". A strict reordering of the sequence of returns would imply making a little pile of your slips of paper beside the bag as they are drawn and only replacing them at the end of the run when you are ready to generate another alternative history.
I would not feel that I would have a fair sample if I had the opportunity to draw my biggest outlier returns / drawdowns several times during one "history".
Do you see something else there that is amiss?
Jake
I would not feel that I would have a fair sample if I had the opportunity to draw my biggest outlier returns / drawdowns several times during one "history".
Do you see something else there that is amiss?
Jake
Jack
It seems to me the the article you mentioned is availabe only to subscribers.
The scramble method you mentioned above doesn't seems compatible with the following definition:
It seems to me the the article you mentioned is availabe only to subscribers.
The scramble method you mentioned above doesn't seems compatible with the following definition:
Equity Curve Scramble: this is the simplest method, and does not involve using the raw trades during the generation of randomized Monte Carlo runs.The Equity Curve scramble works by populating a new equity curve one bar at a time. For each bar, a random bar of the baseline equity curve is selected, and the bar to bar percentage return is applied to the randomized equity curve. Equity Curve Scramble allows returns from the initial equity curve to be resampled (if it did not allow resampling, all resulting equity curves would have the same Net Profit).This method effectively captures the dynamics of the historical testing period, including price shock events, because the effects of multiple positions reacting to events is captured in the baseline equity curve and translated to the randomized Monte Carlo equity curves.
- Attachments
-
- montecarlo2.jpg (178.72 KiB) Viewed 18301 times
-
- Site Admin
- Posts: 1493
- Joined: Fri Sep 12, 2003 10:32 am
- Location: Austin, Texas
You are correct in what you write Kianti. The method I proposed will end up with the same net performance, but drawdowns and profits will be distributed differently over the timeline of the curve.
Given that, perhaps this is not the "fatal mistake" jankiraly refers to. I will have to give it another read through.
Also thanks to Ted for pointing out that "right" and "wrong" ways of doing analysis are quite subjective. The form of analysis that is useful for your circumstance depends on that circumstance.
The type of Monte Carlo analysis I proposed is handy for the trader that wants to answer a question like, "I am happy with my net results, but what would I do if my four worst drawdown months happened back to back? How much would that drawdown be, and could I trade through it?"
Jake
Given that, perhaps this is not the "fatal mistake" jankiraly refers to. I will have to give it another read through.
Also thanks to Ted for pointing out that "right" and "wrong" ways of doing analysis are quite subjective. The form of analysis that is useful for your circumstance depends on that circumstance.
The type of Monte Carlo analysis I proposed is handy for the trader that wants to answer a question like, "I am happy with my net results, but what would I do if my four worst drawdown months happened back to back? How much would that drawdown be, and could I trade through it?"
Jake
Special thanks to everybody for the the help in the Rally of Montecarlo.
Ted, you are correct, the so-called ‘definition’ is one of the WL methods to perform an MC simulation; as you mentioned in your previous posts there are several ways to perform an MC simulation. The major problem with WL Montecarlo, I guess, it’s that is built on the No.2 method you mentioned in another post, the so called ‘returns’. I can resample daily equity curve changes but if I try to resample the single trades the results could be very misleading with futures, take Crude Oil back-adjusted futures as an example. That means that I cannot run all the other more sophisticated methods with WL Montecarlo.
The choices available for me are quite scaring: trying to write my custom C# software, Excel VBA or the infamous Count of MonteCristo.
So far MC for me is more of an art than science
Best regards, as ever
Ted, you are correct, the so-called ‘definition’ is one of the WL methods to perform an MC simulation; as you mentioned in your previous posts there are several ways to perform an MC simulation. The major problem with WL Montecarlo, I guess, it’s that is built on the No.2 method you mentioned in another post, the so called ‘returns’. I can resample daily equity curve changes but if I try to resample the single trades the results could be very misleading with futures, take Crude Oil back-adjusted futures as an example. That means that I cannot run all the other more sophisticated methods with WL Montecarlo.
The choices available for me are quite scaring: trying to write my custom C# software, Excel VBA or the infamous Count of MonteCristo.
So far MC for me is more of an art than science
Best regards, as ever
- Attachments
-
- montecarlo3.jpg (163.01 KiB) Viewed 18140 times
Considering the large DDs in 2003 experienced by some well-know trading systems, I went into the Montecarlo lab and tried the following experiment:
- trading system: aberration plus on midsize portfolio using an approximated 1,5% fixed fractional rule
- 10,000 equity curve scrambles of daily returns using the above 1,5% risk position sizing rule with DD% simulations at 31/12/2002 and 05/02/2004.
The distribution simulation of %DDs returned:
- largest DD% simulated value: -38.82%
- largest 1% DD% simulated value: -28.48%
The trading system simulation at 05/02/2004 returned:
- max %DD: -31,69%
I guess the end result of any MC simulation is always a distribution similar to the one above; I understand also that just one trading system trial is not enough.
I’m asking your opinion on how to read a DD% distribution simulation (tabular result to export into Excel are available to use more complex statistical measures).
Would it be correct to assume the largest 1% as conservative measure or Uncle Bob or Uncle Point…. Or is just a mere coincidence?
Best regards, as ever
- trading system: aberration plus on midsize portfolio using an approximated 1,5% fixed fractional rule
- 10,000 equity curve scrambles of daily returns using the above 1,5% risk position sizing rule with DD% simulations at 31/12/2002 and 05/02/2004.
The distribution simulation of %DDs returned:
- largest DD% simulated value: -38.82%
- largest 1% DD% simulated value: -28.48%
The trading system simulation at 05/02/2004 returned:
- max %DD: -31,69%
I guess the end result of any MC simulation is always a distribution similar to the one above; I understand also that just one trading system trial is not enough.
I’m asking your opinion on how to read a DD% distribution simulation (tabular result to export into Excel are available to use more complex statistical measures).
Would it be correct to assume the largest 1% as conservative measure or Uncle Bob or Uncle Point…. Or is just a mere coincidence?
Best regards, as ever
- Attachments
-
- ABDD-ALL.jpg (44.66 KiB) Viewed 18122 times
-
- ABDD-31122002.jpg (39.51 KiB) Viewed 18123 times
-
- MCDD-31212002 b.jpg (124.06 KiB) Viewed 18125 times
-
- Roundtable Fellow
- Posts: 68
- Joined: Fri May 16, 2003 8:28 am
- Location: Atlanta, GA
Kianti,kianti wrote:The major problem with WL Montecarlo, I guess, it’s that is built on the No.2 method you mentioned in another post, the so called ‘returns’. I can resample daily equity curve changes but if I try to resample the single trades the results could be very misleading with futures, take Crude Oil back-adjusted futures as an example. That means that I cannot run all the other more sophisticated methods with WL Montecarlo.
I'm not sure what you mean here. I wouldn't say that MC-Lab is built on any of the specific methods, you do have several options regarding which method you would like to use. Sorry if I've misunderstood you...
I usually set up MC Lab with the options "Trade Scramble", "Maintain Win/Loss Sequence", "Maintain Serial Correlation" all active. I've found it interesting that the results usually show that the actual drawdown reported in the Simulator almost always falls below the 1% extreme expected drawdown based on MC-Lab runs. I'm not sure if I should interpret this as a flaw in the logic behind this type of MC run, or as evidence that trend trading the stock market was, in reality, as brutal as it could have possibly been in the 2000-02 period.
Jason,
I probably didn’t make myself clear or maybe I was just wrong. I was referring to problems with back-adjusted contracts such as Crude Oil.
The ‘Raw Trades Information’ used by WL Montecarlo use the ‘Percentage Profit’, as you will see in the images below on the first trade you buy at –88 and sell at 323, profit is $4,110 but the percentage change is –467.05%; while back-adjusting doesn’t affect the real $ profit/loss in WL you get a return on the trade of –467.05%.
If you use the equity scramble or the trade scramble doesn’t seem to make any difference; if you use the ‘trade scramble and randomize’ it makes a big difference.
Having said that, I consider myself an absolute beginner on Montecarlo.
Best regards, as ever
I probably didn’t make myself clear or maybe I was just wrong. I was referring to problems with back-adjusted contracts such as Crude Oil.
The ‘Raw Trades Information’ used by WL Montecarlo use the ‘Percentage Profit’, as you will see in the images below on the first trade you buy at –88 and sell at 323, profit is $4,110 but the percentage change is –467.05%; while back-adjusting doesn’t affect the real $ profit/loss in WL you get a return on the trade of –467.05%.
If you use the equity scramble or the trade scramble doesn’t seem to make any difference; if you use the ‘trade scramble and randomize’ it makes a big difference.
Having said that, I consider myself an absolute beginner on Montecarlo.
Best regards, as ever
- Attachments
-
- CL trades.jpg (114.86 KiB) Viewed 18059 times
Thanks for the reminder, I tried to convert the last part of The Count of Monte Carlo .jpg (145KB), these is the result of the conversions:
.png 738KB
.gif 183KB
.pdf 362KB
Any other suggestion to reduce the file sizes?
best regards, as ever
.png 738KB
.gif 183KB
.pdf 362KB
Any other suggestion to reduce the file sizes?
best regards, as ever
- Attachments
-
- montecarlo4.jpg (145.06 KiB) Viewed 17349 times
-
- Roundtable Knight
- Posts: 1842
- Joined: Tue Apr 15, 2003 11:02 am
- Contact:
I think they're pretty funny.
I've been looking at them over dialup all week. With Internet Explorer 6.0 at least, they all load in the background after the text so you can read everything without having to wait for the images. So, I'm not seeing any real adverse effect.
If others don't like this, the easiest solution might be to move them to a separate thread with a warning in the topic.
What effect are the large images having on others?
- Forum Mgmnt
I've been looking at them over dialup all week. With Internet Explorer 6.0 at least, they all load in the background after the text so you can read everything without having to wait for the images. So, I'm not seeing any real adverse effect.
If others don't like this, the easiest solution might be to move them to a separate thread with a warning in the topic.
What effect are the large images having on others?
- Forum Mgmnt
http://www.thisislondon.co.uk/news/lond ... 20Standard£1.3m casino 'sting'
By Nilufer Atik, Evening Standard
22 March 2004
One industry source said: "There have been rumours for several years about a device that could compute the rate at which the speed of a roulette wheel and the ball degrade.
Black Swan or fat tail !?!
best regards, as ever
-
- Roundtable Knight
- Posts: 100
- Joined: Wed Apr 30, 2003 2:27 pm
- Location: Minneapolis, MN
- Contact:
The incident shows that someone finally succeeded at the mission of the brainiacs who starred in Eudaemonic Pie. The same guys behind the The Predictors.
link to book descriptions:
http://www.amazon.com/exec/obidos/ISBN% ... ingblox-20
http://www.amazon.com/exec/obidos/tg/de ... 1?v=glance
link to book descriptions:
http://www.amazon.com/exec/obidos/ISBN% ... ingblox-20
http://www.amazon.com/exec/obidos/tg/de ... 1?v=glance
-
- Senior Member
- Posts: 30
- Joined: Thu Feb 26, 2004 12:50 pm
- Location: Castlegar, BC, Canada
- Contact:
kianti wrote: One industry source said: "There have been rumours for several years about a device that could compute the rate at which the speed of a roulette wheel and the ball degrade.
Black Swan or fat tail !?!
best regards, as ever
I think Edward Thorpe developed the first wearable computer to do this in the 1960's. The man played with many marlbles.
c.f. wrote:
Presume the lack of replies implies that most are not affected. I'm not.What effect are the large images having on others?
- Forum Mgmnt
If any others are indeed affected it would be nice to hear.