Monte Carlo analysis of trading systems

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

The Mistake

Post by kianti »

Can you share with us a little bit more of knowledge about 'the fatal mistake'? :lol:
Attachments
montecarlo1.jpg
montecarlo1.jpg (142.79 KiB) Viewed 18192 times
Jake Carriker
Site Admin
Site Admin
Posts: 1493
Joined: Fri Sep 12, 2003 10:32 am
Location: Austin, Texas

Post by Jake Carriker »

Well, if the idea is to scramble the monthly returns I don't think that the "used" numbers should be placed back in the "bag". A strict reordering of the sequence of returns would imply making a little pile of your slips of paper beside the bag as they are drawn and only replacing them at the end of the run when you are ready to generate another alternative history.

I would not feel that I would have a fair sample if I had the opportunity to draw my biggest outlier returns / drawdowns several times during one "history".

Do you see something else there that is amiss?

Jake
kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti »

Jack

It seems to me the the article you mentioned is availabe only to subscribers.

The scramble method you mentioned above doesn't seems compatible with the following definition:
Equity Curve Scramble: this is the simplest method, and does not involve using the raw trades during the generation of randomized Monte Carlo runs.The Equity Curve scramble works by populating a new equity curve one bar at a time. For each bar, a random bar of the baseline equity curve is selected, and the bar to bar percentage return is applied to the randomized equity curve. Equity Curve Scramble allows returns from the initial equity curve to be resampled (if it did not allow resampling, all resulting equity curves would have the same Net Profit).This method effectively captures the dynamics of the historical testing period, including price shock events, because the effects of multiple positions reacting to events is captured in the baseline equity curve and translated to the randomized Monte Carlo equity curves.
Attachments
montecarlo2.jpg
montecarlo2.jpg (178.72 KiB) Viewed 18108 times
Jake Carriker
Site Admin
Site Admin
Posts: 1493
Joined: Fri Sep 12, 2003 10:32 am
Location: Austin, Texas

Post by Jake Carriker »

You are correct in what you write Kianti. The method I proposed will end up with the same net performance, but drawdowns and profits will be distributed differently over the timeline of the curve.

Given that, perhaps this is not the "fatal mistake" jankiraly refers to. I will have to give it another read through.

Also thanks to Ted for pointing out that "right" and "wrong" ways of doing analysis are quite subjective. The form of analysis that is useful for your circumstance depends on that circumstance.

The type of Monte Carlo analysis I proposed is handy for the trader that wants to answer a question like, "I am happy with my net results, but what would I do if my four worst drawdown months happened back to back? How much would that drawdown be, and could I trade through it?"

Jake
kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti »

Special thanks to everybody for the the help in the Rally of Montecarlo.

Ted, you are correct, the so-called ‘definition’ is one of the WL methods to perform an MC simulation; as you mentioned in your previous posts there are several ways to perform an MC simulation. The major problem with WL Montecarlo, I guess, it’s that is built on the No.2 method you mentioned in another post, the so called ‘returns’. I can resample daily equity curve changes but if I try to resample the single trades the results could be very misleading with futures, take Crude Oil back-adjusted futures as an example. That means that I cannot run all the other more sophisticated methods with WL Montecarlo.
The choices available for me are quite scaring: trying to write my custom C# software, Excel VBA or the infamous Count of MonteCristo.
So far MC for me is more of an art than science

Best regards, as ever
Attachments
montecarlo3.jpg
montecarlo3.jpg (163.01 KiB) Viewed 17947 times
kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti »

Considering the large DDs in 2003 experienced by some well-know trading systems, I went into the Montecarlo lab and tried the following experiment:
- trading system: aberration plus on midsize portfolio using an approximated 1,5% fixed fractional rule
- 10,000 equity curve scrambles of daily returns using the above 1,5% risk position sizing rule with DD% simulations at 31/12/2002 and 05/02/2004.

The distribution simulation of %DDs returned:
- largest DD% simulated value: -38.82%
- largest 1% DD% simulated value: -28.48%

The trading system simulation at 05/02/2004 returned:
- max %DD: -31,69%

I guess the end result of any MC simulation is always a distribution similar to the one above; I understand also that just one trading system trial is not enough.
I’m asking your opinion on how to read a DD% distribution simulation (tabular result to export into Excel are available to use more complex statistical measures).
Would it be correct to assume the largest 1% as conservative measure or Uncle Bob or Uncle Point…. Or is just a mere coincidence?

Best regards, as ever
Attachments
ABDD-ALL.jpg
ABDD-ALL.jpg (44.66 KiB) Viewed 17929 times
ABDD-31122002.jpg
ABDD-31122002.jpg (39.51 KiB) Viewed 17930 times
MCDD-31212002 b.jpg
MCDD-31212002 b.jpg (124.06 KiB) Viewed 17932 times
Jason Czech
Roundtable Fellow
Roundtable Fellow
Posts: 68
Joined: Fri May 16, 2003 8:28 am
Location: Atlanta, GA

Post by Jason Czech »

kianti wrote:The major problem with WL Montecarlo, I guess, it’s that is built on the No.2 method you mentioned in another post, the so called ‘returns’. I can resample daily equity curve changes but if I try to resample the single trades the results could be very misleading with futures, take Crude Oil back-adjusted futures as an example. That means that I cannot run all the other more sophisticated methods with WL Montecarlo.
Kianti,
I'm not sure what you mean here. I wouldn't say that MC-Lab is built on any of the specific methods, you do have several options regarding which method you would like to use. Sorry if I've misunderstood you...

I usually set up MC Lab with the options "Trade Scramble", "Maintain Win/Loss Sequence", "Maintain Serial Correlation" all active. I've found it interesting that the results usually show that the actual drawdown reported in the Simulator almost always falls below the 1% extreme expected drawdown based on MC-Lab runs. I'm not sure if I should interpret this as a flaw in the logic behind this type of MC run, or as evidence that trend trading the stock market was, in reality, as brutal as it could have possibly been in the 2000-02 period.
kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti »

Jason,

I probably didn’t make myself clear or maybe I was just wrong. I was referring to problems with back-adjusted contracts such as Crude Oil.
The ‘Raw Trades Information’ used by WL Montecarlo use the ‘Percentage Profit’, as you will see in the images below on the first trade you buy at –88 and sell at 323, profit is $4,110 but the percentage change is –467.05%; while back-adjusting doesn’t affect the real $ profit/loss in WL you get a return on the trade of –467.05%.
If you use the equity scramble or the trade scramble doesn’t seem to make any difference; if you use the ‘trade scramble and randomize’ it makes a big difference.

Having said that, I consider myself an absolute beginner on Montecarlo.

Best regards, as ever
Attachments
CL trades.jpg
CL trades.jpg (114.86 KiB) Viewed 17866 times
kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti »

Thanks for the reminder, I tried to convert the last part of The Count of Monte Carlo .jpg (145KB), these is the result of the conversions:

.png 738KB
.gif 183KB
.pdf 362KB

Any other suggestion to reduce the file sizes?

best regards, as ever
Attachments
montecarlo4.jpg
montecarlo4.jpg (145.06 KiB) Viewed 17156 times
Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt »

I think they're pretty funny.

I've been looking at them over dialup all week. With Internet Explorer 6.0 at least, they all load in the background after the text so you can read everything without having to wait for the images. So, I'm not seeing any real adverse effect.

If others don't like this, the easiest solution might be to move them to a separate thread with a warning in the topic.

What effect are the large images having on others?

- Forum Mgmnt
leonardo
Roundtable Knight
Roundtable Knight
Posts: 100
Joined: Wed Apr 30, 2003 2:27 pm
Location: Minneapolis, MN
Contact:

Post by leonardo »

I thought the cartoons were great! I access this site exclusively with dial-up and had no problems.

No slowing of text visible.

If currency trading was only as easy as it is for the Count of Monte Cristo.

-Leonardo-
kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti »

£1.3m casino 'sting'
By Nilufer Atik, Evening Standard
22 March 2004
http://www.thisislondon.co.uk/news/lond ... 20Standard

One industry source said: "There have been rumours for several years about a device that could compute the rate at which the speed of a roulette wheel and the ball degrade.


Black Swan or fat tail !?!

best regards, as ever
leonardo
Roundtable Knight
Roundtable Knight
Posts: 100
Joined: Wed Apr 30, 2003 2:27 pm
Location: Minneapolis, MN
Contact:

Post by leonardo »

Would the casino have minded the three using their contraption if they lost?

-Leonardo-
kianti
Roundtable Knight
Roundtable Knight
Posts: 335
Joined: Fri May 02, 2003 6:10 am
Location: Florence - Italy

Post by kianti »

Maybe it was just a good example of real-time good money management (with Ralph Vince or Ed Thorpe on the other side of the scanner).

Or it was simply Ted Annemann playing with his MonteCarlo-Lab ? :lol:
leonardo
Roundtable Knight
Roundtable Knight
Posts: 100
Joined: Wed Apr 30, 2003 2:27 pm
Location: Minneapolis, MN
Contact:

Post by leonardo »

The incident shows that someone finally succeeded at the mission of the brainiacs who starred in Eudaemonic Pie. The same guys behind the The Predictors.

link to book descriptions:

http://www.amazon.com/exec/obidos/ISBN% ... ingblox-20

http://www.amazon.com/exec/obidos/tg/de ... 1?v=glance
lperepol
Senior Member
Senior Member
Posts: 30
Joined: Thu Feb 26, 2004 12:50 pm
Location: Castlegar, BC, Canada
Contact:

Post by lperepol »

kianti wrote: One industry source said: "There have been rumours for several years about a device that could compute the rate at which the speed of a roulette wheel and the ball degrade.


Black Swan or fat tail !?!

best regards, as ever

I think Edward Thorpe developed the first wearable computer to do this in the 1960's. The man played with many marlbles.
pruggera
Contributing Member
Contributing Member
Posts: 8
Joined: Sat Mar 20, 2004 6:47 pm
Location: San Diego

Post by pruggera »

Forum Mgmnt wrote:What effect are the large images having on others?

- Forum Mgmnt
It's having a laughter effect on me.

Phil
batuco9
Roundtable Fellow
Roundtable Fellow
Posts: 53
Joined: Thu May 29, 2003 12:28 am
Location: Chile

c.f. wrote:

Post by batuco9 »

What effect are the large images having on others?

- Forum Mgmnt
Presume the lack of replies implies that most are not affected. I'm not.
If any others are indeed affected it would be nice to hear.
Post Reply