Trading using Random Entries (Van Tharp book method)

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Trading using Random Entries (Van Tharp book method)

Post by sluggo » Wed Feb 14, 2007 12:02 am

The Second Edition of Van K. Tharp's book "Trade Your Way To Financial Freedom" contains an extraordinary claim on pages 253-254. Tharp says that a mechanical system which enters trades at random, then exits using a trendfollowing "trailing stop", 3.0 ATR's below the close, makes money 100 percent of the time on a 10 market portfolio over 10 years of testing, using $100 per contract for slippage and commissions.

So I decided to test this idea in Blox. Blox customers can download my code from the Blox Marketplace on this website, (a link is here), and play around with it themselves. You can see in Figure 1 below that I've reproduced Tharp's test, including his $100 per contract for commissions. Sadly, the bottom of the figure displays the bad news: all 100 tests out of 100, lost money. Their Compound Annual Growth Rate was negative. Exactly opposite of the book: 100 percent of the time, it lost money.

(As its name implies, "This Param Does Nothing!!" doesn't do anything. It merely lets us tell Blox to run 100 different tests of trading-using-random-entries. These appear along the horizontal axis of the plot at the bottom of Figure 1.)

Figure 2 shows a typical winning trade and a typical losing trade. Entry occurs at random, and then a 3*ATR trailing stop below the closes (blue line) determines the exit.

I guess the markets have changed between the writing of the book, and today. Which is too bad; it certainly would be comforting if random entries plus simple trailing exits, produced a profit. But they don't.
Attachments
combo2.png
Figure 2. Random entry produces a winning trade (top) and a losing trade (bottom)
combo2.png (28.12 KiB) Viewed 34856 times
combo.png
Figure 1. System parameters, commission and slippage, and (very bad) trading results
combo.png (67.16 KiB) Viewed 34857 times

BARLI
Roundtable Knight
Roundtable Knight
Posts: 650
Joined: Sat Jan 17, 2004 6:01 pm
Location: USA

Post by BARLI » Wed Feb 14, 2007 3:35 pm

First, you need to define random, are you sure you have same definition of random as Tharp has it?

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Wed Feb 14, 2007 8:15 pm

are you sure you have same definition of random as Tharp has it?
Yes. Quite sure.

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Thu Feb 15, 2007 6:27 am

One other possibility is that the pseudo-random number generator for whatever simulation software he used did not have an adequate dispersion of random values. In other words, it wasn''t as random as it needs to be for this sort of test.

Computer pseudo-randomnumber genrators that are supplied by the providers of ocmpilers like Microsoft's Visual C++ are notoriously poor. They can generate sequences that are not evenly distributed leading to repeatable tests that are not truly random.

This is one of the reasons we spent a lot of time looking at various pseudo-randomnumber algorithms for Trading Blox and chose to use the Mersenne-Twister algorithm since it offered very good dispersion. Good dispersion is especially important for accurate Monte Carlo analysis.

With poor pseudo-random alogorithms if you happen to choose a poor seed you can get very skewed results and very poor spreading (aka dispersion) of the random numbers.

I found that the built-in Visual C++ pseudo-random number algorithm was so bad with certain seed values that it gave me very wrong answers for my tests time after time. One test I tried first was to take a random long or short position and then determine the Maximum Adverse Excursion and Maximum Favorable Excursion for each trade when exiting on the close 10 days later. I then divided each MAE and MFE by the ATR and then averaged the values for each trade.

For tests with thousands of trades, one would expect a truly random number to create a situation where these values would be very close to equal, where the average MFE and the average MAE would be close to equal when normalized for ATR. In my first tests this was not the case. I found that this was because the random numbers were generating far more long trades than short trades using the built-in pseudo-random number algorithm. When I switched this to Mersenne-Twister I got the results I expected.

The Mersenne Twister algorithm works very well with the random numbers spread evenly across the space so you get the types of answers you expect to get. For example, if you test 2 million random numbers from 1 to 1000 you will get very close to 1 million values in the range between 1 and 500 and 1 million values in the range between 501 and 1000.

Paying attention to these types of details is critical and one of the reasons that it is important to have both a strong trading and a strong computer science background if you intend to build your own simulation software. One can easily draw the wrong trading conclusions because of poor computer science if you do not understand the issues here.

- Forum Mgmnt

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Thu Feb 15, 2007 8:31 am

I think the explanation is much simpler: the markets have changed.

Figure 1 below shows the equity curve of the Tharp-Basso trading system with $100 commissions+slippage when the random seed is set by DoNothing=1. I have plotted it from 1970 to today, 36+ years worth of simulation, using the TBB "All Liquid" portfolio. Notice the shape of the equity curve. Sloping upwards, followed by choppy flatness, followed by sloping downwards. Random trading was profitable, followed by breakeven, followed by unprofitable.

I grabbed the equity curves of the first eight tests (DoNothing = 1 to 8 ) of the Tharp-Basso system (using $100 for commission+slippage, just as Tharp-Basso did), optically scaled them down by 50% to save room, and collected them into one picture: Figure 2. The same general shape is seen in all of them: Profitable trading from 1973 to 1988 (in some cases, to 1995), followed by poor performance and losses thereafter.

This suggests to me that the markets before 1988 were kinder to simplistic "trailing stop trendfollowers" than the markets after 1995. The Tharp-Basso system is an especially simplistic example of a trailing stop trendfollower, so it performed well "when the markets were good" and poorly in other conditions.

We don't need to hypothesize some kind of subtle, hidden flaw in a software random number subroutine to explain the results. The markets used to be kind to simplistic trendfollowers, but they aren't so kind any more.

However, for those who wish to doggedly pursue the Random Number Goodness angle, I have tossed you a bone. Across the top of Figure 1 are some statistics. The last 3 statistics are calculated by a little Blok I added. It counts the total number of trades, the number of trades that went Long, and the % of all trades that were Long. Too bad we can't go back in time to the early 1990's and add similar instrumentation to the software that Tharp and Basso used when doing the experiments reported in the book!

(In many repeated trials of flipping 8919 fair coins and counting the number of heads, we would expect a variability of about sqrt(8919) = 94 heads among the trials. The expected number of heads from flipping 8919 fair coins, is 4459 heads, plus or minus 94 heads. Anywhere from 4365 to 4553 heads, i.e., anywhere from 48.94% to 51.05% heads.)

Here is the Blox code that calculates these Statistics. You may notice that it operates skeptically. Just because TBB reports that test.totalTrades = 8919 doesn't mean that we should trust that number. Oh no. The code wants to see that (A) a trade was of nonzero size; and (B) its direction was properly set to either LONG or SHORT, and not some other value, before that trade gets counted in the statistics.

Code: Select all

VARIABLES: i, nlongs, nshorts TYPE: integer
VARIABLES: pctlongs TYPE: floating

nlongs = 0
nshorts = 0

FOR i = 1 to (test.totalTrades) STEP 1
	IF( test.tradeQuantity[i] > 0 ) THEN
		IF( test.tradeDirection[i] == LONG ) THEN
			nlongs = nlongs + 1
		ENDIF
		IF( test.tradeDirection[i] == SHORT) THEN
			nshorts = nshorts + 1
		ENDIF
	ENDIF
NEXT

test.AddStatistic("myNtrades", (nshorts+nlongs))
test.AddStatistic("myNlongs", nlongs)
pctlongs = AsFloating(nlongs + nshorts)
pctlongs = 100.0 * AsFloating(nlongs) / pctlongs
test.AddStatistic("myPctLongs", pctlongs)
Attachments
eight_in_one.png
Figure 2: Equity curves of 8 runs of Tharp-Basso system, from 1/1/1970 to 2/12/2007
eight_in_one.png (31 KiB) Viewed 34617 times
randomtrades.png
Figure 1: Equity curve of Tharp-Basso system, 1/1/1970 to 2/12/2007, DoNothing=1
randomtrades.png (29.55 KiB) Viewed 34618 times

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Thu Feb 15, 2007 10:06 am

Yes, that make sense. I did not realize that you were testing over different years than the Sharp/Basso test.

It looks like if you tested 10 years from 1988 to 1997 there would have been a 100% increase in the acocunt for most of the equity curves which matches the description of the results from the book.

The markets have more medium term noise around trends in my testing. That would make a 3 ATR trailing stop much more likely to get hit in the noise even during good trends. This is also one of the reasons that a Turtle-like medium-term breakout strategy does not work as well over the last 15 years as a very-long-term trend-following strategy.

- Forum Mgmnt

P.S. Of course you got almost exactly 50% long trades. You are using the Blox Basic Mersenne-Twister-based random number generator. I wonder what happens if you run the same test using whaterver tool the used? :)

Of course yours is by far the most likely explanation.

freeman
Roundtable Knight
Roundtable Knight
Posts: 114
Joined: Tue Oct 04, 2005 7:12 pm
Location: Switzerland

Post by freeman » Thu Feb 15, 2007 10:21 am

I forgot to mention,on my test it was done on weekly data (5yrs).
Slippage = 0 & commision = 0.
First 2 equity charts are randomly selected stocks of 10,last 1 is radom on 50 stocks.

Filter
Lookback Days 30
Stregnth Threshold 100%
Long Only
Attachments
10 of the first E.png
10 of the first E.png (9.97 KiB) Viewed 34588 times
30-40.png
30-40.png (10.4 KiB) Viewed 34588 times
50 As.png
50 As.png (9.6 KiB) Viewed 34588 times

BARLI
Roundtable Knight
Roundtable Knight
Posts: 650
Joined: Sat Jan 17, 2004 6:01 pm
Location: USA

Post by BARLI » Thu Feb 15, 2007 2:55 pm

Forum Mgmnt wrote: When I switched this to Mersenne-Twister I got the results I expected.
So was it around 50/50 ?

sluggo , question to you: in semi-conductor company there are 300 employees, what the chance that any 2 of them has birthday on the same day?

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Thu Feb 15, 2007 6:42 pm

the probability that in a group of 300 people, no two people have the same birthday, is 6.245E-82.

the probability that two or more people in a group of 300 do have the same birthday, is (1 - 6.245E-82).

I will let you display the formula with the factorials, which computes this result.

The special cases that puzzle books discuss are:

22 people: prob = 52.43% that no two have same birthday
23 people: prob = 49.27% that no two have same birthday

Old time UNIX hacks like c.f., would not be able to prevent their long-neglected neuron from firing off a dusty but urgent message: Use the "bc" command!!

ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Post by ecritt » Fri Feb 16, 2007 1:15 am

Forum Mgmnt wrote:One other possibility is that the pseudo-random number generator for whatever simulation software he used did not have an adequate dispersion of random values. In other words, it wasn''t as random as it needs to be for this sort of test.

Computer pseudo-randomnumber genrators that are supplied by the providers of ocmpilers like Microsoft's Visual C++ are notoriously poor. They can generate sequences that are not evenly distributed leading to repeatable tests that are not truly random.
- Forum Mgmnt
Forum Mgmnt,

I always feared that the Mersenne Twister algorithm was a way for people to force their distributional assumptions into the simulated data (to get what they thought was randomness). If I remember correctly MT was designed to nullify any serial correlation between adjacent output values.

True randomness results in plenty of lumpy clusters and runs. What are you thoughts on this fear of mine?

Thanks,

ec

BARLI
Roundtable Knight
Roundtable Knight
Posts: 650
Joined: Sat Jan 17, 2004 6:01 pm
Location: USA

Post by BARLI » Fri Feb 16, 2007 2:15 am

the funny thing is that when you go to the kitchen of the company(where all the birthdays are showed on the board), you'll see that every month at least 8-10 people have birthday on the same day

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Fri Feb 16, 2007 4:12 am

This useful thread has further confirmed the conclusions I have reached over several years of extensive backtesting. Markets do change and in today's markets, simple short term trend following systems, such as the 5 and 20 day MA crossover, no longer work. Ditto medium term trend following systems. You have to add bells and whistles.

A very long term simple trend following system avoids the noise and works admirably - for the time being - without bells and whistles.

Using Sluggo's code and a 10 ATR stop on my 50 future portfolio, this purely random system is profitable to this day - the equity curve no longer turns down. Not very profitable, but profitable nonetheless. In other words a much wider stop avoids some of the noise.

So apologies for my slur on Doc Van P - in one respect.

In another respect, I do not think it is terribly impressive that he failed to update his thoughts on this topic.

I think that is the danger of such people who rely on what others tell them and do not trade (or even backtest) themselves. That is the principal problem with people who merely re-iterate to the world what they have been told by others - they are in no position to ascertain (or update) the truth of what they have been told.
Last edited by AFJ Garner on Fri Feb 16, 2007 4:42 am, edited 1 time in total.

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Fri Feb 16, 2007 4:16 am

ecritt wrote:If I remember correctly MT was designed to nullify any serial correlation between adjacent output values.

True randomness results in plenty of lumpy clusters and runs. What are you thoughts on this fear of mine?
Lumpiness does not imply serial correlation. Truly random numbers have both qualities, there are clusters but there is no serial correlation. There are definitely clusters in random events as anyone who like to play craps will attest. The lack of serial correlation means that the runs are unpredicatable, i.e. one roll of the dice has no effect on the next roll of dice, not that there will not be clusters of nearby values in the output data.

The ideal psuedo-random number alogorithm will have output values with characteristics that are virtually indistinguishable from shose of a series of actual random values. There are a large number of specific tests used to measure the randomness of a partcular algorithm's output values. See Diehard Tests for a description of what is purported to be the most stringent array of these tests. Mersenne Twister passes these tests. The typical pseudo-random number generators do not.

Mersenne Twister also has the distinct advantage of being relatively fast to compute new values which makes it excellent for Monte Carlo simulations which often require millions of such values in a single simulation. See: Mersenn Twister for a pretty good writeup on the algorithm.
Barli wrote:So was it around 50/50 ?
Yes, in my case the average MAE and average MFE values were very close to equal and the number of long and short trades were very close to 50/50. Using the C++ compiler's normal random routine the values were sometimes as much as 70/30 skewed.

Anthony brings up an excellent point. Markets do change, yesterday's rules of thumb do not necessarily apply to today's markets. Other traders are not stupid, they notice things, try to anticpate other traders, etc. I cover this idea specifically in my book. This is one of the reaons that I recommend against using a specific pre-made system that has not been adjusted or altered in any way. If many people are doing the same thing, and that algorithm is known then you can anticipate that other traders will try to take advantage of it. I believe this contributed to the large drawdown for the Abberation system, for example.

Had others known our exact system while we traded as Turtles they might not have made money using the system but they certainly could have easily picked up a good short run by buying ahead of the breakouts in the markets we traded. This would have hurt our system results. It is this sort of thing that has contributed to the increase in noise and resulted in the 3 ATR stop no longer working as well, and also in the Turtle System not working as well if left unaltered.

I have a theory, however, that the markets are fairly cyclical in this respect. As markets change certain ideas stop working, so traders eventually stop using those ideas, and that over the long term this can result in old ideas starting to work again.

- Forum Mgmnt

Old European
Roundtable Knight
Roundtable Knight
Posts: 144
Joined: Mon Apr 19, 2004 3:04 pm
Location: Old Europe

Post by Old European » Fri Feb 16, 2007 4:44 am

Although I am well aware that simple ultra-long-term trend following systems worked remarkably well over the last few years, one has to bear in mind that, once the markets change again, it will take a very very long time to come to the conclusion that these systems stopped working (since it takes a long time to have enough trade data to draw statistically significant conclusions). And that can be quite painful, except for investors with investment horizons of 25 years or more.

Old European

P.S. Please don't get the impression that I'm old. I only happen to live in Old Europe.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Fri Feb 16, 2007 8:33 am

AFJ Garner wrote:... thread has further confirmed the conclusions I have reached over several years of extensive backtesting. Markets do change ...
Yes I agree completely. To some people this is an astonishing (or not-believable) result. For example, those who strongly believe that market prices must be fractal and self-similar on all time scales. Another group of astonished unbelievers is the large collection of sheep who have (as AFJ points out later) read some guru's remarks in some book, saying "the markets never change because human nature never changes" and accepted this as gospel truth because an authority figure had said it.
AFJ Garner wrote:I think that is the danger of such people who rely on what others tell them and do not trade (or even backtest) themselves. That is the principal problem with people who merely re-iterate to the world what they have been told by others - they are in no position to ascertain (or update) the truth of what they have been told.
An excellent point. However the more nimble of them might start to appropriate the information in this very thread :D and begin to regurgitate it as though they had made the discovery themselves, or as though they hadn't been saying the opposite for years.
AFJ Garner wrote:In another respect, I do not think it is terribly impressive that (Van K. Tharp) failed to update his thoughts on this topic.
I too am frustrated that the book is out of date, I worry that it may mislead beginners. However if you read it carefully, Tharp presents the whole story as an interesting piece of history: Tom Basso and I coded up this system with random entries and trailing stop exits, we tested it on 10 (undisclosed) markets for 10 (undisclosed) years, and it was profitable. I am sure this historical narrative is true. However, the markets have changed since then, and this "system" is no longer profitable. It no longer supports Tharp's conclusion that profits automatically spring forth from "good exits", while entries are essentially unimportant. He needs to go find a new anecdote that bolsters this claim.
Forum Mgmnt wrote:I have a theory, however, that the markets are fairly cyclical in this respect. As markets change certain ideas stop working, so traders eventually stop using those ideas, and that over the long term this can result in old ideas starting to work again.
Perhaps it is therefore a good idea to trade a "suite" of different systems with different principles of operation and different time horizons. If the markets change in such a way to favor method A and dis-favor method B, it would be nice if one were already trading a suite that included method A!
Forum Mgmnt wrote:See Diehard Tests for a description of what is purported to be the most stringent array of these tests.
Not really. Diehard is a suite of civilian tests. The military has a more stringent bank of tests because they require "Cryptographically Strong Random Numbers". These are numbers which are not only "pseudo random", they are also completely un-reverse-engineer-able. Given the first hundred million output numbers from an RNG, and given unlimited computing power, an opponent still can't predict the next output number with greater-than-random-chance odds. The diehard suite is merely step one of testing for Cryptographically Strong Random Numbers; the easy part. Further tests are more stringent. See this link from the National Institute of Standards and Technology, at a scary "dot gov" URL, for more info.

RedRock
Roundtable Knight
Roundtable Knight
Posts: 941
Joined: Fri Jan 30, 2004 3:54 pm
Location: Chicago

Post by RedRock » Fri Feb 16, 2007 1:21 pm

sluggo wrote:
Forum Mgmnt wrote:I have a theory, however, that the markets are fairly cyclical in this respect. As markets change certain ideas stop working, so traders eventually stop using those ideas, and that over the long term this can result in old ideas starting to work again.
Perhaps it is therefore a good idea to trade a "suite" of different systems with different principles of operation and different time horizons. If the markets change in such a way to favor method A and dis-favor method B, it would be nice if one were already trading a suite that included method A!
This is interesting. It would seem that if one had three systems in a collection. Perhaps one is doing well. One breaking even, one breaking down. This would be three sets of commission and slippage for the lonely winner to overcome. I agree diversification in systems can be wundervoll. Just not necessarily so. I have traded three systems in different time frames different methods and different markets, and still had them all go south at the same time. geez :roll:

Chuck B
Roundtable Knight
Roundtable Knight
Posts: 469
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B » Thu Apr 12, 2007 3:48 pm

I just came across this thread -- haven't been around the board much in a long time.

I was working with Tharp back in late 1993 to 1994 on trading system design issues, so I have a vague recall the details of what transpired -- after all it has been 13-14 years ago now. The software used was Trading Recipes -- more about that in a moment. The challenge for the random entry idea came when Tom made the comment in a Tharp seminar that he bet you could take a long term trendfollowing type of exit and randomly enter and still make money.

Van asked Bob Spear if he would add a keyword function to Trading Recipes to allow it to take a random entry, either long or short, in order to then test exit algorithms. Bob didn't take long to produce the update (and I have no idea about the specifics of how it operated, i.e. how he coded it).

The exit I recall Tom saying was a trailing volatility exit, which at the time, it seemed like few had ever heard of such a thing. He said to use a 3 x ATR trailing exit with a 19 period exponential average of ATR (not that 19 was any magic number). This is what I recall Van using back then.

He ran the tests using Trading Recipes, and I seem to recall he used the free data that Bob Spear used to provide with TR back at that time. I looked at an old notes file, and here is the data that was initially provided with TR and used for the random system in its initial stages (might not be the data used in the published work, but I'm not sure). Ten markets: BP, CL, DM, ED, GC, LC, S, SI, SU (NY sugar), and US. The time period was from 1981 to 1991. It was noted that not all would have a total of 10 years, so use what was available (i.e. I think CL started in 1983). Somewhere, on a diskette, I have that original continuous data, lol.

Regards,
Chuck

P.S. All of the above is to the best of my recall. Also, I have not had any business relationship or contact with Van in almost 9 years now.

BARLI
Roundtable Knight
Roundtable Knight
Posts: 650
Joined: Sat Jan 17, 2004 6:01 pm
Location: USA

Post by BARLI » Fri Apr 13, 2007 1:05 am

hi Chuck, if you have those backtest results, that'd be good if you'd post them here. thanks!

White Cube
Roundtable Fellow
Roundtable Fellow
Posts: 98
Joined: Fri Oct 20, 2006 2:25 pm
Location: London, UK

Post by White Cube » Sun Sep 16, 2007 12:23 pm

As Anthony suggested I run 100 tests of the random entries-trailing stops at 10 ATR from 1982 until September 2007 on a portfolio of 69 futures with $100 for commission and slippage and the system made money 100% of the time as you can see in the following figure.

Sluggo explained that markets have probably changed after 1995 and c.f. confirmed that the medium term noise was more important explaining thus why the 3 ATR stopped working. But when I test on several simple Long to Very Long Term Trend Systems I have noticed that during the last few years (since 2003) the log Equity curve does not climb as fast as it used to do or even goes flat as you can see in the figure joined. Have other forum members noticed this phenomenon too? Have the markets changed again? Could this be explained by the markets being less volatile?
Attachments
Long Term trend following.jpg
Long Term trend following.jpg (60.85 KiB) Viewed 30676 times
Random Entries and 10 ATR trailing Stops.jpg
Random Entries and 10 ATR trailing Stops.jpg (49.14 KiB) Viewed 30676 times

nodoodahs
Roundtable Knight
Roundtable Knight
Posts: 218
Joined: Wed Aug 09, 2006 4:01 pm

Post by nodoodahs » Mon Sep 17, 2007 2:27 pm

I thought that "profitable" was meant in the trivial sense, i.e., it didn't lose money. I assumed that it was trivially profitable but most likely well below any CAGR/STDEV or CAGR benchmarks I might assign, so it never occurred to me to test the random entry + chandelier exit system.

Big difference between "profitable" and "worth trading" – if risk is involved and CAGR is less than a money market, why bother? I think someone mentioned 100% in 10 years, is that 7.2%? I think I can allocate assets with ETFs to get that …

Post Reply