Your advice: what's a reasonable way to test RANDOM EXITS?

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Your advice: what's a reasonable way to test RANDOM EXITS?

Post by sluggo » Thu Mar 22, 2007 1:59 pm

We've seen the Tharp-Basso system that makes trade entries at random, then uses a trailing stop to exit (link). But what about the reverse: a system that makes trade exits at random, but lets us use our favorite type of entry. To my knowledge, nobody has ever discussed such a thing.

It got me wondering: What is the most reasonable way to "test random exits"? What plan of testing is the fairest? On my own I came up with a few ideas, but would love to hear YOUR ideas for how to "test random exits".

I thought of these:
  1. Wait for your entry signal to make a trade entry. Then go ahead and enter the market according to that entry signal. Remain in the trade for "D" days after entry, then exit at the market price at the end of the D'th day. "D" is a number chosen at random, uniformly distributed between D=1 days and D=100 days (or whichever boundaries you like).
  2. Same as #1 except the random numbers D follow a probability distribution of trade holding times that resembles a real-world trading system's holding time distribution. For example, Figure 1 below is taken from straight from the Blox backtest of a futures trading system; it is column U of "Trade Log.csv".
  3. You restrict yourself to entry signals taken from pure-reversal systems that are always in the market, either long or short but never out. The random number generator is used to generate random DATES (rather than random trade holding-times). For each exit date, you look back and see what was the most recent entry signal. Then your "trade" consists of that most recent entry signal, plus the randomly chosen exit date.
I am not so fond of #3 because it seems too restrictive to forbid any kind of entry signals. After all we are trying to answer the question, "How profitable is a system that uses a good entry but exits at random?" and it seems unwise to forbid any kinds of good entries.

Would love to hear your thoughts.

+SLUGGO+
Attachments
histog.png
Figure 1. Holding times
histog.png (9.29 KiB) Viewed 10127 times

Jens Albrecht
Senior Member
Senior Member
Posts: 40
Joined: Thu Feb 08, 2007 3:19 pm
Location: Dortmund, Germany

Post by Jens Albrecht » Thu Mar 22, 2007 3:12 pm

I would like to see added:
Take entry, then given an exit chance x% roll each day wether to leave trade. Would be interesting stepping through x%.

Roscoe
Roundtable Knight
Roundtable Knight
Posts: 250
Joined: Sat Jan 24, 2004 2:06 am
Location: Houston TX

Post by Roscoe » Thu Mar 22, 2007 5:37 pm

Hi Sluggo. I have a form of #1 as part of my standard testing code but it is very simple - it exits at market after D bars but D is just a static value. I never really found a lot of use for it really, it was based on a suggestion from Chuck LeBeau as a means for testing entries (try 5, 10, 20 bars) but my observation was that the interaction between entry and exit was far more relevant.

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Thu Mar 22, 2007 6:50 pm

I've tested using both #1 and a close cousin to #2, namely an exit that was a random number of days between some minimum and maximum. For example, randomly exiting between 20 and 150 days after entry.

I have not tested using a distribution as you outlined in #2 but this is certainly an interesting idea. My only fear with using the distribution is that you skew your results toward the type of system you pick, i.e. long-term trendfollowing, medium-term, super-long-term, medium-term counter-trend, etc.

- Forum Mgmnt

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Thu Mar 22, 2007 7:51 pm

Preliminary results on
  • Portfolio of 10 US futures markets. One of them was "ES", the eMini S&P 500 futures.
  • Test ran for 10 years, from 3/19/1997 to 3/21/2007
  • $100 per contract, round trip, for commissions+slippage
  • Textbook entry signal
  • Random Exits of some kind :D
It's fun, I encourage Blox owners to experiment.

TENTATIVE CONCLUSION: Textbook entry signals are non-negligible contributors towards system overall profits. Even with random exits, good entry signals can make money.
Attachments
SteppedParameterGraph_P0.png
One hundred experimental runs of random exit trading system. Notice that all 100 were profitable (CAGR > 0)
SteppedParameterGraph_P0.png (25.93 KiB) Viewed 10067 times
Last edited by sluggo on Fri Mar 23, 2007 10:11 pm, edited 1 time in total.

td80
Roundtable Fellow
Roundtable Fellow
Posts: 60
Joined: Thu Mar 08, 2007 8:38 pm
Location: Irvine, CA

Post by td80 » Thu Mar 22, 2007 8:33 pm

Excellent work sluggo, very interesting results.

I am experimenting with some volatility concepts myself with the same sort of focus on exit efficiency.

If you get bored try plotting an indicator with:

(20 bar high - 20 bar low) / (50 bar high - 50 bar low)

Notice what happens on a variety of timeframes on a variety of instruments when the percentage drops below 40%. It tends to be either a reversal or an acceleration of the trend right around that area and the instrument normally goes ballistic and the ratio goes over 80% .... I'm experimenting with it trying some things, trying to see if it's possible to develop a system around it or perhaps entries/exits. If it dips below 40%, I consider that a trigger, and then until it goes over 80%, I consider the market to be "springy". Then until another dip below 40, market will probably do average price acceleration....

Thanks again for your insightful posts Sluggo,

-Dave

nickmar
Roundtable Knight
Roundtable Knight
Posts: 191
Joined: Tue Oct 26, 2004 12:38 pm
Contact:

Post by nickmar » Fri Mar 23, 2007 3:09 pm

Here's a system inspired by this discussion:

System 1 Rules:

Code: Select all

1 - Pick a random number X between 20 and 200
2 - Wait X trading days
3 - Pick another random number Y between 20 and 200
4 - Flip a coin. If it is heads, enter a Long position and hold for Y trading days. If it is tails, enter a Short position and hold for Y trading days
5 - Repeat from step 1

System 2 Rules:

Code: Select all

1 - Pick a random number X between 20 and 200
2 - Wait X trading days
3 - Pick another random number Y between 20 and 200
4 - If the current close is greater than the Y day moving average, enter a Long position and hold for Y trading days. If the current close is less than the Y day moving average, enter a Short position and hold for Y trading days.
5 - Repeat from step 1
  • Portfolio Breakdown:

    Currencies - 6
    Energies - 6
    Grains - 7
    Long_Rates - 7
    Meats - 4
    Metals - 9
    Short_Rates - 5
    Softs - 8
    Stock_Indices - 9

    Total Markets - 61
- Position-sizes were volatility-adjusted
- Slippage of 15% and roll slippage of 10%
- Test ran from 1990-03-31 to 2007-02-28

Each system was executed 2000 times - here's an interesting visual:

EDIT: System 1 is represented by the bottom graph and system 2 is represented by the top graph.
Attachments
Sheet 1.JPG
Sheet 1.JPG (127.01 KiB) Viewed 9991 times

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Fri Mar 23, 2007 5:33 pm

nickmar, that's outstanding! It seems that (random entries + random exits) are significantly worse than (moving average entries + random exits). Conversely, moving average entries are better than random! This contradicts the teachings of a few gurus.

svquant
Roundtable Knight
Roundtable Knight
Posts: 126
Joined: Mon Nov 07, 2005 3:39 am
Location: Silicon Valley, CA
Contact:

Perhaps another "random exit"

Post by svquant » Fri Mar 23, 2007 6:24 pm

Just to throw out another idea on the random exit method would be to have a random selection of exit methodologies per trade. In the fixed N-day or distribution based N-day exit we are really testing a fixed time exit criterion. Yes we vary the days but not the method. I'm suggesting to look at picking based on a random variable which exit to use after your entry point, eg N-day min/max, MA-xover... Of course if the exit gives a signal immediately this would be like a 1-day fixed exit. A way of thinking about this is that it is similar to the N-day drawn from a distribution but we are generating the distribution in a more random unbiased way vs using a current trading system which will show a bias as mentioned above. Best to have a good selection of exit methodologies/criterion that include trend, counter trend, profit target...

Have not coded this - but it is food for thought. Thanks to those who have coded and shown some results!

Marc

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Fri Mar 23, 2007 7:14 pm

nickmar's tests above reminded me very much of the research I did for the chapter "Lies, Damn Lies, and Backtests" in Way of the Turtle. Since this is a topic that I had already promised to discuss in detail in a blog entry on my blog site at www.wayoftheturtle.com. I finished that blog entry tonight so those of you who don't have the book yet can see what I was talking about.

In short, my tests showed very similar positive skew to the random exit system if I considered only random long entries when the 25-day moving average was higher than the 350 day moving average, an vice versa for shorts. In my case I randomly exited between 20 and 120 days after entry.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Fri Mar 23, 2007 7:40 pm

svquant / Marc, thanks very much for the suggestions! I certainly agree, what has been discussed so far is only one particular "method" of exit: a time based exit. The amount of time you wait before exiting each particular trade is random, but the fact that it is a time-exit and nothing but, remains.

Your suggestions seem to be along the lines of "Let's create a grab bag of numerous realistic exits, and then for each trade, reach into the bag and pull out an exit." But is this what we really want? We seek to isolate the effects of entry signals by pairing them with "Random" exits; but to only use "realistic" exits may be to reduce the randomness, perhaps?

Another thought: No matter what procedure is used to choose an exit, eventually the exit order is filled at some instant of time. The two instants of time: Entry Fill Time, and Exit Fill Time, uniquely identify this trade. Since the Entry Fill Time is chosen by the entry signal, the purpose of the exit signal is to choose the Exit Fill Time. Why not make this Exit Fill Time be a random number? Does that not meet the objective of creating a "Random Exit?"

sluggo

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Fri Mar 23, 2007 9:01 pm

Test results placed in a new thread: viewtopic.php?t=3779

svquant
Roundtable Knight
Roundtable Knight
Posts: 126
Joined: Mon Nov 07, 2005 3:39 am
Location: Silicon Valley, CA
Contact:

Perhaps Overkill

Post by svquant » Sat Mar 24, 2007 4:27 pm

Sluggo:

Yes in the end all trades are just entry and exit dates... oh but the effort we put into those dates :D

My comments were really in terms of #2 and c.f.' comment about the distribution. Of course another way would be to just look at the distributions from a few different systems/exits and use the N-day drawn from this mixture of distributions. But in the end will it really tell you anything more about your entries then unif-random in a range of days? Perhaps not!

Reminds me of academic papers that always do a bootstrap procedure vs just a plain old t-test since many financial time series or trading system time series violate the normality assumption, but in the end the t-test vs bootstrap does not give different answers. Better to use a more computationally difficult non-parametric test due to the time series properties? Yes, the theory tells us so. Matter in practice - not from what I can tell.

GENX
Full Member
Full Member
Posts: 21
Joined: Thu Oct 12, 2006 7:12 pm

Randomness with Order

Post by GENX » Mon Mar 26, 2007 9:12 am

Flip out the old stethiscope at day of entry and document the following:

ATR @ Entry Signal X Calendar Day
ATR X March 26 (ATR*26)/30days = Stethiscope number (S#)

Running Mean of ATR for last 30 days= 30!ATR

S#/30!ATR and smooth by say 15 days= Exit monitor

When Exit Monitor displaces by 1STD exit trade.


Of course I have not tested this, and I can't tell you pedigree info, it was truly just a random jittle of thoughts itself. So accept it for what it is, something to consider.

Post Reply