Traders' Roundtable Forum Index

Check out our New Equity System Pack

Traders' Roundtable
A forum for mechanical system traders.
 
  FAQFAQ   SearchSearch  UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
Reliable number of trades?

 
Post new topic   Reply to topic    Traders' Roundtable Forum Index -> Testing and Simulation
View previous topic :: View next topic  
Author Message
LeviF
Roundtable Knight
Roundtable Knight


Joined: 22 Dec 2003
Posts: 1421
Location: Des Moines, IA

PostPosted: Thu Jan 31, 2008 2:05 pm    Post subject: Reliable number of trades? Reply with quote

How does one know if a system generates enough trades to be statistically reliable into the future? Would 100 trades over 10 years on a basket of 30 futures be adequate?
Back to top
View user's profile Send private message Send e-mail Visit poster's website
nodoodahs
Roundtable Knight
Roundtable Knight


Joined: 09 Aug 2006
Posts: 218

PostPosted: Thu Jan 31, 2008 2:40 pm    Post subject: Reply with quote

Just some random thoughts on the matter.

Philosophical answer:

You never "know" for sure, because, in theory at least, the conditions that existed in the market during your test period may have been unique, and have changed in the current market. With one trade per contract every three years or so, I guess there's a real possibility of some kind of outlier in that data. Statistics involves assumptions. Not comforting, I know. So much of this is having confidence or a reason to believe we have an edge, without ever knowing for sure. Well, sometimes you almost maybe know for sure.

Short statistical answer: Maybe. More would be better.

Long statistical answer: Statistics involves assumptions, and with only 100 data points, you're not on super ground, but it depends on what kind of results those 100 data points have. Assume that a trade is a trade, regardless of contract or source. BIG assumption. Then with 100 trades you have a payout distribution that can be tested.

You could take the 100 trades, convert them into +/- percent of equity, and categorize them into a distribution that had a mean, standard deviation, skewness, kurtosis, etc., and perform some tests on that, too. With only 100 in the sample, unless it looks really promising from a profitability standpoint, you might want to proceed with some caution. Any big bad losses? Look at the losses and what the contracts were doing, and think if that's a common situation or what the cause might be, how to eliminate those big losses.

Winning percentage could be tested with the binomial approximation to the normal, which assumes some things, like a normal distribution. The population mean winning percentage is estimated by your sample of 100 trades, and the standard deviation is the square root of the product of win %, loss %, and number in the sample (in this case, 100). For example, if your sample was 35% winners, then statistically you can have about 98% confidence that your system has a future winning percentage between 25% and 45%. Assuming your assumptions are good.

Now if you take your average win and average loss amounts as percentages of equity, and *assume* that these distributions are solid, you could calculate your expectancy based on the lower bound (25% wins in the example), the sample amount (35% wins in the example), and the upper bound (45% wins in the example) of the winning percentage. Are all of those numbers "good" to you? Could you take that system at the lower bound of win % and turn over enough trades to make a good return? Is the system even profitable at that lower bound? What's the maximum number of losses in a row you could expect at that lower bound, and with ten trades a year, what are the odds that you go winless in a year? Would that make you stop trading?

Bottom line, we all have to make decisions with very imperfect data. You have to be comfortable with what your analysis of those 100 trades gives you. If you're not, could you backtest longer? More contracts? I don't know.

_________________
One minute you’re up half a million in soybeans and the next, boom, your kids don’t go to college and they’ve repossessed your Bentley.
Back to top
View user's profile Send private message Send e-mail
sluggo
Roundtable Knight
Roundtable Knight


Joined: 11 Jun 2004
Posts: 2941

PostPosted: Thu Jan 31, 2008 5:08 pm    Post subject: Reply with quote

A trading system exploits (profits from) a certain kind of price action. When market prices act this certain way, the system makes profitable trades. And when market prices don't act this certain way, the system makes losing trades.

If you decide to trade a system with real money, you are betting that future price action will be the kind that your system profits from. You are making a prediction: that prices will act a certain way (similar to the way they acted in the past, whose historical data gave you nice backtest results). However there is a problem. Your prediction might not come true. The markets might change (see (this thread) for an example) and if they do, your system can turn unprofitable.

So I don't know of any way to guarantee that a system will be profitable in the future. Nor do I know of any way to prove that a system will be "statistically reliable into the future" as our Original Poster asks. The makets can change. Backtests, performed on past prices, won't help you predict whether or not the markets will change and turn your system into a loser. If the markets change, it doesn't matter how many trades were in your backtest; your backtest becomes irrelevant when future prices don't resemble past prices.

I think about all you can do and all you can hope for is to get comfortable with a system, which is a highly personal & subjective judgement. What would make one person feel comfortable, might not satisfy someone else.

Some things that might or might not make you feel comfortable, include
  • Seeing that the system's performance is "good" (according to your judgment) over a wide range of parameter values
  • Seeing that the system's performance is "good" across many different tradeable instruments
  • Seeing that the system's performance is "good" over a large piece of past history that includes many different market behaviors
  • Seeing that the system's performance is "good" across many different market sectors
  • Seeing that the system's performance is "better" (according to your judgment) than well known reference systems such as buy-and-hold, or 200 day MA, or 20 day Donchian channel breakout, or whatever is the appropriate reference system
  • Seeing that Monte Carlo resampling of your sysem's longterm equity curve, produces an "acceptable" distribution of outcomes (according to your judgment)
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic    Traders' Roundtable Forum Index -> Testing and Simulation All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You can edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB 2.0.23 © 2001, 2008 phpBB Group