Traders' Roundtable
A forum for mechanical system traders.

Author Message
LeviF
Roundtable Knight

Joined: 22 Dec 2003
Posts: 1423
Location: Des Moines, IA

 Posted: Thu Jan 31, 2008 2:05 pm    Post subject: Reliable number of trades? How does one know if a system generates enough trades to be statistically reliable into the future? Would 100 trades over 10 years on a basket of 30 futures be adequate?
nodoodahs
Roundtable Knight

Joined: 09 Aug 2006
Posts: 218

 Posted: Thu Jan 31, 2008 2:40 pm    Post subject: Just some random thoughts on the matter. Philosophical answer: You never "know" for sure, because, in theory at least, the conditions that existed in the market during your test period may have been unique, and have changed in the current market. With one trade per contract every three years or so, I guess there's a real possibility of some kind of outlier in that data. Statistics involves assumptions. Not comforting, I know. So much of this is having confidence or a reason to believe we have an edge, without ever knowing for sure. Well, sometimes you almost maybe know for sure. Short statistical answer: Maybe. More would be better. Long statistical answer: Statistics involves assumptions, and with only 100 data points, you're not on super ground, but it depends on what kind of results those 100 data points have. Assume that a trade is a trade, regardless of contract or source. BIG assumption. Then with 100 trades you have a payout distribution that can be tested. You could take the 100 trades, convert them into +/- percent of equity, and categorize them into a distribution that had a mean, standard deviation, skewness, kurtosis, etc., and perform some tests on that, too. With only 100 in the sample, unless it looks really promising from a profitability standpoint, you might want to proceed with some caution. Any big bad losses? Look at the losses and what the contracts were doing, and think if that's a common situation or what the cause might be, how to eliminate those big losses. Winning percentage could be tested with the binomial approximation to the normal, which assumes some things, like a normal distribution. The population mean winning percentage is estimated by your sample of 100 trades, and the standard deviation is the square root of the product of win %, loss %, and number in the sample (in this case, 100). For example, if your sample was 35% winners, then statistically you can have about 98% confidence that your system has a future winning percentage between 25% and 45%. Assuming your assumptions are good. Now if you take your average win and average loss amounts as percentages of equity, and *assume* that these distributions are solid, you could calculate your expectancy based on the lower bound (25% wins in the example), the sample amount (35% wins in the example), and the upper bound (45% wins in the example) of the winning percentage. Are all of those numbers "good" to you? Could you take that system at the lower bound of win % and turn over enough trades to make a good return? Is the system even profitable at that lower bound? What's the maximum number of losses in a row you could expect at that lower bound, and with ten trades a year, what are the odds that you go winless in a year? Would that make you stop trading? Bottom line, we all have to make decisions with very imperfect data. You have to be comfortable with what your analysis of those 100 trades gives you. If you're not, could you backtest longer? More contracts? I don't know._________________One minute you’re up half a million in soybeans and the next, boom, your kids don’t go to college and they’ve repossessed your Bentley.
sluggo
Roundtable Knight

Joined: 11 Jun 2004
Posts: 2959

 Posted: Thu Jan 31, 2008 5:08 pm    Post subject: A trading system exploits (profits from) a certain kind of price action. When market prices act this certain way, the system makes profitable trades. And when market prices don't act this certain way, the system makes losing trades. If you decide to trade a system with real money, you are betting that future price action will be the kind that your system profits from. You are making a prediction: that prices will act a certain way (similar to the way they acted in the past, whose historical data gave you nice backtest results). However there is a problem. Your prediction might not come true. The markets might change (see (this thread) for an example) and if they do, your system can turn unprofitable. So I don't know of any way to guarantee that a system will be profitable in the future. Nor do I know of any way to prove that a system will be "statistically reliable into the future" as our Original Poster asks. The makets can change. Backtests, performed on past prices, won't help you predict whether or not the markets will change and turn your system into a loser. If the markets change, it doesn't matter how many trades were in your backtest; your backtest becomes irrelevant when future prices don't resemble past prices. I think about all you can do and all you can hope for is to get comfortable with a system, which is a highly personal & subjective judgement. What would make one person feel comfortable, might not satisfy someone else. Some things that might or might not make you feel comfortable, includeSeeing that the system's performance is "good" (according to your judgment) over a wide range of parameter values Seeing that the system's performance is "good" across many different tradeable instruments Seeing that the system's performance is "good" over a large piece of past history that includes many different market behaviors Seeing that the system's performance is "good" across many different market sectors Seeing that the system's performance is "better" (according to your judgment) than well known reference systems such as buy-and-hold, or 200 day MA, or 20 day Donchian channel breakout, or whatever is the appropriate reference system Seeing that Monte Carlo resampling of your sysem's longterm equity curve, produces an "acceptable" distribution of outcomes (according to your judgment)
 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
 All times are GMT - 6 Hours Page 1 of 1

 Jump to: Select a forum General----------------How to Use these Forums - REQUIRED READINGQuestions and Suggestions for the ForumNews and AnnouncementsDiversions and Amusements Trading Psychology----------------Market PsychologyTrader Psychology Trading Systems----------------Testing and SimulationMoney ManagementTrend Indicators and Signals Newbie's Corner----------------Newbie to NewbieTotal Newbie Questions Learning to Trade----------------Books, Courses, and SeminarsQuestions and AnswersOn Becoming a Pro - The Business of TradingTrading for a Living - The Independent Trader Trading Blox™ Customer Support----------------Training and Tutorials Simulation and Testing Platforms; and Trading Software----------------Data Providers and other non testing software Brokers and Markets----------------StocksFutures MarketsForexBrokers
You cannot post new topics in this forum
You cannot reply to topics in this forum
You can edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum

Powered by phpBB 2.0.23 © 2001, 2008 phpBB Group