As someone more comfortable trading stocks I'm finding it very difficult to accurately backtest any further than 3 years. I realize I have much more than 3 years of data at my disposal but then doesn't it really come down to stock selection rather than the system.
Let's say my system is designed to trade the SP 500 stocks. I grab the current components on the SP 500 and backtest last year's data which is fine. But it isn't realistic testing those same stocks 3 years ago because picking stocks that are currently in the SP 500 gives me the advantage of forsight into the future for a test of those stocks 3 years or even worse 5-10 years ago. Obviously those stocks are in the SP for a reason. 10 years ago, they wouldn't have been. And having the advantage of knowing they would eventually make it into the SP 500 obviously makes the system give good results maybe because of the system, but more likely because your stock selection was a group of stocks that would eventually perform so well as to be included in the SP 500.
So, unless I have a list of the SP 500 at the beginning of each year I want to backtest the data the test seems very skewed.
Is there information on the components of the SP 500 for the beginning of each year dating back to x years ago?
Backtesting Equities (Stocks)
-
Paul King
- Roundtable Knight
- Posts: 207
- Joined: Mon Feb 23, 2004 9:13 am
- Location: Vermont, USA
- Contact:
I looked into this a while ago when I was thinking of developing a system based on additions and deletions to the S&P 500.
You can get S&P500 index changes (not all the components - but you can work it out from the deltas) back to 2000 for free at:
http://www2.standardandpoors.com/servle ... cd=500&f=2
For data further back than that you have to pay (don't know what the fees are like - I never got that far):
8. Where can I order historical data for any of the S&P US Indices?
For historical data, call S&P's Central Inquiry at 1-800-233-2310, option 1 or 212-438-2046 from 9 a.m. to 4:30 p.m. weekdays, or email index_services@sandp.com. (This is a fee-based service).
You can get S&P500 index changes (not all the components - but you can work it out from the deltas) back to 2000 for free at:
http://www2.standardandpoors.com/servle ... cd=500&f=2
For data further back than that you have to pay (don't know what the fees are like - I never got that far):
8. Where can I order historical data for any of the S&P US Indices?
For historical data, call S&P's Central Inquiry at 1-800-233-2310, option 1 or 212-438-2046 from 9 a.m. to 4:30 p.m. weekdays, or email index_services@sandp.com. (This is a fee-based service).
The problem with back-testing equities is that the environment is not very reocurrring unlike futures. The reversion to mean in terms of volatility in equities is not as strong.
You might find a good looking equity curve, and it fails completely when you forward test it.
The best candidates should be the ones with the highest consistent volatilities, not necessarily the ones in the S&P.
You might find a good looking equity curve, and it fails completely when you forward test it.
The best candidates should be the ones with the highest consistent volatilities, not necessarily the ones in the S&P.
I completely agree in terms of the consistency of the environment.
You can create a somewhat static environment by filtering the market to return a group of individual equities, but then you run into the problem of being able to acquire the data to backtest according to your filter if the filter isn't directly related to price data.
For instance if your filter is something comparable to the CANSLIM method (used by O'Neill I believe) it's hard to return a group of stocks that met those specific criteria in 1998 to backtest the system. You could return the stocks that meet the criteria today, but obviously wouldn't have met the criteria 5 years ago.
Additionally, backtesting against say all stocks currently in the NASDAQ for the last 10 years gives you the advantage of the foresight of knowing that they were going to be in the NASDAQ ten years later, at least for the furthest away years of the backtest and less so as you move closer to the present.
So how can I fairly put together a group of stocks to backtest against?
Could developing a strategy that trades ETFs instead of individual equities help overcome this problem?
You can create a somewhat static environment by filtering the market to return a group of individual equities, but then you run into the problem of being able to acquire the data to backtest according to your filter if the filter isn't directly related to price data.
For instance if your filter is something comparable to the CANSLIM method (used by O'Neill I believe) it's hard to return a group of stocks that met those specific criteria in 1998 to backtest the system. You could return the stocks that meet the criteria today, but obviously wouldn't have met the criteria 5 years ago.
Additionally, backtesting against say all stocks currently in the NASDAQ for the last 10 years gives you the advantage of the foresight of knowing that they were going to be in the NASDAQ ten years later, at least for the furthest away years of the backtest and less so as you move closer to the present.
So how can I fairly put together a group of stocks to backtest against?
Could developing a strategy that trades ETFs instead of individual equities help overcome this problem?
-
Paul King
- Roundtable Knight
- Posts: 207
- Joined: Mon Feb 23, 2004 9:13 am
- Location: Vermont, USA
- Contact:
ETFs do change composition as well
Using ETFs will mitigate this problem somewhat, but they do still change composition from time-time due to corporate actions (mergers, acquisitions, breakups, spin-offs etc.) and companies going bankrupt.
If you look at www.holdrs.com for example, each one has a list of all historical changes so you could track how the makeup of each one has changed over its' entire history.
Paul
If you look at www.holdrs.com for example, each one has a list of all historical changes so you could track how the makeup of each one has changed over its' entire history.
Paul
What I've done in the past, is to make system simulation backtests a two-pass affair. Pass1 is a custom-programmed application written in C++ which performs the filtering and, oh you're going to positively hate this, writes a flag (OK_to_trade or not_OK_to_trade) right into the daily ASCII price data files. Pass2 is BloxBuilder or Recipes or Metastock or AmiBroker, which runs the actual trading system and acts upon the filter flags produced in Pass1. A new flag value is computed each day for each stock.
Suppose you implement a volume-based filter for pass1. This lets you include Microsoft in your portfolio of items-to-possibly-trade, without allowing hindsight bias. The system won't take the trade of the century (buy MSFT on the IPO) because it fails the volume filter. Later, when MSFT is big enough to pass the volume filter (~1986) your system will be allowed to trade it. So you can happily feed in a list of ALL stocks, knowing that the volume filter will prevent you from trading supremely illiquid issues.
On the other hand if you start with a portfolio containing today's Nasdaq-100 and run a test of 15 years, you've selected (in hindsight) 100 stocks that are going to grow ferociously and eventually become the biggest 100 around. Not wise.
Suppose you implement a volume-based filter for pass1. This lets you include Microsoft in your portfolio of items-to-possibly-trade, without allowing hindsight bias. The system won't take the trade of the century (buy MSFT on the IPO) because it fails the volume filter. Later, when MSFT is big enough to pass the volume filter (~1986) your system will be allowed to trade it. So you can happily feed in a list of ALL stocks, knowing that the volume filter will prevent you from trading supremely illiquid issues.
On the other hand if you start with a portfolio containing today's Nasdaq-100 and run a test of 15 years, you've selected (in hindsight) 100 stocks that are going to grow ferociously and eventually become the biggest 100 around. Not wise.
I think this is perfectly acceptable though. Although the components of an ETF have changed the inherent underlying structure of what the ETF has been built to track has not changed, at least when the ETF has been built to track certain sectors/industries. Or maybe I'm way off base on how ETFs work.Using ETFs will mitigate this problem somewhat, but they do still change composition from time-time due to corporate actions (mergers, acquisitions, breakups, spin-offs etc.) and companies going bankrupt.
Regardless, has anyone out there built a system around trading ETFs?
To me this seems like it might be a good way of dealing with many of the problems presented in trading individual securities.
-
Paul King
- Roundtable Knight
- Posts: 207
- Joined: Mon Feb 23, 2004 9:13 am
- Location: Vermont, USA
- Contact:
When you buy an ETF, say the Merrill Lynch Semiconductor HOLDR (SMH), what you are actualy buying is a number of shares of each component of SMH in the proportions determined by a algorithm specified when the ETF was created.
The obvious advantages of buying SMH instead of each component individually are the transaction costs of doing one trade instead of multiple.
Whether you care that the components of SMH have changed over time (and whether you account for this in your testing) is up to you.
I trade a medium-term trend following system that trades ETFs based on analysis of their components, so it matters to me, but you could trade the ETF as though it is not a derivative instrument and only include the price action of the ETF itself in your trading system.
Paul
The obvious advantages of buying SMH instead of each component individually are the transaction costs of doing one trade instead of multiple.
Whether you care that the components of SMH have changed over time (and whether you account for this in your testing) is up to you.
I trade a medium-term trend following system that trades ETFs based on analysis of their components, so it matters to me, but you could trade the ETF as though it is not a derivative instrument and only include the price action of the ETF itself in your trading system.
Paul
