Traders' Roundtable

When back test my strategy against the S&P 500 over a 10 year period it does extremely well. When I test the same strategy against the Russell 2000 it doesn't perform as well; same with NASDAQ. Can this be due to the fact that the S&P 500 is a diversified weighted portfolio and the the Russell for instance is simply made up of components based on market cap? And that the correlation of the components of the S&P 500 is lower than the other indices? I'm trying to dermtermine if the system is robust or if it's the makeup of portfolio that is causing variances in performance. I should note that stastics of each test yields very similar results in terms of win/loss percentage and ratio of avg win in dollars/avg loss in dollars. Interested to hear any thoughts, suggestions or recommendations. Thank you.

Maybe you would find it enlightening to backtest your trading strategy against 100 (or 1000) portfolios of randomly chosen stocks.

For a moment, suppose you have done this. Here are some possible outcomes

1. 30% of the backtest runs were profitable; 70% of backtests lost money
2. 52% of the backtest runs were profitable
3. 75% of the backtest runs were profitable, and 40% of backtest runs had a Sharpe Ratio greater than +0.5

How might you react to these outcomes?

Maybe you'll say that outcome #1 indicates the strategy is worse than coinflipping, and you won't risk your money on it.

Maybe you'll say that outcome #2 indicates the strategy is a 50-50 proposition, and so you'll only allocate half of your risk-capital to it.

Maybe you'll say that outcome #3 indicates the strategy is not deliberately or accidentally curve-fitted to one particular portfolio or one particular style of portfolio-construction.

I seem to recall that there is free Blox source code for Random Portfolio testing in the Blox Marketplace [useable by Blox Pro or Blox Builder]. Maybe an hour's searching around there, would find it.

Thanks for the comments Sluggo. I'll look into the Blox Marketplace and see what that has to offer.

Hi Ghost,
are you speaking about a test on all 500 S&P stocks or a test against the futures?

Thanks for asking. I have a subscription to CSI end of day data for all US stocks. So in CSI I downloaded the S&P 500 components and transferred those stocks to TB for me to run my suite against. So that was long winded answer to your question - I'm using S&P 500 stocks.

If you are allowing stocks to be traded before they were actually included in the index, I can guarantee your results will be inflated and unrealistic.

I agree with your comment an have takin this in to account by editing the portfolio to include only those securities that haven't changed over the past 10 years. What I'm trying to understand is if the strategy isn't robust because it works well on S&P but not as well on Russell 2000. Not sure if this is due to correlation - maybe?

Additionally, when I run a test it scans all the stocks which are listed alphabetically which means I might be omitting a security that is exhibiting a trend - not sure how to overcome this. I found some sort of ranking/strength feature but not really sure how this works and what the criteria for ranking is.