Page 1 of 1

survivor bias

Posted: Sun May 03, 2009 2:52 pm
by tigertaco
Hi all,

I've recently done an experiment testing a system on a free database of currently traded stocks (yahoo) and purchased database free of survivor bias.

I'm glad I bought that data because the performance on the latter was significantly worse. Even though the system buys supposedly strong stocks (it is buy only system). I wonder what other people's experience is in this regard. Basically, I feel that these free databases are useless, even dangerous.

t.

Re: survivor bias

Posted: Sun May 03, 2009 3:58 pm
by ecritt
tigertaco wrote:Hi all,

I've recently done an experiment testing a system on a free database of currently traded stocks (yahoo) and purchased database free of survivor bias.

I'm glad I bought that data because the performance on the latter was significantly worse. Even though the system buys supposedly strong stocks (it is buy only system). I wonder what other people's experience is in this regard. Basically, I feel that these free databases are useless, even dangerous.

t.
It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.

Free things are often dangerous, or at least lead to danger.

Posted: Sun May 03, 2009 6:10 pm
by CRMCM
If you are testing to put real money at work use paid for data that you know will yield results comparable to the real world. It's shocking that anyone would penny pinch on data - its the equivalent of a fundamentally driven hedge fund having a $0 research budget. Garbage in, Garbage out.

Re: survivor bias

Posted: Sun May 03, 2009 8:07 pm
by tigertaco
It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.
********************

Yes, for a system which buys undervalued stocks that bias would be a real disaster. My system roughly speaking tries to fit the price curve to a polynomial c0+c1*x+c2*x^2 with positive thresholds on c1,c2 coefs ( to ensure uptrend). The idea is that since all indicators/oscillators reduce to either measuring velocity or acceleration, degree 2 polynomial accounts for both but without exotic looking formulas. Basically it buys a strong enough uptrend. The exits are time-based (maybe that's a problem).

On a different note, did you find new all-time high + atr exit reliable? Don't you periodically end up with 100+ stocks in your portfolio or you use some other filter?

Re: survivor bias

Posted: Sun May 03, 2009 8:34 pm
by ecritt
tigertaco wrote:It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.
********************

Yes, for a system which buys undervalued stocks that bias would be a real disaster. My system roughly speaking tries to fit the price curve to a polynomial c0+c1*x+c2*x^2 with positive thresholds on c1,c2 coefs ( to ensure uptrend). The idea is that since all indicators/oscillators reduce to either measuring velocity or acceleration, degree 2 polynomial accounts for both but without exotic looking formulas. Basically it buys a strong enough uptrend. The exits are time-based (maybe that's a problem).

On a different note, did you find new all-time high + atr exit reliable? Don't you periodically end up with 100+ stocks in your portfolio or you use some other filter?
My average number of positions is about 800, although it has gone as low as 80 (like right now) and as high as 1800 (like in the mid 90's). What's the benefit of keeping the number of positions below 100?

The strategy is outlined in detail here: http://www.trendfollowing.com/whitepape ... stocks.pdf

Re: survivor bias

Posted: Sun May 03, 2009 10:49 pm
by tigertaco
ecritt wrote:
tigertaco wrote:It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.
********************

Yes, for a system which buys undervalued stocks that bias would be a real disaster. My system roughly speaking tries to fit the price curve to a polynomial c0+c1*x+c2*x^2 with positive thresholds on c1,c2 coefs ( to ensure uptrend). The idea is that since all indicators/oscillators reduce to either measuring velocity or acceleration, degree 2 polynomial accounts for both but without exotic looking formulas. Basically it buys a strong enough uptrend. The exits are time-based (maybe that's a problem).

On a different note, did you find new all-time high + atr exit reliable? Don't you periodically end up with 100+ stocks in your portfolio or you use some other filter?
My average number of positions is about 800, although it has gone as low as 80 (like right now) and as high as 1800 (like in the mid 90's). What's the benefit of keeping the number of positions below 100?

The strategy is outlined in detail here: http://www.trendfollowing.com/whitepape ... stocks.pdf
Thanks, it's a very interesting paper. One problem with 100+ stocks is capital. I don't have enough capital to have so many open positions without overpaying for shares. The problem which affects anybody is correlation between positions which makes it hard to compute the true risk, but you seem to solve that problem, which is good. Another problem is it makes no guess on general market direction. I think if you add this component you may really boost performance. From my research, it's very important to evaluate the state of general market. I guess the simplicity of your entries/exits just shifts burden on money management which you are not disclosing and this makes perfect sense. My philosophy is the opposite - ideally I would own 2-3 stocks - but that's of course too extreme.

Re: survivor bias

Posted: Sun May 03, 2009 11:14 pm
by ecritt
tigertaco wrote:
ecritt wrote:
tigertaco wrote:It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.
********************

Yes, for a system which buys undervalued stocks that bias would be a real disaster. My system roughly speaking tries to fit the price curve to a polynomial c0+c1*x+c2*x^2 with positive thresholds on c1,c2 coefs ( to ensure uptrend). The idea is that since all indicators/oscillators reduce to either measuring velocity or acceleration, degree 2 polynomial accounts for both but without exotic looking formulas. Basically it buys a strong enough uptrend. The exits are time-based (maybe that's a problem).

On a different note, did you find new all-time high + atr exit reliable? Don't you periodically end up with 100+ stocks in your portfolio or you use some other filter?
My average number of positions is about 800, although it has gone as low as 80 (like right now) and as high as 1800 (like in the mid 90's). What's the benefit of keeping the number of positions below 100?

The strategy is outlined in detail here: http://www.trendfollowing.com/whitepape ... stocks.pdf
Thanks, it's a very interesting paper. One problem with 100+ stocks is capital. I don't have enough capital to have so many open positions without overpaying for shares. The problem which affects anybody is correlation between positions which makes it hard to compute the true risk, but you seem to solve that problem, which is good. Another problem is it makes no guess on general market direction. I think if you add this component you may really boost performance. From my research, it's very important to evaluate the state of general market. I guess the simplicity of your entries/exits just shifts burden on money management which you are not disclosing and this makes perfect sense. My philosophy is the opposite - ideally I would own 2-3 stocks - but that's of course too extreme.
With respect to picking which stocks to include in a narrow portfolio, consider that 90% of the stock market's returns come from a very small minority of approx 12% of stocks. If you miss those you miss the returns. If I had to pick 100 stocks out of 800, with my luck, none of them would end up in the 12% of outperforming stocks. So I buy them all and accept the results.