survivor bias

Discussions specific to trading the stock market.
Post Reply
tigertaco
Full Member
Full Member
Posts: 23
Joined: Fri Feb 27, 2009 1:50 am

survivor bias

Post by tigertaco »

Hi all,

I've recently done an experiment testing a system on a free database of currently traded stocks (yahoo) and purchased database free of survivor bias.

I'm glad I bought that data because the performance on the latter was significantly worse. Even though the system buys supposedly strong stocks (it is buy only system). I wonder what other people's experience is in this regard. Basically, I feel that these free databases are useless, even dangerous.

t.
ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Re: survivor bias

Post by ecritt »

tigertaco wrote:Hi all,

I've recently done an experiment testing a system on a free database of currently traded stocks (yahoo) and purchased database free of survivor bias.

I'm glad I bought that data because the performance on the latter was significantly worse. Even though the system buys supposedly strong stocks (it is buy only system). I wonder what other people's experience is in this regard. Basically, I feel that these free databases are useless, even dangerous.

t.
It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.

Free things are often dangerous, or at least lead to danger.
CRMCM
Full Member
Full Member
Posts: 20
Joined: Sat Apr 04, 2009 8:16 pm
Location: New York

Post by CRMCM »

If you are testing to put real money at work use paid for data that you know will yield results comparable to the real world. It's shocking that anyone would penny pinch on data - its the equivalent of a fundamentally driven hedge fund having a $0 research budget. Garbage in, Garbage out.
tigertaco
Full Member
Full Member
Posts: 23
Joined: Fri Feb 27, 2009 1:50 am

Re: survivor bias

Post by tigertaco »

It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.
********************

Yes, for a system which buys undervalued stocks that bias would be a real disaster. My system roughly speaking tries to fit the price curve to a polynomial c0+c1*x+c2*x^2 with positive thresholds on c1,c2 coefs ( to ensure uptrend). The idea is that since all indicators/oscillators reduce to either measuring velocity or acceleration, degree 2 polynomial accounts for both but without exotic looking formulas. Basically it buys a strong enough uptrend. The exits are time-based (maybe that's a problem).

On a different note, did you find new all-time high + atr exit reliable? Don't you periodically end up with 100+ stocks in your portfolio or you use some other filter?
ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Re: survivor bias

Post by ecritt »

tigertaco wrote:It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.
********************

Yes, for a system which buys undervalued stocks that bias would be a real disaster. My system roughly speaking tries to fit the price curve to a polynomial c0+c1*x+c2*x^2 with positive thresholds on c1,c2 coefs ( to ensure uptrend). The idea is that since all indicators/oscillators reduce to either measuring velocity or acceleration, degree 2 polynomial accounts for both but without exotic looking formulas. Basically it buys a strong enough uptrend. The exits are time-based (maybe that's a problem).

On a different note, did you find new all-time high + atr exit reliable? Don't you periodically end up with 100+ stocks in your portfolio or you use some other filter?
My average number of positions is about 800, although it has gone as low as 80 (like right now) and as high as 1800 (like in the mid 90's). What's the benefit of keeping the number of positions below 100?

The strategy is outlined in detail here: http://www.trendfollowing.com/whitepape ... stocks.pdf
tigertaco
Full Member
Full Member
Posts: 23
Joined: Fri Feb 27, 2009 1:50 am

Re: survivor bias

Post by tigertaco »

ecritt wrote:
tigertaco wrote:It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.
********************

Yes, for a system which buys undervalued stocks that bias would be a real disaster. My system roughly speaking tries to fit the price curve to a polynomial c0+c1*x+c2*x^2 with positive thresholds on c1,c2 coefs ( to ensure uptrend). The idea is that since all indicators/oscillators reduce to either measuring velocity or acceleration, degree 2 polynomial accounts for both but without exotic looking formulas. Basically it buys a strong enough uptrend. The exits are time-based (maybe that's a problem).

On a different note, did you find new all-time high + atr exit reliable? Don't you periodically end up with 100+ stocks in your portfolio or you use some other filter?
My average number of positions is about 800, although it has gone as low as 80 (like right now) and as high as 1800 (like in the mid 90's). What's the benefit of keeping the number of positions below 100?

The strategy is outlined in detail here: http://www.trendfollowing.com/whitepape ... stocks.pdf
Thanks, it's a very interesting paper. One problem with 100+ stocks is capital. I don't have enough capital to have so many open positions without overpaying for shares. The problem which affects anybody is correlation between positions which makes it hard to compute the true risk, but you seem to solve that problem, which is good. Another problem is it makes no guess on general market direction. I think if you add this component you may really boost performance. From my research, it's very important to evaluate the state of general market. I guess the simplicity of your entries/exits just shifts burden on money management which you are not disclosing and this makes perfect sense. My philosophy is the opposite - ideally I would own 2-3 stocks - but that's of course too extreme.
ecritt
Roundtable Knight
Roundtable Knight
Posts: 134
Joined: Sat Aug 28, 2004 3:44 am
Location: Phoenix, AZ
Contact:

Re: survivor bias

Post by ecritt »

tigertaco wrote:
ecritt wrote:
tigertaco wrote:It will really depend on what the system does. I buy new all time highs and trail with a 10 atr stop. Removing delisted stocks from my database marginally inflates the performance results. However, if you have any mean reversion or "buy the dip" or short-term trading component, not including delisted stocks will likely yield results that are very unrealistic.
********************

Yes, for a system which buys undervalued stocks that bias would be a real disaster. My system roughly speaking tries to fit the price curve to a polynomial c0+c1*x+c2*x^2 with positive thresholds on c1,c2 coefs ( to ensure uptrend). The idea is that since all indicators/oscillators reduce to either measuring velocity or acceleration, degree 2 polynomial accounts for both but without exotic looking formulas. Basically it buys a strong enough uptrend. The exits are time-based (maybe that's a problem).

On a different note, did you find new all-time high + atr exit reliable? Don't you periodically end up with 100+ stocks in your portfolio or you use some other filter?
My average number of positions is about 800, although it has gone as low as 80 (like right now) and as high as 1800 (like in the mid 90's). What's the benefit of keeping the number of positions below 100?

The strategy is outlined in detail here: http://www.trendfollowing.com/whitepape ... stocks.pdf
Thanks, it's a very interesting paper. One problem with 100+ stocks is capital. I don't have enough capital to have so many open positions without overpaying for shares. The problem which affects anybody is correlation between positions which makes it hard to compute the true risk, but you seem to solve that problem, which is good. Another problem is it makes no guess on general market direction. I think if you add this component you may really boost performance. From my research, it's very important to evaluate the state of general market. I guess the simplicity of your entries/exits just shifts burden on money management which you are not disclosing and this makes perfect sense. My philosophy is the opposite - ideally I would own 2-3 stocks - but that's of course too extreme.
With respect to picking which stocks to include in a narrow portfolio, consider that 90% of the stock market's returns come from a very small minority of approx 12% of stocks. If you miss those you miss the returns. If I had to pick 100 stocks out of 800, with my luck, none of them would end up in the 12% of outperforming stocks. So I buy them all and accept the results.
Post Reply