## Interesting study on predicting prices

yp
Roundtable Fellow
Posts: 77
Joined: Thu Apr 17, 2003 9:11 am

### Interesting study on predicting prices

On my site I posted a section from a book called "mind traps".
I wanted to share it with members of this forum.

The following text was taken from a popular trading book, Decision Traps, by J. Edward Russo and Paul J.H. Schoemaker. In this book, nine different types of decisions were tested using three different decision methods. The accuracy of the decisions was then compared and analyzed for effectiveness in predicting final outcomes. The investigator looked at different types of decisions, predicting grades, predicting recovery from cancer, performance of life insurance salesmen, as well as predicting changes in stock prices. He used three different decision making processes: an Intuitive Prediction Model, a Subjective Linear Model, and an Objective Linear Model.

Intuitive prediction is defined as making a decision without the use of any objective or quantifiable data. For instance, in trying to predict the academic performance of graduate students, the researches asked their advisors to do so without seeing their grades and just by talking to them. The decision-makers had to rely on their intuitive impressions and any other factors they thought relevant (how the student dressed, their language skills, grooming habits, etc.). This is the same way discretionary traders make trading decisions - using intuition and gut instinct. In predicting the stock prices, it is highly likely that the researcher engaged a discretionary trader to predict the future prices of stocks.

A Subjective Linear Model is a much more complex decision making process. It starts with the interviewing experts in a field and learning how they make decisions. The researcher literally asks the expert how he or she makes decisions and they respond by explaining how they make their predictions. Although these experts are not using quantifiable data, they have enough experience and knowledge in their field to be successful. This decision making process is then outlined by the researcher.

For instance, a physician, highly experienced in treating cancer, has probably become fairly adept at predicting the life expectancy of his patients, even without using any objective data. The researcher interviewed the physician and attempted to determine exactly how the physician made this assessment. Then the researcher put this newly quantified data into a regression model and attempted to predict the life expectancy of cancer patients.

This is very similar to how a technical trader makes decisions. He goes to seminars and reads books to learn how the experts make decisions using technical indicators. He then takes what he learns and attempts to trade like he experts. In a sense, he does his own regression model of the expert's process to make trading decisions.

For the Objective Linear Model, the researcher developed an objective model based on historical tests and observations to predict results. This is defining and using quantifiable data, running historical tests, and then using the results of the tests to predict future outcomes.

For instance, the researcher would look at reams of physical data from terminal ill patients, and correlate the data with how long the patient lived. After running the historical tests, the researcher would then obtain the physical data form cancer patient, and using the historical test data, attempt to predict how long that cancer patient will live.

This is exactly what a system trader does. He runs historical tests and then uses that data to take a position in the market. He uses objective quantifiable data tested historically to make his trading decisions. The following table shows the results of tests.

Types of Judgment Intuitive Prediction Subjective Linear Objective Linear
Life Expectancy of Terminally Ill Patients -.01 .13 .35
Change in Stock Prices .23 .29 .80
Mental Illness using Personality Tests .28 .31 .46
Grades and Attitudes in Psychology Course .48 .56 .62
Business Failure using Financial Ratios .50 .53 .67
Student's Ratings of Teacher's Effectiveness .35 .56 .91
Performance of Life Insurance Salesmen .13 .14 .43
IQ Scores using Rorsach Tests .47 .51 .54
Mean Scores using Rorsach Tests .33 .39 .64

In every case, the Subjective Linear Model outperformed the Intuitive Prediction Model but only by a small margin. If you look at predicting the changes in stock prices, the Subjective Linear Model only slightly outperformed the Intuitive Prediction Model.

The real insight from this study comes when we look at the results of the Objective Linear Model. In every case, the Objective Linear Model outperformed both the Intuitive Prediction model and the Subjective Linear Model. In some cases, the improvement was minor, and in others it was substantial. It is interesting to observe that the greatest improvement came when using the Objective Linear Model in predicting the changes in stock prices.

Howard Brazzil
Roundtable Fellow
Posts: 54
Joined: Wed Apr 16, 2003 12:45 pm
Location: Houston, TX USA

### re: Interesting study on predicting prices

Great post, yp. Thanks.

There was an excellent section on actuarial versus subjective decision making in James P. Oâ€™Shaughnessyâ€™s What Works on Wall Street. The 2nd chapter, entitled â€œThe Unreliable Experts: Getting in the Way of Outstanding Performance,â€

AFJ Garner
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:
Yes, very interesting yp. Here is a little tale in support. For much of the 90's I played the IPO market. My mentor and at that stage financial backer was a wonderful American who taught us to take stock and simply flip it at the open. Perfectly legal of course. Worked wonderfully regardless of market conditions- it was a "mechanical system" pure and simple. We did not give a hoot what the company did or why so long as it was brought by a solid underwriter in the bulge bracket. We took good and bad. On rare occasions over the past three lean years I have run positions for a few weeks or a month, trying to be smart, double guessing the market. I still made money but guess what? The stress was a great deal higher and I would have fared better to have stuck to my tried and trusted "mechanical system" - hit the first bid! Long and the short of it? I will never deviate from my "system" in IPOs again. And that is also why I am such an avid student of systems for the futures market.

edward kim
Roundtable Knight
Posts: 344
Joined: Sun Apr 20, 2003 2:42 pm
Location: Silicon Valley / San Jose, CA USA
Contact:
The Objective Linear Model probably explains why so many systems/people fail. Many statistical analyses (especially from Wall Street) only take a chunk of historical data (from say 1995-1997), and show how accurate their predictors were, and how this same technique SHOULD be applied to investment choices in the present. I'm sure that if they used 30 years of historical data to test their proposition, the results would be quite different.

Most people do NOT adequately backtest, and they should. The data is all there and wants to be used for a good reason. I think that just because a market was hot at the time (like gold in the late 70s), doesn't mean that the particular set of data containing an "abnormal event" should be excluded to "normalize the data". These unusual events make the markets what they are. I'm a firm believer in going through the "reams of data" and developing a system for all scenarios - not just the ones that fit nicely.