Adjusted Prices

[TrendMonkey]

This is a line copied from the CL price file that comes with VeriTrader.

19960105, -0.82, -0.61, -0.92, -0.63, 85001, 387329

Now, I realize there's a lot of complicated math that goes into CSI's algorithms, but at the end of the day, isn't this implying that on this day Crude fluctuated by 50%, which surely is not the case?

It's late Sunday and I'm sunburnt, I apologize if I'm missing something blatantly obvious...

[TK]

I suppose these data are point adjusted so only point differences are preserved. Percentage differences (ratios) are not preserved in such back-adjusted data and in fact they change from one rollover to another.

Here is the Crude Oil Feb 1996 data from Pinnacle Data:

19960105 20.05 20.28 19.97 20.26

As you can see, the point difference between high and low is the same, i.e. 0.31. Also point differences between high, low and close prices are the same. Differences between high or low or close and open are off by 0.02, and my guess is that this is because CSI had the original open price of 20.07 and Pinnacle had 20.05.

[TrendMonkey]

Thanks TK... I find that all very disturbing, though! I will confirm that this anomaly, if that's what it is, exists in my own (non-VeriTrader) generated CSI data and if so ask them to explain it.

[TK]

Thanks TK... I find that all very disturbing, though! I will confirm that this anomaly, if that's what it is, exists in my own (non-VeriTrader) generated CSI data and if so ask them to explain it.

Hi,

It is not an anomaly. Negative prices in back-adjusted data are to be expected and this is perfectly normal. You just have to know what types of back-adjusted data go with what types of systems and why. Perhaps this can help:

viewtopic.php?p=3651#3651

[TrendMonkey]

I'm not questioning the negative prices. I'm questioning the 50% range between Hi and Lo. It's interesting that the difference in CENTS is the same as Pinnacle's, but I can't believe VeriTrader (and any other software) would not get screwed up by gross distortions in the relationship of OHLC.

[damian]

....but I can't believe VeriTrader (and any other software) would not get screwed up by gross distortions in the relationship of OHLC.

Hi Trendmonkey,

Your concern is a valid, but you need to first consider what types of indicators and price relationships a particular system (in VeriTrader or otherwise) uses.

If you have a relationship that signals action based upon something like Momentum/ATR then you might have an issue.

Guess which system has a strength filter based on something like Momentum/ATR?

A moving average system might be ok, same for a HH-LL breakout system.

Obviously if your system signals action based upon % changes in price, you are going to have big problems.

[Jake Carriker]

Backadjusted data retains the integrity of price changes from one bar to the next, not the absolute price levels.

You cannot use backadjusted data to accurately calculate price percentage changes. However, this type of data does not distort indicators like highest high, lowest low breakout channels, moving average crossovers, ATRs, or any number of things that don't depend on comparing point changes to absolute price levels.

There is a rather simple workaround for anyone that does want to compare prices (the closing price, for instance) to point moves and still retain the myriad benefits of using backadjusted data. One might have an extra field in one's data files that contains the actual closing price (or open, or low, or high, or all of them) for the contract month the backadjusted series is derived from at any given time. This requires some cutting and pasting in Excel.

An even simpler solution for those less nitpicky is to tell CSI Unfair Advantage (I don't know if other data vendors have this capability) to export the cash price as a field in the data file. It might be off a bit from the contract month that is active in your backadjusted series at a given time, but it won't be orders of magnitude away and it certainly won't be negative.

When the absolute price level is required, refer to whatever field in the data file holds that value rather than the backadjusted price. If your software does not know how to refer to such fields, you might consider looking for some that does and adding it to your portfolio of trading tools. I don't know if TBB can do this or not. Old school Trading Recipes and new generation Mechanica both handle it just fine.

All this is moot if your system logic does not use indicators that require accurate absolute price levels. Most systems that people categorize as "long term trend following" do not.

Best,
Jake

[Jake Carriker]

I see Damian made some of my points for me as I wrote my last message.

To be clear, Momentum / ATR presents no problem for backadjusted data, if momentum is defined as price minus price some bars back (i.e. a price change described in points).

Momentum / Price or Price / ATR or other such comparisons require the ability to refer to the "real" price level in conjunction with backadjusted data.

Jake

[sluggo]

For those who have purchased the "Ocean" trading indicators from Jim Sloman (originator of the phase-of-the-moon Delta Phenomenon, marketed by J. Welles Wilder):

You may wish to analyse the Ocean indicators and ask yourself whether or not they have this very problem that damian, TM, and JC are discussing.

[damian]

I see Damian made some of my points for me as I wrote my last message.

To be clear, Momentum / ATR presents no problem for backadjusted data, if momentum is defined as price minus price some bars back (i.e. a price change described in points).

oops, you are of course correct, Jake.

Um, in my defence, I did once get it right. From a previous post :

viewtopic.php?p=3606#3606

Trendmonkey, there is also some good discussion in the posts that follows (by sadly absent members)

On a related matter so I thought I would throw this into the mix.

- Take a commodity, sep02 expiry, calculate the 10 day ATR. Lets say it is 15.
- On 1st Sep 02 the close is 50.
- Divide ATR by the close of 50. 15/50=0.3
- Lets say you have a silly rule: if ATR/C<0.35 then buy.
- In real time trading of the sep02 expiry, you would buy on 1 Sep 02.

Now, 1 year later and you are building a continuous back adjusted series.

- There is a 10 point gap between sep02 and dec02.
- Dec is 10 points below sep.
- The splice requires the sep02 series to have 10 points deducted from all data points so as to align with the dec02 relative level.
- Remember that the actual historical close of the Sep02 contract on 1 Sep was 50. However the back adjustment changes it to 40.
- 1 Sep 02 ATR would not have changed as a result of the back adjustment.
- Now on 1 Sep 02, we have ATR/C = 15/40 = 0.375
- in backtesting on this adjusted series you would no longer get a buy signal on 1 Sep 02.

The only difference is the close.

I used a silly rule, yet it is not so hard to think of a realistic rule that would be influenced by the magnitude of the close relative to ATR.

[TrendMonkey]

Thanks for the feedback, all. This whole topic of backadjusting data really bothers me, I really need to invest more energy in understanding it before I can be comfortable. Thanks again.

[Jake Carriker]

...I really need to invest more energy in understanding it before I can be comfortable.

That doesn't seem like a bad thing to do at all. Once you think through the whole process and make the concepts second nature you have a good chance of lessening the "D'oh!!!" factor later on.

As an experiment you might take one or more data files and add or subtract a fixed number to every price in the series, thus creating prices that are most definitely far from the actual prices. You might then run one "manipulated" data file through a simple channel breakout system (i.e.place a buystop at the highest high of the last N bars, and a sellstop at the lowest low of the last N bars). Also run the original file on the same system. Notice any similarities or differences in the trades taken.

Repeat the process with different types of trading math (moving averages, volatility breakouts, indicators that use price % changes, etc.) learn under what circumstances "anomolies" show up and under what circumstances they don't and why.

As an advanced version of this experiment, you might try adding ever larger fixed amounts to the "manipulated" series and notice if your results start changing at any point. Again, if they do, ask why. It may be instructive.

Good luck!

Jake