Volume data in CSI - how stable are the estimates?

Use this forum to discuss data providers like CSI, charting, or other non testing software.
Post Reply
Mudnas
Senior Member
Senior Member
Posts: 25
Joined: Thu Dec 05, 2013 4:21 pm

Volume data in CSI - how stable are the estimates?

Post by Mudnas » Thu Aug 14, 2014 6:08 pm

Imagine a trading rule that incorporates the logic to only trade if the most recent volume is less that the volume of the previous bar.
Obviously we don't want to find that CSI makes significant adjustments the following day to volume data such that an entry signal would no longer be met.

So, is the volume date provided by CSI for the most recent day just an estimate? And more importantly, how accurate or stable is that estimate?

I'm going to research this myself! I will chose 50 liquid futures markets, and for each day over the next week I will make a note of how often the volume size changes such that if it was greater (smaller) than prior day, it is adjusted to now be smaller (greater). I'll also make a note of the average and max percentage adjustments.

I'll post the results back here in a week.

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Fri Aug 15, 2014 11:40 am

Suppose the long term average rate of retroactive-volume-changes in CSI Data is approximately 12 changes per year; one change per month. Would that be a tolerable number, for you and your trade execution team?

It's unlikely (~25% probability) you'll see any retroactive-volume-changes in 1 week of observation, if the long term average rate is 12 per year.

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Sat Aug 16, 2014 4:28 am

You may find this recent reply from Richard Dale of Norgate Investor services helpful:



Hi Anthony,
You raise an interesting question… I’ve done some research and presented how our data works below.
PRICE DATA UPDATE SCHEDULE
Our consolidated tape price data arrives in two batches each day.
1. A closing batch (typically 30-45 minutes after the market closes)
2. A corrections batch (a few hours later)
CLOSING PRICE CORRECTIONS
Here’s some statistics on closing prices changing from the closing batch to the corrections batch in the last 6 months:
NYSE: 5
NASDAQ: 7
NYSE Arca: 19
NYSE Mkt: 2
OTC: 508
In terms of closing price changes that are applied the following day (or beyond) the statistics in the last 6 months are as follows:
NYSE: 0
NASDAQ: 0
NYSE Arca: 1
NYSE Mkt: 1
OTC: 40
OHLC PRICE CORRECTIONS
In terms of a change to any one of OHLC from the closing batch to the corrections batch in the last 6 months:
NYSE: 76
NASDAQ: 45
NYSE Arca: 29
NYSE Mkt: 3
OTC: 610
In terms of OHLC changes that are applied the following day (or beyond) the statistics in the last 6 months are as follows:
NYSE: 0
NASDAQ: 1
NYSE Arca: 3
NYSE Mkt: 0
OTC: 43
VOLUME CORRECTIONS
Volume corrections are common for the corrections batch because the Consolidate Tape Association states that any after market trades should be included in the day’s volume. Therefore there are thousands of volume corrections from the closing batch to the corrections batch.
In terms of volume that is corrected on the following day (or beyond) the 6 month statistics are:
NYSE: 1
NASDAQ: 0
NYSE Arca: 0
NYSE Mkt: 1
OTC: 751
CORPORATE ACTIONS PRICE ADJUSTMENTS
Adjustments for corporate actions are made at the same time as the closing run comes in. However, there are certain chained corporate actions such as spinoffs that may not be calculated immediately, especially if there are overseas entities involved. These are typically adjusted during the early run maintenance at about 05:00-06:00 the next day.
CORPORATE ACTIONS TIMELINESS CONSIDERATIONS
Timeliness of normal corporate actions should definitely be considered as orders that are generated need to take into account the impact of the action on the order. Some of our competitors take until after the close of the NEXT DAY to adjust for the corporate actions. We adjust as soon we have enough information and the security becomes entitled to the corporate action.
DIVIDENDS AND DISTRIBUTIONS
Our data is unique in that the dividends/distributions are attributed to the data on the day BEFORE the ex-date. i.e. on the date you are entitled to the dividend if you held at the close. Most data vendors place the dividend on the ex-date. The consequence of this is you won’t know the dividend has happened until you are no longer entitled to it and any orders generated from other vendors’ data won’t have incorporated the data. This could have serious consequences for your trade.
HYBRID DATABASE
What you’re actually asking for is a hybrid point-in-time database that takes the first print of the price data for a particular date and has the ability to ignore any subsequent corrections. I understand where you’re coming from – however, to add such capabilities to the system would dramatically affect the existing design and performance.
Maybe a alternate way of looking at this sort of thing is to allocate trades that were undertaken as a result of altered data to an a “premature signalâ€

Mudnas
Senior Member
Senior Member
Posts: 25
Joined: Thu Dec 05, 2013 4:21 pm

Post by Mudnas » Tue Aug 19, 2014 11:05 am

Thanks Anthony
My side, I collected some CSI data, and also had a chat with the CSI team.

Here’s what I observed....
Volume data for day T is typically only confirmed by exchanges on T+2.
Some exchanges provide estimates on T+1, but these are only estimates and can change for the final value confirmed on T+2.
Exchanges which provide good estimates on T+1 are CME (incl Nymex, Comex, CBT) and ICE (incl NYCE and CSCE), and Montreal.
Some exchanges tend not to provide estimates on T+1, and hence CSI simply provides us with the same volume number as for T-1. They then “correctâ€

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Tue Aug 19, 2014 11:31 am

My only use for volume figures is (a) to avoid trading low volume stocks in back tests and (b) in real trading avoid trading or holding more than x% of the turnover.

But chopping out parameters sounds like a good idea to me.

fab1usa1
Roundtable Knight
Roundtable Knight
Posts: 383
Joined: Sat May 21, 2011 8:30 pm
Location: New York

Post by fab1usa1 » Sun Aug 24, 2014 8:45 am

I would like to expand upon what Mr. Garner said.

I run a mean-reversion system on stocks that looks for opportunities in a universe of about 7,200 stocks that trade at $1 or more and that meet a minimum liquidity threshold (i.e. moving average of price * volume).

The system enters positions with a limit order. I let the price come to me. No slippage there, fortunately. The problem comes at the exit in the form of a Market on Open order. Most times I get the opening price but other times I get a horrendous fill that turns out to be the low of the day rather than the official open.

I tried various techniques with limited success, most notably a limit order at 1 cent below the prior day's close with a Market on Close if the limit was never hit. That worked well for a while but needed additional logic that seemed reasonable at the time I wrote it. Recently, I got a fill that made me question this approach, and now I am searching for something new.

My broker is IB, so I began looking at their various order types. I focused upon their "Algo Accumulation/Distribution" as a possible solution. Right about that time TradingBlox announced its new release that supports that Algo. How fortunate for me but it requires a few parameters to make it go. I decided to develop a rational solution via spreadsheet before trying to automate it.

So let me state the objective. I am trying to avoid moving the market. I do like the simplicity of a Market order but if the market cannot quickly absorb it then I will submit an Algo Distribution order.

Allow me to offer an example of when I would feel comfortable submitting a Market order: Apple stock symbol AAPL. The three-month average volume was 53,119,500 shares per day (according to Yahoo Finance for the market ending 8/22/2014). 25% of 53,119,500 is 13,279,875. 13,279,875 divided by 23,400 (the number of seconds in the regular session) is 567 shares per second. My system wants to sell 217 shares. 217 divided by 567 is 0.4 seconds. So the market can absorb my 217 shares in 0.4 seconds. Under these circumstances I am comfortable submitting a Market on Open order.

And now an example of when to avoid a Market order: BBVA Banco Franc stock symbol BFR. The three-month average volume was 464,249 per day. 25% of 464,249 is 116,062. 116,062 divided by 23,400 is 5 shares per second. My system wants to sell 2,312 shares. 2,312 divided by 5 is 466 seconds, that is 7.8 minutes. I would be out of my mind trying to unload those shares at Market.

Once I decide that a Market order is insane, I must calculate the parameters needed for the Algo. The idea is to subdivide the total number of shares into bite-size slices that the market can more readily absorb. For example, I can choose to sell 100 shares every 20.2 seconds, or 50 shares every 10.1 seconds, or 200 shares every 40.3 seconds, etc. In each of these cases the total number of shares will be slowly introduced to the market after 7.8 minutes.

So far I am feeling comfortable with this solution but it would be nice if I could run a parallel experiment so that I could compare my average fill with the Algo versus a Market on Open order.

Any comments or suggestions are greatly appreciated.

wguan
Roundtable Fellow
Roundtable Fellow
Posts: 64
Joined: Sun Feb 27, 2011 1:52 pm

Re: Volume data in CSI - how stable are the estimates?

Post by wguan » Sat Apr 11, 2015 11:47 am

Hello fab1usa1,

I am trying to scan for CSI stock data for stocks that meet some liquidity and price criteria. How do you approach this? Do you use the market scanner? Would be really helpful if you could list some of the steps you took to come up with stocks using CSI UA.

Right now I am essentially creating a portfolio and selecting all stocks that are in the NYSE and I am writing custom scripts to select ones that meet my criteria. It would be great if this step is done in CSI.

Thanks,

fab1usa1
Roundtable Knight
Roundtable Knight
Posts: 383
Joined: Sat May 21, 2011 8:30 pm
Location: New York

Re: Volume data in CSI - how stable are the estimates?

Post by fab1usa1 » Sun Apr 12, 2015 7:55 am

Good Morning wguan,

Every weekend my routine includes the following:
1. Backup hard drive.
2. Windows Update.
3. Defragment hard drive.
4. Run CSI Market Scanner.

That last step prepares my swing trading system for next week's trading. It's purpose is to:
1. Update my CSI Stock Portfolio with symbols that meet a specific criteria.
2. Create a Trading Blox .SET file with those symbols.

These are the steps I execute in the Market Scanner:

Click Trading Tools -> Market Scanner.
Scan Settings:
Include 200 Days
Current Date: Today's Date
Default Number of Digits: 4
Max Bars Back: 200
Series to Show: All in Exchange
AMEX
NASDAQ(CM)
NASDAQ(GM)
NASDAQ(GS)
NYSE
Check: Skip issues which do not yet have this many bars of data
Check: Represent futures as a perpetual rather than back-adjust
Click: Save and Rebuild

Note that my Trading Blox indicators require a minimum of 200 bars so that is why use see the number 200 in my scanner settings. I figure why burden Trading Blox with symbols that do not meet this minimum requirement.

Next, I want to eliminate all stocks that closed below $1:

Click File -> Open Layout
Select "c:\ua\archives\WatchLists\RA1 Stock Scanner.scr"
This shows columns for:
Csinum
Symbol
Name
Sector
Open
High
Low
Close
Vol
Exchange
We want to eliminate all stocks having a closing price of under $1.00
Right-click on the Close column and select Filter by Column.
In the dialog:
Show only when > 1.00
Click OK
Click Export Table -> Save Table to ASCII
Save it to your desktop with a name like foo.csv

Note that I attached the SCR file to this post. (I hope that it is compatible with your version of CSI UA.)

Next, I want to eliminate all stocks that contain a "+" or "." in the symbol. My experience is that Interactive Brokers doesn't much care for these kinds of symbols:

Open foo.csv with OpenOfficeCalc
Do this by inserting a a column to the right of "Symbol" and name it "Symbol Filtered".
Type this formula: =IF(ISNUMBER(FIND("+";B2));"";IF(ISNUMBER(FIND(".";B2));"";IF(ISNUMBER(FIND("-";B2));"";B2))
Copy that formula for all rows and then sort the worksheet by "Symbol Filtered".
Select all the symbols that pass and copy to the clipboard.
Open Notepad and paste.
Save the file to "c:\TradingBlox\Stock Sets" with a name like "UA 2015-04-10.set"

Next, we need to import all of those symbols into CSI Unfair Advantage:

Click Portfolio -> Import Portfolio -> Ascii Import.
Select the file that we just saved with the Browse button (it may take 30 secs after you open the file)
Click the Import button.
In the Set Portfolio Details dialog, choose Portfolio: Stocks. Keep all other choices, and then click OK.
Depending on your version of CSI UA this could take a few hours.
The dialog that it shows will likely say (Not Responding) but give no indication of the progress.
However if you launch Task Manager you will see substantial CPU utilization.
Have patience.
When complete, you are done!

This weekend's scan included 6,957 stocks which is comfortably below the 7,500 limit that CSI places on me.

As far as a liquidity filter, I do that in my Trading Blox Portfolio Manager. I calculate a 20-day SMA of Price * Volume, and if the value is greater than 6,000,000 I allow trades. I should try to see if I can accomplish that in Market Scanner.

Happy trading!
Attachments
RA1 Stock Scan.txt
Before using this file, rename it from "RA1 Stock Scan.txt" to "RA1 Stock Scan.scr"
(1.54 KiB) Downloaded 97 times

wguan
Roundtable Fellow
Roundtable Fellow
Posts: 64
Joined: Sun Feb 27, 2011 1:52 pm

Re: Volume data in CSI - how stable are the estimates?

Post by wguan » Sun Apr 12, 2015 11:14 am

Thank you so much. You have no idea how much this has streamlined the process.

Another question I have, which may not seem to be that quite on topic for this thread is how you are backtesting your strategy? Did you build your own survivorship bias free database for stocks? Right now I am testing a market neutral strategy on the list of stocks that I come up with in the scanner. Also would love to hear how you approach swing trading in stocks too. Thanks again.

fab1usa1
Roundtable Knight
Roundtable Knight
Posts: 383
Joined: Sat May 21, 2011 8:30 pm
Location: New York

Re: Volume data in CSI - how stable are the estimates?

Post by fab1usa1 » Sun Apr 12, 2015 12:11 pm

Glad to be of assistance!

I am a terrible designer of systems so I constantly evaluate other peoples' systems. The vendor of this particular system codes in EasyLanguage but honestly his documentation is so good that you can code in Blox Basic directly from it. Since I do not (yet) have delisted data I rely on his backtest results. Even without delisted data I can come very close to his equity curve when I limit the test period to this past year. I have been running his systems for about three years and am very happy. Send me a private message and I will give you his contact information.

wguan
Roundtable Fellow
Roundtable Fellow
Posts: 64
Joined: Sun Feb 27, 2011 1:52 pm

Re: Volume data in CSI - how stable are the estimates?

Post by wguan » Sun Apr 12, 2015 12:38 pm

I can't seem to send private messages....can you send it to this email: 833636321abc@gmail.com

thanks

Post Reply