Page 1 of 1

Volume data in CSI - how stable are the estimates?

Posted: Thu Aug 14, 2014 6:08 pm
by Mudnas
Imagine a trading rule that incorporates the logic to only trade if the most recent volume is less that the volume of the previous bar.
Obviously we don't want to find that CSI makes significant adjustments the following day to volume data such that an entry signal would no longer be met.

So, is the volume date provided by CSI for the most recent day just an estimate? And more importantly, how accurate or stable is that estimate?

I'm going to research this myself! I will chose 50 liquid futures markets, and for each day over the next week I will make a note of how often the volume size changes such that if it was greater (smaller) than prior day, it is adjusted to now be smaller (greater). I'll also make a note of the average and max percentage adjustments.

I'll post the results back here in a week.

Posted: Fri Aug 15, 2014 11:40 am
by sluggo
Suppose the long term average rate of retroactive-volume-changes in CSI Data is approximately 12 changes per year; one change per month. Would that be a tolerable number, for you and your trade execution team?

It's unlikely (~25% probability) you'll see any retroactive-volume-changes in 1 week of observation, if the long term average rate is 12 per year.

Posted: Sat Aug 16, 2014 4:28 am
by AFJ Garner
You may find this recent reply from Richard Dale of Norgate Investor services helpful:

Hi Anthony,
You raise an interesting question… I’ve done some research and presented how our data works below.
Our consolidated tape price data arrives in two batches each day.
1. A closing batch (typically 30-45 minutes after the market closes)
2. A corrections batch (a few hours later)
Here’s some statistics on closing prices changing from the closing batch to the corrections batch in the last 6 months:
NYSE Arca: 19
NYSE Mkt: 2
OTC: 508
In terms of closing price changes that are applied the following day (or beyond) the statistics in the last 6 months are as follows:
NYSE Arca: 1
NYSE Mkt: 1
OTC: 40
In terms of a change to any one of OHLC from the closing batch to the corrections batch in the last 6 months:
NYSE: 76
NYSE Arca: 29
NYSE Mkt: 3
OTC: 610
In terms of OHLC changes that are applied the following day (or beyond) the statistics in the last 6 months are as follows:
NYSE Arca: 3
NYSE Mkt: 0
OTC: 43
Volume corrections are common for the corrections batch because the Consolidate Tape Association states that any after market trades should be included in the day’s volume. Therefore there are thousands of volume corrections from the closing batch to the corrections batch.
In terms of volume that is corrected on the following day (or beyond) the 6 month statistics are:
NYSE Arca: 0
NYSE Mkt: 1
OTC: 751
Adjustments for corporate actions are made at the same time as the closing run comes in. However, there are certain chained corporate actions such as spinoffs that may not be calculated immediately, especially if there are overseas entities involved. These are typically adjusted during the early run maintenance at about 05:00-06:00 the next day.
Timeliness of normal corporate actions should definitely be considered as orders that are generated need to take into account the impact of the action on the order. Some of our competitors take until after the close of the NEXT DAY to adjust for the corporate actions. We adjust as soon we have enough information and the security becomes entitled to the corporate action.
Our data is unique in that the dividends/distributions are attributed to the data on the day BEFORE the ex-date. i.e. on the date you are entitled to the dividend if you held at the close. Most data vendors place the dividend on the ex-date. The consequence of this is you won’t know the dividend has happened until you are no longer entitled to it and any orders generated from other vendors’ data won’t have incorporated the data. This could have serious consequences for your trade.
What you’re actually asking for is a hybrid point-in-time database that takes the first print of the price data for a particular date and has the ability to ignore any subsequent corrections. I understand where you’re coming from – however, to add such capabilities to the system would dramatically affect the existing design and performance.
Maybe a alternate way of looking at this sort of thing is to allocate trades that were undertaken as a result of altered data to an a “premature signalâ€

Posted: Tue Aug 19, 2014 11:05 am
by Mudnas
Thanks Anthony
My side, I collected some CSI data, and also had a chat with the CSI team.

Here’s what I observed....
Volume data for day T is typically only confirmed by exchanges on T+2.
Some exchanges provide estimates on T+1, but these are only estimates and can change for the final value confirmed on T+2.
Exchanges which provide good estimates on T+1 are CME (incl Nymex, Comex, CBT) and ICE (incl NYCE and CSCE), and Montreal.
Some exchanges tend not to provide estimates on T+1, and hence CSI simply provides us with the same volume number as for T-1. They then “correctâ€

Posted: Tue Aug 19, 2014 11:31 am
by AFJ Garner
My only use for volume figures is (a) to avoid trading low volume stocks in back tests and (b) in real trading avoid trading or holding more than x% of the turnover.

But chopping out parameters sounds like a good idea to me.

Posted: Sun Aug 24, 2014 8:45 am
by fab1usa1
I would like to expand upon what Mr. Garner said.

I run a mean-reversion system on stocks that looks for opportunities in a universe of about 7,200 stocks that trade at $1 or more and that meet a minimum liquidity threshold (i.e. moving average of price * volume).

The system enters positions with a limit order. I let the price come to me. No slippage there, fortunately. The problem comes at the exit in the form of a Market on Open order. Most times I get the opening price but other times I get a horrendous fill that turns out to be the low of the day rather than the official open.

I tried various techniques with limited success, most notably a limit order at 1 cent below the prior day's close with a Market on Close if the limit was never hit. That worked well for a while but needed additional logic that seemed reasonable at the time I wrote it. Recently, I got a fill that made me question this approach, and now I am searching for something new.

My broker is IB, so I began looking at their various order types. I focused upon their "Algo Accumulation/Distribution" as a possible solution. Right about that time TradingBlox announced its new release that supports that Algo. How fortunate for me but it requires a few parameters to make it go. I decided to develop a rational solution via spreadsheet before trying to automate it.

So let me state the objective. I am trying to avoid moving the market. I do like the simplicity of a Market order but if the market cannot quickly absorb it then I will submit an Algo Distribution order.

Allow me to offer an example of when I would feel comfortable submitting a Market order: Apple stock symbol AAPL. The three-month average volume was 53,119,500 shares per day (according to Yahoo Finance for the market ending 8/22/2014). 25% of 53,119,500 is 13,279,875. 13,279,875 divided by 23,400 (the number of seconds in the regular session) is 567 shares per second. My system wants to sell 217 shares. 217 divided by 567 is 0.4 seconds. So the market can absorb my 217 shares in 0.4 seconds. Under these circumstances I am comfortable submitting a Market on Open order.

And now an example of when to avoid a Market order: BBVA Banco Franc stock symbol BFR. The three-month average volume was 464,249 per day. 25% of 464,249 is 116,062. 116,062 divided by 23,400 is 5 shares per second. My system wants to sell 2,312 shares. 2,312 divided by 5 is 466 seconds, that is 7.8 minutes. I would be out of my mind trying to unload those shares at Market.

Once I decide that a Market order is insane, I must calculate the parameters needed for the Algo. The idea is to subdivide the total number of shares into bite-size slices that the market can more readily absorb. For example, I can choose to sell 100 shares every 20.2 seconds, or 50 shares every 10.1 seconds, or 200 shares every 40.3 seconds, etc. In each of these cases the total number of shares will be slowly introduced to the market after 7.8 minutes.

So far I am feeling comfortable with this solution but it would be nice if I could run a parallel experiment so that I could compare my average fill with the Algo versus a Market on Open order.

Any comments or suggestions are greatly appreciated.

Re: Volume data in CSI - how stable are the estimates?

Posted: Sat Apr 11, 2015 11:47 am
by wguan
Hello fab1usa1,

I am trying to scan for CSI stock data for stocks that meet some liquidity and price criteria. How do you approach this? Do you use the market scanner? Would be really helpful if you could list some of the steps you took to come up with stocks using CSI UA.

Right now I am essentially creating a portfolio and selecting all stocks that are in the NYSE and I am writing custom scripts to select ones that meet my criteria. It would be great if this step is done in CSI.


Re: Volume data in CSI - how stable are the estimates?

Posted: Sun Apr 12, 2015 7:55 am
by fab1usa1
Good Morning wguan,

Every weekend my routine includes the following:
1. Backup hard drive.
2. Windows Update.
3. Defragment hard drive.
4. Run CSI Market Scanner.

That last step prepares my swing trading system for next week's trading. It's purpose is to:
1. Update my CSI Stock Portfolio with symbols that meet a specific criteria.
2. Create a Trading Blox .SET file with those symbols.

These are the steps I execute in the Market Scanner:

Click Trading Tools -> Market Scanner.
Scan Settings:
Include 200 Days
Current Date: Today's Date
Default Number of Digits: 4
Max Bars Back: 200
Series to Show: All in Exchange
Check: Skip issues which do not yet have this many bars of data
Check: Represent futures as a perpetual rather than back-adjust
Click: Save and Rebuild

Note that my Trading Blox indicators require a minimum of 200 bars so that is why use see the number 200 in my scanner settings. I figure why burden Trading Blox with symbols that do not meet this minimum requirement.

Next, I want to eliminate all stocks that closed below $1:

Click File -> Open Layout
Select "c:\ua\archives\WatchLists\RA1 Stock Scanner.scr"
This shows columns for:
We want to eliminate all stocks having a closing price of under $1.00
Right-click on the Close column and select Filter by Column.
In the dialog:
Show only when > 1.00
Click OK
Click Export Table -> Save Table to ASCII
Save it to your desktop with a name like foo.csv

Note that I attached the SCR file to this post. (I hope that it is compatible with your version of CSI UA.)

Next, I want to eliminate all stocks that contain a "+" or "." in the symbol. My experience is that Interactive Brokers doesn't much care for these kinds of symbols:

Open foo.csv with OpenOfficeCalc
Do this by inserting a a column to the right of "Symbol" and name it "Symbol Filtered".
Type this formula: =IF(ISNUMBER(FIND("+";B2));"";IF(ISNUMBER(FIND(".";B2));"";IF(ISNUMBER(FIND("-";B2));"";B2))
Copy that formula for all rows and then sort the worksheet by "Symbol Filtered".
Select all the symbols that pass and copy to the clipboard.
Open Notepad and paste.
Save the file to "c:\TradingBlox\Stock Sets" with a name like "UA 2015-04-10.set"

Next, we need to import all of those symbols into CSI Unfair Advantage:

Click Portfolio -> Import Portfolio -> Ascii Import.
Select the file that we just saved with the Browse button (it may take 30 secs after you open the file)
Click the Import button.
In the Set Portfolio Details dialog, choose Portfolio: Stocks. Keep all other choices, and then click OK.
Depending on your version of CSI UA this could take a few hours.
The dialog that it shows will likely say (Not Responding) but give no indication of the progress.
However if you launch Task Manager you will see substantial CPU utilization.
Have patience.
When complete, you are done!

This weekend's scan included 6,957 stocks which is comfortably below the 7,500 limit that CSI places on me.

As far as a liquidity filter, I do that in my Trading Blox Portfolio Manager. I calculate a 20-day SMA of Price * Volume, and if the value is greater than 6,000,000 I allow trades. I should try to see if I can accomplish that in Market Scanner.

Happy trading!

Re: Volume data in CSI - how stable are the estimates?

Posted: Sun Apr 12, 2015 11:14 am
by wguan
Thank you so much. You have no idea how much this has streamlined the process.

Another question I have, which may not seem to be that quite on topic for this thread is how you are backtesting your strategy? Did you build your own survivorship bias free database for stocks? Right now I am testing a market neutral strategy on the list of stocks that I come up with in the scanner. Also would love to hear how you approach swing trading in stocks too. Thanks again.

Re: Volume data in CSI - how stable are the estimates?

Posted: Sun Apr 12, 2015 12:11 pm
by fab1usa1
Glad to be of assistance!

I am a terrible designer of systems so I constantly evaluate other peoples' systems. The vendor of this particular system codes in EasyLanguage but honestly his documentation is so good that you can code in Blox Basic directly from it. Since I do not (yet) have delisted data I rely on his backtest results. Even without delisted data I can come very close to his equity curve when I limit the test period to this past year. I have been running his systems for about three years and am very happy. Send me a private message and I will give you his contact information.

Re: Volume data in CSI - how stable are the estimates?

Posted: Sun Apr 12, 2015 12:38 pm
by wguan
I can't seem to send private messages....can you send it to this email: