A few weeks back I decided that I wanted to do something about the lack of a reasonable format for Trading Data. Problems like not having a uniform format and where the value of the tick, big point value, underlying market currency, etc. are different for each market; or survivorship bias in indices like the S&P 500 which changes over time and therefore needs a set of historical adds and deletes; symbols that change meaning over time as companies get acquired and stock symbols are reused etc.In another topic, rwk wrote:TT data format was a huge breakthrough in the early days, but it has some serious limitations now. I have not found an open (or semi-open) format that is any better, including Computrac/Metastock IMHO. The proprietary formats, such as TC2000 and CSI, are as only good as the vendor and the interface as c.f. pointed out. Chartbook is the main reason for staying with TT format, and I haven't found anything that comes within light-years of Chartbook for power and ease-of-use. It is a shame that it isn't more readily available.
We are going to form an industry consortium for an open standard for data. Then we will modify VeriTrader to support that new format. Further we will promote the new format with the small and large data providers all around the world.
We will also develop and release an open source multi-platform data library and sample application and give it to the consortium which will then manage continued development.
I've done this very successfully before. Check out www.hr-xml.org which is the consortium I started for defining standardized ways to exchange human-resources data like resumes, job requisitions, compensation and benefits changes, payroll updates, etc. You'll note that we were able to recruit all the major players.
Problems associated with the hodgepodge of data formats is our biggest single support issue. It is worth quite a bit for us to solve this problem which is why we are going to put significant resources into eliminating the older data formats.
The goal of the standards is that the data and associated dictionary information including symbol, portfolio construction, index membership (and changes over time), fundamentals, etc. is standardized so there is no need to do anything to install the data and use it in a variety of situations.
We aren't going to do much until 2.0 ships but I thought I'd start the discussion here to solicit our member's advice about the format since we are going to create the initial format standard ourselves.
If we do it right, this will solve a big problem for many people.
- Forum Mgmnt