davidib wrote: [snip]
But it seems that maybe such a system exists already? Take a look at this thread
viewtopic.php?t=1790
Well I don't want to reinvent the wheel and if someone has such software and it works I'd say go with it, Maybe Roger has more info on such a tool?
In the above link "Davidib" references there is discussion that points to our Windows Data Manager program in use daily. That software provides an interface for manually adding data to contracts and for making backwards adjustments. This image shows the main screen:
http://tinyurl.com/3dkhz4
In this data entry screen I don't add Volume or Open Interest, but that is going to change along with the ability to record Contract Information and Unadjusted Close values.
When working in this screen, the data input is manual, punch it in with the keyboard, that will change as well, but the feature that has been really helpful has been the ability to enter information as fractions or decimals depending upon how the data trades.
In another part of the software is a section for manually stitching contracts together:
http://tinyurl.com/2th6lw
In this Join-File screen File#1 is usually a Continuous Back-Adjusted file. To find a matching date in the second file, the user just clicks a row and the matching date appears so the user can decide if they want to move on that date. Here again I've got some changes in the works to make this part even easier and automated so that the stitching process isn't so time consuming when the timing of the stitch doesn't have any account impact.
Helping to make some of this manual stuff easier is a database that keeps track of the information for all the ASCII files it loads. Some of the details stored reflect the file name and path location along with the normal market data like this:
http://tinyurl.com/3xpkha
Along with maintaining any data is to know when there isn't any data. This is important in this software because it checks the date records and price relationships in each file to keep the data as clean as possible. To know what trades when is this screen:
http://tinyurl.com/2urgvy
Unfortunately this Days-Off process is localized to the US so international markets are being handled with more effort. Maybe some day I'll find the time to figure out a process for managing this better, but for now it is all lookup for markets not traded in the US.
As I moved towards automation all the issues Sluggo mentions had to be addressed and the process for automating must have a fail-safe date trigger for when things are just not normal. Here are a couple of screen images for setting automated roll timing triggers:
http://tinyurl.com/2hrvl9
http://tinyurl.com/2e6jp9
Right now these are two separate screens, but there is another version of this process in the works that will combine the process so the user will better understand that the process can be combined.
All of the above is moving towards this kind of interface where the data is presented in a more visual manner:
OLD:
http://tinyurl.com/243pub
Current:
http://tinyurl.com/ylqo44b
In this screen layout the user gets to see the raw data coming in from CSI and gets to look ahead in a visual manner to see what is happening with Prices, Volume and Open Interest. For example here are the Volume and Open Interest expanded detail screens:
http://tinyurl.com/yvfpzh
http://tinyurl.com/22tlvm
In this screen set we are only looking ahead 2-contracts, but the process will look ahead 6-contracts for markets like the Eurodollar Interest Rate.
There are still some problems in how this is working that need to be worked on, but the insight I get and the time I save to feed my need to add skill to the roll timing is making progress and it goes a long way towards satisfying my need to be aware of what is happening in the market everyday.
One of the areas not working well enough is one of the issues Sluggo mentions above:
http://tinyurl.com/3a2eqk
This screen controls which markets are excluded from consideration and it isn't part of the roll timing setup screen. This screen shows a disjoint that is making the process clumsy. When this screen was designed I didn't have all the mechanics of how to handle the date fail-safe process so I disabled the roll trigger sections on this screen, but left out the contract exclusion process because I didn't have the time back then to get it all stuffed into on better laid out dialog.
All of this effort competes with my trading and system development efforts. Over these last 8-months Trading Blox has been consuming any time not being taken by the markets so the work on this gets to run at near Glacial speeds, even with the recent speed increase the ice flows are demonstrating.
What isn't in any of the tools I use is the ability to do any of the other types of data stitching. Mostly because I find that Back-Adjusting works very well at paying the rent, but to some extent I've never tried to go down those other paths because of early 90's period disappointments with other approaches. For certain I've not tried everything, but then I've never seen anything meaningful to elevate my interest.
NOTE:
20100326 - Added updated image of Quote Board.