Futures Market Data Pitfalls - Newbies Beware

General discussions about futures.
Post Reply
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Sat May 06, 2006 10:53 am

I despair of CSI. I reported to them that the contract size is rupee 100 x the index as specifically confirmed to me by the exchange. The have updated the "Contract Size" column of the factsheet but left point value as 2. Their "Price Format" is 2.

So 2 / 0.01 = 200.

I am finding this a real worry. See separate entry re DJ Stoxx 50

Roscoe
Roundtable Knight
Roundtable Knight
Posts: 250
Joined: Sat Jan 24, 2004 2:06 am
Location: Houston TX

Post by Roscoe » Sun May 07, 2006 10:17 pm

Hi AG. This leads me back to my hobby horse of wanting a stand-alone application that can handle rollovers and formatting with data from varying sources – has anyone seen anything in their travels? (Sorry to be going off-topic Mr G.)

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Tue May 09, 2006 5:49 am

You are not going off topic my friend. The trouble I have had recently makes me very frustrated. I will start a new thread to share my frustration.
A

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Tue May 09, 2006 10:09 am

I looked at the CSI contract information a few years ago and found it so riddled with errors for the Big Point Value that I decided not to use their data as the basis for our starting numbers.

Instead, I went to the exchange web sites and looked at the contract specifications and then manually determined the Big Point Values from there. This took quite a few hours, but I was confident in my own figures when I was done. Over the years that followed I sometimes had others do this for me and there would often be errors. The reason is that it is not a simple calculation in some cases, especially with the interest rate instruments. It is easy to make an error.

I suggest that anyone who actually trades manually verify the Big Point Values for any markets that they trade or are contemplating trading. It doesn't take that much time and it is good to get to know what each tick up and down actually means.

Once you begin trading the errors are obvious as they start showing up in your account, as an order of magnitude error. Hopefully one won't let this sort of thing go unnoticed for more than the first day.

CSI seems to have reliable historical data but unreliable contract information. I suspect this is because so many people have bypassed their data that when errors occur the saavy traders are not affected therefore they don't report the errors. Price errors are reported and corrected quickly since that is information that will remain incorrect in even the most careful user's data until corrected by CSI themselves.

- Forum Mgmnt

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Tue May 09, 2006 11:06 am

You are right in everything you say. I have done precisely the same myself - looked up details on the relevant exchange website, not only for every contract I trade but for every contract I follow. And still, it is easy to make a mistake. I accept that.

Look, this is really a subject for a different heading and feel free to move this post if you will..

But I will say this to anyone who is new to the game:

You need to check every single setting not only in TB and the futuresinfo.txt file but also in the horrendously complex CSI UA software. And you need to make sure they match.

And you need to make double d*** sure that your systems assist broker also matches such settings – exactly. Otherwise there will be screw-ups and misunderstandings. And its going to cost you, not the broker.

It is lucky indeed that I have paid close attention to my account over the past couple of months and religiously checked orders and fills etc every day. I hope that I will have a smoother ride going forward.

For instance, oh lucky Newbie, say you decide to trade Crude, HO and HU on the quarterly contracts, HMUZ. In CSI you must ensure you choose HMUZ as the months to trade, ditto in the futuresinfo.txt file. You or your broker may mistakenly choose all 12 monthly contracts in CSI UA and assume that by putting HMUZ into the futuresinfo.txt file, the desired result would be achieved. Or there again one party or the other may just have forgotten you were supposed to be trading HMUZ. Easily done and must be checked by the client – hopefully before any signals are executed. Oh deary me.

You or your broker may not appreciate that a totally different data series is obtained depending on which months you choose to include in the concatenated contract. Or there again you or your broker might be aware but might simply have made a mistake. Oh dear again.

Mistakes are easily made but I suspect that there are few brokers who will have spent the hundreds and hundreds of hours that we “anoraksâ€

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Tue May 09, 2006 11:21 am

I should add that CSI users need to be especially careful. The meaning of the "portfile.adm" which CSI uses internally to define the settings for a portfolio, including months and rolling algorithm file can change as CSI releases new versions of Unfair Advantage.

So sometimes even having the same "portfile.adm" as your broker won't ensure that you have the same data settings. Several times in the past, CSI has changed the meaning of their default values so that there would be different settings with the same "portfile.adm" file with different versions of CSI's Unfair Advantage software.

So make sure the versions of UA match along with the "portfile.adm" file.

- c.f.
Last edited by Forum Mgmnt on Wed May 10, 2006 8:28 am, edited 1 time in total.

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Tue May 09, 2006 11:52 am

An extremely good point. I have just had a long and very helpful telephone conversation with CSI on this very topic.

It may be best if the broker sends the client his portfile.adm file which the client puts somewhere OTHER than in his UA files - IE he avoids overwriting his own settings. I am told that the client can then import the broker' portfolios into his own version of UA (provided they are using compatible versions) and can then manually inspect, one by one, both the portfolio settings and the settings for each individual future.

From what you are saying, it will be important to repeat the exercise if and when the broker upgrades CSI UA.

I did exactly this with my broker with the futuresinfo.txt file and communicated my findings. I spent many hours looking through his futuresinfo.txt file. Unfortunately, certain changes were not made and the results have been..................er...............well, what can I say?

We are all human and all prone to error but this is a dangerous game and great attention to detail is essential. And even then mistakes will occur from time to time. Sometimes your fault, sometimes not. Yet another reason why backtesting and actual trading will never perfectly match.

I have no doubt this will get sorted out but I felt a discussion of these sort of problems was well worth airing. And thank you for your input.

Old European
Roundtable Knight
Roundtable Knight
Posts: 144
Joined: Mon Apr 19, 2004 3:04 pm
Location: Old Europe

Post by Old European » Tue May 09, 2006 12:35 pm

It is my experience that doing one's homework properly is more than worthwhile.

Before I started trading my system two years ago I spent a great deal of time checking all CSI data settings and files and choosing very realistic assumptions for slippage etc.

I don't use a robobroker, but place all the orders myself (in order to keep in control as much as possible) and try to do it exactly as I'm supposed to do it.

It has been and is a big effort, but it has been very rewarding, since my real time real life trading results have been unbelievably close (divergence of less than 1.25%) to my theoretical system results over the last two years!

Cheers,

Old European

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Tue May 09, 2006 1:55 pm

Congratulations, that is some achievement. I too would benefit from placing my own orders. It would greatly speed my learning curve.

The problem I have is backup. Or lack thereof. You may well be a lone individual as I am (?)

But I have it in mind from some of your previous posts that you are building a track record so as to launch your own fund at some stage. I suspect therefore that you probably have a partner or colleague who can and does cover for you.

It is not that I am immensely busy on other things at the moment. But I maintain houses in London and Switzerland and constantly dart between the two. I have no assistant and no business partner. I would have to maintain two internet connections at each house and at least 3 laptops - 1 backup at each place and one to travel with.

So, what happens if I am in hospital, stuck in central Europe on some snowy French motorway etc etc.

I have noted with interest the measures taken by Sluggo, Jake and others on at least the equipment backup side. I have no doubt that you are every bit as professional.

I have no problem handling my share trading on my own - I keep in touch wherever I am and check allocations of stock by mobile or whatever. Equally I check prices and give orders when on the fly. In the heyday of the IPO and junk bond market I would travel everywhere with a portable Bloomberg on my tiny laptop.


You have given me good food for thought however. There is perhaps no valid reason for me not to take over my own trading............and my broker would no doubt be relieved if I was handing him the orders rather than him waiting in dread for my next complaint.

Do you use software such as Mtrade or J Trader or give your orders by fax/e-mail/telephone?

Roscoe
Roundtable Knight
Roundtable Knight
Posts: 250
Joined: Sat Jan 24, 2004 2:06 am
Location: Houston TX

Post by Roscoe » Tue May 09, 2006 8:55 pm

AG I could not agree more with your sentiments – it is imperative that one understands every detail of each contract that one intends to use for testing and for trading. Anything less will simply not do.

Just to add my $0.02 to c.f.’s post: I too have expended some not inconsiderable effort in researching each exchange in order to get the correct details for all the contracts that I use for testing and trading, the results of which I have stored in an Access database that I use to ‘feed’ my trading software. I use Pinnacle Data (having been completely put off CSI by the horrendously messy interface that they persist in hobbling their users with) and I have been well pleased with them.

Old European
Roundtable Knight
Roundtable Knight
Posts: 144
Joined: Mon Apr 19, 2004 3:04 pm
Location: Old Europe

Post by Old European » Wed May 10, 2006 12:48 am

AFJ,

Yes, I paid a lot of attention to backup procedures. I have hardware and internet connection backups (internet went down last night because of heavy weather and thunderstorms!). I can count on a friend of mine for backup. I'm spending some time at the moment on writing down all contingency procedures up to the very last detail.

While you are dreaming of placing your orders yourself, I am dreaming of using a robobroker in the future. The daily work is very boring (especially since I'm forcing myself not to follow the financial and commodity markets on a daily basis, because I have chosen for a relatively long investment horizon).

Although I'm very happy with my current broker (I suspect that we are working with the same broker), I'm really afraid of exactly the issues you are mentioning (like sync of CSI settings) and don't feel ready yet to ask him to auto-execute my system. I continue to place all the orders myself for the time being via the MTrade platform.

Cheers,

Old European

TrendMonkey
Roundtable Knight
Roundtable Knight
Posts: 154
Joined: Fri Apr 22, 2005 9:14 pm
Location: Vancouver, Canada

Post by TrendMonkey » Wed May 10, 2006 2:33 pm

Speaking of CSI, I see they are now in the trading systems vendor business directly. In the newsletter I just got they are flogging their 'Sierra Hotel' system for $995. It looks like it is completely undisclosed, but I just quickly skimmed it. The only thing that piqued my interest a little was the money-back guarantee, but a 30 day guarantee on a system that generates 8-16 trades per year does not seem all that useful.

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Wed May 10, 2006 2:44 pm

Believe me the money back guarantee on something like this will be so vague as to be unenforceable.

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Thu May 11, 2006 10:03 am

Another little point which occurs on crunching dull detail by dull detail through my settings and my broker's.

Your backtesting software updates your stops every single day. That is what your backtesting results are based on.

You discover:

A) your broker does not update stops EVERY day and
B) in some instances, if your stop is far enough away from the market he will be unable to enter a GTC (or perhaps even a GDO) stop because his software says "hey, you are trying to enter a stop that is beyond the daily allowable limit move".

Shucks..........I must check whether the cash market for the relevant stock index has an identical limit move. Because I seem to recall that some of the daily moves in the cash markets back in the ghastly days of 200/2001 were greater than the current daily limits for the relevant futures contract.

So does your broker check every day to see whether a stop can now be entered since you are in range of the daily limit move? Seems to me he needs to.

BARLI
Roundtable Knight
Roundtable Knight
Posts: 650
Joined: Sat Jan 17, 2004 6:01 pm
Location: USA

Post by BARLI » Mon May 22, 2006 5:44 pm

AFG, you as a major trend follower were hit by today's SENSEX crash? anyone got a chart of what happened?from what I heard it's been 10.16% down move

AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2040
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner » Mon May 22, 2006 6:16 pm

I don't trade it.

But my comments above came home to roost when I discovered a stop had not been entered when it could/should have been 3 trading days ago.. It has now been entered but for the wrong number of contracts.

It does make me wonder...............

BARLI
Roundtable Knight
Roundtable Knight
Posts: 650
Joined: Sat Jan 17, 2004 6:01 pm
Location: USA

Post by BARLI » Mon May 22, 2006 6:44 pm

you wanted to buy more or less? I think they bought you more than you wanted :twisted:

rajivm
Roundtable Knight
Roundtable Knight
Posts: 129
Joined: Fri Jul 25, 2003 12:50 am
Location: New Delhi, India

The Sensex Charts - Nothing EarthShattering

Post by rajivm » Tue May 23, 2006 3:11 am

BARLI wrote:AFG, you as a major trend follower were hit by today's SENSEX crash? anyone got a chart of what happened?from what I heard it's been 10.16% down move
Hi Barli,
It might just be the end of a 3 year long bull market in India.
Attachments
sensex.doc
(77 KiB) Downloaded 425 times

BARLI
Roundtable Knight
Roundtable Knight
Posts: 650
Joined: Sat Jan 17, 2004 6:01 pm
Location: USA

Post by BARLI » Tue May 23, 2006 9:12 pm

thanks rajivm!

Post Reply