Master ETF list to emulate futures universe?

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
squaredQ
Full Member
Full Member
Posts: 23
Joined: Wed Feb 01, 2012 5:27 pm

Master ETF list to emulate futures universe?

Post by squaredQ »

Is there a good master list of ETFs that would emulate the major say top 100 futures by liquidity and diversity? Similar to the futures data that is provided by TradingBlox for back-testing portfolios.
jas-105
Roundtable Knight
Roundtable Knight
Posts: 130
Joined: Sat Aug 02, 2008 2:32 am
Location: London, England.

Post by jas-105 »

The ETF Database is a good place to start your search:

www.etfdb.com

Yahoo Finance is one place where you can download free backadjusted data but there may be others.

Hope that helps.
mojojojo
Roundtable Knight
Roundtable Knight
Posts: 131
Joined: Mon Oct 31, 2005 2:07 pm

Post by mojojojo »

Can go here and sort by "Asset Class"

http://www.etfg.com/research/scanner

Free version doesn't allow any filtering

You will have to go to someplace like yahoo for historical data though
squaredQ
Full Member
Full Member
Posts: 23
Joined: Wed Feb 01, 2012 5:27 pm

Post by squaredQ »

thanks for replies. I saw the etfdb and it has nice lists, but no historical data (that I could see). Maybe I'd have to take a list and then get yahoo data from the list. I was hoping there would be a zip file with the common portfolio ingredients somewhere. There's so many books on the subject, but not a lot of data readily available (Garner talks about this conundrum a bit).
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

The best source of information I found was Morgan Stanley's now outdated world guidebook attached. No price data but you have got all the tickers etc.
Attachments
ETF Worldwide Guidebook - May 2008.pdf
(1.61 MiB) Downloaded 936 times
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

As to data, in 2008 there were few country ETFs with a track record much above 10 years. And those were pretty well limited to Barclays iShares.
Frankly, anyone doing research on index tracking ETFs is likely to need to pay the closest attention to tracking error. If tracking error is low and charges are low then you can probably simply use index data to test back in time so long as you add back in dividends and deduct fees/tracking error.

If tracking error and fees are high, you can of course still back test using appropriately adjusted index data but you are likley to want to avoid these funds!

I have no interest in actively managed or theme ETFs and so make no comment on those.
mojojojo
Roundtable Knight
Roundtable Knight
Posts: 131
Joined: Mon Oct 31, 2005 2:07 pm

Post by mojojojo »

The problem is that most ETPs that track commodities do not trade that often and have sizable spreads. The other major issue is that many don't have a lot of history, as AFJ mentions.

Also, a lot are ETNs so you have realize that your taking basically credit risk with them.
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

mojojojo wrote:The problem is that most ETPs that track commodities do not trade that often and have sizable spreads. The other major issue is that many don't have a lot of history, as AFJ mentions.

Also, a lot are ETNs so you have realize that your taking basically credit risk with them.
Absolutely agree with all of this.
mojojojo
Roundtable Knight
Roundtable Knight
Posts: 131
Joined: Mon Oct 31, 2005 2:07 pm

Post by mojojojo »

Here is a list I created. Hope it helps. Note the trading volumn column.
Attachments
Commodities.xlsx
(36.12 KiB) Downloaded 471 times
squaredQ
Full Member
Full Member
Posts: 23
Joined: Wed Feb 01, 2012 5:27 pm

Post by squaredQ »

Thanks for the replies! And I completely agree with the liquidity and lack of history issues.

But, it looks as if many (smaller?) funds seem to say they are moving to ETFs (e.g. see 'winning with etf strategies,' max issacman) . Isn't it possible they are monitoring tracking of etfs and their corresponding long term indices, and if there is strong correlation, that is sufficient until shown otherwise (what better test than 08 data for liquidity issues)?

There are also a lot of hassles, IMO, with putting together good long-term historical futures continuous data . Even the free blox data has re-constructed negative price data to deal with; and for someone that does a lot of returns based backtesting that is a pain, IMO.

ps. @garner great book! And very happy to see your pos. amzn rec on
'following the trend,; one of the best books I've come across in that space in a while! Only he also doesn't provide too much details on reproducible data gathering and back-testing. It's as if you somehow have to jump through tons of hoops, just in gathering, massaging, and cleaning proper data, in order to even begin contemplating replicating and validating the work.

If you ever decide to write a text dedicated to gathering and cleaning back-testing data or know of one , please let me know! Thomas stridesman kind of touched on it, but again, very lacking in detailed re-producible guidance. The nice thing about a lot of the rotational ETF books is that they can at least demonstrate reproducable research (even though there are short histories and other warts, heck -- you can reproduce and validate exactly what they are telling you).

Anyways, best of luck again on your new site!
rgd
Roundtable Knight
Roundtable Knight
Posts: 396
Joined: Thu Aug 10, 2006 5:21 pm

Post by rgd »

keep in mind that the spreads and liquidity one sees on the screen is often not indicative of the true liquidity in a lot of these ETPs. A rule of thumb: if the underlying is liquid, the authorized participant or market makers will usually make a decent market for you. This is assuming you are trading in volumes big enough to make it worth their while and big enough to hedge. Of course, when the crap hits the fan, they might not be so friendly to you.

Also, the management fees on these ETPs are certainly a drag on performance.

After the initial upsurge in ETP offerings, we are now seeing them contract, and many specialty ETPs shutting down.
squaredQ
Full Member
Full Member
Posts: 23
Joined: Wed Feb 01, 2012 5:27 pm

Post by squaredQ »

rgd wrote:keep in mind that the spreads and liquidity one sees on the screen is often not indicative of the true liquidity in a lot of these ETPs. A rule of thumb: if the underlying is liquid, the authorized participant or market makers will usually make a decent market for you. This is assuming you are trading in volumes big enough to make it worth their while and big enough to hedge. Of course, when the crap hits the fan, they might not be so friendly to you.

Also, the management fees on these ETPs are certainly a drag on performance.

After the initial upsurge in ETP offerings, we are now seeing them contract, and many specialty ETPs shutting down.
Good to know. Thank you.

Although, again I would assume crap hitting fan and bad liquidity to smaller players should somewhat be reflected in the historical data series during the worst parts of 08, unless they were back-editing the data records. Is that a bad assumption? Or do you mean to say the aggregate records are not reliable-- i.e. even thought the record denotes prices were hit, only big guys got hit at that price.
stopsareforwimps
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Sun Oct 10, 2010 1:47 am
Location: Melbourne Australia

Post by stopsareforwimps »

squaredQ wrote:again I would assume crap hitting fan and bad liquidity to smaller players should somewhat be reflected in the historical data series during the worst parts of 08, unless they were back-editing the data records. Is that a bad assumption?
If you compare the volatility of the ETF versus the underlying, the answer becomes clear very quickly. At least this is what I found for share index ETFs, where in bad times the volatility was often multiples higher in the ETFs. My suspicion was that it was the mug punters like us - not the market makers - who were paying these gaping spreads. Not someone that is easy to prove.
mojojojo
Roundtable Knight
Roundtable Knight
Posts: 131
Joined: Mon Oct 31, 2005 2:07 pm

Post by mojojojo »

rgd wrote:keep in mind that the spreads and liquidity one sees on the screen is often not indicative of the true liquidity in a lot of these ETPs. A rule of thumb: if the underlying is liquid, the authorized participant or market makers will usually make a decent market for you. This is assuming you are trading in volumes big enough to make it worth their while and big enough to hedge. Of course, when the crap hits the fan, they might not be so friendly to you.

Also, the management fees on these ETPs are certainly a drag on performance.

After the initial upsurge in ETP offerings, we are now seeing them contract, and many specialty ETPs shutting down.
This is very true. The liquidity of an ETF is based on:
1) it's actual trading volume
2) The liquidity of the underlying holdings
3) Time of day

#3 is somewhat related to #2. They will be most liquid during the real market hours of the exhange that the underlying trades on. Mostly true for equity funds than commodity based ones.

I don't know if the ETP market place is contracting. Using US domiciled ETP data:

New funds in 2012: 163
Closed funds in 2012: 94
---------------------------
Net growth: 69

We can argue if it's "good" or not, but you will see a lot firms launching active ETFs this year IMHO. They will be the next wave of products.
michaelt
Senior Member
Senior Member
Posts: 37
Joined: Wed Feb 03, 2010 10:53 am
Location: princeton,nj

Post by michaelt »

I tested our systems on ETF's. I used approximately 20 ETF's to represent every market in our futures portfolio for which an ETF was available, or what seemed a reasonable proxy.

There was a significant trading performance deviation. At the time (about 18 months ago) one could see the price deviation in some ETF's vs. their underlying futures market. And that deviation showed up in my tests via underperformance.

One understanding I got is that some ETF's are shifting money due to inflows and outflows thus changing their beta to the underlying. Another issue I understood was the specific terms of some ETF's to the underlying.

It's attractive but I found elusive.
Post Reply