CTA performance data

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
stopsareforwimps
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Sun Oct 10, 2010 1:47 am
Location: Melbourne Australia

Post by stopsareforwimps »

jankiraly wrote:Hello stopsareforwimps, I think I have found a photograph. Is it right? http://tinyurl.com/3dexs2n
This one is more flattering I think.

In case anyone is wondering how someone "outed" me, I posted my name in an earlier thread.

A beginning futures trader, having only traded currencies and index futures and CFDs for 2-3 years so far (though profitably!). Invested in stocks for many years with some success until the 2007 crash opened my eyes to many other things. As an old guy I prefer to avoid reinventing the wheel, or as Isaac Newton put it to "stand.. on the shoulder of giants", if I can, but if necessary I will do the work to find things out.

I run a Seykota-style trading tribe in Melbourne Australia and attended one of Ed's workshops.

As a retired computer programmer I write all my own code in Lisp, and do not yet have a copy of Blox though it would probably make sense if I did.

Tim Josling
Attachments
tj_hplus_small.jpg
tj_hplus_small.jpg (72.17 KiB) Viewed 5749 times
AFJ Garner
Roundtable Knight
Roundtable Knight
Posts: 2071
Joined: Fri Apr 25, 2003 3:33 pm
Location: London
Contact:

Post by AFJ Garner »

stopsareforwimps wrote: As a retired computer programmer I write all my own code in Lisp, and do not yet have a copy of Blox though it would probably make sense if I did.
Tim Josling
Talking of LISP, I have long been fascinated by the search for AI and the reflections it forces us to engage in regarding consciousness, qualia, free will and the nature of humanity in general.

It is a pity that human life is so short; if I had another lifetime ahead of me AI is very probably what I would focus on.
stopsareforwimps
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Sun Oct 10, 2010 1:47 am
Location: Melbourne Australia

Post by stopsareforwimps »

AFJ Garner wrote:It is a pity that human life is so short; if I had another lifetime ahead of me AI is very probably what I would focus on.
Yes I think we about to enter the golden age of AI as computers finally catch up with the human brain's processing capabilities. It's been a long time coming. Last year I left my last job and decided not to look for another one. One reason was so I could devote more time to AI and related issues. There is so much good material around these days for the price of your time.

http://www.ai-class.com/
http://agi-conf.org/2010/2010/11/28/agi-10-videos/ ('11 videos coming up soon too)

Related
http://www.singularitysummit.com/

I also helped organize this
http://summit2011.singinst.org.au/

This is relevant to blox forum because I think that in many fields, including finance, AI will play a massively increasing role in the next 10-20 years.
M20J
Senior Member
Senior Member
Posts: 39
Joined: Tue Aug 12, 2008 4:49 am

Post by M20J »

I have to admit to being sorely tempted to attend the Singularity Summit, which looks to have a really interesting program this year. I'm sometimes a bit tight-fisted with discretionary spending on myself though, so I haven't quite committed myself to it yet.

In a finance-related aside, one of the speakers this year is John Mauldin, who also writes an excellent, free weekly investment newsletter, available to all. It's absolutely nothing to do with systematic trading, but I find always well worth a read for thinking about the big picture. A quick Google would lead to a sign-up page. He's a bit of a perennial bear, but none the worse for that.
stopsareforwimps
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Sun Oct 10, 2010 1:47 am
Location: Melbourne Australia

Post by stopsareforwimps »

stopsareforwimps wrote:I will report back in due course.
Sluggo I can reproduce all your results except for the profitability of the system. (You are right about the tenuous relationship between the skew on a per trade basis and the skew of periodic returns).

The only things I can think of are

a) I am using stops and profit targets that only trigger at the end of day. Are you using intraday stops ie sell stops; ditto for the profit targets? Consistent with this I am getting much worse worst case losses.

b) What is the assumption re slippage and fees? I am using 30% slippage and $7 fee per contract. Without fees or slippage it is just marginally profitable.

c) In my runs I tend to get peaks in trade profits near or just past the stop and target. In your graph there is a peak at -1 ATR and no peak near -2 ATRs. The only exception is where the stop is quite wide which means the system gets out before the stop triggers.

d) What time period are you using to compute the ATRs and are they exponential or straight averages?

Any ideas?
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

Are you using the identical "International-38" portfolio? Including the forward contracts for Nickel and Copper, traded on the London Metals Exchange? A trading system's profitability can vary when you modify the portfolio of markets traded; some examples from the Roundtable are (REF1) , (REF2) . Thumbnail plots of examples are shown below.

Please note the difference between Average True Range ("ATR"), and R-multiples. Some of the histograms I presented, use R-multiples on the horizontal axis. None of the histograms I presented have ATR anywhere.
Attachments
Change the portfolio, change the system's performance
Change the portfolio, change the system's performance
pair.png (62.03 KiB) Viewed 5402 times
stopsareforwimps
Roundtable Knight
Roundtable Knight
Posts: 199
Joined: Sun Oct 10, 2010 1:47 am
Location: Melbourne Australia

Post by stopsareforwimps »

stopsareforwimps wrote:I will report back in due course.
TLDR: A heap of research showed me that when stops and profit targets are quite tight, they can affect the skew and kurtosis of the monthly returns. But otherwise they seem not to. So Sluggo was right. My flippant comment was flawed.

Ultimately the fact that the long term CTA survivors have positive skewness of monthly returns and positive kurtosis of monthly returns, may reflect nothing more than the fact that they have survived.

You can probably stop reading here unless you are a masochist.

Possible Explanation - Hypothesis 1

Unless stops and targets are quite tight, the periodic fluctuations in equity are dominated by the daily moves of the underlying markets not by the stops and targets.

For a tight stop loss of 1.1 ATRs there was a strong positive relationship between the skew of monthly returns and of the trades as we increase the profit target. Both skews start negative at low targets and go positive as the targets are increased. The skew of trade returns goes positive at a lower PT than the skew of monthly returns. In between those two values, the skews are of different signs.

In contrast, with a tight Profit Target of 2.1 ATRs there was a strong inverse relationship between the skew of trade returns and the skew of monthly returns as we increase the stop. The skew of the trades returns starts off positive and the skew of monthly returns starts off negative. As the stop is increased, the skew of trade returns declines and goes negative, while the skew of monthly returns increases and goes positive.

So you can indeed generate the kinds of phenomena Sluggo described.

In my simulations - with different data than the international set originally described - having such tight stops and targets reduced risk-adjusted returns markedly eg -30% PA. This result may vary depending on the underlying strategy.

Possible Explanation - Hypothesis 2

When trade or periodic returns are compounded, the distribution tends to increasingly resemble a normal distribution, with zero skewness and kurtosis - provided certain conditions are met. This means that letting winners run and cutting losses would not generate positive skew when measured over long intervals. Instead it could generate higher compound returns by reducing volatility drag and increasing positive-skew lift. In that way it might increase your chances of survival.

With financial market return data, these conditions tend not to be met. For example, according to my calculations, the US stock market's real annual returns from 1820 (sic) to 2009 show a fair bit of skew (in the low single digit range) even for 30-year returns measured over this 189 year period. According to theory the skew and kurtosis should be minuscule at these time frames.

The issues of a) What are the underlying statistical distributions of financial market returns, and b) How do these returns compound to longer periods, are not at all resolved in the theory so it is hard to say anything much about this.

Note

I did not test trailing stops or trailing profit targets. The results there may be different. I used the blox free data as at January this year for the test (Futures only).

Apart from learning possibly more than I wanted to know about the statistical distributions of financial market returns, I also found and fixed a significant bug in my trading code during this exercise which was costing me 2% pa. So it was time very well spent. Which also raises some troubling questions. I will be hanging out in the testing forum a bit.

See

Jarrod Wilcox "Investing by the Numbers"
Jean-Phillipe Bouchaud "Theory of Financial Risks"
Hudson, Richard L.; Mandelbrot, Benoît B. The (Mis)Behavior of Markets: A Fractal View of Risk, Ruin, and Reward.
Attachments
This pic shows the fact the the skewness of returns for 22-day periods can be negative even though the skewness of trade returns is positive.
This pic shows the fact the the skewness of returns for 22-day periods can be negative even though the skewness of trade returns is positive.
skewness-v-pt-sl1.1.png (26.72 KiB) Viewed 5310 times
In this pic we see the skewness of the 22 day returns moving in the opposite direction to the skewness of the trade returns as the stop loss is increased.
In this pic we see the skewness of the 22 day returns moving in the opposite direction to the skewness of the trade returns as the stop loss is increased.
skewness-v-stop-loss-pt2.1.png (29.22 KiB) Viewed 5310 times
Eventhorizon
Roundtable Knight
Roundtable Knight
Posts: 229
Joined: Thu Jul 08, 2010 2:36 pm
Location: Boulder, CO
Contact:

Post by Eventhorizon »

stopsAreForWimps,

Very interesting work - thank you for your continuing excellent contributions!
stopsareforwimps wrote:When trade or periodic returns are compounded, the distribution tends to increasingly resemble a normal distribution, with zero skewness and kurtosis - provided certain conditions are met.
Why are you making this assumption? I am not sure that there is an expectation that this distribution should be normal with zero skew and kurtosis.

One issue that occurs to me regarding your analysis is the choice of "returns". If one wants to answer the question "what is the expected return selecting any period at random", furthermore what is the standard deviation, skew, kurtosis, etc, then the data points are the arithmetic returns for the period in question, which you have used. But aren't we more interested in geometric returns when considering the "goodness" of a system / manager?

If one is considering issues such as long term return. survivorship, etc, then the underlying datapoints for the analysis should be Ln(1 + return). We are interested in the compounded returns.

Using simple returns, we are under-weighting negative returns and over-weighting positive returns in terms of their impact on compounded returns and thus likelihood of survival: with a large enough standard deviation of returns, a system / manager with +ve arithmetic returns can have negative compound returns. I will try to work up some "thought experiment" figures to support this argument.

Bottom line, your exploration might be even better if you use continously compounded returns as your basis for analysis. I don't think you can reliably do any of this on a per trade basis as one needs to reflect trade frequency and trade duration to get anything meaningful from the distribution of per trade returns.

edit: ordering paragraphs
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

Found this in an earlier (Posting) ,
Once upon a time long ago, I did a study on the distribution of (net profits per trade (expressed in R multiples)) on a handful of garden variety trading systems. I seem to recall that the Normal Distribution (aka Gaussian Distribution) was a very poor fit to the data. Executive Summary: Trade Outcomes Are Not Normally Distributed.

At least, that's what I think I remember.

Then, using software that lets you simulate trading lots of instruments at once out of the same account, using dynamic position sizing, I did another study on the distribution of (daily equity returns). i.e., ((total account equity today) / (total account equity yesterday)). I seem to recall that the Normal Distribution was a very good fit to the data. Not perfect, of course, but very good. Executive Summary: When trading a portfolio of instruments, Daily Returns Are Very Close To Normally Distributed.

At least, that's what I think I remember.

At the time, I attributed the very good fit to these details of the particular study I performed: (1) I tend to trade, and hence to test, with lots of different instruments in the portfolio. (2) I tend to trade, and test, using futures contracts rather than stocks or forex pairs or fixed income securities. (3) Futures tend to have significantly lower instrument-to-instrument correlation than other tradeables, notably, than stocks. Put these together and you approximately meet the requirements of the (Central Limit Theorem), which states that if you add up a large number of independent random variables, the resulting sum has a Normal Distribution.
And here's a message that contains a bit of data viewtopic.php?p=44141&highlight=skew#44141 shown below.

Ambitious and enthusiastic readers are encouraged to try this on a few of their own trading systems. Are your trade outcome distributions normally distributed? Are your daily returns distributions normally distributed?
Attachments
histogram_356.png
histogram_356.png (96.68 KiB) Viewed 5251 times
Post Reply