Optimizing with Veritrader? 19billion!

[Erwin Dicker]

I was optimizing with veritrader and i used the demo portfolio.

Results: 19.000.000.000

Ending balance $19,301,562,247.62
CAGR 501.4%
MAR 8.32
Sharpes 1.83 & 1.13
Drawdowns 60.3% & 44.0% (oké, is much)
periods DD 6.7
trades 2268

WHAT NOW?

Erwin

[Erwin Dicker]

I made a nice system with the turtle futures set.

CAGR % 86.74%
Maximum Total Equity Drawdown % 46.13%
MAR Ratio 1.88
Modified Sharpe Ratio 1.58
Annual Sharpe Ratio 1.45
Annual Sortino Ratio + ∞  
Monthly Sharpe Ratio 0.43
Monthly Sortino Ratio 0.92
Calmar Ratio 2.08

To keep it simple i included all futures, so max number of instrument was 15, and the other correlation items too.
No adding of positions was allowed. The idea was that it is very difficult to find the best portfolio. I optimized the demo set and came to 19 billion. Changing PA by SI reduced it to 130 mill.
So trade as much as possible without looking/checking for correlations.
When i change max number of instruments from 15 to 14 a huge drop in performance take place. I don't expect this.

Can anyone explain this?

Thanks for your comments!

Erwin

[Forum Mgmnt]

There is way too little data included with the Demo version for one to draw any conclusions about likely performance.

This concept has been discussed here before and will come up again.

The sensitivity to portfolio and particular parameters is far to high if you don't test with enough data. I always recommend as much data as you can get, at least 20 years worth.

- Forum Mgmnt

[Erwin Dicker]

There is way too little data included with the Demo version for one to draw any conclusions about likely performance.

This concept has been discussed here before and will come up again.

The sensitivity to portfolio and particular parameters is far to high if you don't test with enough data. I always recommend as much data as you can get, at least 20 years worth.

- Forum Mgmnt

Thanks for your reaction. I understand your point. Of course you would give away too much if it was enough in your opinion. But....

the last system values I posted where not at all (profit) optimized! When optimizing the (profit) results be much, much better. I took the Turtle set, because this was the most extensive set. It had 2629 trades. In my feeling that are sufficiently high numbers to have statistical evidence.....?

When 20 years of data (i only have 10 from CSI), has other results then my point is incorrect....or the markets sets have rotated.... My opinion is too trade as many markets as possible.

By the way, in reading i discovered a little mistake in the User's Guide on page 21 - 22. The long example has to be short example vice versa. Slippage give a worse price in the direction of the trend. I hope it is not in the software .

I go on investigating.....

Erwin

[Forum Mgmnt]

Erwin,

It is important to understand the rationale behind the concept of a statistically siginificant sample as it relates to trading.

In most contexts, statistical significance and confidence intervals are discussed with reference to samples taken from a population (polls, quality control samples from a production line, etc), where measurements are taken of the samples in an attempt to draw conclusions about the entire population. In these instances, a sample is used because polling or testing the entire population is too expensive.

If the samples taken are representative of the overal population, then the size of the sample determines the confidence that the measurements of the sample are representative of the underlying population.

In the context of trading, the expression: "If the samples taken are representative of the overal population" is especially important.

In trading system discussions where trades represent the samples, the population is all trades a given system would have taken in the past, and will take in the future.

If one looks only at the number of trades, and not at whether or not those trades are representative of the entire population (i.e. the past and future trades) then one is likely to end up drawing erroneous conclusions.

One example: there are plenty of stock trading systems with 10,000 trades or more over the years 1997 through 1999 that have excellent results over that period but that performed very poorly after the crash. Conclusions drawn from tests using only those years would have seemed to be very "statistically significant" based on the number of trades in the simulation.

However, when one asks the question: "How representative are the years 1997 to 1999 of the stock market behavior, in general?", one quickly gets to the crux of the problem with using number of trades as the only measure of confidence.

So the reason to use many years of data is not so much to get a certain number of trades, but because we want to have the best chance that our tests include a sufficient variety of market conditions that they are representative of what is likely to occur in the future.

In trading we don't have the luxury of being able to test the future. So our tests are, at best, a sample of the population of all trades. If we don't use enough data, we run the risk of testing over a period that is not typical.

Then when the future comes, it will be very different from our testing.

This phenomenon generally results in trading losses, either because we end up trading a system that only works under certain types of markets, or because we stop trading a successful system because our actual results are so much worse than our testing indicated.

(see also: viewtopic.php?t=559)

[Erwin Dicker]

I understand your point (s). Thanks for your extensive answer.

My idea was that trading a set of 23 futures for a period of 5 years would be enough to have enough trends and enough non-trending times. I cannot prove my idea. If the system performs well on the period before the testset, it is better....but i have only the demo veritrader so i could not test it.

I wanted to isolate the problem (making money with reasonable risk/reward), to get a clean look upon the working of the system. I left out correlation and ADDING positions.

I left the max units (all) on 15, because i wanted to know what results one could get, leaving out correlations and in my opinion trade as much markets as possible, in order to get the most out of the demo .

The idea about ADDING POSITIONS is, although i practise it, still strange to me. The "signal" to add positions is actually reacting on another system which is slower. So adding at +1N after a 33 day breakout could have a comparable result as the system which trades 37 day (or so) breakouts.

And to me the money management is not correct then. Because HOW MUCH you have to risk depends on the statistics of that (slower) system. Every system has another optimum in "percent account for N", because it has other profit and loss statistics.

The question which rises is : Why trading on a system that logically takes trades which are worse than the system which trades the first position? That is why i left out ADDING POSITIONS, to keep it as simple as possible.

When you study the outcomes of performance, MAR, CAGR%, Sharpe Ratio as function of "percent account for N", there is a interval in which you get very smooth variations in the risk and performance parameters.

Now the data:

I made an excell file in which random graphs are generated. I will put this file as an attachement to this post. You will be astonished how many patterns, and trends you can see! Try it.

What i hope/want to do is to generate random price data. If one can make a system which makes money then...... i think you are there.

Could i change the files of the demo? Who wants to help?


Erwin

[Erwin Dicker]

Hello Brendo,

Yes you are right, partially i believe. ADDING is done rather crude, but it can be done like the turtles say, and i even practise it. So it is not that i don not agree, but i described some qualities of the added positions.

And, when you compare e.g. the performance of the quadriga funds (www.quadriga.at), you can see an equivalent equity development, and they don't ADD positions. So i think it is not necessary to add in order to achieve similar results.

Thanks for the tips about the authors. Ralph Vince can be added to the list.

Erwin

[d-b]

First, thanks Erwin for raising this issue, the subject of “when do we have a large enough sample?â€

[batuco9]

Erwin,
Just to add to what has been said on the matter of length of testing: you might take a look at a dow chart and see how different things come up as you go longer term (you might try http://www.dogsofthedow.com/dow1900log.htm) or commodity charts, such as metals (might try http://www.kitco.com/charts/). Pity CSI doesn't at least include the indices on a longer term basis - I think they should. (remember to use log scales wherever possible for this in order not to lose out on the details just because of scale).

But anyway, since you have CSI, why don't you use that 10 year data for testing with Veritrader and compare results?

I use CSI to build as ASCII (mark 'ASCI' under preferences when you're selecting the data series for your portfolio), I then use a batch file (*.bat) to automatically copy the 'wierdly named' CSI txt files to the Veritrader data directory with names I can understand, and than I use TextPipe to get rid of all those incredible 0's that come up (though I understand that the newest CSI release allows you to get rid of them - I beleive by unmarking 'display decimals with maximal precision' in the ASCII Misc tab ). (Ahh yes, and for my forex data I have to use TextPipe to add in dummy volume and open interest numbers, though I can imagine that with a bit of patience you could use Excel for that, speeding up a bit with some macros).
Of course, you have to prepare a *.set file or files, so you can choose your new data with Veritrader, and you have to prepare new entries in the FuturesInfo.txt or StockInfo.txt files in the Veritrader directory in order for Veritrader to use the data - its explained in the manual.

Best of luck,
batuco9

[cyphrograph]

So the reason to use many years of data is not so much to get a certain number of trades, but because we want to have the best chance that our tests include a sufficient variety of market conditions that they are representative of what is likely to occur in the future.

Precisely. This is exactly where one of the intraday systems' advantages occurs: more different market conditions in the history used for testing. For example, if you base your system on 30 min intervals within 6 year S&P history you'll have 13*250*6 = 19,500 bars which represent "market conditions". In EOD testing with 20 years - only 5,000 bars. In 20,000 bars we have plenty of trending, choppy & flat periods, gaps, price spikes, etc.