Optimal f

Discussions about Money Management and Risk Control.
enigma
Roundtable Fellow
Roundtable Fellow
Posts: 99
Joined: Tue Apr 29, 2003 6:56 am
Location: UK

Optimal f

Post by enigma »

How useful is this concept in practice?
Mark Johnson
Roundtable Knight
Roundtable Knight
Posts: 122
Joined: Thu Apr 17, 2003 9:49 am

Post by Mark Johnson »

In case it isn't screamingly obvious, what follows is my opinion. Other people will have different opinions. Occasionally someone might represent their opinions as "facts" :P , but they're just opinions.

I think Ralph Vince's book Portfolio Management Formulas, which invented and explained the concept of "optimal f", is incredibly useful in practice. I think his idea of making a graph of TWR versus f is brilliant.

However, I think the real value of this graph is not the single point representing the maximum (the one Mr. Vince labels "optimal f"); rather, I think its value is showing you a map of The Cliff of Death. This plot will help you decide just how close you wish to get, to blowing up your account.

Here are a few recommendations for exercises to perform:
1. Plot TWR vs. f for your trade outcomes (from a backtest)
2. Delete your biggest winning trade and insert a 2nd copy of your biggest losing trade, then plot TWR vs. f again. Compare these two plots.
3. Plot some measure of "pain" versus f: Max Drawdown, Downside Drawdown, annual standard deviation of returns, avg-of-5-worst-drawdowns, whatever you like. Line up this plot below plot #1 and now you have TWR vs. Pain vs. f. This is exactly what you want to see (in my opinion), when making your betsize decisions
4. Run your system test again with a different value of commissions + slippage and plot TWR vs. f (and Pain vs. f) again. Compare against item 3. above
5. Use your imagination! Think of other experiments. Do these experiments. Ponder your results. Make some decisions. Trade.
Kiwi
Roundtable Knight
Roundtable Knight
Posts: 513
Joined: Wed Apr 16, 2003 1:18 am
Location: Nowhere near

Post by Kiwi »

My opinion is that Mark is more often right than most. 8)

I would add one thing. Choose a point with considerably less pain than you think you can stand because most people find that this is the point where they do something very wrong and terminate their trading career. Its hard to think straight when you are worried that your system has failed and that you're going to have to explain your failure to your wife and family. Bet smaller until you know how you feel when the pain really starts.

If you trade up to your point of pain and are still comfortable then increase your bet size.
gbos
Senior Member
Senior Member
Posts: 26
Joined: Wed May 21, 2003 1:06 pm
Location: Athens Greece
Contact:

Post by gbos »

An excellent paper about betting size and “optimal fâ€
Attachments
Money-1.xls
Drawdowns formulas
(36 KiB) Downloaded 3515 times
Hiramhon
Roundtable Fellow
Roundtable Fellow
Posts: 98
Joined: Fri May 09, 2003 12:45 am

Post by Hiramhon »

In real life, of course, there are more than two possible outcomes. (You could make $100 profit. You could make $350 profit. You could lose $200. You could lose $4250. etc.) Good old MJ posted a nice programming problem in which there are SIX possible outcomes rather than two. Your job: find the optimum fixed fraction to bet in this case. Naturally he also gives the solution and software code that solves the problem.

http://traderclub.com/discus/messages/1 ... 20020734am
gbos
Senior Member
Senior Member
Posts: 26
Joined: Wed May 21, 2003 1:06 pm
Location: Athens Greece
Contact:

Post by gbos »

Hi Hiramhon

That’s true but the calculation of expectation and variance remains straightforward.

Expectation = p1 * w1 + p2*w2 + p3*w3 + …..
Variance = p1*w1^2 + p2*w2^2 + p3*w3^2 + ….. – Expectation^2

The approximation f = mean/variance remains valid for small f not matter how many possible outcomes you have even if you have a continuous probability density function.

When f is not small comparing with 1 there is not a general valid approximation and in order to find optimal f you must maximize a sum

Sum pi * ln ( 1 + xi *f)

where pi are the probabilities for each outcome and xi the payoffs per unit bet (positive for profits and negatives for loses).
In most practical cases however f=mean/variance remains a good conservative approximation.

Best Regards
masmit
Full Member
Full Member
Posts: 20
Joined: Sun Oct 12, 2003 7:28 pm
Location: London, England

Post by masmit »

Interestingly, your coin toss example gives the same result as the Kelly formula:

K=W-((1-W)/(w/l))

where W is the percentage of wins, w is the average dollar win and l is the average dollar loss.

As in your optimal f calculation:

.5 -((1-.5)/(2/1)) = .25

Cheers,

Mark
Dan G

Post by Dan G »

Van Tharp plans on releasing a program soon that will allow you to easily test different position sizing systems. I don't have a beta of it, so I do not know how it works, or exactly what it can do. From the data they provided, it seemed to have lots of interesting and useful features, at least the ability to plot different PS systems on the same graph(using the same trading system), and a full range of evaluation statistics.

When I find out more about the program(which should be soon), it will be posted here. I just spoke with IITM, and they said that the program has become much more advanced than they originally planned, so they are delaying the beta release.
edward kim
Roundtable Knight
Roundtable Knight
Posts: 344
Joined: Sun Apr 20, 2003 2:42 pm
Location: Silicon Valley / San Jose, CA USA
Contact:

Post by edward kim »

Hi Mickslam,

I am currently helping the developers at IITM with their software called Know Your System (NOT A PLUG!). There are two major things that the simulator does: it evaluates your system over many trials so that you can see the different performance ranges, and it also can help you determine the optimal bet size for your system.

You do have to know the R-Multiples for your system, and the counts for each R-Multiple. For example, you can use the R-Multiple results from VeriTrader and run as many as 10,000 simulations over a specified time period. You will then see the actual performance ranges for your system, which often is quite surprising. I had a system in VT that was like this:

80% CAGR
2.55 MAR
40% MaxDD

When I plugged in the R Multiples from VeriTrader into Van Tharp's simulator, there was a 0.4% chance that I had a 0% return over a 10-year period. Although 0.4% is unlikely, it shows me how my R-Multiple distribution can yield different results when it is simulated many many times.

I use both VT and Know Your System to test my systems.

Edward
Ghostrider
Roundtable Fellow
Roundtable Fellow
Posts: 83
Joined: Thu Jul 31, 2003 11:46 am
Location: USA

Optimal f

Post by Ghostrider »

In my view trading is NOT about fancy mathematics any more than running a business is about knowing tax laws (of course, I am assuming that a basic knowledge of statistics and probability concepts is not considered 'fancy' here). Trading is about trading, you have to have a basic concept of math but you don't have to be a quant, unless you are trying your hand at some high powered form of arbitrage. Ralph Vince's theory is off in its assumptions because it overlooks the fact that controlling risk is the absolute number one priority, NOT optimization. This very topic is discussed and wrapped up nicely in a few paragraphs in Jack Schwager's interview with scientist/money manager William Eckhardt in 'New Market Wizards' (Eckhardt has a few hundred million under management; I don't know how much Vince does, if any).

The idea that money management is a mysterious and complex issue is proliferated by books like this. I can sum up effective money management in two steps: First take your worst case loss scenario, which will be based on a run of lossses with a less than one percent probability of occurring, as calculated by your expected win/loss ratio. For example, if you reasonably expect to win 40% of the time and lose 60% of the time, there is a less than 1% or "worst case" probability, that you will see ten losses in a row at some point in your trading (this may look like hard math but the calculations are actually grade school level). Next, determine your max desired drawdown. What's the biggest hit you could possibly stand? Ten percent down? Twenty five? Fifty? Let's say you are moderately aggressive and able to deal with a twenty percent drawdown without losing your nerve. Divide twenty percent by ten, and you see that your max allowable risk is 2% of your account balance, including calculated slippage and commissions per trade. If you can stomach a 40% drawdown you don't risk more than 4%, and so forth. Simple, straightforward, no hidden gimmicks, gizmos or geekspeak. The only other bogey you have to deal with is the once in a blue moon nasty price shock that blows your stop to kingdom come (a simple and emphatic argument for less risk per trade, not more).

Most CTA's could double their returns very easily, simply by doubling the amount of risk they take. Why don't they do it? Because the reward is not worth the risk. If you get a huge win, it will help your profits but it will not change your world. A string of fat losses or a single huge loss, however, can kill you, take you out for years, maybe for good, leave you wandering the streets muttering 'if only I had taken a smaller risk on that trade with my name on it, I would have survived'....
If you double your money in eleven months and then take a 50% loss in December, where have you gotten? Nowhere. I recall an anecdote a while back about how George Soros' fired one of his currency traders after a huge score because the trader took on way too much risk with the trade. Even though he won, he got canned for being reckless. If I had someone trading for me and he wasn't thinking defense first, I would can him too. So you've had a reliable win/loss ratio in the past, so what. How do you know a string of losses or a price shock isn't going to bite you when you least expect it? You don't know, and no one is completely immune to a streak of misfortune. Optimization is for gamblers, not professionals. The 'F' in Optimal F should stand for a word that rhymes with 'shucked.'


A trader wrote this, and I couldn't agree more.

GR
Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt »

GR,

I too couldn't agree more.

A former partner used to love to pontificate about the Kelly Formula and optimal bet size but when it came down to it he never traded at anywhere near the level implied by those formulas. Trading at sizes suggested by the Kelly Formula is a good way to go bust.

I don't know why no one talks about this but there's a dirty little secret of the formula as it relates to trading, the Kelly Formula is based on a "non-terminating game" of gamblers betting infinite times. That's why he uses the math associated with infinite sums. See www.bjmath.com/bjmath/kelly/kelly.pdf for the original 1956 paper by J. L. Kelly Jr.

The infinte-series-based math in the Kelly Formula considers all the outcomes possible, even those that involve very long series of losers followed by winners.

This is a real problem for all of us who have finite amounts of trading capital. Those long or even moderate series of losers can result in zero capital, or at least so little that trading is no longer viable. In this event, we don't get to keep betting like Kelly's gamblers playing their "non-terminating game".

If you have infinite capital, go ahead and use the Kelly Formula. If you're like me, with a fixed sum in the bank, I suggest using bet sizes based on much more conservative analysis.

Rules of thumb like: "use 80% of the Kelly Formula size", are likewise wishful thinking without any basis other than supposition.

- Forum Mgmnt

P.S. I do think the formula is useful in a theoretical sense and for understanding the relationship between payoff and probability but not for determing how much to trade.
shakyamuni
Roundtable Knight
Roundtable Knight
Posts: 113
Joined: Wed Jun 04, 2003 9:44 am
Location: Somewhere, Hyperspace

Optimize This

Post by shakyamuni »

Agreed. It's just as unwise to trade on a purely theoretical optimum as it is to arbitrarily select a position sizing scheme based on your zodiac sign.
Last edited by shakyamuni on Sat Jan 08, 2005 4:43 pm, edited 1 time in total.
lperepol
Senior Member
Senior Member
Posts: 30
Joined: Thu Feb 26, 2004 12:50 pm
Location: Castlegar, BC, Canada
Contact:

Post by lperepol »

[quote="gbos"]An excellent paper about betting size and “optimal fâ€
ksberg
Roundtable Knight
Roundtable Knight
Posts: 208
Joined: Fri Jan 23, 2004 1:39 am
Location: San Diego

Optimal-f, Monte Carlo, and Kelly

Post by ksberg »

Iperepol, warm greetings from a former web-footed Seattlite.
I was wondering if any one compared Monte Carlo methods to Ralph Vince's formula? Ralph does spend some ink in mentioning Monte Carlo as a means. I read his book "The New Money Management" and was disappointed. I am confident that I could find a collection of stocks (a portfolio) that will disprove his formula. On the other hand, if I found a portfolio that proved it correct I would be condemned to the school of "Cooks For Books."
Yes, I have done such comparisons; but I think there may be some misunderstanding between method and outcome.

Optimal-f is fixed fraction position sizing that is optimal in a very precise, mathematical sense. The methodology returns the value which maximizes geometric growth. Whether this is true or not has little bearing on trading, and would not be proved or disproved by finding a collection of stocks. You disprove Ralph Vince by finding alternate mathematics. So far, his body of work stands.

Monte Carlo methods can also be used to find a fixed fraction value that is optimal according to input constraints. There is a whole body of theoretical statistics behind their derivation (Google on Markov Chains Monte Carlo methods). Monte Carlo methods require me to define what I mean precisely by "optimal". For instance, I may want to find the highest CAGR's with drawdown less than 40%. Then, using MC, I run hundreds or thousand of sample sets drawn at random from my original test data. The resulting set of set of trades are useful for asking statistical questions, like choosing the best fixed fraction value that will result in a probable return of X% with an N% chance of never seeing a 40% drawdown.

So, in this sense, Monte Carlo and Optimal-f both describe methods for finding an optimal fixed-fraction value. Their mechanics behind each approach are different, and the definition of optimal are different, but they each can produce a fixed fraction result.

My recommendation is that for trading, NEVER optimize solely on geometric growth. That, not Ralph Vince, is the road to ruin.

The MC optimization I suggested will typically produce a fixed fraction result that is a small fraction of Optimal-f. It will be very different for every system and situation you test.
A Kelly fraction is very nebulous requiring large sample spaces and many samples to accurately find. If one maximizes the coin flip example Kelly would equal 1 – bet it all. That is the shortest time to reach riches or death. When one deals with stocks, distributions are unknown . The Kelly fraction has some interesting properties. It can reduce drawdown -- bet small and increase diversity.
Remember that road to ruin? Take and graph the Kelly derived bet size. You will find that most often it lies to the right of Optimal-f. By that I mean that Kelly criteria is more aggressive than Optimal-f. So, I find it amusing that you're interested in Kelly, but somehow think Ralph is a crook.

Mark mentions that Optimal-f is The Cliff-of-Death, a line in the sand no trader should ever cross. That's precisely what Optimal-f means. It is the peak after which everything is downhill. Guess what that means for Kelly?

If you're using fractional-Kelly, it probably makes better sense to at least switch to Fractional-f (some fraction of Optimal-f). That way you at least ensure that you are trading some bet size less than a cliffhanger.

Moi? Man, I'd get nosebleeds even thinking about trading those kind of levels!

There's a heck of a lot to be said about Richard Dennis' sizing work. Those Turtle recommendations aren't accidental.

BTW: Forum Mgmnt, your suggestion to shrink bet size to 0.7% was great. I find 0.6% works terrific with my turtle-inspired system.

Cheers,

Kevin
lperepol
Senior Member
Senior Member
Posts: 30
Joined: Thu Feb 26, 2004 12:50 pm
Location: Castlegar, BC, Canada
Contact:

Optimal-f, Monte Carlo, and Kelly

Post by lperepol »

Hi ksberg,

I thanks for the feed back. In a nut shell what I was implying was that if one derives a fractional bet size (by what ever method) that comes out too high, there is something wrong. One should look further and test smarter.

>> Mark mentions that Optimal-f is The Cliff-of-Death, a line in the sand no trader should ever cross. That's precisely what Optimal-f means. It is the peak after which everything is downhill. Guess what that means for Kelly?
This does not follow from c.f.' paper on optimization -- generally speaking. Gambler ruin does occur on the right side of the peak, but the one point to the right side and one point to left should be logically equivalent No? There is variance involved no?

I was looking to see if some one calculated the variances between Vince's method and other methods.
Something like this:

repeat many times
Select set of stocks;
Calculate Vince's fractional bet
Other Method fractional bet
store Vince fraction results in array Vince
store other fraction results in array Other
end

average (Vince)
average (Other)
sd(Vince)
sd(Other)

I did not think Vince covered the topic adequately in his book "The New Money Management." I can play the fool and ask more questions than a thousand wise men can answer.
Neil
Full Member
Full Member
Posts: 11
Joined: Sun Dec 21, 2003 3:35 pm
Location: London

optimal f

Post by Neil »

All,

Before I was aware of Kelly and just starting out learning to trade it crossed my mind that if you run a series of trades using 2% account size and then the same series using 80% the results were going to be very different to put it mildly. I decided to test it using excel. The system was a very simple moving average crossover with only one trade open at a time.

To answer the question "how much?" I plotted an equity chart for a few different percentages and the results were enlightening to say the least.

Optimal f (though I didn't know it was called that!) for the system turned out to be 26%.

Now look at the chart below. I've plotted various sizes up to and including 26%. Putting volatility and variances aside, before the "cliff of death" (which by the way is more af a gentle hill) the bottom of each drawdown period is higher than the previous. The red line is 40% position size as an illustration of what happens when you bet the ranch.

Now I'm no genius but I can see that risking 20% is going to make me richer than if I was risking 5% regardless of how you calculate it. Do I care if I have a bigger drawdown if at the end of it I'm still better off? No doubt a hedge fund or CTA is going to frighten customers betting like this but I'm not a hedge fund manager. We are all here to make money, aren't we?

My conclusion was (and is) this-

1. Fixed fraction betting is good.

2. If you bet beyond optimal f this will not help your account size. Love it or hate it, optimal f is a number you NEED TO KNOW.

3. So, if you can stomach it, back off from the cliff (hill) of death by a safe distance and make your money work for you.

As an aside, the turtle system that we were all brought to this forum by is far from conservative. With up to 24 units at 2% each I would say that the turtles may be closer to optimal f than they admit to.

Regards,

Neil
Attachments
Variable size_20270_image001.gif
Variable size_20270_image001.gif (15.96 KiB) Viewed 47473 times
Last edited by Neil on Wed Mar 03, 2004 7:42 am, edited 1 time in total.
lperepol
Senior Member
Senior Member
Posts: 30
Joined: Thu Feb 26, 2004 12:50 pm
Location: Castlegar, BC, Canada
Contact:

Perhaps I am indeed a fool.

Post by lperepol »

Perhaps I am not confident that I can choose stocks in the order (in respect to time) that maximizes E log Xn (rate of asset growth).
Perhaps I am not confident that I can choose stocks that maximize E log Xn (rate of asset growth).
Perhaps the data I am using is not so good.
Perhaps I am risk adverse.
Perhaps the system I wrote is a piece of garbage.
Perhaps I lack objectivity.
Perhaps I trade illiquid stocks.
Perhaps my transaction costs are low.
Perhaps I am “The Timid Trader.â€
ksberg
Roundtable Knight
Roundtable Knight
Posts: 208
Joined: Fri Jan 23, 2004 1:39 am
Location: San Diego

Optimal-f stuff

Post by ksberg »

Gambler ruin does occur on the right side of the peak, but the one point to the right side and one point to left should be logically equivalent No? There is variance involved no?
lperepol, I think you've hit the nail on the head. You can take and graph what is optimal for any particular run. If you change the set of trades slightly, say by sliding the test window forward, you will get a slightly different answer for what is optimal. This kind of variance will continue as you change the set of trades. So, not only is Optimal-f the Cliff of Death, but the cliff itself keeps moving underneath your feet! Your notion of looking at variance is very pertinent.

Neil, thanks for the graphic. If you continued incrementing the bet size, you will get equity curves that are guaranteed to hit zero. I've attached a sample sizing comparison between methods applied to a simple S&P 30-minute breakout. In this example, you can see how the account blows up using the Kelly ratio. For the sample, Fractional-f was 25% Optimal-f, and Williams Ratio was 10%. I don't have notes on what I used for Fixed Fraction, but I recall the settings were derrived from the trade set according to Ryan Jones rules. At any rate, the point is that there will be a values right of Optimal-f that will destroy the account.
As an aside, the turtle system that we were all brought to this forum by is far from conservative. With up to 24 units at 2% each I would say that the turtles may be closer to optimal f than they admit to.
Neil, it would be highly unusual to get 24*2% risk in Turtle. That condition could only occur with 24 separate markets, precisely 12 long and 12 short, all having no correlation. In practice, scaling and correlation rules cut this number to something far less. For instance, in scaling, you'll note 4 units does not equate to 4*2% = 8%, but a maximum of 2.0+1.5+1.0+0.5 = 5%. The correlation rules limit full load to 6 highly correlated units, or 10 loosely correlated units. Note that when scaling and correlation are combined, as often happens, 6 units != 6*2% = 12%. If we had 4 units in one market, and 2 units in another where both were highly correlated, then we'd have 5% + 3.5% = 8.5%; much less than 12%. You'll find that c.f. mentions that correlation rules are very important, and often overlooked.

Cheers,

Kevin
Attachments
Comparison among sizing methods. At a certain bet size, there is a high probability of the account going to zero.
Comparison among sizing methods. At a certain bet size, there is a high probability of the account going to zero.
SizingComparison.jpg (79.3 KiB) Viewed 47741 times
Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt »

Kevin,

Just for the sake of grounding this discussion which I find hard to follow since we haven't agreed on definitions of either approach or the formulas, would you mind outlining the formulas you used to generate the above graph, what was required as inputs for each bet size type, and where you got those inputs?

- Forum Mgmnt
Neil
Full Member
Full Member
Posts: 11
Joined: Sun Dec 21, 2003 3:35 pm
Location: London

optimal? f

Post by Neil »

Kevin,

It's good to see optimal f get a good airing. I get the impression that Ralph Vince has just confused the issue. I find clarity in Van Tharp (trade your way to financial freedom, and Special report on money management) as well as Nauser Balsara's Money Management for Futures Traders.

I appreciate the maximum risk on a single market is going to be 5%. But lets look at the current situation. You could be fully loaded long in any of the soyabean complex allowing 6 units. long any of copper, silver, platinum. Corn adds another 4 to the beans (loosely correlated). Not to mention crude oil, bonds etc. Easily 12 units long say, 15% risk in total. what about short 4 units FCOJ? another 5%. We're not even fully loaded short and we've got 20% risk. Sounds like optimal f numbers to me.



Dare I say it but is this why turtle systems make so much money?

I was also under the impression that the Kelly criterion and optimal f were the same thing. I agree that optimal f floats around a bit. Which is why I am a big fan of your fractional f, to me it smacks of Vulcan-like logic, backing away from where optimal f might be floating, yet still enough risk to make healthy profits. I like 80%, it makes good money.

Also optimal f is hardly the edge of the "cliff of death". If you ignore Vince and his peculiar TWR and concentrate on hard cash, you will find that if you exceed f by 5% or so, you find yourself on the "gentle slope of less profits" The f zone on an equity chart is actually quite flat. Take a look.

Lawrence,

If you risk .001 per position on the turtle system, that equates to a total of 1% risk in the above situation. Might it be more profitable to put your money in a bank account or bonds? You certainly won't get a big drawdown tradng as you are, which unfortunately guarantees small profits. How about $900k in the bank and $100k with your futures broker at 2% a trade. I don't wish to sound harsh.

Regards,

Neil
Attachments
Size.xls
ending equity after a series of trades. risk per trade varied in increments of 1%
(15 KiB) Downloaded 2088 times
Post Reply