Reverse Engineering System Parameters

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
ATB
Contributor
Contributor
Posts: 1
Joined: Fri Apr 29, 2011 5:16 pm

Reverse Engineering System Parameters

Post by ATB »

Does anybody have any experience reverse engineering system parameters given an input signal (price data) and an output signal (equity curve data)?

If I had at my disposal every possible output signal (equity curve) that could ever be generated for trading end of day Gold futures (there's an infinite number of possibilities, but lets assume), does anybody know anything about reverse engineering and generating trading system parameters using the end of day price data as an known input signal and a randomly picked equity curve as a known output signal?

It's a complicated problem no doubt, but I thought somebody here might have some experience doing something like this or might be able to provide a lead into how to go about accomplishing something like this.
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

Most folks on this site plot portfolio level equity curves, because their systems trade several instruments at once -- a "portfolio" of instruments. These equity curves include the profit and loss contributions from simul-trading gold and soybeans and 5 year treasury notes and cocoa and Australian dollars and feeder cattle and reformulated blendstock oxygenated gasoline, all at the same time.

So I think if you seek to deconstruct the equity curve from a one-instrument portfolio (gold only) then the portfolio trading / testing folks you meet here, might not be the ones with the experience you seek.

Finally, I suspect you may be too pessimistic when you suggest there are an infinite number of possible equity curves. Prices are quantized (the quanta are called "ticks") and each historical price bar has got a finite number of prices (ticks) between the high of the bar and the low of the bar. If the total number of ticks from low to high on bar number "j" is N[j], then the total number of possible trades on that bar is 1 + (2*N[j]) which is obviously a finite number.

The system can sell at any one of the N[j] ticks, the system can buy at any one of the N[j] ticks, or the system can do nothing: N[j] + N[j] + 1 different actions. For each bar there are a finite number of possible actions (trades), and the total number of bars is finite, ergo, the total number of possible trades across all bars is also a finite number. Very large, but finite. Conclusion: the number of possible equity curves is infinitely smaller than infinity :)

---------


edit added after lunch -- Since the number of possible trades is finite, the straightforward approach to finding them is to write a computer program that searches through the possibilities. I bet the programmers for hire on (THIS) page could write just such a program for you in less than 50 billable hours.

The key idea is to "color" each bar of the equity curve (red=down, green=up, black=unchanged) and to also "color" each bar of the historical prices of the single instrument portfolio.

Then, start a-searchin' For each bar
  • Match the colors of the equity curve bar and the price bar. This reveals long, short, or flat.
  • Do an exhaustive search (among the finite number of price ticks) for the trade price;
  • Do another exhaustive search for the position size, BAR COMPLETE.
Repeat for each bar.

This produces a full list of all trades (half turns), where each item of the list includes
  • (BarNumber, TradePrice, Trade action (one of LE/SE/LX/SX/noAction), PositionSize).
I imagine you will find it far easier to fit system parameters starting from a tradelist, rather than starting from an equity curve.[/i]
Last edited by sluggo on Sat May 07, 2011 12:40 pm, edited 1 time in total.
babelproofreader
Roundtable Knight
Roundtable Knight
Posts: 138
Joined: Wed Nov 10, 2004 4:36 pm

Post by babelproofreader »

Whilst I think Sluggo's approach is eminently doable, I would ask to what purpose? Even if you knew the exact day and price of entries and exits you wouldn't be able to discern the logic behind them. Would you even be able to tell whether the trades were systematic or discretionary, technical or fundamental etc?
leslie
Roundtable Knight
Roundtable Knight
Posts: 135
Joined: Tue Apr 12, 2011 11:46 am

Post by leslie »

Hi ATB

Regarding the possibility of "reverse engineering system parameters given an input signal (price data) and an output signal (equity curve data)?" the answer is NO.

First the practical: if it were possible, you could obtain the daily closing prices of your favorite Mutual Fund --your favorite since it performed best according to Morning Star or Value Line-- and get their prospectus, see what they invested in, reverse engineer the equity curve (read the closing prices from the Wall St. Journal) and away you go. You can be as good as they are... Right?

Theoretical: given an input X and an output Y such that trading algorithm deterministically select action with result(Y) amounts to an informal definition of a Finite State Machine. The inputs are known as the 'control tape', applied by the Automaton to the Current State, S. Since the current state depend on the previous state, the first time you input X, the Y may be different then the second time you input X. The second time, you may get Y2.

Note that in Comp. Sci. 'deterministic' does not mean that YOU can determine it; it means that the ALGORITHM determines what to do based on *declarative* rule set.

The rule set can be complex and MAY result in infinite sub states, so for each input of the same X, you may get an infinite variety of Y's. Oh, well :-(

Also, in practical terms, experienced programmers may code algorithms MORE complex then deterministic FSM, or code algorithms that SEEM more complex -- say someone coded a Hidden Markov Chain heuristic to deal with extended runs of profits or losses. In that case the FSM *appears* non-deterministic, yet is is deterministic!

Other issues: I code most of my trading ENTRIES as FSMs, however, code most of my EXITS as Push Down Automatons. Push Down Automatons are very similar to FSM, except can 'jump' from one FSM to another.

A trading models which use Push Down Automatons for exits generally have a lovely equity curves, but impossible to decipher by looking at the results. That is just how FSMs and PDAs work.

If the trading programs is truly TRIVIAL, you can make a guess. See how Tushar Chande did it for entire portfolios (published performance of Hedge Fund Managers).

Chapter 6 "Equity Curve Analysis" in Chande, Tushar: Beyond Techincal Analysis, 2nd Edition, 2001. Wiley. (has to be 2nd Edition.)

L
rgd
Roundtable Knight
Roundtable Knight
Posts: 396
Joined: Thu Aug 10, 2006 5:21 pm

Post by rgd »

From a practical perspective, I have gotten good ideas from looking at the equity curves and/or trades of other managers. While it may be impossible to exactly reverse engineer the system, it is possible to obtain useful information from such an exercise. At the very least you might come to the conclusion that other managers are doing things which you don't understand, yet are adding value. To me, that may be the most useful of all as it serves as an example of what is possible and what may yet be discovered in the research process.
Chuck B
Roundtable Knight
Roundtable Knight
Posts: 481
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B »

rgd wrote:From a practical perspective, I have gotten good ideas from looking at the equity curves and/or trades of other managers. While it may be impossible to exactly reverse engineer the system, it is possible to obtain useful information from such an exercise. At the very least you might come to the conclusion that other managers are doing things which you don't understand, yet are adding value. To me, that may be the most useful of all as it serves as an example of what is possible and what may yet be discovered in the research process.
I know my equity curve would be very difficult to decipher the underlying trades. Entries you could figure out pretty easily I suppose (but big deal); however, exits are so multifaceted with dependencies other than the price data for a given asset (i.e. open position risk as a % of portfolio equity, group %risk, portfolio total %risk, etc). I would imagine trying to gather anything meaningful from a manager's equity curve is a total waste of time.
rgd
Roundtable Knight
Roundtable Knight
Posts: 396
Joined: Thu Aug 10, 2006 5:21 pm

Post by rgd »

Hi Chuck,

From my perspective, any exercise which can generate research ideas is not "a total waste of time".
rabidric
Roundtable Knight
Roundtable Knight
Posts: 243
Joined: Mon Jan 09, 2006 7:45 am

Post by rabidric »

Easy Tiger, i'm sure Chuck wasn't making a personal slight on what you do with your time. 8)

Besides, if an exercise that generates research ideas takes up lots of time and yields very low informational gain, then that time could probably be spent on another exercise that generated BETTER, USEABLE ideas.

maybe you could even try and look at how much time people spent on various tasks and then try and reverse engineer that to see what is most productive. :lol:

always more rabbit holes to explore.... question is will you live long enough to look through them all? The truth is you might need more time than humankind has known to date.

Time IS short it seems.

Don't waste it! lol.
Chuck B
Roundtable Knight
Roundtable Knight
Posts: 481
Joined: Thu Apr 17, 2003 6:34 am

Post by Chuck B »

Yeah, my point was really that many managers equity curves are due to risk (position) management decisions which often times are not market specific but portfolio specific. Sort of like the old days in the pits when you would hear that "so and so" was buying (selling) T-bonds as if that was useful information in some way when in fact it could be a portfolio heat decision that has nothing to do with the underlying system on a specific market. Trying to extract meaning from that action wouldn't be too useful.

For example, one of the scale out routines I've used to manage open position risk is based on a real time calculation of open risk in a specific market to the current exit (based on yesterday's closing data) along with where that exit would be on today's close if the market closed right then. When one or the other of these values exceeded my threshold for open risk as a % of portfolio equity, the position size would be reduced to get below that threshold.

Stuff like that is just littered through professional manager's equity curves along with conditional things (i.e. if the AUDJPY has just traded under yesterday's low, reduce long risk in stock futures by 50%, etc.), so trying to figure out how any given manager's equity curve was created will not likely yield a repeatable or useful system of any sort.

Just my opinions above though, so if it doesn't fit your plan, just ignore it.

Cheers,
Chuck
rgd
Roundtable Knight
Roundtable Knight
Posts: 396
Joined: Thu Aug 10, 2006 5:21 pm

Post by rgd »

Good points. Everyone decides for themselves what is worthwhile and what is not.

For me, knowing what is possible is a great motivator in my research. For example, quite a few managers post daily equity numbers. If I find my daily numbers are correlated to theirs, then, abruptly, they diverge, mine for the worse, theirs for the better. If I find this repeats in certain environments, then I have a very valuable piece of information. Whether I can engineer exactly what they are doing is not as important as what I learn as I try to engineer what they are doing. What is it about a certain environment that they are taking into account that I am not?

Another example at a broader level, most trend followers struggled in 2004-2005 (myself included), yet some found a way to generate solid returns. This alone led me to embark on a research project which yielded 3 new systems, a trend following system which buys pullbacks, and 2 pattern recognition models, all of which performed very well in 30 year backtests, and specifically in this time period.

Also, I would argue, that portfolio level risk management measures are triggered by market conditions that are very obvious.

Realistically, I am not so much trying to reverse engineer signals or systems, but concepts, approaches, and ideas. Of which, I believe, there is a finite supply that can be applied effectively to markets.
babelproofreader
Roundtable Knight
Roundtable Knight
Posts: 138
Joined: Wed Nov 10, 2004 4:36 pm

Post by babelproofreader »

One way to practise one's reverse engineering skills, or to glean new trading ideas, is to go to the Collective2 site, choose a system which interests you, download the hypothetical track record which, by the way, explicitly gives times and dates of entries and exits, read the system description for clues as to the system's approach, and then try to recreate the system.
sluggo
Roundtable Knight
Roundtable Knight
Posts: 2987
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo »

Here's a profitable system with downloadable blox source code, whose rules may not be easy to discern from studying its equity curve. Beware, it trades a portfolio of 10 futures markets simultaneously. Portfolio trading makes it difficult to tease out the individual trade entries, trade exits, and as Chuck B reminds us, trade partial-entries ("pyramiding" aka "scaling in") and trade partial-exits ("scaling out").

viewtopic.php?p=22806&highlight=oscillator#22806
Post Reply