Trade sequencing

Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.
Post Reply
SL
Senior Member
Senior Member
Posts: 30
Joined: Tue Feb 10, 2004 7:02 pm
Location: New Zealand

Trade sequencing

Post by SL » Thu Feb 10, 2005 6:34 am

This is a subject we may have looked at before or at least danced around in other posts. None-the-less it maybe useful to gauge opinon on this.

Lets say I have a system. It trades using EOD data. Today the system presents 5 new trades. All these trades are inside the defined percentage of risk. That is the margin is less than the risk per trade in each case. But lets say the margin rules say we can only trade two of these markets out of the five that could be traded.

The question is what two markets today shall we trade from the five? What is the selection criteria for which two markets get picked out of the five. For example in TR the order of the portfolio dictates which trade gets preference. The order of the portfolio is assembled in my case as arranged in CSI.

From my testing re-ordering possible trades sets up path dependencies within the system. In general terms I can choose 5^2 (25) different combinations of trades for the 5 possible markets up for grabs. Depending on what markets are selected for trading, results could greatly vary at the end of the test period.

The next question I would put to c.f. or Dan. Could Veritrader 2.0 randomly mix the trade selection crtieria so that one could derive a standard deviation of results against any combination of tradeable markets? It would be interesting to observe the variation of outcomes against what markets were selected.

Also would it be possible to provide rules for creating a criteria for eligible markets and what is traded against a background of possible opprotunities? I believe the Turtle rules had such a critieria by using a method that picked the strongest markets over a set period. Several forum members are playing around with this concept on various paltforms already unfortunately my present platform has its limits in this regard.

Cheers

Stephen

sluggo
Roundtable Knight
Roundtable Knight
Posts: 2986
Joined: Fri Jun 11, 2004 2:50 pm

Post by sluggo » Thu Feb 10, 2005 8:43 am

The situation is even more complicated than you describe. Your mechanical trading system might present those five new trades that you mention, using Stop orders (or Limit orders). For example the Original Turtle System would definitely present you the new trades using Stop orders.

Your computer software tells you "Here are five new trades for tomorrow" and you know that you are only allowed to take two of them. But you have no way of telling how many of those five Stop orders, will actually be filled tomorrow. The market(s) could blast right past all five of your stops and you'd be filled in all five. Or the markets could stay quiet and you wouldn't be filled in any of the five. You also have no way of telling which ones of those five Stop orders will actually be filled tomorrow.

If you're using End-Of-Day data, you have no idea what time during the day the various fills occurred, nor in what order.

I recommend you figure out what you will do in real life, trading the markets with real money, when presented with this situation. Then, when you know what you actually want, ask your software vendor to write his backtest program to operate that way. I'm only guessing here but I suspect that SL doesn't yet know exactly how he would handle this situation when trading his account real-time, so he doesn't yet know what he wants backtest software to do. If I'm wrong, I apologize in advance.

verec
Roundtable Knight
Roundtable Knight
Posts: 162
Joined: Mon Jun 09, 2003 7:04 pm
Location: London, UK
Contact:

Post by verec » Thu Feb 10, 2005 7:14 pm

Not to be such a nit-picker but ...
sluggo wrote:[...]I'm only guessing here but I suspect that SL doesn't yet know exactly how he would handle this situation when trading his account real-time, [...]
Isn't SL precisely saying he doesn't know how to handle market selection in his post? And isn't he asking for methods people use when trying to select markets when faced with the situation of dropping some of them due to risk management?

Regarding the answer, I'm not there yet myself, but I would venture that unless you have strong and back-tested reasons to favor market X over market Y in such a situation, a random selection might not be the worst choice :idea:

DrHendricks
Roundtable Fellow
Roundtable Fellow
Posts: 81
Joined: Tue May 13, 2003 12:21 am

Post by DrHendricks » Thu Feb 10, 2005 11:59 pm

strong and back-tested reasons
Hmmm... sounds like a good argument for leaving it as user defined code in a trading blox rather than something Veritrader does automatically. Another alternative to randomization would be to rank them based on some measure of technical strength, RSI, Efficiency Ratio, etc.

verec
Roundtable Knight
Roundtable Knight
Posts: 162
Joined: Mon Jun 09, 2003 7:04 pm
Location: London, UK
Contact:

Post by verec » Fri Feb 11, 2005 4:25 pm

SL, I apologize in advance if you feel like I am putting words in your mouth. I am not, but what follows is how I perceived your post, right or wrong... :arrow:

When I first read SL post I could sense some kind of emotion between the lines that I'm all too familiar with. In a nutshell: you've done your homework, you've done your research and your back-testing, you've laid down a plan and rules for risk and money management. You think that you've got the beast well cornered... and suddenly some kind of obscure thought drives its way into your mind: you forgot about that issue.

What looked rosy and promiseful, suddenly is jeopardized by this recurring thought: you missed the point, you're off target, and because, so far, your only drive is based on belief and not on actual successful trading experience, that very belief is being shaken. Maybe all the Nassim Taleb and John Allen Paulos of the world are right, after all!
Maybe this all lunacy, and only sheer luck sorts the winners from the loosers... Maybe...

If such was the case, my only advice is this: keep digging. Until such point where you can either succeed or confirm the "efficient market theory". Don't give up until you can answer that one for yourself.

Sorry if this post is a bit off-topic.

SL
Senior Member
Senior Member
Posts: 30
Joined: Tue Feb 10, 2004 7:02 pm
Location: New Zealand

Post by SL » Sun Feb 13, 2005 12:06 am

Thankyou to all that have replied to this post. Here's some more comments and thoughts on this...

I think there is a subtle but important difference between what I am refering to and what Sluggo is refering to. One is the issue of skipped or missed trades, the other is a decision process that lay outside of the markets control and within your own system design.

In my case there is a hidden rule-set I have dug up, which is not within my easy control. I have a system that I believe is robust etc. I back test it a zillion times. But the markets used for back testing are always ordered the same each run. What this lends itself to is a testing bias. For example AD is at the top of the list so this market for better of worse is going to get preferential treatment over JY or W etc. So what in effect occurs is that AD swallows some or all of the margin before JY or W can get a crack at it some of the time. If I move W to the head of the que when the program starts, what difference would this reshuffle do to the outcome of future trades and ending equity and drawdown?

All three points below I believe have a similar effect on system performance in the longer term because of the way they alter the path of subsequent trades.

1. Skipped trades might occur becuase the margin requirements for the trade on offer are outside your risk defined boundaries. This is almost certain to happen unless you are trading large amounts of equity.

2. The market blows through your stops. That's the market for you. You have little control over this one.

3. You have several opportunities on offer within your risk critieria but you can't trade all of them. So you are left with defining a new rule set over which of the trades to take. This action is independent of the market.

All of these activities result in missed opportunities and missed disasters depending on how you look at it. For example the market that was skipped by whatever means might have been the on the wrong side of a huge move that would have severly dented your account. So chance always plays a role. The money the system collects or looses on this trade(s) sets in motion what can occur on the subsequent trades.

Even if one has suffcient equity to trade all opportunities on a given day, you might decide that you will only trade the 3 strongest based on some critieria, RSI, DMI or whatever. You might want to test how your ranking critiera compares to a purely random portfolio selection as a benchmark.

Cheers

Stephen

verec
Roundtable Knight
Roundtable Knight
Posts: 162
Joined: Mon Jun 09, 2003 7:04 pm
Location: London, UK
Contact:

Post by verec » Sun Feb 13, 2005 4:45 am

An alternative method (and it might be what Sluggo was hinting at) is to assume you are to trade each market at the very instant the signal is issued by your system. Obviously, for back testing purposes, this requires that you have access to intraday data, but this is probably the only method that doesn't introduce additional bias into the selection of which markets to drop:

first come, first traded, in signal time of arrival order doesn't need any other information, and doesn't favor any market more than any other.

SL
Senior Member
Senior Member
Posts: 30
Joined: Tue Feb 10, 2004 7:02 pm
Location: New Zealand

Post by SL » Sun Feb 13, 2005 8:03 am

Thanks Verec you have a relevant point. Using EOD data we cannot know which trade would have occured first. We take the first trade, but we do not know what the first trade was. The problem is not the quality of the EOD data but that it lacks the neccessary information to determine which trade would have occured first.

What I think is neccessary in back testing, is to compensate for the inadequacy of the EOD data by altering the order the trades are taken and looking at the range of possible outputs. From that we can gauge how sensitive the system is to the order of trades and whether the pass was a fluke series of trades that lead to a particular figure. I am not sure this is quite what an MC simulation does. I think and correct me if I am wrong but MC when typically applied to trading system testing jumbles up trades after they have been processed. I have my doubts about this method.

(I am looking for a suitable metahpor here.) Individual markets have a these kind of 'strange attractors' to each other that seem to bind them together. One market will move and then another in some strong or weak correlated fashion. At least by reordering the trades at around the same point in time you are preserving as best possible the relative correlations (attraction properties) that is not present in typical randomised MC analysis at the output from a single series of runs. Markets aren't random but appear to contain these rubber band like properties to each other that should be preserved during testing. IMHO MC simulations are bogus since they do not preserve the intermarket relationships on which markets move. The outputs of standard MC simulations totally ignore this fact. Is this making any sense? I would rather jumble the order of the inputs not jumble the outputs in testing to preserve these relationships to determine a range of possible outcomes.

Perhaps I am straying into new territory with this.

Stephen

Forum Mgmnt
Roundtable Knight
Roundtable Knight
Posts: 1842
Joined: Tue Apr 15, 2003 11:02 am
Contact:

Post by Forum Mgmnt » Sun Feb 13, 2005 9:40 am

This is just one of the issues that confronts those who are trying to test using insufficient information. As discussed, end-of-day data (actually anything but actual tick data) is insufficient to determine the order of events within the day (or bar for intraday data).

The specific problem of trade ordering is just one instance where this lack of information results in a mismatch between what is simulated in a back test, and what actually happens in real life.

In this instance, if one tests over a long period of time, I haven't found significant differences between random trade ordering and one based on a fixed order (whether alphabetic or some other scheme).

We have the capability to order trades using an abitrary mechanism in VeriTrader 1.6.2 and 2.0, so when those ship you'll be able to try out other trade orderings. This capability was put in place to affect the order of the markets for order generation. The default ordering numbers are the times of the market openings, this is probably the "best" order if one wants to get as realistic as possible given the constraints of insufficient information.

This new feature could be extended to offer a pseudo random daily ordering since we already have the code for sorting the markets according to a numeric ranking. Random ordering would be something that one would likely only use occasionally, as it would likely cause repeated executions of the same test to have differing results. There are ways around this since computer-generated random numbers aren't truly random, and they can be repeated if one starts with the same numeric value (known in the business as the random "seed").

From my testing, I believe this is really one of those areas where the issue is not knowing whether there is an effect, rather than there being some large problem with a fixed order based approach; more of a theoretical problem than an actual one. So any new feature would likely be used only occasionally as a verification step.

What one is really interested in, is the answer to the question of whether or not the market order affects results signficantly. Once one arrives at an answer of "no" for their particular system, the question loses it's attraction. It could be that for certain types of systems there might be a significant difference, but I haven't found one ofr medium or long-term trend following.

- Forum Mgmnt

Post Reply