Testing Results Question
Testing Results Question
I've been testing a pretty simple system over about 30 years data in 65 different worldwide futures. I've been using in sample and out of sample data to test for robustness, etc.
My question is that my results I feel are decent (for me) but if you look at the profit contribution all of the winners and losers are within 1R. In addition, my win/loss rate is nearly 50% which is a red flag for me as I would expect something more like 30/70 losses to wins or similar. So basically I am getting profitable results but not the big huge winners (10-15R).
It seems that if I'm happy with the results then it shouldn't matter but I have to say I'm a bit surprised there weren't bigger wins.
To be clear this is a long only system with no profit target.
Would like to hear some thoughts on this and maybe some ideas how I'm missing these bigger moves or whatnot.
'Preciate it.
My question is that my results I feel are decent (for me) but if you look at the profit contribution all of the winners and losers are within 1R. In addition, my win/loss rate is nearly 50% which is a red flag for me as I would expect something more like 30/70 losses to wins or similar. So basically I am getting profitable results but not the big huge winners (10-15R).
It seems that if I'm happy with the results then it shouldn't matter but I have to say I'm a bit surprised there weren't bigger wins.
To be clear this is a long only system with no profit target.
Would like to hear some thoughts on this and maybe some ideas how I'm missing these bigger moves or whatnot.
'Preciate it.
- Attachments
-
- EquityCurve.png (35.94 KiB) Viewed 11195 times
-
- ProfitContribution.png (13.7 KiB) Viewed 11193 times
-
- Win_LossStats.png (24.98 KiB) Viewed 11193 times
I have and indicator that when it reaches a threshold it goes long and then exits when it reverses to another level.Robwynge wrote:Well, how are you instructing the system to exit trades? And when you look at the trades tab, is the system exiting based on how you expected it to?
Double checked and yes it is entering and exiting how I'm instructing.
Right? That's why I posted this question. It's not a pre-supplied system but one I picked up through the Blox Marketplace and combined with some pre-supplied blox.rossb34 wrote:It seems odd to me that all winners are less than 1R for a system with no profit target. Are you using a pre-supplied system or a custom trading system? Another thing that jumps out is the Expectation is 0.0 and Average Risk Percent is N/A. There could be a bug in your code, data, or both.
My entry/exits look ok but just doesn't make sense why everything is so clustered around the mean so to speak.
So then I am confused about why you are confused.HWG wrote:I have and indicator that when it reaches a threshold it goes long and then exits when it reverses to another level.
Double checked and yes it is entering and exiting how I'm instructing.
If it's doing what you want by following the signals of your indicator faithfully, is your indicator such that it only has small winners? If not, and your indicator sometimes triggers exits after small moves, and other times triggers exits after large moves, then clearly something must be wrong with your code. Maybe soemthing is wrong with the risk manager forcing you out of trades when your risk gets "too large"? Just throwing something out there...
Right, its not that its not working properly its just not what I expected so trying to figure out why. The more interesting part is what rossb34 said where the expectation is 0.0 and Average Risk Percent is N/A. That doesn't seem right to me. I will go through checking again. Cheers.Robwynge wrote:So then I am confused about why you are confused.HWG wrote:I have and indicator that when it reaches a threshold it goes long and then exits when it reverses to another level.
Double checked and yes it is entering and exiting how I'm instructing.
If it's doing what you want by following the signals of your indicator faithfully, is your indicator such that it only has small winners? If not, and your indicator sometimes triggers exits after small moves, and other times triggers exits after large moves, then clearly something must be wrong with your code. Maybe soemthing is wrong with the risk manager forcing you out of trades when your risk gets "too large"? Just throwing something out there...
With an average holding period of ~198 days and an average trade return of ~1.75%, I would venture there's a bug in the code, or he's betting so small he can't fully capitlize on the moves.rossb34 wrote:It seems odd to me that all winners are less than 1R for a system with no profit target. Are you using a pre-supplied system or a custom trading system? Another thing that jumps out is the Expectation is 0.0 and Average Risk Percent is N/A. There could be a bug in your code, data, or both.
I'm using starting test equity of 1mm and 2% risk per trade so seems big enough to make a difference. I did go back and look to see what percentage wise some of my biggest winners and losers were. Had some in the 60-70% range and a bunch in the 30s. Biggest losers were in the 5s and a few in the 6-8% range so seems as if some of those would be more than 1R or maybe that 1R is not registering properly.Aaron01 wrote:With an average holding period of ~198 days and an average trade return of ~1.75%, I would venture there's a bug in the code, or he's betting so small he can't fully capitlize on the moves.rossb34 wrote:It seems odd to me that all winners are less than 1R for a system with no profit target. Are you using a pre-supplied system or a custom trading system? Another thing that jumps out is the Expectation is 0.0 and Average Risk Percent is N/A. There could be a bug in your code, data, or both.
This is what I'm using for risk:
estimated_risk = 3.0 * MAX(myATR, (5.0 * instrument.minimumTick))
ncars = riskPerTrade * system.totalEquity / (estimated_risk * instrument.BigPointValue)
order.setQuantity(ncars)
At least one of the presupplied systems that ships with Trading Blox software, exhibits the same behavior. See figure below.
If anybody is really interested, I suppose they can try all eleven presupplied systems, one by one, to find out which of them behave this way.
If anybody is really interested, I suppose they can try all eleven presupplied systems, one by one, to find out which of them behave this way.
- Attachments
-
- Presupplied_with_Blox.png (23.99 KiB) Viewed 11085 times
Correct, no stops. System exits when the indicator changes a certain way.LeviF wrote:I assume you are not using (or properly using) stops. You cant calculate R-multiples, expectancy, avg risk, etc without stops.
So if that's the case then it can't really calculate the R-multiples as far as profit contribution right? That makes sense as I said before I have some winning trades in the 30-70% range and losing trades in the 5-8% range so this would appear to be further out on the curve then what its showing.
-
- Roundtable Knight
- Posts: 138
- Joined: Wed Nov 10, 2004 4:36 pm
I assume you are not using (or properly using) stops. You cant calculate R-multiples, expectancy, avg risk, etc without stops.
I'm afraid I disagree with these two statements; it might not be possible for TB to calculate R-multiples, expectancy, average risk etc. without a hard coded stop, but these calculations can be done if you have access to the history of the losses. These losses will form a distribution and thus measurements can be taken:If the exit (stop) is undefined when (before) an entry signal is triggered, then you have no means to define the risk on the trade or size the position based on that risk.
- the average loss can be taken as 1R
the average loss plus 1/2 standard deviations can be taken as 1R
if the loss distribution is highly non-normal, bootstrap it and take the bootstrap average, with or without +1/2 bootstrap standard deviations as 1R
take a suitably high nth percentile value as 1R
some other "robust" measure of the distribution can be taken as 1R
My point about having no means to define the risk at entry was in regard to this specific case where the OP is trying to evaluate a system and seeing strange results. No doubt you can come up with many different possibilities to "create" an initial exit to apply to each entry as you point out. My point was that he/she needs *something* defined as an exit for each occurrence of an entry.
HWG wrote:This is what I'm using for risk:
estimated_risk = 3.0 * MAX(myATR, (5.0 * instrument.minimumTick))
ncars = riskPerTrade * system.totalEquity / (estimated_risk * instrument.BigPointValue)
order.setQuantity(ncars)
Chuck B wrote:No doubt you can come up with many different possibilities to "create" an initial exit to apply to each entry as you point out. My point was that he/she needs *something* defined as an exit for each occurrence of an entry.
I am pleased to present some good news: Trading Blox has a feature which makes this possible. In fact, easy. For those with access to the "Customers Only" section of the website, see (THIS thread) . HWG can command Trading Blox software to use his variable "estimated_risk" in its risk calculations, including the R-multiple calculations, expectancy calculations, etc. Without placing stop orders in the market. It's exactly what you want, and mirabile dictu, it's exactly what MS-DOS "Trading Recipes" software offered in 1997 with its keyword "SYSTEM" (later replaced by "MANAGER[n]").
Hmmm..thanks Sluggo. The Entry Day Replacement is a % correct? I changed it to -1% however didn't see any change as far as calculating the risk and R-Multiples. Will keep poking around unless I'm going in the wrong direction.sluggo wrote:HWG wrote:This is what I'm using for risk:
estimated_risk = 3.0 * MAX(myATR, (5.0 * instrument.minimumTick))
ncars = riskPerTrade * system.totalEquity / (estimated_risk * instrument.BigPointValue)
order.setQuantity(ncars)Chuck B wrote:No doubt you can come up with many different possibilities to "create" an initial exit to apply to each entry as you point out. My point was that he/she needs *something* defined as an exit for each occurrence of an entry.
I am pleased to present some good news: Trading Blox has a feature which makes this possible. In fact, easy. For those with access to the "Customers Only" section of the website, see (THIS thread) . HWG can command Trading Blox software to use his variable "estimated_risk" in its risk calculations, including the R-multiple calculations, expectancy calculations, etc. Without placing stop orders in the market. It's exactly what you want, and mirabile dictu, it's exactly what MS-DOS "Trading Recipes" software offered in 1997 with its keyword "SYSTEM" (later replaced by "MANAGER[n]").