Unbeknown to us, it’s just behaving as it always has and should. We just happened
to do our research on a set of markets and a period that wasn’t representative for
the types of results this system really produced.
Remember that a one standard deviation interval around the mean should
contain approximately 68 percent of all observations, and a two standard deviation
interval around the mean should contain approximately 95 percent of all observa-
tions. Now we can twist that reasoning around a little bit and say that we can be
95 percent sure the true average value of the variable we’re testing is between 2
standard errors around the just-computed average of averages. We do that using
the following three-step process:
First, calculate the average profit per trade for each market tested (this is the
variable we’re investigating), an average for all those averages (this is the average
to estimate the true average), and the standard deviation for those averages, using
the standard deviation formula.
Second, use the standard deviation just calculated to calculate the standard
error for all averages of averages. For example, for a standard deviation of $700
over thirty observations (markets tested), the standard error will be $128 [700 /
Sqrt(30)].
Third, multiply the standard error by 2 (for two standard errors, to produce a
95 percent confidence interval), and add and deduct the product to and from the
average profit per trade, calculated earlier as the average of several market aver-
ages. For example, with an average profit per trade of $300 and a standard error of
$128, we now can say that we can be 95 percent sure that the true (always unknown)
average profit per trade will fall somewhere in the interval $244 to $756.
Because the true standard deviation will always be unknown, the factor we’re
multiplying the standard error by actually should be slightly higher than the stan-
dard error interval for which we’re creating the confidence interval, depending on
the number of observations we’re working with (30, in this case). You can look up
the exact value in a so-called t-tablein any statistics book. However, because 30
to 60 observations is considered to be a fairly high number of observations, this
number will be very close to the standard error value anyway (2.0423 for 30 obser-
vations).OTHER STATISTICAL MEASURES
Another way to make sure that a system is not too dependent on any outliers is to
exclude them when you are calculating the average return. You can do this in Excel
with the trimmeanfunction, which excludes a certain percentage of observations
from each end of the distribution. Ideally, when building a trading system, we prefer
the trimmean to remain above zero, so that we make sure that the bulk of our trades
behave as they should. If the trimmean is larger than the average return, it indicates
that the positive outliers are larger than the negative outliers, and vice versa.30 PART 1 How to Evaluate a System