Technical Trading Systems
one that has an extremely tight stop of 2 pips. For very tight stops, the probability
of being taken out by the market is significantly higher than with a system of with
a stopsize of 50 pips. As such, it will experience many more losses on average, and
this will introduce a statistical bias toward more degenerate results. Therefore, the
smaller the stopsize, the less accurate will be the assumption that 30 trades would
produce a statistically reliable result. In short, statistical significance is a function
of stopsize.
It is possible during the development process to continue tweaking the system
parameters until the system exhibits positive expectancy. In fact any trade method-
ology can be made to show positive expectancy with sufficient adjustment of the
trade parameters. This is referred to as curve fitting. To avoid or reduce the effects
of curve fitting, out‐of‐sample data is used to test the system. If the system fails to
perform well using out‐of‐sample data, it will undergo additional re‐optimization.
Unfortunately, the out‐of‐sample data would now be used re‐optimize the system.
This process gradually reduces the amount of out‐of‐sample data available, which
increases the likelihood of curve fitting with the system.
Another way to test for curve fitting is to reorder the data during the backtest.
This simply means to shuffle the data around so that the order of the trades does
not occur according to their original sequence. A robust and reliable trading system
should continue to display the same level of positive performance under such ran-
domization of trade sequences. Extreme underperformance during the randomiza-
tion of trade sequences is an indication of potential curve fitting affecting the data.
One big mistake that many novice traders commit is to back test a system by
looking for the parameter settings that produce the greatest profit, only to dis-
cover that the system fails to replicate its earlier performance when tested using
out‐of‐sample data.
For example, let us assume that a trader is testing a moving average cross-
over strategy where the system goes long every time price crosses above a moving
average and goes short when price crosses below it. Stoplosses are placed below
the moving average when the system goes long and above it when it goes short.
The trader is careful to ensure that enough trade data is tested across a variety
of market conditions to ensure that the system is robust. A test is then conducted
to identify the moving average with the lookback period that will produce the
greatest profit. Occasionally this approach may work in producing a system that
seems to display positive performance when tested with out‐of‐sample data. But
in most cases it will fail to maintain its positive performance. The main reason has
to do with the behavior of the equity curve. See Figure 29.3.
In Figure 29.3, we observe equity curves associated with moving averages
with various lookback periods tested for returns over the in‐sample test period.
The moving average associated with equity curve B displays the greatest returns
over the in‐sample test period, indicated by the equity level at point 1. Most
novice traders would therefore regard this moving average as the moving aver-
age with the optimum lookback period. Unfortunately they fail to realize that the
equity curve associated with this moving average is in fact displaying diminishing
returns over most of the in‐sample testing period. Therefore, when they subse-
quently test the performance of the moving average using out‐of‐sample data,