THE HAnDbook of TEcHnicAl AnAlysismaximizes the system’s profit and minimizes its losses. Unfortunately, it is possible
to over‐optimize a system, producing unrealistic results that would not likely be
replicated when tested using new data.
When testing a trading system, data is employed to develop the trading strategy
and test for profitability and longer term positive expectancy. The original data set
used to develop and test the strategy is called the in‐sample data. The strategy is then
back tested using past data that was not used during the development process. This
data is referred to as out‐of‐sample data. Finally, the system is tested using live data.
The process of testing the system with live data is referred to as forward testing or
walkthrough. The data used during a forward test is also referred to as out‐of‐sample
data since such data was not used during the development process. See Figure 29.2.
Generally speaking, the greater the amount of data used to back and forward
test the system, the more reliable will be the results of the tests in gauging the ex-
pectancy and consistency of performance of the system. Although it is generally
accepted that data corresponding to at least 30 trades is required for any test to
be statistically significant, it is more realistic to employ data based on at least 300
to 500 trades to determine the true longer‐term performance of a trading system.
For example, the minimum 30 trades may consist of data that is entirely related
to a series of trades that occurred during a very strong market rally. Such data
may be biased and may not account for situations where the market was bearish
or volatile. Hence testing a system under such limited market conditions will not
produce a system that is flexible and robust enough to handle changing market
conditions. Therefore, it is recommended that sufficient data be collected under a
variety of market conditions, which should ideally include:
■ (^) Ranging markets
■ (^) Strong bullish and bearish markets
■ (^) Highly volatile markets
■ (^) Markets exhibiting gapping prices
The commonly held assumption that 30 trades is sufficient to yield statistically
significant results is not true, and it varies with the stopsize. For example, a foreign
exchange system that trades with a stop size of 50 pips cannot be compared with
Figure 29.2 In‐Sample and Out‐of‐Sample Data.