246 CHAPTER ◆ 2 7 Define Performance Controls
27.1. LOOP 1: Benchmark Single Performance
Calculations
Determining which metrics to calculate depends on the nature of the system. For single
performance calculations, many or all of the following are important:
● Mean/median P & L.
● Average returns.
● Standard deviation of returns.
● Sharpe ratio.
● Sortino ratio.
● Percentage of winning days.
● Number of winning trades and losing trades, winning days and losing days.
● Drawdowns.
● Average holding period.
● T-statistics for trigger and filter trades for comparison of universe to selected trades.
● Information coefficient/Spearman correlation.
● Hurst index.
For SPC charting purposes, the mean value for these metrics will be that experienced
in the backtest. However, these values may be smoothed using group averages, rolling
means, and/or exponential moving averages due to the need for normality in classical
SPC. Trading/investment systems normally have substantially more output variation com-
pared to traditional SPC controlled processes in the manufacturing world.
Many of these metrics will vary wildly. The average of the sum of a large number of
independent, identically distributed random variables converges in distribution to a nor-
mally distributed random variable. So, use a cumulative sum over five or more samples.
When quality principles are applied to trading/investment system performance met-
rics, data-related problems often occur. Non-normally distributed data is a typical issue.
We often use time series data of cumulative averages to estimate process efficiency in a
trading/investment system.
The mean and standard deviation of the metric is combined with the upper and lower con-
trol limits (UCL, LCL) from the backtest with the result reported graphically. The data usually
results from the cumulative average of errors of random variation and follows a normal distri-
bution. The areas falling outside of the control limits represent a defect within the process.27.2. LOOP 2: Benchmark Attribution Calculations
Some systems have a well-defined benchmark such as the large cap, relative-value (filter-
style) system and long/short Nasdaq 100 (signal-strength style) system. Many hedge fund
managers claim they are out only for absolute return. But, every system should be com-
pared against a benchmark alternative; to beat the benchmark a trader has to place bets
that have a higher chance of winning than the bets in the benchmark.
Here is an example. It would be easy to make a lot of money on a volatility disper-
sion trade, if for some good fortune you were short volatility on companies with high
credit and long volatility on companies with poor credit during a credit crunch. The credit
market default spreads would open up for poor credit companies as people bought credit
default spreads, and credit default traders would purchase leaps in the puts to hedge their