16 CHAPTER ◆ 2 Key Concepts and Definitions of Terms
winner of the section as the winner overall for, say, the past year. This process is
repeated year after year with a new winner in each.
In sailboat racing, the sailors that win their section are considered to be excel-
lent sailors; the sailors that win overall are just lucky. (Incidentally, this is why the
Olympics has very stringent rules on the boats, crew weight, and sail material, and
runs multiple races before declaring a winner.) The same thinking should hold for fund
managers. Applying quality by way of our methodology will not necessarily make
your trading/investment system the winner overall, but over time it will make you one
of the rare traders or money managers that consistently beats your sector benchmark.
2. Process benchmarking is, according to the American Society for Quality, “ the
search for best practices, the ones that will lead to superior performance. ”^2 This
definition requires organizations to investigate industry practices and to accurately
assess their own performance and that of their trading/investment systems. Process
benchmarking demands that firms understand the best practices of its competitors
to the extent possible and to research new ideas and methods, in order to gain and
maintain competitive superiority.So, process benchmarking is not just copying what some other firm does, it is rather a
continuous self-improvement process in which firms measure performance against that of
best-of-breed competition, determine how those competitors have achieved their perform-
ance levels, and use the information to improve the firm ’ s own strategies, operations, and
business processes. In trading/investment system design and development, we will bench-
mark four processes:● Quantitative methods, including trade selection and execution algorithms.
● Data cleaning and backtesting processes.
● Technological architecture.
● Portfolio and risk management techniques.Benchmarking is not just a buzzword. Management must actively structure benchmarking
methods to ensure thorough and accurate investigations.^3 In any case, benchmarking will
usually require several iterations to arrive at a best practice.
Process benchmarking requires a significant expenditure of resources: time, people,
and money. It is possible to expend large amounts of time in a process benchmarking
study and receive little in return for the effort. We know of trading firms that spend years
researching and millions of dollars on salaries, hardware, and software, only to end up
with nothing. The selected process benchmarking project must have the potential to result
in a return on the investment.^4
Process benchmarking focuses effort on how to improve a trading/investment system
by exploiting best practices, not best performances. A best practice may not be the same
for every system because each has its own data, technology, and underlying stochastic
processes. As trading/investment systems most often integrate off-the-shelf components,
we define component benchmarking as the method of comparing the performance of
third-party hardware or software against known best practices.
In the dynamic, fast-paced financial markets, finding and adapting best practices is no
longer an option; it is a necessity for firms that intend to survive.^5 A sought-after trading
algorithm may be unique; the true best practice is unknown, maybe even unknowable.
Nevertheless, we will use the term to mean the continuous search for better practices that
will yield a competitive advantage.