188 CHAPTER ◆ 2 0 Plan and Document Technology Specifications
● Warranty replacement for nonconforming products.
● Existence of complete documentation.
● Vendor experience, reputation, and viability and references.
● Support and effectiveness of response to problems.
● Ability to comply with laws and regulations.The advantage of custom-made systems is that products are optimized for the application
at hand, which is particularly important for high frequency trading systems. Unfortunately,
design and development of one-off hardware and software is costly and time-consuming.
As prebuilt tools become more sophisticated, it will become more and more difficult to compare
the performance of various components simply by reading their specifications. Tests must
be developed that allow for comparison across multiple vendors. For example, the team may
purchase a statistical package from a third-party vendor, but calculations may be different
from their own, and assumptions may not be clearly defined. Doing the hard work of build-
ing benchmarks and a system that can be benchmarked against other people ’ s assumptions
certainly pays dividends.
Vendors who sell trading system components have a long history setting up their
tests to show unrealistically high performance that may not be replicated in real usage.
Vendors of course only report those tests that show their components in the best light.
They also have been known to misrepresent the significance of underlying assumptions,
again to show their products ’ best side under specific configurations.
We recommend that product teams establish benchmarks, and take claims regard-
ing performance (particularly those provided by vendors themselves) with ample grains
of salt unless the underlying assumptions are explicitly provided. According to ISO
9001:2000 clause 4.1: “ Where an organization chooses to outsource any process that
affects product conformity with requirements, the organization shall ensure control over
such processes. ” Amanat Hussain adds that “ the overhead of developing, supporting and
enhancing an in-house system needs to be assessed against the cost, functionality, tech-
nology, flexibility, and the level of support available from third-party vendors. ” Trading
and money management firms seeking to gain a competitive advantage understand that
proprietary systems enable them to differentiate their trading/investment systems and that
when buying third-party systems, platform compatibility and integration issues must be
considered carefully.^3
Technology benchmarks furthermore give developers the ability to measure perfor-
mance attributes and make trade-offs in buy versus build decisions. For example, a bench-
mark could extract the key algorithms that contain the performance generating aspects of a
system. Running the much smaller, prototyped benchmark can give clues about the vendor ’ s
assumptions and furthermore how to improve performance.20.4. Hitting Deadlines
Development teams naturally want to discuss their elegant technology and the significant
progress they are making. But, teams also need to frankly discuss development risks with
both the product team and top management. Risk factors might involve cost and schedule
overruns, technological feasibility issues, performance issues, or other factors that may
cause the reality of the final implementation to be lower than expectations. Development
team members may feel threatened by conversation that shines light on solutions that did
not work or estimates they wish to reconsider. But, if both the development team and