121
the risk of the system and the market. As before, checking the performance of the sys-
tem will prevent additional time and resources from being spent on unprofitable projects.
The team may need to loop back to the initial research stage and reassess the quantitative
methods and algorithms after the onset of trading.
If trading does occur, we recommend that for systems consisting of spreadsheet pro-
totypes the product team at a minimum employ Tom Grossman ’ s six application develop-
ment features:● Protect source code from user changes.
● Prevent user customization.
● Restrict linking user spreadsheets into the application.
● Prevent users from sharing the application.
● Provide easy control of protection features using a well-designed user interface.
● Prevent user tampering with protection features by employing passwords.^9Following these guidelines will at least make it very difficult for users to mess around
with the code and introduce errors or untested modifications.11.7. Performance Runs and Control
Performing multiple test runs of a trading/investment system can lead to confusion about
which versions generated which results, and which conclusions depended upon which
assumptions. Normally, programmers use a tool, such as Microsoft SourceSafe, to control
versioning. However, when using prototyping languages, the team cannot easily use these
types of tools, so the team must control versioning itself. Over the successive tests, prod-
uct teams should generate individual run packs, computer folders with unique names for
each run. Included in the run pack should be a copy of the prototype software and data
used to generate the run, and a run control document, containing:● The unique run number and date/time.
● A summary of run pack contents, signed off on by all team members.
● Reports, graphs, tables of trading system performance results.
● Details of key assumptions.
● Recommendations for further development.11.8. Summary
K|V delivers rudimentary unit prototypes of the trading system within the first few weeks
and then strives to continue to deliver systems of increasing functionality. Traders or seed
capital providers may in theory choose to put these systems into production if they think
they are worthy. Research shows that skimping on testing leads to errors and failure. The
goal of the iterative process is to deliver tested, working prototypes for use as oracles,
gold standards, and run packs for regression testing of later implementations.
Prototypes will indicate what data to collect. Teams collect that data and analyze it
to validate the continued suitability and effectiveness of the trade selection algorithms,11.8. SUMMARY