119
and may point the way to more robust designs. However, because white box tests require
such intimate knowledge of the system, they generally do not scale well to large applica-
tions with thousands of lines of code or spreadsheet cells. Also, white box testing will not
account for errors caused by omission.
Despite its shortcomings, white box testing is critical as it allows skilled team mem-
bers to develop test cases. Test case data should be inputs and outputs against which
future regression tests can be run for the life of the development project. The more test
cases, the higher is the quality of the prototypes, and eventually the implementation soft-
ware, and in the end the overall quality of the trading system. Any changes to the proto-
types or fully functional software must be tested to ensure “ fixed ” versions generate the
same output values as the white box test cases.
White box testing is also good for team development since it is normally accom-
plished by having a programmer and financial engineer sit at a single computer and walk
through the code in debug mode to test calculations and outputs. This type of close testing
builds trust between team members; when the team encounters complex problems down
the road, they will have working relationships to solve problems. Everyone is equal on a
team and white box testing can be an effective management tool for preventing cultural
problems. Early testing helps avoid the blame culture before it becomes a major issue in
the later stages of development, where the team cannot afford to have isolated members
who believe their way is the only way.
In the end, white box testing is only the start. For a complete examination, gray box
and black box tests are also required.11.4. STEP 4, LOOP 2: Perform Gray Box Testing with
Inputs and Outputs
All team members should test the performance of the trading system. At this point, how-
ever, the model code may very well be sufficiently complex so as to require not white
box, but gray box testing. Gray box testing is a software testing technique where testers
have limited knowledge of the internals or source code of the system, relying instead on
architecture diagrams, state-based models, interface definitions, and functional specifica-
tions. Based on what knowledge they do have, testers can derive intelligent test cases. Of
course, without knowledge of the code, the tester must deduce test cases.
For trading system gray box testing, we recommend both the trader and marketer sit
with the financial engineer and/or programmers and perform unit testing using examples
selected by the trader or marketer. This type of testing builds teamwork and understand-
ing of the system along with finding errors. Traders tend to test extreme events, such as
gap moves and highly volatile instruments. (Every trader remembers that extreme day
where they lost or made a lot of money.) Marketers also tend to test extreme cases. In
many ways, this is actually the start of value at risk testing of the system.11.5. STEP 4, LOOP 3: Perform Black Box Testing,
Documenting Results Using Inputs and Outputs
Black box testing, also known as functional testing, is a software testing technique whereby
the internal workings of the software are unknown to the testers. For example, in a black11.5. STEP 4, LOOP 3: PERFORM BLACK BOX TESTING