117
with known calculated values. In some cases, however, there may be no known comparable
calculation. In nearly all spreadsheets, calculations are extended well beyond what has
been done previously with manual calculations. In such cases, there is no oracle other than
the spreadsheet calculations themselves. This lack of a readily available oracle is the most
serious problem in spreadsheet execution testing. Without a strong and easy-to-apply oracle,
execution testing simply makes no sense for error-reduction testing. A major problem in
finance is that much of the research is correct only for the limited sample data presented and
may be incorrect over larger, real-world samples. Also, many papers deliberately withhold
full documentation of equations to keep proprietary information secret. In such cases, the
team has no choice but to build its own test cases and oracles.
Regression testing is attractive for spreadsheet execution testing. Regression testing is
undertaken after modification to a spreadsheet to ensure that the spreadsheet still works
properly in parts where no changes were made. In a regression test, suites of input values
previously used in the model are reentered and the bottom-line, output figures are compared
to the values experienced before the spreadsheet was changed. In regression testing, the
fact that the only realistic oracle is the spreadsheet itself is not a problem. Spreadsheet
testing is important, because the spreadsheet (or model in other software) becomes the
oracle for later stages of development in K|V.
In a spreadsheet logic inspection, team members examine all formula cells. In con-
trast to auditing, which frequently focuses only on the parts of the spreadsheet that seem
risky, logic inspection is comprehensive. Logic inspection at the prototyping stage, before
conversion to code begins in later stages, allows errors to be detected again when the cost
of fixing them is small. In software development, Fagan 6,7 developed the code inspection
methodology for IBM.
Although there are several variants, we will discuss Fagan inspection as a basic model
for spreadsheet logic inspection. Fagan mandates team inspection rather than individual
inspection, because in several laboratory experiments, individuals typically catch fewer
than half of all errors. Having multiple inspectors raises this detection rate substantially.
Fagan ’ s inspection process has seven steps:
● Planning. Preparing materials, obtaining participants, and scheduling meetings.
● Overview meeting. Introduces the software, lays out roles, and describes the process.
● Preparation. Inspectors, working alone, examine the spreadsheet. The goal is not
to identify errors but to understand the software module.
● Inspection meeting. The goal is to detect errors. There should not be discussions
beyond those needed to identify and clarify errors. Should be limited to no more
than two hours to prevent loss of diligence.
● Process improvement. The inspection also should provide feedback to the firm ’ s
inspection process guidelines. Each inspection must generate statistics on time
spent, errors found, and error severity.
● Rework. Fixing the software is done after the meeting.
● Follow-up. To ensure that changes are made appropriately.The need for team logic in spreadsheet testing has also been demonstrated in the labora-
tory. Panko shows that gains from team inspection are unsurprising in view of teamwork ’ s
long-proven ability to reduce errors more than individual work.^8
Code inspection does not entirely eliminate errors; error-reduction methods never
eliminate all errors. Typically, a round of team-oriented logic inspection is likely to
eliminate 60–80% of all errors. That is why we condone iterative development. The
11.2. SPREADSHEET TESTING