(^162) 97 Things Every Programmer Should Know
Test Precisely
and Concretely
Kevlin Henney
iT iS iMPORTANT TO TEST for the desired, essential behavior of a unit of code,
rather than for the incidental behavior of its particular implementation. But
this should not be taken or mistaken as an excuse for vague tests. Tests need to
be both accurate and precise.
Something of a tried, tested, and testing classic, sorting routines offer an illus-
trative example. Implementing a sorting algorithm is not necessarily an every-
day task for a programmer, but sorting is such a familiar idea that most people
believe they know what to expect from it. This casual familiarity, however, can
make it harder to see past certain assumptions.
When programmers are asked, “What would you test for?”, by far and away
the most common response is something like, “The result of sorting is a sorted
sequence of elements.” While this is true, it is not the whole truth. When
prompted for a more precise condition, many programmers add that the result-
ing sequence should be the same length as the original. Although correct, this is
still not enough. For example, given the following sequence:
3 1 4 1 5 9The following sequence satisfies a postcondition of being sorted in non-
descending order and having the same length as the original sequence:
3 3 3 3 3 3Although it satisfies the spec, it is also most certainly not what was meant!
This example is based on an error taken from real production code (fortu-
nately caught before it was released), where a simple slip of a keystroke or a
momentary lapse of reason led to an elaborate mechanism for populating the
whole result with the first element of the given array.
The full postcondition is that the result is sorted and that it holds a permuta-
tion of the original values. This appropriately constrains the required behavior.