Palgrave Handbook of Econometrics: Applied Econometrics

1294 Testing Econometric Software

is important because econometric software does not always produce accurate
answers. Sometimes the inaccuracy can be traced to a bug, an incorrectly written
line of code; at other times the algorithm is simply not up to the task of calcu-
lating the correct answer (e.g., as will be discussed in section 28.2, do not use the
“calculator formula” to compute the sample variance, and, as will be discussed in
section 28.3, do not invert the(X′X)matrix when calculating the least squares
regression coefficientsβˆ; neither of these approaches is numerically sound for eco-
nomic data). Unless the software is tested, the user has nothing more than mere
hope or the software developer’s blandishments on which to base his or her con-
fidence that the computer program’s output is correct. As we shall see, such blind
confidence is misplaced.
Forty years ago, back when mainframe computers took their input from lines
of code on punchcards, and a program consisted of a stack of punchcards, Lon-
gley (1967) worked out by hand the solution to a linear regression problem with
a constant and six independent variables, for 16 observations, and he did so to
several digits of accuracy. The dependent variable was “total employment” and
the independent variables were: implicit price deflator, gross national product,
unemployment, size of armed forces, population, and time. When he compared
his hand-calculated results to those from the computer programs, he found wildly
disparate results. As he put it (ibid., p. 822):

Test problems were run on available programs for use on the following electronic

computers: IBM 1401, 7070/7074/ 360 model 50, 7090/7094, and General Elec-

tric 235. With identical inputs, all except four programs produced outputs which

differed from each other in every digit.

To convey the essence of what Longley found, below we reproduce (to only three
decimals) some of the regression coefficients produced by four computers/software
packages from his Table 10 in our Table 28.1.
Within several years, most (if not all) linear regression programs could meet the
so-called “Longley benchmark.” In the present day, it is almost unheard of for
any program not to meet the Longley benchmark. One may think that several
different packages giving several different answers to the same problem is a relic of
the “old” days, but it is very much a problem that still persists. In 1999, McCullough
and Vinod (1999, p. 534) published three widely divergent sets of full information

Table 28.1 Some of Longley’s regression results

Defl. GNP Unem. Mil. Pop. Time Constant

Correct 15.026 −0.036 −2.020 −1.033 −0.051 1829 − 3482258

IBM 7074 −36.187 0.059 −0.593 −0.607 −0.344 183 − 269126

G.E. 235 13.944 −0.346 −2.00 −1.028 −0.055 1809 − 344297

BMD 27.082 −0.032 −1.946 −0.987 −0.013 1653 146264

NIPD −0.039 0.536 −0.068 −0.064 −3.44 31.5 −5.22