28

Testing Econometric Software

B.D. McCullough

Abstract

The first part of this chapter is a non-technical survey of the relatively sparse literature on testing
the accuracy of econometric software. Accuracy is primarily assessed by taking a test problem, with
known inputs and outputs, giving it to the software, and comparing the software’s output with
the output of the test problem. We discuss the various types of tests (introductory, intermediate,
and advanced) and the types of errors that these tests have uncovered. The reader is directed
to specific resources for further information. The second part, which is technical, constructs a
test problem (i.e., benchmark) for autoregressive moving average (ARMA) estimation. In 1994 it
was reported in the literature that different packages give different answers to the same ARMA
estimation problem. To date, this open problem has been unresolved. We provide benchmarks for
conditional least squares and unconditional least squares ARMA estimation.

28.1 Introduction 1293
28.2 Computer arithmetic 1296
28.3 Introductory tests 1297
28.4 Intermediate tests 1299
28.4.1 StRD 1300
28.4.2 Random number generators 1302
28.4.3 Statistical Distributions 1304
28.5 Advanced tests 1305
28.6 Benchmarks for ARMA models 1306
28.6.1 Definitions and notation 1307
28.6.2 Calculation of derivatives 1309
28.6.3 Conditional least squares 1311
28.6.4 Unconditional least squares 1315
28.6.5 Final thoughts on ARMA benchmarking 1316
28.7 Conclusions 1316

28.1 Introduction

McCullough (2000a) rhetorically entitled an article, “Is it safe to assume that soft-
ware is accurate?” The answer, of course, is “No.” Testing econometric software

1293