1314 Testing Econometric Software
Table 28.7 How close to the benchmark?Package Tolerance μφ θBenchmark – 17.093752390 0.90658703600 0.56880910281
Package X Default 17.09375129 0.90658475 0.56880310
Package X 1E-6 17.09375234 0.90658690 0.56880876
Package X 1E-8 17.09375234 0.90658690 0.56880876
Package X 1E-10 17.09375234 0.90658690 0.56880876Table 28.8 CLS benchmark – standard errors (constant omitted)Package φθBenchmark 0.045388753586 0.086811221485
Package X 0.0457 0.0874
Package Y 0.0547 0.1186
Package Z 0.0433 0.0894digits. How much more accuracy can be squeezed out of a package by varying the
default options? We consider only the case of Package X.
The primary option in this case is the convergence tolerance which, by default,
is set to 0.00001, i.e., 1E-5. What this tolerance controls is unknown, because the
package’s extensive documentation does not say! (This is typical of econometric
software packages.) Let us vary this convergence tolerance nonetheless.
As can be seen in Table 28.7, tightening up to 1E-6 yields a small improvement in
about the sixth digit (which is negligible since the difference between the bench-
mark with 50 digits and the benchmark with double-precision occurs in about the
same place), and no further improvement occurs with more tightening.
We now turn to the question of standard errors. Since all packages do not use the
same parameterization for the constant term, the standard errors thereof are not
directly comparable and are omitted. NAM noted that even when the parameters
were the same, different standard errors could be observed, and we find the same
with our three packages, as seen in Table 28.8. There are many possible sources for
this (see McCullough and Renfro, 2000, for a discussion); here we mention just one.
There is no single method for computing standard errors for nonlinear estimators;
the product of the gradient (recommended by BJ, as discussed at the very end of
sub-section 28.2.2), which is used in our benchmark program, the inverse of the
Hessian, and the information matrix are all prime candidates.
Though the documentation for Package X gives no indication of how its stan-
dard errors are computed, we can see that Package X probably uses the product
of the gradient method. Similarly, the documentation for both Packages Y and Z
are silent on this important point. We have no idea whether these standard errors
are incorrect or based on some other approach, e.g., inverse of the Hessian or the