B.D. McCullough 1301
βˆ=(X′X)−^1 X′ywill fail many of the linear regression tests. The reason is that
computing the inverse of(X′X)is like a squaring operation that produces a great
deal of cumulated rounding error, sometimes enough cumulated rounding error
to completely ruin the estimateβˆ. A better method is to use what is called the
QR decomposition, which avoids the squaring in the computation ofβˆ. If a pack-
age fails many of these linear problems, it is safe to say that the package does not
use a good QR decomposition, and almost certainly calculates the least squares
coefficients by inverting(X′X).
Most nonlinear solvers offer various options to the user: different algorithms,
convergence tolerance, etc. Two major lessons have been learned from the early
application of the StRD nonlinear suite. First, the default options for most nonlin-
ear solvers will not find the correct answer. One primary example is the convergence
tolerance. Suppose the solver is set, at default, to stop when the difference between
the sum of squares on two successive iterations changes by less than 0.001. This
might not be tight enough and the solver might give a “failure to converge” mes-
sage; it might be necessary to set the tolerance at 0.000001. For a nonlinear least
squares problem, changing the tolerance by this amount may well dramatically
alter the coefficients and markedly reduce the sum of squares. Do not rely on the
default options for nonlinear solvers!
Normally, this first problem would not be of much consequence; simply change
the options until convergence is declared. However, there is a common second
problem that greatly complicates the matter: many packages have a tendency to
stop at a point that is not a solution and claim that they have found a solution.
(How is a user to know whether his/her solver has this second problem? See whether
anyone has applied the StRD nonlinear suite to his/her software package!) If a
user happens to have such a defective solver, then a reasonable strategy is to keep
decreasing the convergence tolerance – getting a “convergence achieved message”
every time – until the user verifies that the coefficients have stopped changing,
that the solver really has achieved a stationarity point. For a detailed example of
applying this strategy, see McCullough (2004b). For extended discussion of using
nonlinear solvers to find solutions to nonlinear problems, see McCullough and
Renfro (2000) and McCullough and Vinod (2003a, 2004).
To belabor this important point, almost any nonlinear problem solved using a
package that has this second problem will produce an incorrect answer. The user
simply accepts the default options, the solver produces an incorrect answer, and the
user accepts the incorrect results. The literature is filled with results from such pack-
ages, and because journals typically do not even compel authors to identify their
software packages, let alone make their data and code available, there is little hope
of purging these incorrect results from the literature. As will be discussed in the
conclusions, replication of published results and software accuracy are intimately
connected.
One interesting aspect of the nonlinear StRD suite that has yet to be published
in the literature is the following. The StRD nonlinear suite is for nonlinear least
squares problems. Nonlinear least squares solvers make use of the special structure
of nonlinear least squares problems. Reformulating these problems as maximum