20 Numerical methods
20.1 Concepts
A numerical method is a method for obtaining the solution of a mathematical problem
in the form of numbers. In many cases the problem can be solved ‘exactly’ in terms of
known functions by means of the analytical methods discussed in the earlier chapters
of this book. Numerical methods are then required only if a solution is required in
numerical form. Many other problems, on the other hand, cannot be solved analytically,
and numerical methods are required for the process of solution itself. Because
numerical methods are commonly implemented on computers, they are sometimes
described as ‘methods for solving problems on a computer’. The development of
computers over the past fifty years has been accompanied by the development of
sophisticated numerical procedures for the solution of a wide range of problems in
the physical sciences, in engineering and in statistics. These procedures have been
collected in ‘mathematical libraries’ or ‘numerical packages’, available on most standard
computers. In this chapter we discuss the general principles underlying some of the
more important numerical methods, and treat in detail only the very simplest.
Nearly all numerical operations are necessarily accompanied by errors, and the
analysis of these errors is an integral part of any numerical method.
20.2 Errors
Errors in numerical computations arise largely in three ways.
- Mistakes.These include errors in the carrying out of a numerical operation or an
error (‘bug’) in a computer program. Such errors are in principle removable by
checking. In the experimental sciences, removable errors also arise from faulty or
incorrectly calibrated apparatus. Mistakesare not counted as errorsin numerical
analysis.
- Mathematical truncation errors.These are due to the use of approximate
representations of functions; for example, as discussed in Section 7.7, when a Taylor
series expansion of a function is truncated after a given number of terms. Taylor’s
theorem then provides a method of determining the error bounds of the result.
Many numerical methods have their origin in the Taylor expansion and are
therefore necessarily accompanied by truncation errors; numerical methods are
approximate methods.
- Rounding errors.As discussed in Section 1.4, numbers are in practice nearly
always approximated by rounding, either to a given number of decimal places in
the fixed-point representation, or to a given number of significant figures in the
floating-point representation. It is the consequences of this type of error that we
consider in this section.
Because of the conventions for rounding (Section 1.4) a number like a 1 = 1 1.234,
obtained by rounding to three decimal places, represents all the numbers between