620 Chapter 21Probability and statistics
Simple least squares fitting
The simplest and one of the most popular methods of fitting a set of data points to a
model function (21.49) is to minimize the quantity
(21.50)
with respect to the parametersa
1
, a
2
, =, a
k
:
(21.51)
As shown in Figure 21.11, each term of the sum is the
square of the difference
ε
i
1 = 1 y
i
1 − 1 y(x
i
; a)
between the actual and predicted values of yfor each
value ofx 1 = 1 x
i
.
This method of least squares can be regarded either
as an empirical curve-fitting procedure that is justified
on the grounds that it often gives sensible results, or it
can be derived from the theory of random errors for
the special case that theσ
i
values are all equal; that is, when all they
i
have the same
precision. It is also the method to be used when no information about precision is
available.
5
The straight-line fit
The quantity to be minimized for a fit to the straight liney 1 = 1 mx 1 + 1 cis
(21.52)
Then
∂
∂
=− − − = − −
===
∑∑∑
D
c
ymxc ym xc
i
N
ii
i
N
i
i
N
i
i
20
111
()
==
∑
=
1
10
N
∂
∂
=− − − = −
===
∑∑
D
m
xy mx c xy m
i
N
ii i
i
N
ii
i
N
20
111
()or
∑∑∑
−=
=
xcx
i
i
N
i
2
1
0
Dymxc
i
N
ii
=−+
=
∑
1
2
()
∂
∂
=,
∂
∂
=, ,
∂
∂
=
D
a
D
a
D
a
12 k
00 0...
Dyyx
i
N
ii
i
N
i
=−;
=
==
∑∑
1
2
1
2
()a ε
5
The method of least squares (la méthode des moindres quarrés) was developed by Legendre in 1805 in connection
with the determination of the orbits of comets, and quickly became a standard method for solving problems in
astronomy and geodesy. The method also appeared in 1809 in Gauss’ Theory of motion of the heavenly bodies.
Gauss did not quote Legendre, and claimed that he had been using the method since 1795. Legendre argued that
precedence in scientific discoveries could only be established by publication, and was still accusing Gauss in 1827
of appropriating the discoveries of others.
x
i
y
i
y(x
i
;a)
y(x;a)
..
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
..
...
...
..
...
..
..
...
...
..
...
....
...
...
..
...
...
..
...
...
..
...
...
..
...
..
...
...
..
..
...
..
...
...
..
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
...
..
....
..
...
...
..
...
.
..
..
...
...
..
...
.
...
........
........
..
.......
........
....
.....
..
...
...
..
...
...
...
..
...
...
..
...
...
i
.
..
...
..
.
..
...
..
.
..
...
..
.
................................................................................
................................................................................
...
...........
...........
..........
..........
...........
.........
..........
..........
.........
.........
.........
.........
........
.........
.........
........
.........
.......
........
.........
.......
........
........
.......
........
........
.......
.......
........
.......
.......
.......
.......
.......
.......
.......
.......
.......
......
.......
.......
.......
......
.......
......
......
.......
.......
......
.......
......
......
.......
......
......
......
......
......
.......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
.......
.....
.....
Figure 21.11