Mathematical Methods for Physics and Engineering : A Comprehensive Guide

STATISTICS

hence does not possess an inverse. Inserting the form forRin (31.100) into the expression
(31.101), we find
(
ˆc
mˆ

)

=

(∑

i^1

∑

∑ ixi

ixi

∑

ix

2

i

)− 1 (∑

∑iyi

ixiyi

)

=

1

N(x^2 − ̄x^2 )

(

x^2 − ̄x

− ̄x 1

)(

N ̄y

Nxy

)

.

We thus obtain the LS estimates

mˆ=

xy− ̄x ̄y

x^2 − ̄x^2

and ˆc=

x^2 ̄y− ̄xxy

x^2 − ̄x^2

=y ̄−mˆ ̄x, (31.102)

where the last expression forˆcshows that the best-fit line passes through the ‘centre
of mass’ ( ̄x, ̄y) of the data sample. To find the standard errors on our results, we must
calculate the covariance matrix of the estimators. This is given by (31.99), which in our
case reduces to

V=σ^2 (RTR)−^1 =

σ^2

N(x^2 − ̄x^2 )

(

x^2 − ̄x

− ̄x 1

)

. (31.103)

The standard error on each estimator is simply the positive square root of the corresponding
diagonal element, i.e.σˆc=

√

V 11 andσmˆ=

√

V 22 , and the covariance of the estimatorsmˆ
andcˆis given by Cov[ˆc,mˆ]=V 12 =V 21. Inserting the data sample averages and moments
into (31.102) and (31.103), we find

c=ˆc±σˆc=0. 40 ± 0. 62 and m=mˆ±σmˆ=1. 11 ± 0. 17.

The ‘best-fit’ straight liney=mxˆ +cˆis plotted in figure 31.9. For comparison, the true
values used to create the data werem=1andc=1.

The extension of the method to fitting data to a higher-order polynomial, such

asf(x;a)=a 1 +a 2 x+a 3 x^2 , is obvious. However, as the order of the polynomial

increases the matrix inversions become rather complicated. Indeed, even when the

matrices are inverted numerically, the inversion is prone to numerical instabilities.

A better approach is to replace the basis functionshm(x)=xm,m=1, 2 ,...,M,

with a set of polynomials that are ‘orthogonal over the data’, i.e. such that

∑N

i=1

hl(xi)hm(xi)=0 forl=m.

Such a set of polynomial basis functions can always be found by using the Gram–

Schmidt orthogonalisation procedure presented in section 17.1. The details of this

approach are beyond the scope of our discussion but we note that, in this case,

the matrixRTRis diagonal and may be inverted easily.

31.6.2 Non-linear least squares

If the functionf(x;a)isnotlinear in the parametersathen, in general, it is

not possible to obtain an explicit expression for the LS estimatesaˆ.Instead,one

must use an iterative (numerical) procedure, which we now outline. In practice,