Mathematical Methods for Physics and Engineering : A Comprehensive Guide

31.6 THE METHOD OF LEAST SQUARES

where{h 1 (x),h 2 (x),...,hM(x)}is some set of linearly independent fixed functions

ofx, often called thebasis functions. Note that the functionshi(x) themselves may

be highly non-linear functions ofx. The ‘linear’ nature of the model (31.92) refers

only to its dependence on theparametersai. Furthermore, in this case, it may

be shown that the LS estimatorsaˆihave zero bias and are minimum-variance,

irrespective of the probability density function from which the measurement errors

niare drawn.

In order to obtain analytic expressions for the LS estimatorsaˆLS,itisconvenient

to write (31.92) in the form

f(x;a)=

∑M

j=1

Rijaj, (31.93)

whereRij=hj(xi) is an element of theresponse matrixRof the experiment. The

expression forχ^2 given in (31.90) can then be written, in matrix notation, as

χ^2 (a)=(y−Ra)TN−^1 (y−Ra). (31.94)

The LS estimates of the parametersaare now found, as shown in (31.91), by

differentiating (31.94) with respect to theaiand setting the resulting expressions

equal to zero. Denoting by∇χ^2 the vector with elements∂χ^2 /∂ai, we find

∇χ^2 =− 2 RTN−^1 (y−Ra). (31.95)

This can be verified by writing out the expression (31.94) in component form and

differentiating directly.

Verify result (31.95) by formulating the calculation in component form.

To make the derivation less cumbersome, let us adopt the summation convention discussed
in section 26.1, in which it is understood that any subscript that appearsexactlytwice in
any term of an expression is to be summed over all the values that a subscript in that
position can take. Thus, writing (31.94) in component form, we have

χ^2 (a)=(yi−Rikak)(N−^1 )ij(yj−Rjlal).

Differentiating with respect toapgives

∂χ^2

∂ap

=−Rikδkp(N−^1 )ij(yj−Rjlal)+(yi−Rikak)(N−^1 )ij(−Rjlδlp)

=−Rip(N−^1 )ij(yj−Rjlal)−(yi−Rikak)(N−^1 )ijRjp, (31.96)

whereδijis the Kronecker delta symbol discussed in section 26.1. By swapping the indices
iandjin the second term on the RHS of (31.96) and using the fact that the matrixN−^1
is symmetric, we obtain

∂χ^2

∂ap

=− 2 Rip(N−^1 )ij(yj−Rjkak)

=−2(RT)pi(N−^1 )ij(yj−Rjkak). (31.97)

If we denote the vector with components∂χ^2 /∂ap,p=1, 2 ,...,M,by∇χ^2 and write the
RHS of (31.97) in matrix notation, we recover the result (31.95).