Applied Statistics and Probability for Engineers

12-8

After simplification, the normal equations are

These equations are not linear in and , and no simple closed-form solution exists. In gen-

eral, iterative methodsmust be used to find the values of and. To further complicate the

problem, sometimes there are multiple solutions to the normal equations. That is, there are

multiple stationary values for the residual sum of squares function L().

A widely used method for nonlinear regression is linearizationof the nonlinear function

followed by the Gauss-Newton iteration method of parameter estimation. Linearization is ac-

complished by a Taylor series expansionof f(xi, ) about the point  

0 [ 10 ,  20 ,p, p 0 ]

with only the linear terms retained. This yields

(S12-10)

If we set

we note that the nonlinear regression model can be written as

(S12-11)

That is, we now have a linear regression model. We usually call  0 the starting values for the

parameters.

We may write Equation S12-10 as

(S12-12)

so the estimate of  0 is

Now since  0  0 , we could define

as revised estimates of . Sometimes is called the vector of increments.We may now

replace the revised estimates in Equation S12-10 (in the same roles played by the initial

estimates  0 ) and then produce another set of revised estimates, say ˆ 2 and so forth.

ˆ 1

ˆ 0

ˆ 1 ˆ 0  0

 1 Z¿ 0 Z 02 ^1 Z¿ 01 yf 02

ˆ 0  1 Z ¿ 0 Z 02 ^1 Z¿ 0 y 0

y 0 Z 0  0 

yifi^0  a

p

j 1

^0 jZ^0 iji, i1, 2,p, n

Z (^0) ijc
f 1 xi,  2
j
d
 0
^0 jjj 0
fi^0 f 1 xi,  02
f 1 xi,  2 f 1 xi,  02 a
p
j 1
c
f 1 xi,  2
j
d
 0
1 jj 02
ˆ 1 ˆ 2
ˆ 1 ˆ 2
a
n
i 1
yixieˆ^2 xiˆ (^1) a
n
i 1
xie^2 ˆ^2 xi 0
a
n
i 1
yie
ˆ 2 xi
ˆ (^1) a
n
i 1
e^2 
ˆ 2 xi
 0
PQ220 6234F.CD(12) 5/20/02 10:52 M Page 8 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH114 FIN L:Quark