Applied Statistics and Probability for Engineers

12-7

The derivatives of the expectation function with respect to  1 and  2 are

and

Since the derivatives are a function of the unknown parameters  1 and  2 , the model is nonlinear.

Parameter Estimation in a Nonlinear Model

Suppose that we have a sample of nobservations on the response and the regressors, say yi,

xi 1 ,xi 2 , p, xik, for i1, 2, p, n. We have observed previously that the method of least squares

in linear regression involves minimizing the least-squares function

Because this is a linear regression model, when we differentiate S() with respect to the

unknown parameters and equate the derivatives to zero, the resulting normal equations are

linearequations, and consequently, they are easy to solve.

Now consider the nonlinear regression situation. The model is

where now xi[1, xi 1 , xi 2 , p, xik] for i1, 2, p, n. The least squares function is

(S12-8)

To find the least squares estimates we must differentiate Equation S12-8 with respect to each

element of . This will provide a set of pnormal equations for the nonlinear regression situa-

tion. The normal equations are

(S12-9)

In a nonlinear regression model the derivatives in the large square brackets will be functions

of the unknown parameters. Furthermore, the expectation function is also a nonlinear func-

tion, so the normal equations can be very difficult to solve.

To illustrate this point, consider the nonlinear regression model in Equation S12-5:

The least squares normal equations for this model are

a

n

i 1

3 yiˆ 1 e

ˆ 2 xi

4 ˆ 1 xie

ˆ 2 xi

 0

a

n

i 1

3 yiˆ 1 e

ˆ 2 xi

4

ˆ 2 xi

 0

Y 1 e^2 x   

a

n

i 1

3 yif 1 xi, 24c

f 1 xi,  2



j

d

ˆ

 0 for j1, 2,p, p

L 1  2  a

n

i 1

3 yif 1 xi,  242

Yif 1 xi,  2  i i1, 2,p, n

L 1  2  a

n

i 1

cyia 0 a

k

j 1

(^) jxijbd
2
f 1 x,  2

 2
 1 xe^2 x
f 1 x,  2

 1
e^2 x
PQ220 6234F.CD(12) 5/20/02 3:54 PM Page 7 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH114 FIN L:Quark F