Applied Statistics and Probability for Engineers

12-6.6 Nonlinear Regression

Linear regression models provide a rich and flexible framework that work extremely well in

many problems in engineering and science. However, linear regression models are not appro-

priate for all situations. There are many problems where the response variable and the predic-

tor variables are related through a known nonlinearfunction. This leads to a nonlinear

regression model.When the method of least squares is applied to such models, the resulting

normal equations are nonlinear and, in general, difficult to solve. The usual approach is to

directly minimize the residual sum of squares by an iterative procedure. We now give a very

brief introduction to nonlinear regression models.

Linear or Nonlinear Models

We have focused in Chapter 12 on the linear regression model

(S12-2)

These models can include not only the first-order relationships, such as Equation S12-2, but

polynomial models, and other more complex relationships as well. In fact, we could write the

linear regression model as

(S12-3)

where zirepresents any functionof the original regressors x 1 , x 2 , p, xk, including trans-

formations such as exp(xi), , and sin(xi). These models are called linearregression models

because they are linear in the unknown parameters,the j, j1, 2,p, k.

We may write the linear regression model (Equation S12-2) in a general form as

f 1 x,  2  (S12-4)

Yx¿

1 xi

Y 0  1 z 1  2 z 2 pr zr

Y 0  1 x 1  2 x 2 pk xk

12-5

Figure S12-1 Ridge trace for the cement data.

–60

–50

–40

–30

–20

–10

0

10

20

30

40

50

60

70

0 .05 .10 .15 .20 .25 .30 .35 .40 .45 .50 .55

j( )

θ

β*θ

β* 3 ( )θ

β* 2 ( )θ

β* 4 ( )θ

β* 1 ( )θ

PQ220 6234F.CD(12) 5/20/02 10:52 M Page 5 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH114 FIN L:Quark