Applied Statistics and Probability for Engineers

414 CHAPTER 12 MULTIPLE LINEAR REGRESSION

These plots indicate that the expected change in Ywhen x 1 is changed by one unit (say) is a

function of both x 1 and x 2. The quadratic and interaction terms in this model produce a mound-

shaped function. Depending on the values of the regression coefficients, the second-order

model with interaction is capable of assuming a wide variety of shapes; thus, it is a very flex-

ible regression model.

12-1.2 Least Squares Estimation of the Parameters

The method of least squaresmay be used to estimate the regression coefficients in the mul-

tiple regression model, Equation 12-2. Suppose that nkobservations are available, and let

xijdenote the ith observation or level of variable xj.The observations are

It is customary to present the data for multiple regression in a table such as Table 12-1.

Each observation (xi 1 , xi 2 ,p, xik, yi), satisfies the model in Equation 12-2, or

(12-7)

The least squares function is

(12-8)

We want to minimize Lwith respect to  0 ,  1 , p , k. The least squares estimatesof  0 ,

 1 , p , kmust satisfy

(12-9a)

and

(12-9b)

Simplifying Equation 12-9, we obtain the least squares normal Equations

(12-10)

Note that there are pk1 normal Equations, one for each of the unknown regression

coefficients. The solution to the normal Equations will be the least squares estimatorsof the

ˆ (^0) a
n
i 1
̨xikˆ (^1) a
n
i 1
̨xikxi 1 ˆ (^2) a
n
i 1
xikxi 2 p ˆk (^) a
n
i 1
xik^2 a
n
i 1
xikyi
o o o o o
ˆ (^0) a
n
i 1
̨xi 1 ˆ (^1) a
n
i 1
̨xi^21 ˆ (^2) a
n
i 1
xi 1 xi 2 p ˆk (^) a
n
i 1
xi 1 xik a
n
i 1
xi 1 yi
nˆ 0 ˆ (^1) a
n
i 1
̨xi 1 ˆ 2 ̨a
n
i 1
̨xi 2 p ˆka
n
i 1
̨xik  a
n
i 1
̨yi
L
j
`
ˆ 0 ,ˆ 1 , p, ˆk
 (^2) a
n
i 1
ayiˆ 0  a
k
j 1

ˆj xijb xij 0 j1, 2,p, k

L

     0

`

ˆ 0 ,ˆ 1 , p , ˆk

 (^2) a
n
i 1
ayiˆ 0 a
k
j 1
ˆjxijb 0
L a
n
i 1
̨
2
i^ a^
n
i 1
̨ayi 0 a
k
j 1
̨ jxijb
2
 0 a
k
j 1

jxiji i1, ̨ 2,p, ̨n

y ̨i 0  1 xi 1  2 xi 2 pkxiki

1 xi 1 , ̨xi 2 ,p, xik, ̨yi 2 , i1, 2,p, ̨n and nk

c 12 .qxd 5/20/02 9:31 M Page 414 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH114 FIN L:Quark Files: