Applied Statistics and Probability for Engineers

12-1 MULTIPLE LINEAR REGRESSION MODEL 421

Computers are almost always used in fitting multiple regression models. Table 12-4 pre-

sents some annotated output from Minitab for the least squares regression model for wire bond

pull strength data. The upper part of the table contains the numerical estimates of the regres-

sion coefficients. The computer also calculates several other quantities that reflect important

information about the regression model. In subsequent sections, we will define and explain the

quantities in this output.

Estimating ^2

Just as in simple linear regression, it is important to estimate ^2 , the variance of the error term

, in a multiple regression model. Recall that in simple linear regression the estimate of ^2 was

obtained by dividing the sum of the squared residuals by n2. Now there are two parame-

ters in the simple linear regression model, so in multiple linear regression with pparameters a

logical estimator for ^2 is

ˆ^2  (12-16)

a

n

i 1

̨e^2 i

np

SSE

np

This is an unbiased estimatorof ^2. Just as in simple linear regression, the estimate of ^2 is

usually obtained from the analysis of variancefor the regression model. The numerator of

Equation 12-16 is called the erroror residual sum of squares, and the denominator npis

called the erroror residual degrees of freedom.Table 12-4 shows that the estimate of ^2 for

the wire bond pull strength regression model is ˆ^2 115.2 22 5.2364. The Minitab output

rounds the estimate to ˆ^2 5.2.

12-1.4 Properties of the Least Squares Estimators

The statistical properties of the least squares estimators may be easily found,

under certain assumptions on the error terms  1 ,  2 ,p, n, in the regression model. Paralleling

the assumptions made in Chapter 11, we assume that the errors iare statistically independent

with mean zero and variance ^2. Under these assumptions, the least squares estimators

are unbiased estimatorsof the regression coefficients  0 ,  1 ,p, k. This

property may be shown as follows:

since E() 0 and (XX)^1 XX I,the identity matrix. Thus, is an unbiased estimator of .

The variances of the ’s are expressed in terms of the elements of the inverse of the

matrix. The inverse of times the constant ^2 represents the covariance matrixof the

regression coefficients .The diagonal elements of are the variances of

and the off-diagonal elements of this matrix are the covariances. For example, if

we have k2 regressors, such as in the pull-strength problem,

C 1 X¿X 2 ^1 £

C 00 C 01 C 02

C 10 C 11 C 12

C 20 C 21 C 22

§

ˆ 1 ,p, ˆk,

ˆ ^2 1 X¿X 2 ^1 ˆ 0 ,

X¿X

ˆ X¿X

ˆ



E 31 X¿X 2 ^1 X¿X 1 X¿X 2 ^1 X¿ 4

E 31 X¿X 2 ^1 X¿ 1 X 24

E 1 ˆ 2 E 31 X¿X 2 ^1 X¿Y 4

ˆ 0 , ˆ 1 ,p, ̨ ˆk

ˆ 0 , ˆ 1 ,p, ˆk

c 12 .qxd 8/6/02 2:32 PM Page 421