Applied Statistics and Probability for Engineers

11-5.2 Analysis of Variance Approach to Test Significance of Regression

A method called the analysis of variancecan be used to test for significance of regression.

The procedure partitions the total variability in the response variable into meaningful compo-

nents as the basis for the test. The analysis of variance identityis as follows:

11-5 HYPOTHESIS TESTS IN SIMPLE LINEAR REGRESSION 387

The two components on the right-hand-side of Equation 11-24 measure, respectively, the

amount of variability in yiaccounted for by the regression line and the residual variation left

unexplained by the regression line. We usually call the error sum of

squaresand the regression sum of squares.Symbolically, Equation

11-24 may be written as

SSRg

n

i 1 1 yˆiy^2

2

SSEg

n

i 1 1 yiyˆi^2

2

where SST g

n

i 1 is the total corrected sum of squaresofy. In Section 11-2 we

noted that SSE SST  1 Sxy(see Equation 11-14), so since SST  1 Sxy SSE, we note that the

regression sum of squares in Equation 10-26 is SSR  1 Sxy. The total sum of squares SSThas

n1 degrees of freedom, and SSRand SSEhave 1 and n2 degrees of freedom, respectively.

We may show that and that and

are independent chi-square random variables with n2 and 1 degrees of freedom, re-

spectively. Thus, if the null hypothesis H 0 :  1 0 is true, the statistic

SSR^2

E 3 SSE 1 n 224 ^2 , E 1 SSR 2 ^2  12 Sx x SSE^2

ˆ

ˆ ˆ

1 yiy 22

follows the F1,n 2 distribution, and we would reject H 0 if f 0     f ,1,n 2. The quantities MSR

SSR1 and MSESSE(n2) are called mean squares.In general, a mean square is always

computed by dividing a sum of squares by its number of degrees of freedom. The test proce-

dure is usually arranged in an analysis of variance table,such as Table 11-3.

a (11-24)

n

i 1

1 yiy 22  a

n

i 1

1 yˆiy 22  a

n

i 1

1 yiyˆi 22

SSTSSRSSE (11-25)

F 0  (11-26)

SSR 1

SSE 1 n 22



MSR

MSE

Table 11-3 Analysis of Variance for Testing Significance of Regression

Source of Sum of Degrees of Mean

Variation Squares Freedom Square F 0

Regression 1 MSR MSRMSE

Error SSE SST Sxy n 2 MSE

Total SST n 1

Note that MSE ˆ^2.

ˆ 1

SSRˆ 1 Sx y

c 11 .qxd 5/20/02 1:59 PM Page 387 RK UL 6 RK UL 6:Desktop Folder:TEMP WORK:MONTGOMERY:REVISES UPLO D CH114 FIN L:Quark Files: