Introduction to Probability and Statistics for Engineers and Scientists

360 Chapter 9: Regression

Consequently, the maximum likelihood estimators ofαandβare precisely the values ofα
andβthat minimize

∑n

i= 1 (yi−α−βxi)^2. That is, they are the least squares estimators.

Notation

If we let

SxY=

∑n

i= 1

(xi−x)(Yi−Y)=

∑n

i= 1

xiYi−nxY

Sxx=

∑n

i= 1

(xi−x)^2 =

∑n

i= 1

xi^2 −nx^2

SYY=

∑n

i= 1

(Yi−Y)^2 =

∑n

i= 1

Yi^2 −nY^2

then the least squares estimators can be expressed as

B=

SxY

Sxx

A=Y−Bx

The following computational identity forSSR, the sum of squares of the residuals, can
be established.

Computational Identity forSSR

SSR=

SxxSYY−S^2 xY

Sxx

(9.3.4)

The following proposition sums up the results of this section.

PROPOSITION 9.3.1 Suppose that the responsesYi,i=1,...,nare independent normal
random variables with meansα+βxi and common varianceσ^2. The least squares
estimators ofβandα

B=

SxY

Sxx

, A=Y−Bx

are distributed as follows:

A∼N





α,

σ^2

∑

i

xi^2

nSxx







B∼N(β,σ^2 /Sxx)