Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

540 Inferences About Normal Linear Models

grade in calculus. Frequently, one of these variables, sayx, is known in advance
of the other and there is interest in predicting a future random variableY.Since
Y is a random variable, we cannot predict its future observed valueY =ywith
certainty. Thus let us first concentrate on the problem of estimating the mean
ofY,thatis,E(Y). NowE(Y) is usually a function ofx; for example, in our
illustration with the calculus grade, sayY, we would expectE(Y)toincreasewith
increasing mathematics aptitude scorex. SometimesE(Y)=μ(x) is assumed to
be of a given form, such as a linear or quadratic or exponential function; that is,
μ(x) could be assumed to be equal toα+βxorα+βx+γx^2 orαeβx.Toestimate
E(Y)=μ(x), or equivalently the parametersα, β,andγ, we observe the random
variableYfor each ofnpossible different values ofx,sayx 1 ,x 2 ,...,xn,whichare
not all equal. Once thenindependent experiments have been performed, we have
npairs of known numbers (x 1 ,y 1 ),(x 2 ,y 2 ),...,(xn,yn). These pairs are then used
to estimate the meanE(Y). Problems like this are often classified underregression
becauseE(Y)=μ(x) is frequently called a regression curve.

Remark 9.6.1.A model for the mean such asα+βx+γx^2 is called alinear
modelbecause it is linear in the parametersα, β,andγ.Thusαeβxis not a linear
model because it is not linear inαandβ. Note that, in Sections 9.2 to 9.5, all the
means were linear in the parameters and hence are linear models.

For the most part in this section, we consider the case in whichE(Y)=μ(x)is
a linear function. Denote byYithe response atxiand consider the model

Yi=α+β(xi−x)+ei,i=1,...,n, (9.6.1)

wherex=n−^1

∑n
i=1xiande^1 ,...,enare iid random variables with a common
N(0,σ^2 ) distribution. HenceE(Yi)=α+β(xi−x), Var(Yi)=σ^2 ,andYihas
N(α+β(xi−x),σ^2 ) distribution. The major assumption is that the random errors,
ei, are iid. In particular, this means that the errors are not a function of the
xi’s. This is discussed in Remark 9.6.3. First, we discuss the maximum likelihood
estimates of the parametersα,β,andσ.

9.6.1 MaximumLikelihoodEstimates.................

Assume that thenpoints (x 1 ,Y 1 ),(x 2 ,Y 2 ),...,(xn,Yn) follow Model 9.6.1. So the

first problem is that of fitting a straight line to the set of points; i.e., estimating

αandβ. As an aid to our discussion, Figure 9.6.1 shows ascatterplotof 60

observations (x 1 ,y 1 ),...,(x 60 ,y 60 ) simulated from a linear model of the form (9.6.1).

Our method of estimation in this section is that of maximum likelihood (mle). The

joint pdf ofY 1 ,...,Ynis the product of the individual probability density functions;

that is, the likelihood function equals

L(α, β, σ^2 )=

∏n

i=1

1

√

2 πσ^2

exp

{

−

[yi−α−β(xi−x)]^2

2 σ^2

}

=

(

1

2 πσ^2

)n/ 2

exp

{

−

1

2 σ^2

∑n

i=1

[yi−α−β(xi−x)]^2

}

.