Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

9.6. A Regression Problem 547

Model (9.6.1) can be expressed equivalently as

Y = α 1 +βxc+e

=[1xc]

(

α

β

)

+e

= Xβ+e, (9.6.13)

whereXis then×2 matrix with columns 1 andxcandβ=(α, β)′.Next,let
θ=E(Y)=Xβ. Finally, letVbe the two-dimensional subspace ofRnspanned by
the columns ofX; i.e.,V is the range of the matrixX. Hence we can also express
the model succinctly as
Y=θ+e, θ∈V. (9.6.14)

Hence, except for the random error vectore,Ywould lie inV.Itmakessense
intuitively then, as suggested by Figure 9.6.3, to estimateθby the vector inVthat
is “closest” (in Euclidean distance) toY,thatis,byˆθ,where

θˆ=Argmin

θ∈V‖Y−θ‖

(^2) , (9.6.15)
where the square of theEuclidean normis given by‖u‖^2 =
∑n
i=1u
2
i,foru∈R
n.
As shown in Exercise 9.6.13 and depicted on the plot in Figure 9.6.3,ˆθ=ˆα 1 +βˆxc,
where ˆαandβˆare the least squares estimates given above. Also, the vectorˆe=
Y−θˆis the vector of residuals andnˆσ^2 =‖ˆe‖^2. Also, just as depicted in Figure
9.6.3, the angle between the vectorsˆθandˆeis a right angle. In linear models, we
say thatθˆis the projection ofYonto the subspaceV.
Y
V
e
0 ^
^
Figure 9.6.3: The sketch shows the geometry of least squares. The vector of
responses isY, the fit iŝθ, and the vector of residuals isˆe.