Multiple Linear Regression 45

by the regression is indicated by the gray plane. The intercept b 0 is indicated
by the dashed arrow while the slopes in the directions of x 1 and x 2 are indi-
cated by the arrows b 1 and b 2 , respectively.^2 Now, we extend vertical lines
between the hyperplane and the observations, e 1 , e 2 , and e 3 , to show by how
much we have missed approximating the observations with the hyperplane.
Generally, with the ordinary least squares regression method described
in Chapter 2, the estimates are, again, such that Σ−(yyˆ)^2 is minimized with
respect to the regression coefficients. For the computation of the regres-
sion estimates, we need to indulge somewhat in matrix computation. If we
write the minimization problem in matrix notation, finding the vector β
that minimizes the squared errors looks like^3

(^) ∑ −=−β −β

(yyi ˆ)(yX yX)( )
i
n
2 T
1

(3.9)

Differential calculus and matrix algebra lead to the optimal regression
coefficient estimates and estimated residuals given by

=()

−

bXTTXXy

1

(3.10)

and

ey=−=−XbT yyˆ (3.11)

where b in equation (3.10) and e in equation (3.11) are (k + 1) × 1 and n × 1
column vectors, respectively. One should not worry however if this appears
rather complicated and very theoretical. Most statistical software have these
computations implemented and one has to just insert the data for the vari-
ables and select some least squares regression routine to produce the desired
estimates according to equation (3.10).

Designing the Model

Although in the previous chapter we introduced the simple linear regres-
sion model, we did not detail the general steps necessary for the design of

(^2) The arrow b 0 is dashed to indicate that it extends from our point of view vertically
from the point (0,0,0) behind the hyperplane.
(^3) The transpose and matrix inverse are explained in Appendix D. When we use the
matrix inverse, we implicitly assume the matrix of interest to be full rank, a require-
ment for the inversion of a matrix.