Simple Linear Regression 19
it such that the sum of the vertical distances of the y-values from the line is
minimized. However, the problem is that in the scatter plot, positive errors
will cancel out negative errors. To address this, one could just look at the
absolute value of the error terms (i.e., ignore the negative sign). An alterna-
tive, and the method used here, is to square the error terms to avoid positive
and negative values from canceling out.
What we need is a formal criterion that determines optimality of
some linear fit. Measuring the errors in terms of the squared errors, we
want to minimize the total sum of the squared errors. Mathematically,
we have to solve
min
ab, i ii
n
()ya−−bx
=∑
2
1(2.5)
That is, we need to find the estimates a and b of the parameters α and β,
respectively, that minimize the total of the squared errors. Here, the error
is given by the disturbance between the line and the true observation y. By
taking the square, not only do we avoid having positive and negative errors
from canceling out, but we also penalize larger disturbances more strongly
than smaller ones.
The estimation approach given by equation (2.5) is the ordinary least
squares (OLS) methodology, which we describe in more detail in Chapter 13.
FIGUre 2.2 Scatter Plot of Data with Two Different Lines as Linear Fits