13ChAPter
ChAPter
2
Simple Linear regression
a
fter reading this chapter you will understand:■ (^) How to estimate a simple linear regression.
■ (^) What is meant by the residual or error of a regression model.
■ (^) The distributional assumptions of a regression model.
■ (^) The assumptions about mean and variance of the error term in a regres-
sion model.
■ (^) How to measure the goodness-of-fit of a regression model.
■ (^) How to estimate a linear regression for a nonlinear relationship.
In this chapter, we introduce methods to express joint behavior of two vari-
ables. It is assumed that, at least to some extent, the behavior of one vari-
able is the result of a functional relationship between the two variables. In
this chapter, we introduce the linear regression model including its ordinary
least squares estimation, and the goodness-of-fit measure for a regression.
Although in future chapters covering econometric tools we will not focus
on estimating parameters, we will do so here in order to see how some of
the basic measures are calculated. We devote Chapter 13 to explaining the
various methods for estimating parameters.
Before advancing into the theory of regression, we note the basic idea
behind a regression. The essential relationship between the variables is
expressed by the measure of scaled linear dependence, that is, correlation.
The Role of Correlation
In many applications, how two entities behave together is of interest. Hence,
we need to analyze their joint distribution. In particular, we are interested in
the joint behavior of those two entities, say x and y, linearly. The appropri-
ate tool is given by the covariance of x and y. More exactly, we are inter-
ested in their correlation expressed by the correlation coefficient explained
in Appendix A. Generally, we know that correlation assumes values between