Fundamentals of Probability and Statistics for Engineers

that E Y is a function x. In any single experiment, x will assume a certain
value xi and the mean of Y will take the value

Random variable Y is, of course, itself a function of x. If we define a random
variable E by

we can write

where E has mean 0 and variance^2 , which is identical to the variance of Y. The
value of^2 is not known in general but it is assumed to be a constant and not
a function of x.
Equation (11.4) is a standard expression of a simple linear regression model.
The unknown parameters and are called regression coefficients, and random
variable E represents the deviation of Y about its mean. As with simple models
discussed in Chapters 9 and 10, simple linear regression analysis is concerned
with estimation of the regression parameters, the quality of these estimators,
and model verification on the basis of the sample. We note that, instead of
a simple sample such as Y 1 ,Y 2 ,...,Yn as in previous cases, our sample in the
present co ntext takes the form of pairs (x 1 ,Y 1 ), (x 2 ,Y 2 ),...,(xn,Yn). For each
value xi assigned to x, Yi is an independent observation from population Y
defined by Equation (11.4). Hence, (xi,Yi),i 1,2,...,n, may be considered
as a sample from random variable Y for given values x 1 ,x 2 ,..., and xn of x;
these x values need not all be distinct but, in order to estimate both and ,
we will see that we must have at least two distinct values of x represented in
the sample.

11.1.1 Least Squares M ethod of Estimation

As one approach to point estimation of regression parameters and , the
method of least squares suggests that their estimates, and , be chosen so
that the sum of the squared differences between observed sample values
yi and the estimated expected value of Y, is minimized. Let us
write

336 Fundamentals of Probability and Statistics for Engineers

fg

EfYigˆ ‡ xi: … 11 : 2 †

EˆY… ‡ x†; … 11 : 3 †

Yˆ ‡ x‡E; … 11 : 4 †





ˆ

^ ^

^ ^xi,

eiˆyi… ^‡ ^xi†: … 11 : 5 †

‡