Introduction to Probability and Statistics for Engineers and Scientists

9.9Polynomial Regression 391

9.9Polynomial Regression

InsituationswherethefunctionalrelationshipbetweentheresponseYandtheindependent
variablexcannot be adequately approximated by a linear relationship, it is sometimes
possible to obtain a reasonable fit by considering a polynomial relationship. That is, we
might try to fit to the data set a functional relationship of the form

Y=β 0 +β 1 x+β 2 x^2 +···+βrxr+e

whereβ 0 ,β 1 ,...,βrare regression coefficients that would have to be estimated. If the
data set consists of thenpairs (xi,Yi),i=1,...,n, then the least square estimators of
β 0 ,...,βr— call themB 0 ,...,Br— are those values that minimize

∑n

i= 1

(Yi−B 0 −B 1 xi−B 2 xi^2 −···−Brxir)^2

To determine these estimators, we take partial derivatives with respect toB 0 ...Br
of the foregoing sum of squares, and then set these equal to 0 so as to determine the
minimizing values. On doing so, and then rearranging the resulting equations, we obtain
that the least square estimatorsB 0 ,B 1 ,...,Brsatisfy the following set ofr+1 linear
equations called the normal equations.

∑n

i= 1

Yi=B 0 n+B 1

∑n

i= 1

xi+B 2

∑n

i= 1

xi^2 +···+Br

∑n

i= 1

xir

∑n

i= 1

xiYi=B 0

∑n

i= 1

xi+B 1

∑n

i= 1

x^2 i+B 2

∑n

i= 1

xi^3 +···+Br

∑n

i= 1

xri+^1

∑n

i= 1

xi^2 Yi=B 0

∑n

i= 1

xi^2 +B 1

∑n

i= 1

x^3 i+···+Br

∑n

i= 1

xir+^2

..

.

..

.

..

.

∑n

i= 1

xirYi=B 0

∑n

i= 1

xir+B 1

∑n

i= 1

xir+^1 +···+Br

∑n

i= 1

xi^2 r

In fitting a polynomial to a set of data pairs, it is often possible to determine the necessary
degree of the polynomial by a study of the scatter diagram. We emphasize that one should
always use the lowest possible degree that appears to adequately describe the data. [Thus,
for instance, whereas it is usually possible to find a polynomial of degreenthat passes
through all thenpairs (xi,Yi),i=1,...,n, it would be hard to ascribe much confidence
to such a fit.]