9.9Polynomial Regression 391
9.9Polynomial Regression
InsituationswherethefunctionalrelationshipbetweentheresponseYandtheindependent
variablexcannot be adequately approximated by a linear relationship, it is sometimes
possible to obtain a reasonable fit by considering a polynomial relationship. That is, we
might try to fit to the data set a functional relationship of the form
Y=β 0 +β 1 x+β 2 x^2 +···+βrxr+e
whereβ 0 ,β 1 ,...,βrare regression coefficients that would have to be estimated. If the
data set consists of thenpairs (xi,Yi),i=1,...,n, then the least square estimators of
β 0 ,...,βr— call themB 0 ,...,Br— are those values that minimize
∑n
i= 1
(Yi−B 0 −B 1 xi−B 2 xi^2 −···−Brxir)^2
To determine these estimators, we take partial derivatives with respect toB 0 ...Br
of the foregoing sum of squares, and then set these equal to 0 so as to determine the
minimizing values. On doing so, and then rearranging the resulting equations, we obtain
that the least square estimatorsB 0 ,B 1 ,...,Brsatisfy the following set ofr+1 linear
equations called the normal equations.
∑n
i= 1
Yi=B 0 n+B 1
∑n
i= 1
xi+B 2
∑n
i= 1
xi^2 +···+Br
∑n
i= 1
xir
∑n
i= 1
xiYi=B 0
∑n
i= 1
xi+B 1
∑n
i= 1
x^2 i+B 2
∑n
i= 1
xi^3 +···+Br
∑n
i= 1
xri+^1
∑n
i= 1
xi^2 Yi=B 0
∑n
i= 1
xi^2 +B 1
∑n
i= 1
x^3 i+···+Br
∑n
i= 1
xir+^2
..
.
..
.
..
.
∑n
i= 1
xirYi=B 0
∑n
i= 1
xir+B 1
∑n
i= 1
xir+^1 +···+Br
∑n
i= 1
xi^2 r
In fitting a polynomial to a set of data pairs, it is often possible to determine the necessary
degree of the polynomial by a study of the scatter diagram. We emphasize that one should
always use the lowest possible degree that appears to adequately describe the data. [Thus,
for instance, whereas it is usually possible to find a polynomial of degreenthat passes
through all thenpairs (xi,Yi),i=1,...,n, it would be hard to ascribe much confidence
to such a fit.]