Pattern Recognition and Machine Learning

172 3. LINEAR MODELS FOR REGRESSION

lnα

−5 0 5

lnα

−5 0 5

Figure 3.16 The left plot showsγ(red curve) and 2 αEW(mN)(blue curve) versuslnαfor the sinusoidal
synthetic data set. It is the intersection of these two curves that defines the optimum value forαgiven by the
evidence procedure. The right plot shows the corresponding graph of log evidencelnp(t|α, β)versuslnα(red
curve) showing that the peak coincides with the crossing point of the curves in the left plot. Also shown is the
test set error (blue curve) showing that the evidence maximum occurs close to the point of best generalization.

formulae, because they do not require evaluation of the eigenvalue spectrum of the

Hessian.

Figure 3.17 Plot of the 10 parameters wi

from the Gaussian basis function

model versus the effective num-

ber of parametersγ, in which the

hyperparameterαis varied in the

range 0 
α
∞causingγto

vary in the range 0 
γ
M.

9

7

1

3

6

2

5

4

8

0

γ

wi

0 2 4 6 8 10

−2

−1

0

1

2

3.6 Limitations of Fixed Basis Functions

Throughout this chapter, we have focussed on models comprising a linear combina-

tion of fixed, nonlinear basis functions. We have seen that the assumption of linearity

in the parameters led to a range of useful properties including closed-form solutions

to the least-squares problem, as well as a tractable Bayesian treatment. Furthermore,

for a suitable choice of basis functions, we can model arbitrary nonlinearities in the