##### 172 3. LINEAR MODELS FOR REGRESSION

`lnα`

`−5 0 5`

lnα

`−5 0 5`

Figure 3.16 The left plot showsγ(red curve) and 2 αEW(mN)(blue curve) versuslnαfor the sinusoidal

synthetic data set. It is the intersection of these two curves that defines the optimum value forαgiven by the

evidence procedure. The right plot shows the corresponding graph of log evidencelnp(t|α, β)versuslnα(red

curve) showing that the peak coincides with the crossing point of the curves in the left plot. Also shown is the

test set error (blue curve) showing that the evidence maximum occurs close to the point of best generalization.

`formulae, because they do not require evaluation of the eigenvalue spectrum of the`

Hessian.

`Figure 3.17 Plot of the 10 parameters wi`

from the Gaussian basis function

model versus the effective num-

ber of parametersγ, in which the

hyperparameterαis varied in the

range 0 α∞causingγto

vary in the range 0 γM.

`9`

`7`

`1`

`3`

`6`

`2`

`5`

`4`

`8`

`0`

`γ`

`wi`

`0 2 4 6 8 10`

`−2`

`−1`

`0`

`1`

`2`

### 3.6 Limitations of Fixed Basis Functions

`Throughout this chapter, we have focussed on models comprising a linear combina-`

tion of fixed, nonlinear basis functions. We have seen that the assumption of linearity

in the parameters led to a range of useful properties including closed-form solutions

to the least-squares problem, as well as a tractable Bayesian treatment. Furthermore,

for a suitable choice of basis functions, we can model arbitrary nonlinearities in the