# Pattern Recognition and Machine Learning

(Jeff_L) #1
##### 172 3. LINEAR MODELS FOR REGRESSION

``lnα``

``````−5 0 5
lnα``````

``−5 0 5``

Figure 3.16 The left plot showsγ(red curve) and 2 αEW(mN)(blue curve) versuslnαfor the sinusoidal
synthetic data set. It is the intersection of these two curves that defines the optimum value forαgiven by the
evidence procedure. The right plot shows the corresponding graph of log evidencelnp(t|α, β)versuslnα(red
curve) showing that the peak coincides with the crossing point of the curves in the left plot. Also shown is the
test set error (blue curve) showing that the evidence maximum occurs close to the point of best generalization.

``````formulae, because they do not require evaluation of the eigenvalue spectrum of the
Hessian.``````

``````Figure 3.17 Plot of the 10 parameters wi
from the Gaussian basis function
model versus the effective num-
ber of parametersγ, in which the
hyperparameterαis varied in the
range 0 α∞causingγto
vary in the range 0 γM.``````

``9``

``7``

``1``

``3``

``6``

``2``

``5``

``4``

``8``

``0``

``γ``

``wi``

``0 2 4 6 8 10``

``−2``

``−1``

``0``

``1``

``2``

### 3.6 Limitations of Fixed Basis Functions

``````Throughout this chapter, we have focussed on models comprising a linear combina-
tion of fixed, nonlinear basis functions. We have seen that the assumption of linearity
in the parameters led to a range of useful properties including closed-form solutions
to the least-squares problem, as well as a tractable Bayesian treatment. Furthermore,
for a suitable choice of basis functions, we can model arbitrary nonlinearities in the``````