Pattern Recognition and Machine Learning

7.1. Maximum Margin Classifiers 341 Figure 7.7 Illustration of SVM regression, showing the regression curve together with the - ...

342 7. SPARSE KERNEL MACHINES L ̃(a,̂a)=−^1 2 ∑N n=1 ∑N m=1 (an−̂an)(am−̂am)k(xn,xm) − ∑N n=1 (an+̂an)+ ∑N n=1 (an−̂an)tn (7.61 ...

7.1. Maximum Margin Classifiers 343 an=̂an=0. We again have a sparse solution, and the only terms that have to be evaluated in t ...

344 7. SPARSE KERNEL MACHINES Figure 7.8 Illustration of theν-SVM for re- gression applied to the sinusoidal synthetic data set ...

7.2. Relevance Vector Machines 345 case, because they apply toanychoice for the distributionp(x,t), so long as both the training ...

346 7. SPARSE KERNEL MACHINES whereβ=σ−^2 is the noise precision (inverse noise variance), and the mean is given by a linear mod ...

7.2. Relevance Vector Machines 347 in the predictions made by the model and so are effectively pruned out, resulting in a sparse ...

348 7. SPARSE KERNEL MACHINES γi=1−αiΣii (7.89) in whichΣiiis theithdiagonal component of the posterior covarianceΣgiven by (7.8 ...

7.2. Relevance Vector Machines 349 Figure 7.9 Illustration of RVM regression us- ing the same data set, and the same Gaussian ke ...

350 7. SPARSE KERNEL MACHINES t 1 t 2 t C t 1 t 2 t C φ Figure 7.10 Illustration of the mechanism for sparsity in a Bayesian lin ...

7.2. Relevance Vector Machines 351 basis vectorsφ 1 ,...,φMa similar intuition holds, namely that if a particular basis vector i ...

352 7. SPARSE KERNEL MACHINES Figure 7.11 Plots of the log marginal likelihood λ(αi) versus lnαishowing on the left, the single ...

7.2. Relevance Vector Machines 353 EvaluateΣandm, along withqiandsifor all basis functions. Select a candidate basis functionφi ...

354 7. SPARSE KERNEL MACHINES whereσ(·)is the logistic sigmoid function defined by (4.59). If we introduce a Gaussian prior over ...

7.2. Relevance Vector Machines 355 mation, we have p(t|α)= ∫ p(t|w)p(w|α)dw  p(t|w
)p(w
|α)(2π)M/^2 |Σ|^1 /^2. (7.114) If we su ...

356 7. SPARSE KERNEL MACHINES −2 0 2 −2 0 2 Figure 7.12 Example of the relevance vector machine applied to a synthetic data set, ...

Exercises 357 Exercises 7.1 ( ) www Suppose we have a data set of input vectors{xn}with corresponding target valuestn∈{− 1 , 1 } ...

358 7. SPARSE KERNEL MACHINES 7.8 ( ) www For the regression support vector machine considered in Section 7.1.4, show that all t ...

8 Graphical Models Probabilities play a central role in modern pattern recognition. We have seen in Chapter 1 that probability t ...

360 8. GRAPHICAL MODELS Complex computations, required to perform inference and learning in sophis- ticated models, can be expr ...