Pattern Recognition and Machine Learning

Exercises 321 6.11 ( ) By making use of the expansion (6.25), and then expanding the middle factor as a power series, show that ...

322 6. KERNEL METHODS 6.18 ( ) Consider a Nadaraya-Watson model with one input variablexand one target variablethaving Gaussian ...

Exercises 323 6.25 ( ) www Using the Newton-Raphson formula (4.92), derive the iterative update formula (6.83) for finding the m ...

...

7 Sparse Kernel Machines In the previous chapter, we explored a variety of learning algorithms based on non- linear kernels. One ...

326 7. SPARSE KERNEL MACHINES encouraged to review the key concepts covered in Appendix E. Additional infor- mation on support v ...

7.1. Maximum Margin Classifiers 327 y=1 y=0 y=− 1 margin y=1 y=0 y=− 1 Figure 7.1 The margin is defined as the perpendicular dis ...

328 7. SPARSE KERNEL MACHINES does not depend onn. Direct solution of this optimization problem would be very complex, and so we ...

7.1. Maximum Margin Classifiers 329 EliminatingwandbfromL(w,b,a)using these conditions then gives thedual representationof the m ...

330 7. SPARSE KERNEL MACHINES In Appendix E, we show that a constrained optimization of this form satisfies the Karush-Kuhn-Tuck ...

7.1. Maximum Margin Classifiers 331 Figure 7.2 Example of synthetic data from two classes in two dimensions showing contours of ...

332 7. SPARSE KERNEL MACHINES Figure 7.3 Illustration of the slack variablesξn  0. Data points with circles around them are sup ...

7.1. Maximum Margin Classifiers 333 where{an 0 }and{μn 0 }are Lagrange multipliers. The corresponding set of Appendix E KKT co ...

334 7. SPARSE KERNEL MACHINES model (7.13). The remaining data points constitute the support vectors. These have an> 0 and he ...

7.1. Maximum Margin Classifiers 335 Figure 7.4 Illustration of theν-SVM applied to a nonseparable data set in two dimensions. Th ...

336 7. SPARSE KERNEL MACHINES Section 1.4 mensionality. This is not the case, however, because there are constraints amongst the ...

7.1. Maximum Margin Classifiers 337 Figure 7.5 Plot of the ‘hinge’ error function used in support vector machines, shown in blue ...

338 7. SPARSE KERNEL MACHINES For comparison with other error functions, we can divide byln(2)so that the error function passes ...

7.1. Maximum Margin Classifiers 339 Another approach is to trainK(K−1)/ 2 different 2-class SVMs on all possible pairs of classe ...

340 7. SPARSE KERNEL MACHINES Figure 7.6 Plot of an -insensitive error function (in red) in which the error increases lin- early ...