Pattern Recognition and Machine Learning

4.1. Discriminant Functions 181 (McCullagh and Nelder, 1989). Note, however, that in contrast to the models used for regression, ...

182 4. LINEAR MODELS FOR CLASSIFICATION Figure 4.1 Illustration of the geometry of a linear discriminant function in two dimensi ...

4.1. Discriminant Functions 183 R 1 R 2 R 3 ? C 1 notC 1 C 2 notC 2 R 1 R 2 R 3 C (^1)? C 2 C 1 C 3 C 2 C 3 Figure 4.2 Attemptin ...

184 4. LINEAR MODELS FOR CLASSIFICATION Figure 4.3 Illustration of the decision regions for a mul- ticlass linear discriminant, ...

4.1. Discriminant Functions 185 where ̃Wis a matrix whosekthcolumn comprises theD+1-dimensional vector w ̃k=(wk 0 ,wTk)Tand ̃xis ...

186 4. LINEAR MODELS FOR CLASSIFICATION −4 −2 0 2 4 6 8 −8 −6 −4 −2 0 2 4 −4 −2 0 2 4 6 8 −8 −6 −4 −2 0 2 4 Figure 4.4 The left ...

4.1. Discriminant Functions 187 − 6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6 − 6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6 Figure 4.5 Example of a sy ...

188 4. LINEAR MODELS FOR CLASSIFICATION −2 2 6 −2 0 2 4 −2 2 6 −2 0 2 4 Figure 4.6 The left plot shows samples from two classes ...

4.1. Discriminant Functions 189 J(w)= wTSBw wTSWw (4.26) whereSBis thebetween-classcovariance matrix and is given by SB=(m 2 −m ...

190 4. LINEAR MODELS FOR CLASSIFICATION the weights becomes equivalent to the Fisher solution (Duda and Hart, 1973). In particul ...

4.1. Discriminant Functions 191 4.1.6 Fisher’s discriminant for multiple classes We now consider the generalization of the Fishe ...

192 4. LINEAR MODELS FOR CLASSIFICATION These covariance matrices have been defined in the originalx-space. We can now define si ...

4.1. Discriminant Functions 193 where the nonlinear activation functionf(·)is given by a step function of the form f(a)= { +1,a ...

194 4. LINEAR MODELS FOR CLASSIFICATION whereMdenotes the set of all misclassified patterns. The contribution to the error assoc ...

4.1. Discriminant Functions 195 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 −1 −0.5 0 0.5 1 ...

196 4. LINEAR MODELS FOR CLASSIFICATION Figure 4.8 Illustration of the Mark 1 perceptron hardware. The photograph on the left sh ...

4.2. Probabilistic Generative Models 197 Figure 4.9 Plot of the logistic sigmoid function σ(a)defined by (4.59), shown in red, t ...

198 4. LINEAR MODELS FOR CLASSIFICATION Note that in (4.57) we have simply rewritten the posterior probabilities in an equivalen ...

4.2. Probabilistic Generative Models 199 Figure 4.10 The left-hand plot shows the class-conditional densities for two classes, d ...

200 4. LINEAR MODELS FOR CLASSIFICATION −2 −1 0 1 2 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 Figure 4.11 The left-hand plot shows ...