Pattern Recognition and Machine Learning

Exercises 421 8.16 ( ) Consider the inference problem of evaluatingp(xn|xN)for the graph shown in Figure 8.38, for all nodesn∈{ ...

422 8. GRAPHICAL MODELS 8.25 ( ) In (8.86), we verified that the sum-product algorithm run on the graph in Figure 8.51 with node ...

9 Mixture Models and EM If we define a joint distribution over observed and latent variables, the correspond- ing distribution o ...

424 9. MIXTURE MODELS AND EM view of mixture distributions in which the discrete latent variables can be interpreted Section 9.2 ...

9.1.K-means Clustering 425 assigned vectorμk. Our goal is to find values for the{rnk}and the{μk}so as to minimizeJ. We can do th ...

426 9. MIXTURE MODELS AND EM (a) −2 0 2 −2 0 2 (b) −2 0 2 −2 0 2 (c) −2 0 2 −2 0 2 (d) −2 0 2 −2 0 2 (e) −2 0 2 −2 0 2 (f) −2 0 ...

9.1.K-means Clustering 427 Figure 9.2 Plot of the cost functionJgiven by (9.1) after each E step (blue points) and M step (red p ...

428 9. MIXTURE MODELS AND EM Section 2.3.7 but it can also make the determination of the cluster means nonrobust to outliers. We ...

9.1.K-means Clustering 429 K=2 K=3 K=10 Original image Figure 9.3 Two examples of the application of theK-means clustering algor ...

430 9. MIXTURE MODELS AND EM The image segmentation problem discussed above also provides an illustration of the use of clusteri ...

9.2. Mixtures of Gaussians 431 Figure 9.4 Graphical representation of a mixture model, in which the joint distribution is expres ...

432 9. MIXTURE MODELS AND EM instead of the marginal distributionp(x), and this will lead to significant simplifica- tions, most ...

9.2. Mixtures of Gaussians 433 (a) 0 0.5 1 0 0.5 (^1) (b) 0 0.5 1 0 0.5 (^1) (c) 0 0.5 1 0 0.5 1 Figure 9.5 Example of 500 point ...

434 9. MIXTURE MODELS AND EM Figure 9.7 Illustration of how singularities in the likelihood function arise with mixtures of Gaus ...

9.2. Mixtures of Gaussians 435 identifiability(Casella and Berger, 2002) and is an important issue when we wish to interpret the ...

436 9. MIXTURE MODELS AND EM We can interpretNkas the effective number of points assigned to clusterk. Note carefully the form o ...

9.2. Mixtures of Gaussians 437 −2 (^0) (a) 2 −2 0 2 −2 (^0) (b) 2 −2 0 2 (c) L=1 −2 0 2 −2 0 2 (d) L=2 −2 0 2 −2 0 2 (e) L=5 −2 ...

438 9. MIXTURE MODELS AND EM Gaussian components are shown as blue and red circles. Plot (b) shows the result of the initial E s ...

9.3. An Alternative View of EM 439 3.M step. Re-estimate the parameters using the current responsibilities μnewk = 1 Nk ∑N n=1 γ ...

440 9. MIXTURE MODELS AND EM family, the marginal distributionp(X|θ)typically does not as a result of this sum- mation. The pres ...