Pattern Recognition and Machine Learning

Exercises 601 12.14 (*) Thenumberofindependentparametersinthecovariancematrixfortheproba- bilisticPCAmodelwithanM-dimensionallat ...

602 12.CONTINUOUSLATENTVARIABLES 12.22 (**) Writedownanexpressionfortheexpectedcomplete-dataloglikelihoodfunc- tionforthefactora ...

Exercises 603 12.29 (**)EmSupposethattwovariablesZlandZ2areindependentsothatp(zl'Z2)= P(Zl)P(Z2)' Showthatthecovariancematrixbet ...

...

13 Sequential Data So far in this book, we have focussed primarily on sets of data points that were as- sumed to be independent ...

606 13. SEQUENTIAL DATA Figure 13.1 Example of a spectro- gram of the spoken words “Bayes’ theo- rem” showing a plot of the inte ...

13.1. Markov Models 607 Figure 13.2 The simplest approach to modelling a sequence of ob- servations is to treat them as independ ...

608 13. SEQUENTIAL DATA Figure 13.3 A first-order Markov chain of ob- servations{xn}in which the dis- tributionp(xn|xn− 1 )of a ...

13.1. Markov Models 609 Figure 13.5 We can represent sequen- tial data using a Markov chain of latent variables, with each obser ...

610 13. SEQUENTIAL DATA The joint distribution for this model is given by p(x 1 ,...,xN,z 1 ,...,zN)=p(z 1 ) [N ∏ n=2 p(zn|zn− 1 ...

13.2. Hidden Markov Models 611 Figure 13.6 Transition diagram showing a model whose la- tent variables have three possible state ...

612 13. SEQUENTIAL DATA Figure 13.7 If we unfold the state transition dia- gram of Figure 13.6 over time, we obtain a lattice, o ...

13.2. Hidden Markov Models 613 k=1 k=2 k=3 0 0.5 1 0 0.5 1 0 0.5 1 0 0.5 1 Figure 13.8 Illustration of sampling from a hidden Ma ...

614 13. SEQUENTIAL DATA Figure 13.9 Example of the state transition diagram for a 3-state left-to-right hidden Markov model. Not ...

13.2. Hidden Markov Models 615 Figure 13.11 Top row: examples of on-line handwritten digits. Bottom row: synthetic digits sam- p ...

616 13. SEQUENTIAL DATA exponentially with the length of the chain. In fact, the summation in (13.11) cor- responds to summing o ...

13.2. Hidden Markov Models 617 and make use of the definitions ofγandξ, we obtain Q(θ,θold)= ∑K k=1 γ(z 1 k)lnπk+ ∑N n=2 ∑K j=1 ...

618 13. SEQUENTIAL DATA Gaussian emission densities we havep(x|φk)=N(x|μk,Σk), and maximization of the functionQ(θ,θold)then giv ...

13.2. Hidden Markov Models 619 the messages that are propagated along the chain (Jordan, 2007). We shall focus on the most widel ...

620 13. SEQUENTIAL DATA represents a vector of lengthKwhose entries correspond to the expected values of znk. Using Bayes’ theor ...