Pattern Recognition and Machine Learning

-, o , -, o , -, o ,

o

,

", •

, , ,<,

• ,

(^0) ~,

0 0

....

-, -, -,

, 0 , -, 0 -, 0 ,

,

",

, ,

-,

Flgu..12.12 Synt"'elic<lataillustratingtooEMalgorithm!ofPCAdefinedby(12.58)and(1259).(8)Adata

setXwiththedatapointsshownin1JI'e«l,t"ll"tM'W'i1!1l!>et'IMpMdpal""""""",IS(shownaseigenveclor1
scaledbyIt>esquafll'OOIS 04 theeigeJ'l\lllluel).(b)Initialconfigurat"'" 01 tooprincipalsul>sl>a<:<tdefinedbyW,
showninmd,tOO"lhfIrwiththefK'(Ijeclions 01 thelatll<11pointsZinlotoo<lataspace,giItoobyZWT,shownin
cyan,(ojAlter"""Mstep,toolaten!SI'B«lP>asbeenupdate<!wiIhZr>el(lnxed.(d)Me'tt>esuccess....E Slep,
It>e""'-'eo 01 Zhavubeenup<!atll<:1.~'"Ihogoooalr>rojecliQn$,withW h&k!fixed. (e)Aft....tilese<:o<l<lM
S!flp.<')Afterl!IeMC()<>;lEst"l'

S,uionI.J

Be<:au,",th~pm/xlhili>licPeAmoddhasa well·definedlikelilloodf"flCtion,we

<wIdemploycros,-,-.1idationtodelerminethe\"aJueofdi"",nsiooa!ityby"'Iecting

tit<:large,tloglikelihoodt>I1a '-alidationdatasetSuchanopprooch.hov.·~\-er.can

becomecomputationallyro<lly.p3rticularl)'ifweCQnsid<:,• probabilisticmiXlUre

ofPeAmodds(Tippingand Bishop. 1999a)in"hichweseek 10 <!etermi'"the

appropriatedimen,ionalily",paraltlyfortochcomponenlinlt1emixm""

Gi'-enthaiw.ha,-ea probabilislicformulalionofPeA,ils«msnatural 10 s«k

u Buye,ianapproach 10 modelseleclion. Todothi,.,,"'enee<! 10 marginalize 001

themodelparamele"/'.\V.und,,'wilh""peeltoappropriatepriordistribution'.

ThisCanbedonebyu,inga ,-ariation.lframeworkto.pproxim'letheallulylic.lly

intractablemurginaliUOi;oo,(Bi,hop.1mb).1lIcmarginallikelihoodv.lues.given

byttle,'ari.,ionallowerbour.d,cunlhenbec<>mpun:dforar.ngeofdifferent'"Tue'

"f;\Iar.dItie'"IuegivingIhtlargestmarginallikelihood",Iecloo_

l1ereweconsider.simplerapproachintroducoobyb.asedontherddmu"p-