Pattern Recognition and Machine Learning

580 12.CONTlNlJOlJSI"Ht;I'ITVi\RIARlES

Fig"..12.11 ProbabilisticPCAvisoo,zsbon 01 aportion0I1he""!lowdatasetlo<Ihe!irsl 100 (lata»einls,The
left..,...ndplotoIIOWSIheI'O"leoo<meanproj9c1ionsoIlhfI(latapoimsonlheprincipalsubspace.The,;gtrI·hi\nd
plotis obtainedbyfirslran<lomlyomitting30%0I1hevariable.aloo.andlhenus>rlgEM 10 MndIeI""mi......

values.NoteI!IaIeac/1datapoinl1henNoSallea.,onemissingmea.u,ementbutlhoallheploti.""ry..mia,to

lheonaobtainedwit"""lmiss....valL>ll$

Ewrrise /2,/7

subspacetominimize!hesquaredreoonslruCtiooerrorin'oIhichtheproje<:tion,are

C.,N.

Weeangh'ea,implephysicalanalogyforthisEM algorithm.whichiseasily

visualizedforD = 2 andM = 1. Coo,idera collectioonfdatapoint',n tWI)

dimension',aodlettileu""'-dimensiunalprincipalsubspaceberepresentedbya <ohd

rod.NowatlaCheachdatapointtothenxIviaa,pringoo<:)"ingHooI;:e',law("umJ

energyi,propol1ional 10 ,liesquareoflilespring".length). Inll1eE'tel',wekeep

thenxIhedandallowtheattachmentpoint'tn,Iideupand<I<,wnll1enxI'"a,to

minimizell1ee",,'llY,Thiscau",.eachattachmentpoint(independently) 10 position

Itselfattheorthogonalpmjeclionofthec~spondingdatapointontothenxI. In

theM'tel'.wekeeptheattachmentpoiOl'fil<edandthenreleasetilenxIandallowit

tom'>,'e 10 tileminimumenergyposilion.11IeE andM'tepsarethenrepeateduntil

a,uitablec""vergencecri.eri""is..,isfled.a.isilluSlratedinFigure12.12.

12.2.3 BayesianpeA

S<JfarinOIlrdi",""iooofPeA.wehave",'.nledIhal tile'"Ine,IIfor,lie

dl,nen,ionalit)"oftileprincipal.ubspaceisgi"en, Inpraclice.".-enlmtcOOosea

suilable,..I""according 10 theapplication. For,isuali,a,ion.wege""",nychoose

.\1 = 2.whereasforOIherapplication,theapprorrialCchoicefor,1/ma)"beless

dea,. Oneappmao:hi. 10 pi",theeigen"alue'peclrumforlhedataset.analog,•."

10 theexampleinFigure12.4fortheoff_linedigitsdalaSCI,andlooktoseeif lite

eige",,,I....nmurallyformtwogroupscomprisinga setof,mall,'aluesseparatedby

a ,ign;flcantgapfroma",tofrelativel)"large,'alues,indicatinga naturalcholccf<>r

AI,Inpractice.suchagapi,oflen'''''seen