Pattern Recognition and Machine Learning

12.1.PrincipalC01n[>OM"1AnalJsis 561 Flgu,e12.2 P'if>cipalcompooonta",,~"seeks"$pace 01 !owe,dimensionality.kt"(>WIlas! ...

562 12.CONTINUOUSLATENTVARIABLES chapter,weshallconsidertechniquestodetermineanappropriatevalueofIV!from thedata. Tobeginwith,co ...

Exercise 12.1 Section12.2.2 AppendixC 12.1.PrincipalComponentAnalysis 563 amongstallpossibledirectionsorthogonaltothosealreadyco ...

564 12.CONTINUOUSLATENTVARIABLES wherethe{Zni}dependontheparticulardatapoint,whereasthe{bdareconstants thatarethesameforalldatap ...

12.1.PrincipalComponentAnalysis 565 minimizeJ=UISU2'subjecttothenormalizationconstraintuIU2=1.Usinga LagrangemultiplierA2toenfor ...

566 12.COl\'TINUOliSLATf;I\'T\'ARIAIILES Figure12.3 Themean~'"xaklogwith!heII"'tlou'PCAe;gerrvecl<)rllUl,. ..'" lorthe011-... ...

11.1.I'rindpall.:"l11pon~nt Anal~·.i. 567 FIIIUr. 1 :1:.5 An",>gi",,1~mpIeIromlI>e011·_digilsdata...ttOll"1herwithitsPeAre ...

568 12.CONTINUOUSLATENTVARIABLES (^10022) 90 00' B O 80 000 0 0 70 0 08 0 0 ,=~o 0 cPO^00 tj ~ 60 50 O~ ~OOID -2 -2 40 (^246) -2 ...

12.1.I'<incipalCl)m..."n~ntAnal}'s;, 569 .,':----;!---;-" _.S 0 3 ."., '~'" ~.•••._', " ..,-. .'..'~.•~'''' ':r-'---~+·_..-:- ...

570 12.CONTINUOUSLATENTVARIABLES dimensionalcentreddatamatrix,whosenthrowis givenby(xn- X)T.Thecovari- ancematrix(12.3)canthenbe ...

Section12.2.2 Section12.2.3 Section8.1.4 12.2.ProbabilisticpeA 571 WecanderiveanEMalgorithmforPCAthatiscomputationallyefficient ...

572 11.CONTINUOUSLAT!::NTVANIM1LI::S , / ,.- ,, ,, ,, , Flgu..12.9 I\n~I"'tfat""oIlt>eII"""fativevi&woI1t>epabi!;st",P ...

12.2.ProbabilisticpeA wheretheDxDcovariancematrixCis definedby C=WWT+0-^2 1. 573 (12.36) Thisresultcanalsobederivedmoredirectlyb ...

574 12.CONTINUOUSLATENTVARIABLES Figure12.10 TheprobabilisticpeAmodelfora datasetofNobser- vationsofxcanbeexpressedasa directedg ...

12.2.ProbabilisticpeA 575 Again,weshallassumethattheeigenvectorshavebeenarrangedinorderofdecreas- ingvaluesofthecorrespondingeig ...

576 12.CONTINUOUSLATENTVARIABLES Therotationalinvarianceinlatentspacerepresentsa formofstatisticalnoniden- tifiability,analogous ...

12.2.ProbabilisticpeA 577 dimensionality.Ifwerestrictthecovariancematrixtobediagonal,thenit hasonlyD independentparameters,andso ...

578 12.CONTINUOUSLATENTVARIABLES areassumedindependent,thecomplete-dataloglikelihoodfunctiontakestheform N Inp(X,ZIJL,W,(J2)= L{ ...

(12.58) 12.2.ProbabilisticpeA 579 eigenvectordecompositionofthesamplecovariancematrix, theEMapproachis iterativeandsomightappear ...

580 12.CONTlNlJOlJSI"Ht;I'ITVi\RIARlES Fig"..12.11 ProbabilisticPCAvisoo,zsbon 01 aportion0I1he""!lowdatasetlo<Ihe!irsl 100 ( ...