Pattern Recognition and Machine Learning

12.2.l'ru":ohilislkI'CA 585

FllIure12.15 Gillbs.,,,,>p!j"lllo<Bay<lslan

PCAsh<Mingplots oj Ino,

versus ~eralion number br

three " values. showing

tr"""tions betw..... tbe

th"'"moOts<A!heposterior

distribution.

,-"riabl,i'gi,-,nby1'(x)~N(Xlj',C)whe...now

C=WWT+'i'. (12,6~)

Eurr"e 12,19

S"na"12.4

AswithprobabilisticPCA,thi,moMIisim-"ri.rrlto'Olalionsin11><0latent'pace.

Histoocally,factoranal)',;shasbeenlhe,ubjerlofcOl1tro,-ersywroe"a!tempt<

h",-ebttn"'a<k:toplaceanintc'P"'t"liooontheind;vidualfaclon(thecOOfdinates

in z_space).whichh3.\ pm"enproblematicduetolr.e"""i<lcmifiabililyoffactOf

analysisassocimedwithMation'inthis'pace. Fromoorperspeoh-e,howe,-er.we

shall.iewfactoranalysisasa formoflalent"ariabledensilymodel.inwhichthe

formoftl>clalent'pacei'ofinterestbutnO!the particularchoiccofcoordinates

usedtodescrit>cil.Ifwewishtoremovethedegeneracya'sociatedwithlatent'pace

roIations.""emu,tcon'idernon-Gaussianlatent,-"riabledi'tribution,.gi"irrgrise 10

independentcomponent.n.lysi,(lCA)models.

Wecandetenni"etheparametersI'.\V."nd....inthefac!Ofan.ly,i,modelby

muimumlikelihood. 11 ..solutionforI'i'ag"ingivenbythe",,,,pie"'ean.How·

eyC'."nli~eprobabili,ticl'CA.lllcrei'nolongera closed-formmaximumlikelihood

solutionfor\V.",'hichmu.\ltherdorcbefoundi'er.li,'c1)'_Becausefacloranal)',i.is

a latentvariablemodeLthi'canbedon.usinganEMalgorilhm(R.binandThayer.

1982)!h"tis "nalogou,totheoneused(Ofpml>;lbili.tiePeA.Specihcally.lheE-'lep

eqnJtioo,areg;'-enby

where""eh",'edefi""d

E[zoJ = GWT",-'(xn- xl

E[z"z~J _ G+E[zo]E[z,,]T

(12,66)

(1267)

(l2,6H)

NOieth"tthi'i'e.pre,<edina for'"thaiin,-oh'esinycrsi"nofmalrices"fSilO,\I x,If

rathe'lhanDxD(ex"",,,,fortbeDxDdiagooalmatrixoJ'"'hosein,-ersei.''ri"ial