Pattern Recognition and Machine Learning

600 12.CONTINUOUSLATENTVARIABLES

theeigenvectorsofS.Becausethesesolutionsareallequivalent,itisconvenientto

choosetheeigenvectorsolution.

12.3 (*) Verifythattheeigenvectorsdefinedby(12.30)arenormalizedtounitlength,

assumingthattheeigenvectorsVihaveunitlength.

12.4 (*)Imm Supposewereplacethezero-mean,unit-covariancelatentspacedistri-

bution(12.31)intheprobabilisticPCAmodelbya generalGaussiandistributionof

theformN(zlm,~).Byredefiningtheparametersofthemodel,showthatthisleads

toanidenticalmodelforthemarginaldistributionp(x)overtheobservedvariables

foranyvalidchoiceofmand~.

12.5 (**) Letxbea D-dimensionalrandomvariablehavinga Gaussiandistribution

givenbyN(xIJL,~),andconsidertheM-dimensionalrandomvariablegivenby

y = Ax+b whereAisanM xD matrix. Showthaty alsohasa Gaussian

distribution,andfindexpressionsforitsmeanandcovariance.Discusstheformof

thisGaussiandistributionforM <D,forM= D,andforM>D.

12.6 (*)Imm Drawa directedprobabilisticgraphfortheprobabilisticPCAmodel

describedinSection12.2inwhichthecomponentsoftheobservedvariablex are

shownexplicitlyasseparatenodes.HenceverifythattheprobabilisticPCAmodel

hasthesameindependencestructureasthenaiveBayesmodeldiscussedinSec-

tion8.2.2.

12.7 (**) Bymakinguseoftheresults(2.270)and(2.271)forthemeanandcovariance

ofa generaldistribution,derivetheresult(12.35)forthemarginaldistributionp(x)

intheprobabilisticPCAmodel.

12.8 (**)Imm Bymakinguseoftheresult(2.116),showthattheposteriordistribution

p(zlx)fortheprobabilisticPCAmodelis givenby(12.42).

12.9 (*) Verifythatmaximizingtheloglikelihood(12.43)fortheprobabilisticPCA

modelwithrespecttotheparameterJLgivestheresultJLML = xwherexisthe

meanofthedatavectors.

12.10 (**) Byevaluatingthesecondderivativesoftheloglikelihoodfunction(12.43)for

theprobabilisticPCAmodelwithrespecttotheparameterJL,showthatthestationary

pointJLML= xrepresentstheuniquemaximum.

12.11 (**)Imm Showthatinthelimit(Y2-.0,theposteriormeanfortheprobabilistic

PCAmodelbecomesanorthogonalprojectionontotheprincipalsubspace,asin

conventionalPCA.

12.12 (**) For(Y2 >0 showthattheposteriormeanintheprobabilisticPCAmodelis

shiftedtowardstheoriginrelativetotheorthogonalprojection.

12.13 (**) Showthattheoptimalreconstructionofa datapointunderprobabilisticPCA,

accordingtotheleastsquaresprojectioncostofconventionalPCA,is givenby

(12.94)