Pattern Recognition and Machine Learning

(Jeff_L) #1
562 12.CONTINUOUSLATENTVARIABLES

chapter,weshallconsidertechniquestodetermineanappropriatevalueofIV!from
thedata.
Tobeginwith,considertheprojectionontoa one-dimensionalspace(M= 1).
Wecandefinethedirectionofthisspaceusinga D-dimensionalvectorUl,which
forconvenience(andwithoutlossofgenerality)weshallchoosetobea unitvector

sothatufUl = 1 (notethatweareonlyinterestedinthedirectiondefinedbyUl,


notinthemagnitudeofUlitself).EachdatapointXnisthenprojectedontoa scalar
valueufXn.Themeanoftheprojecteddataisufxwherex is thesamplesetmean
givenby

(12.1)

andthevarianceoftheprojecteddatais givenby

whereS is thedatacovariancematrixdefinedby

1 N
S= -NLJ"(xn- x)(xn- x)T.
n=l

(12.2)

(12.3)

AppendixE


WenowmaximizetheprojectedvarianceUfSUlwithrespecttoUl.Clearly,thishas

tobea constrainedmaximizationtopreventIlulll.....00.Theappropriateconstraint


comesfromthenormalizationconditionufUl = 1. Toenforcethisconstraint,


weintroducea LagrangemultiplierthatweshalldenotebyAI,andthenmakean
unconstrainedmaximizationof

(12.4)

BysettingthederivativewithrespecttoUlequaltozero,weseethatthisquantity
willhavea stationarypointwhen

( 12.5)

whichsaysthatUlmustbeaneigenvectorofS.Ifweleft-multiplybyufandmake


useofufUl= 1,weseethatthevarianceis givenby


(12.6)

andsothevariancewillbea maximumwhenwesetUlequaltotheeigenvector
havingthelargesteigenvalueAI. Thiseigenvectorisknownasthefirstprincipal
component.
Wecandefineadditionalprincipalcomponentsinanincrementalfashionby
choosingeachnewdirectiontobethatwhichmaximizestheprojectedvariance
Free download pdf