Pattern Recognition and Machine Learning

12.1.I'<incipalCl)m..."n~ntAnal}'s;, 569

.,':----;!---;-"

_.S 0 3

."., '~'" ~.•••._',

" ..,-. .'..'~.•~''''

':r-'---~+·_..-:--'~-J

~,

.,

."

Acomparison 01 pro:ipalcompo-

Mntanalysis.... 111 Fisha(slinaar

discriminant 101 """",<*man""'"

alityr&duclion. Heretoodatain

twodimansions,belongingtotwo

classessIIOWI1inredandblue.is

tobePfOI"Cled ontoas.ingledi·

mension. PCAc/>xlsasthedirec·

tion 01 maximumvaria""e.sIIOWI1

trythama9""taCo""'.wt11chleads

tostrongclassoverlap. whereas

!heFisl>efIiMardiSCfOrnillanttakes

accoun1 <:A tooclasslabels and

leadsto aprojectionontotheg<ean

CUM! giving much t>etler class

separation

Fig"",12.7

Fig"",12.8 Visualilatlon 01 !heoill'low<latalIetobtained
tryprojoectingthe<lataontothelirsttwoprin.
cipalcompone<1ts.The<ed,blue,and9r&en
pointscorre-spondto!he'IamiNI(,'t>omo-
genoous',and'8nnula~flowoonligurations
",specriveIy.

12.1.4 peAforhigh-dimensionaldata

Insomeapplication.ofpliTlCipalcomponentanalysis.thenumberofdatapoints

is smallerthant!>cdimensionalityoftroedata'pace.FOI"example.",emightwantto

applyPeAtoadata<elofa fewhundredimages,eachof,,'hichrorrespoOOstoa

"eetorina'paceofpoIentially.....mlmilliondimensiOlls(COITespondingtnthfl'e

enlour"aluesforeachofthepi.",lsintroeimage),NOIethatina D-<limen,ionalspace

a setofjYpoints.",'hereN <D.definesa linearsubspa::e",hosedimensi"nality

isat ""'stN - 1,andSOthereislinlepointinapplyingPeAfor,'alue<ofM

thaI"'"greaterthanN - I, Indeed,if"'epelf"",,PeAwewillfindthatatleast

D- N+Ioftheeigen".luesartlero.eorrespnndingtQeigenvectorsaloog",hose

direclioosthedata<elhas10mvarianee.Funhem>ore.typicalalgOl"ithm,forfinding

theeigen,'eet""ofaDxDmatrixha"ea computatiooaleoslthmscaleslikeO(D~J.

aOOsoforappliealionssuchastheimagee,ample.adirec'applicationofPeAwill

becomputatiooallyinfe,,-sibJe.

W.canresoh'ethisproblemasfoIl",",'" Fir;l.letusdefineXtobethe(N"DJ·