Pattern Recognition and Machine Learning

AppendixA

InChapter9,wediscussedprobabilisticmodelshavingdiscretelatentvariables,such

asthemixtureofGaussians.Wenowexploremodelsinwhichsome,orall,ofthe

latentvariablesarecontinuous. Animportantmotivationforsuchmodelsisthat

manydatasetshavethepropertythatthedatapointsalllieclosetoa manifoldof

muchlowerdimensionalitythanthatoftheoriginaldataspace. Toseewhythis

mightarise,consideranartificialdatasetconstructedbytakingoneoftheoff-line

digits,representedbya 64x 64pixelgrey-levelimage, andembeddingit ina larger

imageofsize 100 x 100bypaddingwithpixelshavingthevaluezero(corresponding

towhite pixels)inwhichthelocationandorientationofthedigitisvariedat random,

asillustratedinFigure12.1.Eachoftheresultingimagesisrepresentedbya pointin

the 100 x 100=10,OOO-dimensionaldataspace.However,acrossa datasetofsuch

images,thereareonlythreedegreesoffreedomofvariability,correspondingtothe

verticalandhorizontaltranslationsandtherotations. Thedatapointswilltherefore

liveona subspaceofthedataspacewhoseintrinsicdimensionalityisthree. Note

559