Pattern Recognition and Machine Learning

1 Introduction The problem of searching for patterns in data is a fundamental one and has a long and successful history. For ins ...

2 1. INTRODUCTION Figure 1.1 Examples of hand-written dig- its taken from US zip codes. tackled using handcrafted rules or heuri ...

1. INTRODUCTION 3 also preserve useful discriminatory information enabling faces to be distinguished from non-faces. These featu ...

4 1. INTRODUCTION Figure 1.2 Plot of a training data set ofN= 10 points, shown as blue circles, each comprising an observation o ...

1.1. Example: Polynomial Curve Fitting 5 sin(2πx)and then adding a small level of random noise having a Gaussian distri- bution ...

6 1. INTRODUCTION Figure 1.3 The error function (1.2) corre- sponds to (one half of) the sum of the squares of the displacements ...

1.1. Example: Polynomial Curve Fitting 7 x t M=0 0 1 −1 0 1 x t M=1 0 1 −1 0 1 x t M=3 0 1 −1 0 1 x t M=9 0 1 −1 0 1 Figure 1.4 ...

8 1. INTRODUCTION Figure 1.5 Graphs of the root-mean-square error, defined by (1.3), evaluated on the training set and on an ind ...

1.1. Example: Polynomial Curve Fitting 9 x t N=15 0 1 −1 0 1 x t N= 100 0 1 −1 0 1 Figure 1.6 Plots of the solutions obtained by ...

10 1. INTRODUCTION x t lnλ=− 18 0 1 −1 0 1 x t lnλ=0 0 1 −1 0 1 Figure 1.7 Plots ofM=9polynomials fitted to the data set shown i ...

1.1. Example: Polynomial Curve Fitting 11 Table 1.2 Table of the coefficientsw
forM= 9 polynomials with various values for the r ...

12 1. INTRODUCTION give us some important insights into the concepts we have introduced in the con- text of polynomial curve fit ...

1.2. Probability Theory 13 Figure 1.10 We can derive the sum and product rules of probability by considering two random variable ...

14 1. INTRODUCTION from (1.5) and (1.6), we have p(X=xi)= ∑L j=1 p(X=xi,Y=yj) (1.7) which is thesum ruleof probability. Note tha ...

1.2. Probability Theory 15 and is simply “the probability ofX”. These two simple rules form the basis for all of the probabilist ...

16 1. INTRODUCTION p(X, Y) X Y=2 Y=1 p(Y) p(X) X X p(X|Y=1) Figure 1.11 An illustration of a distribution over two variables,X, ...

1.2. Probability Theory 17 Suppose instead we are told that a piece of fruit has been selected and it is an orange, and we would ...

18 1. INTRODUCTION Figure 1.12 The concept of probability for discrete variables can be ex- tended to that of a probability dens ...

1.2. Probability Theory 19 that the probability ofxfalling in an infinitesimal volumeδxcontaining the pointx is given byp(x)δx. ...

20 1. INTRODUCTION finite sum over these points E[f] 1 N ∑N n=1 f(xn). (1.35) We shall make extensive use of this result when w ...