Pattern Recognition and Machine Learning

Exercises 61 Note that the precise relationship between thew ̃coefficients andwcoefficients need not be made explicit. Use this ...

62 1. INTRODUCTION 1.17 (

) www The gamma function is defined by Γ(x)≡ ∫∞ 0 ux−^1 e−udu. (1.141) Using integration by parts, pr ...

Exercises 63 1.20 (

) www In this exercise, we explore the behaviour of the Gaussian distribution in high-dimensional spaces. C ...

64 1. INTRODUCTION 1.24 (

) www Consider a classification problem in which the loss incurred when an input vector from classCki ...

Exercises 65 Table 1.3 The joint distributionp(x, y)for two binary variables xandyused in Exercise 1.39. y 01 x 0 1/3 1/3 1 0 1/ ...

66 1. INTRODUCTION 1.40 (
) By applying Jensen’s inequality (1.115) withf(x)=lnx, show that the arith- metic mean of a set of re ...

2 Probability Distributions In Chapter 1, we emphasized the central role played by probability theory in the solution of pattern ...

68 2. PROBABILITY DISTRIBUTIONS damentally ill-posed, because there are infinitely many probability distributions that could hav ...

2.1. Binary Variables 69 where 0
μ
1 , from which it follows thatp(x=0|μ)=1−μ. The probability distribution overxcan therefore ...

70 2. PROBABILITY DISTRIBUTIONS Figure 2.1 Histogram plot of the binomial dis- tribution (2.9) as a function ofmfor N=10andμ=0. ...

2.1. Binary Variables 71 given by (2.3) and (2.4), respectively, we have E[m]≡ ∑N m=0 mBin(m|N, μ)=Nμ (2.11) var[m]≡ ∑N m=0 (m−E ...

72 2. PROBABILITY DISTRIBUTIONS μ a=0. 1 b=0. 1 0 0.5 1 0 1 2 3 μ a=1 b=1 0 0.5 1 0 1 2 3 μ a=2 b=3 0 0.5 1 0 1 2 3 μ a=8 b=4 0 ...

2.1. Binary Variables 73 μ prior 0 0.5 1 0 1 2 μ likelihood function 0 0.5 1 0 1 2 μ posterior 0 0.5 1 0 1 2 Figure 2.3 Illustra ...

74 2. PROBABILITY DISTRIBUTIONS large data set. For a finite data set, the posterior mean forμalways lies between the prior mean ...

2.2. Multinomial Variables 75 So, for instance if we have a variable that can takeK=6states and a particular observation of the ...

76 2. PROBABILITY DISTRIBUTIONS We can solve for the Lagrange multiplier∑ λby substituting (2.32) into the constraint kμk=1to gi ...

2.2. Multinomial Variables 77 Figure 2.4 The Dirichlet distribution over three variablesμ 1 ,μ 2 ,μ 3 is confined to a simplex ( ...

78 2. PROBABILITY DISTRIBUTIONS Figure 2.5 Plots of the Dirichlet distribution over three variables, where the two horizontal ax ...

2.3. The Gaussian Distribution 79 N=1 0 0.5 1 0 1 2 3 N=2 0 0.5 1 0 1 2 3 N=10 0 0.5 1 0 1 2 3 Figure 2.6 Histogram plots of the ...

80 2. PROBABILITY DISTRIBUTIONS functional dependence of the Gaussian onxis through the quadratic form ∆^2 =(x−μ)TΣ−^1 (x−μ) (2. ...