Pattern Recognition and Machine Learning

3.4. Bayesian Model Comparison 161 be simplŷy(x)=1, from which we obtain (3.64). Note that the kernel function can be negative ...

162 3. LINEAR MODELS FOR REGRESSION different models, and we shall examine this term in more detail shortly. The model evidence ...

3.4. Bayesian Model Comparison 163 Figure 3.12 We can obtain a rough approximation to the model evidence if we assume that the p ...

164 3. LINEAR MODELS FOR REGRESSION Figure 3.13 Schematic illustration of the distribution of data sets for three models of diff ...

3.5. The Evidence Approximation 165 a Bayesian approach, like any approach to pattern recognition, needs to make as- sumptions a ...

166 3. LINEAR MODELS FOR REGRESSION From Bayes’ theorem, the posterior distribution forαandβis given by p(α, β|t)∝p(t|α, β)p(α, ...

3.5. The Evidence Approximation 167 whereMis the dimensionality ofw, and we have defined E(w)=βED(w)+αEW(w) = β 2 ‖t−Φw‖^2 + α 2 ...

168 3. LINEAR MODELS FOR REGRESSION Figure 3.14 Plot of the model evidence versus the orderM, for the polynomial re- gression mo ...

3.5. The Evidence Approximation 169 Multiplying through by 2 αand rearranging, we obtain αmTNmN=M−α ∑ i 1 λi+α =γ. (3.90) Since ...

170 3. LINEAR MODELS FOR REGRESSION Figure 3.15 Contours of the likelihood function (red) and the prior (green) in which the axe ...

3.5. The Evidence Approximation 171 single variablexis given by σ^2 ML= 1 N ∑N n=1 (xn−μML)^2 (3.96) and that this estimate is b ...

172 3. LINEAR MODELS FOR REGRESSION lnα −5 0 5 lnα −5 0 5 Figure 3.16 The left plot showsγ(red curve) and 2 αEW(mN)(blue curve) ...

Exercises 173 mapping from input variables to targets. In the next chapter, we shall study an anal- ogous class of models for cl ...

174 3. LINEAR MODELS FOR REGRESSION 3.2 ( ) Show that the matrix Φ(ΦTΦ)−^1 ΦT (3.103) takes any vectorvand projects it onto the ...

Exercises 175 together with a training data set comprising input basis vectorsφ(xn)and corre- sponding target vectorstn, withn=1 ...

176 3. LINEAR MODELS FOR REGRESSION Show that the corresponding posterior distribution takes the same functional form, so that p ...

Exercises 177 3.20 ( ) www Starting from (3.86) verify all of the steps needed to show that maxi- mization of the log marginal l ...

...

4 Linear Models for Classification In the previous chapter, we explored a class of regression models having particularly simple ...

180 4. LINEAR MODELS FOR CLASSIFICATION ways of using target values to represent class labels. For probabilistic models, the mos ...