Pattern Recognition and Machine Learning

Exercises 557

Figure 11.15 A probability distribution over two variablesz 1
andz 2 that is uniform over the shaded regions
and that is zero everywhere else.

z 1

z 2

11.11 ( ) www Show that the Gibbs sampling algorithm, discussed in Section 11.3,

satisfies detailed balance as defined by (11.40).

11.12 ( ) Consider the distribution shown in Figure 11.15. Discuss whether the standard

Gibbs sampling procedure for this distribution is ergodic, and therefore whether it

would sample correctly from this distribution

11.13 ( ) Consider the simple 3-node graph shown in Figure 11.16 in which the observed

nodexis given by a Gaussian distributionN(x|μ, τ−^1 )with meanμand precision

τ. Suppose that the marginal distributions over the mean and precision are given

byN(μ|μ 0 ,s 0 )andGam(τ|a, b), whereGam(·|·,·)denotes a gamma distribution.

Write down expressions for the conditional distributionsp(μ|x, τ)andp(τ|x, μ)that

would be required in order to apply Gibbs sampling to the posterior distribution

p(μ, τ|x).

11.14 ( ) Verify that the over-relaxation update (11.50), in whichzihas meanμiand

varianceσi, and whereνhas zero mean and unit variance, gives a valuez′iwith

meanμiand varianceσ^2 i.

11.15 ( ) www Using (11.56) and (11.57), show that the Hamiltonian equation (11.58)

is equivalent to (11.53). Similarly, using (11.57) show that (11.59) is equivalent to

(11.55).

11.16 ( ) By making use of (11.56), (11.57), and (11.63), show that the conditional dis-

tributionp(r|z)is a Gaussian.

Figure 11.16 A graph involving an observed Gaussian variablexwith
prior distributions over its meanμand precisionτ.

μ τ

x