Pattern Recognition and Machine Learning

Exercises 419

Table 8.2 The joint distribution over three binary variables. a b c p(a, b, c)

0 0 0 0.192

0 0 1 0.144

0 1 0 0.048

0 1 1 0.216

1 0 0 0.192

1 0 1 0.064

1 1 0 0.048

1 1 1 0.096

8.3 ( ) Consider three binary variablesa, b, c∈{ 0 , 1 }having the joint distribution

given in Table 8.2. Show by direct evaluation that this distribution has the property

thataandbare marginally dependent, so thatp(a, b) =p(a)p(b), but that they

become independent when conditioned onc, so thatp(a, b|c)=p(a|c)p(b|c)for

bothc=0andc=1.

8.4 ( ) Evaluate the distributionsp(a),p(b|c), andp(c|a)corresponding to the joint

distribution given in Table 8.2. Hence show by direct evaluation thatp(a, b, c)=

p(a)p(c|a)p(b|c). Draw the corresponding directed graph.

8.5 ( ) www Draw a directed probabilistic graphical model corresponding to the

relevance vector machine described by (7.79) and (7.80).

8.6 ( ) For the model shown in Figure 8.13, we have seen that the number of parameters

required to specify the conditional distributionp(y|x 1 ,...,xM), wherexi∈{ 0 , 1 },

could be reduced from 2 MtoM+1by making use of the logistic sigmoid represen-

tation (8.10). An alternative representation (Pearl, 1988) is given by

p(y=1|x 1 ,...,xM)=1−(1−μ 0 )

∏M

i=1

(1−μi)xi (8.104)

where the parametersμirepresent the probabilitiesp(xi=1), andμ 0 is an additional

parameters satisfying 0 
μ 0 
 1. The conditional distribution (8.104) is known as

thenoisy-OR. Show that this can be interpreted as a ‘soft’ (probabilistic) form of the

logical OR function (i.e., the function that givesy=1whenever at least one of the

xi=1). Discuss the interpretation ofμ 0.

8.7 ( ) Using the recursion relations (8.15) and (8.16), show that the mean and covari-

ance of the joint distribution for the graph shown in Figure 8.14 are given by (8.17)

and (8.18), respectively.

8.8 ( ) www Show thata⊥⊥b, c|dimpliesa⊥⊥b|d.

8.9 ( ) www Using the d-separation criterion, show that the conditional distribution

for a nodexin a directed graph, conditioned on all of the nodes in the Markov

blanket, is independent of the remaining variables in the graph.