Pattern Recognition and Machine Learning

10.7. Expectation Propagation 513

−5 (^05) θ 10 −5 (^05) θ 10
Figure 10.16 Examples of the approximation of specific factors for a one-dimensional version of the clutter
problem, showingfn(θ)in blue,efn(θ)in red, andq\n(θ)in green. Notice that the current form forq\n(θ)controls
the range ofθover whichefn(θ)will be a good approximation tofn(θ).
pass through all factors is less than some threshold. Finally, we use (10.208) to
evaluate the approximation to the model evidence, given by
p(D)(2πvnew)D/^2 exp(B/2)
∏N
n=1
{
sn(2πvn)−D/^2
}
(10.223)
where
B=
(mnew)Tmnew
v

−

∑N

n=1

mTnmn

vn

. (10.224)

Examples factor approximations for the clutter problem with a one-dimensional pa-

rameter spaceθare shown in Figure 10.16. Note that the factor approximations can

have infinite or even negative values for the ‘variance’ parametervn. This simply

corresponds to approximations that curve upwards instead of downwards and are not

necessarily problematic provided the overall approximate posteriorq(θ)has posi-

tive variance. Figure 10.17 compares the performance of EP with variational Bayes

(mean field theory) and the Laplace approximation on the clutter problem.

10.7.2 Expectation propagation on graphs

So far in our general discussion of EP, we have allowed the factorsfi(θ)in the

distributionp(θ)to be functions of all of the components ofθ, and similarly for the

approximating factors ̃f(θ)in the approximating distributionq(θ). We now consider

situations in which the factors depend only on subsets of the variables. Such restric-

tions can be conveniently expressed using the framework of probabilistic graphical

models, as discussed in Chapter 8. Here we use a factor graph representation because

this encompasses both directed and undirected graphs.