Sampling 291r(xk+1|xk)< ξ, the move is rejected, and xk+1is set equal to xk. Remember, however,
that even when a move is rejected, the value xk to which xk+1is “reset” must be
considered as the next point in the Markov chain and must be used in the calculation
of estimators. Thus, in any Markov chain, there will be points that are repeated and
used more than once in the calculation of averages.
In the Markov chain generated by the M(RT)^2 algorithm, each of the points
x 1 ,x 2 ,...,xn will have an associated probabilityπ 1 (x),π 2 (x),...,πn(x). We wish to
prove that in the limit of an infinite chain, limn→∞πn(x) =f(x), for which we will
use inductive reasoning. That is, we will show that ifπn(x) =f(x) for somen, then it
follows thatπn+1(x) =f(x). The proof proceeds first by finding a recursive relation
between forπn+1(x) in terms ofπn(x), and then by showing thatf(x) is afixed-point
of the recursion.
The recursion is derived as follows:πn+1(x) has a contribution from attempted
moves that start at y, lead to x, and are accepted. It also has contributions from
attempted moves that start at x, lead to y, and are rejected. Let us begin with the
former. The quantityπn(y)dy is the probability that a given point is in a neighborhood
dy about the value y in thenth step of the Markov chain. Thus, the probability that
a trial move yields a new point x starting from y in dy isA(x|y)T(x|y)πn(y)dy, which
involves a product of the trial and acceptance probabilities, as expected. Therefore, the
probability that a point x is reached from any starting point is obtained by integrating
the product over all y: ∫
A(x|y)T(x|y)πn(y)dy.Similarly, the probability that an attempted move to any y starting from a particular
x is rejected is ∫
[1−A(y|x)]T(y|x)dy,which needs to be multiplied byπn(x), the probability density for being at x to begin
with. When these two expressions are combined, the total probability densityπn+1(x)
becomes
πn+1(x) =∫
A(x|y)T(x|y)πn(y)dy +πn(x)∫
[1−A(y|x)]T(y|x)dy. (7.3.30)From eqn. (7.3.30), it is possible to show that the distributionf(x) is a fixed point
of the recursion. We substitute the assumed condition of the induction,πn(x) =f(x),
into the recursion relation, which yields
πn+1(x) =∫
A(x|y)T(x|y)f(y)dy +f(x)∫
[1−A(y|x)]T(y|x)dy. (7.3.31)However, due to the detailed balance condition in eqn. (7.3.23), eqn.(7.3.31) reduces
to
πn+1(x) =f(x)∫
T(y|x)dy =f(x), (7.3.32)which follows from eqn. (7.3.24) and completes the proof.