Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

676 Bayesian Statistics

pdff(x|yi), whereyiis the observed value ofYi. In advanced texts, it is shown that

Yi

D

→ Y∼fY(y)

Xi

D

→ X∼fX(x), (11.3.8)

asi→∞,and
1
m

∑m

i=1

W(Xi)

P

→E[W(X)],asm→∞. (11.3.9)

Note that the Gibbs sampler is similar but not quite the same as the algorithm
given by Theorem 11.3.1. Consider the sequence of generated pairs

(X 1 ,Y 1 ),(X 2 ,Y 2 ),...,(Xk,Yk),(Xk+1,Yk+1).

Note that to compute (Xk+1,Yk+1), we need only the pair (Xk,Yk) and none of the
previous pairs from 1 tok−1. That is, given the present state of the sequence, the
future of the sequence is independent of the past. In stochastic processes such a
sequence is called aMarkov chain. Under general conditions, the distribution of
Markov chains stabilizes (reaches an equilibrium or steady-state distribution) as the
length of the chain increases. For the Gibbs sampler, the equilibrium distributions
are the limiting distributions in the expression (11.3.8) asi→∞. How large should
ibe? In practice, usually the chain is allowed to run to some large valueibefore
recording the observations. Furthermore, several recordings are run with this value
ofiand the resulting empirical distributions of the generated random observations
are examined for their similarity. Also, the starting value forX 0 is needed; see
Casella and George (1992) for a discussion. The theory behind the convergences
given in the expression (11.3.8) is beyond the scope of this text. There are many
excellent references on this theory. A discussion from an elementary level can be
found in Casella and George (1992). An informative overview can be found in
Chapter 7 of Robert and Casella (1999); see also Lehmann and Casella (1998). We
next provide a simple example.

Example 11.3.2.Suppose (X, Y) has the mixed discrete-continuous pdf given by

f(x, y)=

{ 1

Γ(α)

1

x!y

α+x− (^1) e− 2 y y>0;x=0, 1 , 2 ,...
0elsewhere,
(11.3.10)
forα>0. Exercise 11.3.5 shows that this is a pdf and obtains the marginal pdfs.
The conditional pdfs, however, are given by
f(y|x)∝yα+x−^1 e−^2 y (11.3.11)
and
f(x|y)∝e−y
yx
x!

. (11.3.12)

Hence the conditional densities are Γ(α+x, 1 /2) and Poisson (y), respectively. Thus
the Gibbs sampler algorithm is, fori=1, 2 ,...,m,

1. GenerateYi|Xi− 1 ∼Γ(α+Xi− 1 , 1 /2).


2. GenerateXi|Yi∼Poisson(Yi).