11.3. Gibbs Sampler 675Example 11.3.1.Suppose the random variableXhas pdffX(x)={
2 e−x(1−e−x)0<x<∞
0elsewhere.
(11.3.5)SupposeY andX|Yhave the respective pdfs
fY(y)={
2 e−^2 y 0 <x<∞
0elsewhere
(11.3.6)fX|Y(x|y)={
e−(x−y) y<x<∞
0elsewhere.
(11.3.7)Suppose we generate random variables by the following algorithm:
- GenerateY ∼ fY(y) as in expression (11.3.6).
- GenerateX ∼ fX|Y(x|Y) as in expression (11.3.7).
Then, by Theorem 11.3.1,Xhas the pdf (11.3.5). Furthermore, it is easy to generate
from the pdfs (11.3.6) and (11.3.7) because the inverses of the respective cdfs are
given byFY−^1 (u)=− 2 −^1 log(1−u)andFX−|^1 Y(u)=−log(1−u)+Y.
As a numerical illustration, the R functioncondsim1(found at the site listed
in the Preface) uses this algorithm to generate observations from the pdf (11.3.5).
Using this function, we performedm=10,000 simulations of the algorithm. The
sample mean and standard deviation werex=1.495 ands=1.112. Hence a 95%
confidence interval forE(X)is(1. 473 , 1 .517), which traps the true valueE(X)=
1 .5; see Exercise 11.3.4.
For the last example, Exercise 11.3.3 establishes the joint distribution of (X, Y)
and shows that the marginal pdf ofXis given by (11.3.5). Furthermore, as shown in
this exercise, it is easy to generate from the distribution ofXdirectly. In Bayesian
inference, though, we are often dealing with conditional pdfs, and theorems such as
Theorem 11.3.1 are quite useful.
The main purpose of presenting this algorithm is to motivate another algorithm,
called theGibbs Sampler, which is useful in Bayes methodology. We describe it
in terms of two random variables. Suppose (X, Y)haspdff(x, y). Our goal is to
generate two streams of iid random variables, one onXand the other onY.The
Gibbs sampler algorithm is:
Algorithm 11.3.1(Gibbs Sampler).Letmbe a positive integer, and letX 0 ,an
initial value, be given. Then fori=1, 2 , 3 ,...,m,- GenerateYi|Xi− 1 ∼f(y|x).
- GenerateXi|Yi∼f(x|y).
Note that before entering theith step of the algorithm, we have generatedXi− 1.
Letxi− 1 denote the observed value ofXi− 1. Then, using this value, generate se-
quentially the newYifrom the pdff(y|xi− 1 ) and then draw (the new)Xifrom the