Robert_V._Hogg,_Joseph_W._McKean,_Allen_T._Craig

666 Bayesian Statistics

11.2MoreBayesianTerminologyandIdeas

SupposeX′=(X 1 ,X 2 ,...,Xn) represents a random sample with likelihoodL(x|θ)
and we assume a prior pdfh(θ). The joint marginal pdf ofXis given by

g 1 (x)=

∫∞

−∞

L(x|θ)h(θ)dθ.

This is often called the pdf of thepredictive distributionofXbecause it provides
the best description of the probabilities aboutXgiven the likelihood and the prior.
An illustration of this is provided in expression (11.1.6) of Example 11.1.1. Again
note that this predictive distribution is highly dependent on the probability models
forXand Θ.
In this section, we consider two classes of prior distributions. The first class is
the class of conjugate priors defined by:

Definition 11.2.1.A class of prior pdfs for the family of distributions with pdfs
f(x|θ), θ∈ Ω, is said to define aconjugate family of distributionsif the
posterior pdf of the parameter is in the same family of distributions as the prior.

As an illustration, consider Example 11.1.5, where the pmf ofXigivenθwas
Poisson with meanθ. In this example, we selected a gamma prior and the resulting
posterior distribution was of the gamma family also. Hence the gamma pdf forms
a conjugate class of priors for this Poisson model. This was true also for Example
11.1.2 where the conjugate family was beta and the model was a binomial, and for
Example 11.1.3, where both the model and the prior were normal.
To motivate our second class of priors, consider the binomial model,b(1,θ),
presented in Example 11.1.2. Thomas Bayes (1763) took as a prior the beta dis-
tribution withα=β=1,namelyh(θ)=1, 0 <θ<1, zero elsewhere, because he
argued that he did not have much prior knowledge aboutθ. However, we note that
this leads to the estimate of
(
n
n+2

)(

y

n

)

+

(

2

n+2

)(

1

2

)

.

We often call this ashrinkageestimate because the estimatey/nis pulled a lit-
tle toward the prior mean of 1/2, although Bayes tried to avoid having the prior
influence the inference.
Haldane (1948) did note, however, that if a prior beta pdf exists withα=β=0,
then the shrinkage estimate would reduce to the mley/n. Of course, a beta pdf
withα=β= 0 is not a pdf at all, for it would be such that

h(θ)∝

1

θ(1−θ)

, 0 <θ< 1 ,

zero elsewhere, and ∫

1

0

c

θ(1−θ)

dθ