untitled

356 15 Combining Information

temperature, obtaining 30.2◦±0.3◦C. One now has two independent normal

distributions. Combining the two measurements is the same as combining

the two distributions.

In this chapter the process of meta-analysis is formally defined and proven.

Combining discrete distributions is covered first, followed by the continuous

case. Stochastic inference is a special case of meta-analysis. More gener-

ally, one can combine two Bayesian networks (BNs). Conversely, the meta-

analysis process can itself be expressed in terms of BNs. This is shown in

section 15.3. The temperature measurement example above is an example

of the combination of observations that are continuous distributions. PDs

are not only a means of expressing the uncertainty of a observation, they

can themselves be observations. In other words, PDs can have PDs. A large

number of statistical tests are based on this idea, which is discussed in sec-

tion 15.4. The last section introduces an interesting variation on information

combination, called Dempster-Shafer theory.

15.1 Combining Discrete Information

We first consider the case of combining two discrete PDs. That means we

have two independent random variables X and Y, whose values are discrete

rather than continuous. For example, a patient might seek multiple inde-

pendent opinions from practitioners, each of which gives the patient their

estimates of the probabilities of the possible diagnoses. Combining these two

discrete random variables into a single random variable is done as follows:

Discrete Information CombinationTheorem

LetXandYbe two discrete random variables that represent two independent obser-

vations of the same phenomenon. If there exists a valuevsuch that bothPr(X=v)

andPr(Y =v)are positive, then there is a random variableZthat combines the

information of these two observations, whose PD is

Pr(Z=v)=

∑Pr(X=v)Pr(Y=v)

wPr(X=w)Pr(Y=w)

ProofSinceXandYare independent, their JPD is given byPr(X=u, Y=

v)=Pr(X=u)Pr(Y =v). The random variablesXandYare combined

by conditioning on the event(X=Y). This will be well defined if and only

ifPr(X=Y)is positive, which is the case when there is some valuevsuch

thatPr(X=Y =v)is positive. When this is true, the distribution of the