Palgrave Handbook of Econometrics: Applied Econometrics

John DiNardo 119

Then there is the “conditional” version of the law of total probability: as before,

let theAjbe mutually exclusive events, forj= 1 ...kand

∑k
j= 1 P(Aj)=1 and, if
0 <P(B)<1:

P(B)=

∑k

j= 1

P(B|Aj)P(Aj).

What this says is that ifP(B)is the probability of some event, and it can be
accompanied by some of thekmutually exclusive eventsAjin some way, then
the probability thatP(B)occurs is merely the sum of the different waysBcan occur
withAjtimes the probability ofP(Aj).
Using equations (3.2), (3.3) and (3.1), rearranging, and applying this last
operation to the denominator yields:

P(Ai|B)=

P(B|Ai)P(Ai)

P(B)

(3.4)

P(Ai|B)=

P(B|Ai)P(Ai)

∑k

j= 1 P(B|Aj)P(Aj)

. (3.5)

So far, there seems nothing particularly remarkable. However, here the agreement
ends. Consider a “Note on Bayes’ rule” by the non-Bayesian Feller (1950, p. 125):

In [the above formulas] we have calculated certain conditional probabilities

directly from the definition. The beginner is advised always to do so and not to

memorize the formula [Bayes’ rule, equation (3.5)]...Mathematically, [Bayes’

rule] is a special way of writing [the definition of conditional probability] and

nothing more. The formula is useful in many statistical applications of the type

described in [the above] examples and we have used it there. Unfortunately,

Bayes’s rule has been somewhat discredited by metaphysical applications...

In routine practice this kind of argument can be dangerous. A quality control

engineer is concerned with one particular machine and not with an infinite

population of machines from which one was chosen at random. He has been

advised to use Bayes’s rule on the grounds that it is logically acceptable and cor-

responds to our way of thinking. Plato used this type of argument to prove the

existence of Atlantis, and philosophers used it to prove the absurdity of New-

ton’s mechanics. But for our engineer the argument overlooks the circumstance

that he desires success and that he will do better by estimating and minimizing

the sources of various types of errors in prediction and guessing.

Feller’s suggestion that the engineer will do better by minimizing the various
types oferrorsis one issue where, at least rhetorically, non-Bayesians differ from
Bayesians. For Feller, the focus is on using statistics (or other methods) to put
ideas to the test, rejecting those that fail and advancing provisionally with those
that survive. Bayes rule is a formula aboutrevisingone’s epistemic probabilities
incrementally. This distinction will become apparent when we apply Bayes’ rule to
estimation.