Palgrave Handbook of Econometrics: Applied Econometrics

John DiNardo 107

descends. What no scientist would do is to divide the earth’s surface into

small plots and select some of these at random for the places to perform the

experiments. Randomizers might take one of two attitudes to this behavior of

scientists. They could either say it is irrational and ought to be changed or else

claim that experiments in physics and chemistry are, in some crucial respect,

unlike those in biology and psychology, neither of which would appear to

be very promising lines of defence. (Urbach, 1985, p. 273)

6. Probability does not exist:

The abandonment of superstitious beliefs about the existence of the Phlogis-

ton, the Cosmic Ether, Absolute Space and Time,...or Fairies and Witches

was an essential step along the road to scientific thinking. Probability, too, if

regarded as something endowed with some kind of objective existence, is no

less a misleading misconception.... (deFinetti, 1974, p. 3)

3.2.2 An introductory puzzle

One of the most unusual aspects of metastatistics is that people on different
sides of the debate citethe same exampleto make the case that the other side
is wrong.
Consider the following example. Mayo (1979) and Mayo and Kruse (2002) have
cited it as an example of a flaw in the usefulness of Bayesian reasoning while
Bayesians routinely cite such examples (see Poirier, 1995) to argue that this is evi-
dence of a flaw in non-Bayesian reasoning! It consists of a comparison of what
inferences are justified in two different “experiments.”
In both cases, suppose you are interested in the fraction of black ballsμin a huge
urn (we ignore the complications arising from issues of sampling with or without
replacement) that is “well-mixed” and has only red and black balls. Denote the
null hypothesis asH 0 :μ=0.5 and the alternative asH 1 :μ>0.5. Denote the
random variable “number of black balls” byXand the sample size asn.

Experiment A Experiment B

Method:Declare in advance that you

are going to pick 12 balls randomly

from the urn.

Method:Instead of predesignating or

decidingin advance of the experiment

that you are going to draw 12 observa-

tions, you decide that you are going to

keep drawing balls from the urn until

you get at least 3 red balls.

Result:9 of the 12 balls are black. The

usual estimate isμ=^34.

Result:You draw the third red ball on

the 12th attempt. 9 of the 12 are black

and the usual estimate isμ=^34.

In both experiments, 12 balls were drawn. In both experiments, 9 of the 12 were
black. There are several different “loaded” questions one can ask when comparing