Palgrave Handbook of Econometrics: Applied Econometrics

John DiNardo 135

The possibility of accidentally drawing a Knut Vik square or accidentally putting
just the junior rabbits into the control group and the senior ones into the experi-
mental group illustrates a flaw in the usual...argument that sees randomization
as injecting “objective” or gambling-device probabilities into the problem of
inference. (Savageet al.,1962, p. 88)
Savage’s example of having all the young rabbits in the control group and all
the older rabbits in the treatment group is perhaps more recognizable than the
distinction between Latin squares and Knut Vik squares, which come from clas-
sical agricultural experimentation. It is also related to the problem of random or
pseudo-random number generation. If generating a sequence of random 0s and
1s, for instance, by chance (if only infrequently), some of these sequences will be
undesirable – for example, if drawing a sequence of 1,000 numbers, it is possible
that one draws all zeroes or all ones.
In the context of the RCT, the analogous problem, loosely based on our ECMO
example, is described in Table 3.2. Assignment to the two groups was randomized
but “bad luck” happened and the control group was comprised of the lightest birth-
weight babies (and viewed by the doctors as usually the least healthy) who were,
on average, potentially in need of ECMO at much older ages (again, viewed by the
doctors as an indicator of general frailty).

Table 3.2 Bad luck in a hypothetical RCT on

the efficacy of ECMO pre-treatment values of key

variables (standard errors in parentheses)

Pre-treatment variable Treatment Control

Birth weight (grams) 3.26 2.1

(0.22) (0.23)

Age (days) 52 140

(13) (14)

In this example, the problem is that the treatment and control groups are not
“balanced.” The treatment babies are (before treatment) healthier on average than
the control babies. The typical non-Bayesian would generally find the numbers
in Table 3.2 evidence against the validity of the design.^39
For Bayesians, this suggests that the logic of randomization is flawed. If “balance”
is the primary reason for randomization, why not deliberately divide into groups
which look similar (and would “pass” a balancing test) without randomizationper
se. How does introducing uncertainty into treatment assignment help? Indeed, to
some Bayesians, all it can do is lower the value of the experiment. From Berry and
Kadane (1997):

Suppose a decision maker has two decisions available,d 1 andd 2. These two

decisions have current (perhaps posterior to certain data collection) expected