100 Metastatistics for the Non-Bayesian Regression Runner
That discussion about metastatistics often dwells in the “airy realms” is unfortu-
nate. First, many of the issues discussed in this literature are of practical import. In
economics, Bayesian approaches are becoming increasingly popular: in the United
States, for example, the Food and Drug Administration (FDA) issued a call for com-
ments on a proposal about increased use of Bayesian methods (Food and Drug
Administration, US Department of Health and Human Services, 2006). Second,
it seems to me that much of the debate among practically minded researchers
is rooted in (frequently) unstated assumptions about the underlying philosoph-
ical justification for statistical procedures being debated. Consider the following
statement of the advantages of adopting a Bayesian approach to FDA testing:
- If we turn to Bayesian methods, difficult issues will be discussed in the right way
by the right people. - Some of the dilemmas that FDA decision makers face are artifacts of the (non-
Bayesian) statistical methods they use, and not due to demands of the scientific
method. - The Bayesian perspective provides the best way to think about evidence
(Goodman, 2004). - (In contrast to the usual approach) the Bayesian approach is ideally suited to
adapting to information that accrues during a trial, potentially allowing for
smaller, more informative, trials and for patients to receive better treatment.
Accumulating results can be assessed at any time, including continually, with
the possibility of modifying the design of the trial: for example, by slowing (or
stopping) or expanding accrual, imbalancing randomization to favour better-
performing therapies, dropping or adding treatment arms, and changing the
trial population to focus on patient subsets that are responding better to the
experimental therapies (Berry, 2006).
Such arguments are becoming increasingly common in domains outside of
medicine and are most easily understood by using some of the metastatistical
background.
3.1.1 Life, death, and statistical philosophy: an example
The issue of whether to use Bayesian or non-Bayesian methods has sometimes
quite literally involved life or death issues. The case of ECMO (extracorporeal
membrane oxygenation) is a useful example for those skeptical of the potential
importance of the debate.
ECMO was a therapy developed for use with infants with persistent pulmonary
hypertension: an ECMO machine circulates blood through an artificial lung back
into the bloodstream. The idea is described as providing adequate oxygen to the
baby while allowing time for the lungs and heart to rest or heal. The mortality
rate using conventional therapy was believed to be 40% (Ware, 1989), although
there is debate about whether that number was reasonable.^4 A possibly important
consideration is that the notion of providing additional oxygen for infants was
not obviously “safe.” See theBritish Journal of Ophthalmology(1974) and Silverman