John DiNardo 993.5.3 If the DGP is irrelevant is the likelihood really everything? 127
3.5.4 What probabilities aren’t – the non–bayesian view 129
3.5.5 What should “tests” do? 131
3.5.6 Randomization and severity 132
3.6 Case study 1: “medication overuse headache” 136
3.6.1 What is medication overuse headache? Nosology and
dubious ontology 137
3.6.2 Some salient background 137
3.6.2.1 Early history 137
3.6.2.2 The evidence 138
3.6.2.3 First criticism 139
3.6.3 Redefining MOH to avoid a severe test 139
3.7 Case study 2: “union wage premium” 142
3.7.1 Early history 142
3.7.2 A battery of severe tests 142
3.8 Concluding remarks 146
3.8.1 Bayesian doesn’t have to mean “not severe” 146
3.8.2 Non-Bayesian doesn’t have to mean “severe” 148
3.1 Introduction
“Everything has already been said, but perhaps notbyeveryone andto
everyone.”^1The purpose of the somewhat silly title of this chapter is to warn the reader what
not to expect. This is not intended as a “proper” introduction to metastatistics,
which I could not write, of which there are several very good ones.^2 Given the
enormous amount of writing on the subject, it is not surprising then that none of
the ideas or arguments will be original.^3
An even sillier title that some might use to describe the following is: “A jaundiced
appraisal of some extreme Bayesian views by someone who just doesn’t get it.”
That is, of course, not my intent. Rather, I think that it is sometimes useful for
the practically minded non-Bayesian regression runner (like myself) to consider
some of the basic “philosophical” issues at the heart of statistics and econometrics.
My purpose is also to bring some of the issues debated in metastatistics or the “phi-
losophy of induction” literature “back to earth” from the somewhat airy realms in
which they often dwell and toward the more messy realms of the low sciences,
addressing them to an audience, like myself, who aren’t philosophers but don’t
think that philosophy is necessarily a synonym for “useless.” It is true that the
discussion is often very mathematical, sometimes filled with obscure polysyllabic
words of Greek or Latin origin, and pages and pages of definitions where the reader
is expected to suspend disbelief before something of practical import seems to enter
the discussion. Partly as a consequence, I will talk about ideas that a philosopher
would define with much more precision: if you have philosophical inclinations,
consider yourself forewarned.