148 Metastatistics for the Non-Bayesian Regression Runner
Consequently, Kline and Tobias (2008) provide an “informal test” of the valid-
ity of the “exclusion restriction” by asking whether, after usingoneinstrumental
variable, a model which excludes the other instrumental variable from the “struc-
tural equation” (equation (3.16) above) is well supported. Indeed, they calculate
the Bayes factor associated with the hypothesis that the effect of father’s BMI on
wages is zero while maintaining the validity of mother’s BMI as an instrumental
variable and find strong support for that hypothesis. They also find strong support
for the reverse.
While this does not exhaust the “specification testing” performed in the study, it
does indicate an attempt to put a hypothesis to the severest test possible. Interest-
ingly, although the study is clearly a “Bayesian” analysis, the authors found it useful
to conduct such a test in an “informal” way – without attempting to “shoehorn”
the specification testing into a complete Bayesian analysis.^66
3.8.2 Non-Bayesian doesn’t have to mean “severe”
My personal view is that statistical theory is often useful for situations in which we
are attempting to describe something that looks like a “chance set-up.” How one
might go from information gleaned in such situations to draw inferences about
otherdifferentsituations, however, is not at all obvious. Some Bayesians might
argue that one merely needs to formulate a prior, impose a “window on the world”
(a.k.a. a likelihood) and then use Bayes’ rule to revise our posterior probability. I
am obviously uncomfortable with such a view and find LeCam’s summary to the
point: “The only precept or theory which seems relevant is the following: ‘Do the
best you can’. This may be taxing for the old noodle, but even the authority of
Aristotle is not an acceptable substitute” (LeCam, 1977). This view also comports
well with C.S. Peirce’s classic description of a “severe test” I discussed earlier.
However, even at this level of vagueness and generality, it is worth observing
that such views are not shared by non-Bayesians, or if they are, there is no com-
mon vision of what is meant by severe testing. Except for the most committed
Bayesians, nothing in statistical theory tells you how to “infer the truth of var-
ious propositions.” As I have argued elsewhere (DiNardo, 2007), often the types
of theories economists seem interested in are so vague that it is often impossi-
ble to know what, in principle, would constitute “evidence” even in an “ideal”
situation.
Certainly it is the case that non-Bayesian researchers are frequently unwilling
to use statistical tools to change their views aboutsomeassessments. For one clear
example, compare Glaeser and Luttmer (1997) and Glaeser and Luttmer (2003).
The latter paper is a revised version of the former paper. The paper “develops a
framework to empirically test for misallocation. The methodology compares con-
sumption patterns for demographic subgroups in rent-controlled and free-market
places. [They] find that in New York City, which is rent-controlled, an economi-
cally and statistically significant fraction of apartments appears to be misallocated
across demographic subgroups”(Glaeser and Luttmer, 2003, p. 1027).
A significant difference between the two papers is that the latter, Glaeser and
Luttmer (ibid.), includes an interesting falsification test (not included in Glaeser