90 MEANING AND METHODOLOGY
❖❖❖I feel nervous when researchers claim that their accounts of reality are far too rich and
complex to be expressed as measurement.
—APSA Political Methodology Section e-mail list participant (February 2003)With methods, as with people, if you focus only on
their limitations you will always be disappointed.
—Ian Shapiro (2002, 612)Consider the following thought experiment. Imagine that you are a traditionally trained political
scientist, steeped in what might be called the “variables gestalt,” which encompasses, among
other things, a commitment to measurement, hypothesis testing, and causal analysis. You have
been asked to review a manuscript comparing two government agencies, based on interpretive
scholarship. I construct the thought experiment using political science because the interpretive
epistemic community is not as well established in that discipline as it is in some other social
sciences, making this a plausible scenario.^1 But the scenario could as well hold for sociology or
organizational studies or any other social scientific area of study.
Setting aside epistemological and ontological presuppositions, as well as choices of the par-
ticular theoretical lens informing your work (e.g., realism versus neorealism in international rela-
tions theory or rational choice theory versus political psychology in American politics), when
you sit down to read an empirical study, you bring with you a set of standard expectations about
the logic of research, as well as a developed, discipline-directed set of critical reading skills honed
through training in your field. You are expecting to encounter independent and dependent vari-
ables (whose operationalization can be critically assessed); you are expecting to see causal rea-
soning and perhaps a causal model (the internal logic of which you know how to evaluate); you
are expecting to find statistical analysis—be it a regression equation, ANOVA, or some other
technique (you might even turn immediately to the tables, before reading any of the text, in order
to see what sort of analysis is involved); and you are trained to assess the appropriateness of
techniques as a function of nominal, ordinal, or interval-level data, to know whether underlying
statistical assumptions are met, and so on. Over time, you have become quite practiced at reading
and critiquing research conducted within this gestalt, and your application of evaluative criteria is
almost second nature. A particular reading experience results from these expectations and prac-
tices; its particularity is evinced by the fact that both graduate and undergraduate students must
acquire these critical reading skills.
The interpretive study you have been sent as part of the peer review process is based on sixteen
months of participant-observation, numerous supplementary in-depth interviews, and document
analysis. The manuscript offers a radically different reading experience: None of the variables
have been operationalized in the ways to which you are accustomed—in fact, what the variables
are is not even clear, let alone which ones are dependent and which are independent; no causal
model is offered; there are no tables reporting statistical analyses; there is no discussion of
generalizability.
In short, given your customary reading experience, this study is unrecognizable as a piece of
scientific research: It does not fit your sense of what rigorous, objective research looks like. Your
standard set of evaluative criteria simply does not apply, and you question whether such research
qualifies as social science. Your experience is similar to that of the e-mail list participant quoted
in the first epigraph: the “rich and complex” data reported in the study make you “nervous.” What