EVALUATIVE CRITERIA AND EPISTEMIC COMMUNITIES 91sort of evaluation can you, would you, send to the journal editor or to the author? As Shapiro
implies (in the second epigraph), if you are unaware or dismissive of research gestalts other than
your own, you will clearly be disappointed in this manuscript. To assess it on its own terms, you
must reorient yourself, recognize the legitimacy of such a study as science, and ask, “What are the
strengths of interpretive methodologies? What are the purposes of this kind of research?” Such a
reorientation is essential to understanding the interpretive gestalt and the evaluative criteria asso-
ciated with it.
One of the obstacles encountered by those seeking to strengthen interpretive epistemic com-
munities in political science, sociology, and similarly constituted disciplines is the allegation that
there are no criteria for judging the quality of interpretive research—the “anything goes” charge.
This allegation is disputable in light of the substantial, growing body of literature spanning sev-
eral social science disciplines that addresses the question of appropriate criteria for judging inter-
pretive research. In this chapter I assess this body of work. My purpose is to provide researchers
new to interpretive research with a vocabulary that they can use and discuss with (and convey to)
those interested in participating in and strengthening interpretive epistemic communities within
their respective disciplines.
The scholarly enterprise is built on the exercise of judgment. Whether it is assessing a dis-
sertation, evaluating a manuscript in the peer review process, or judging a research proposal for
funding, scholars sit in judgment of others and, likewise, submit their own scholarship to such
judgment. There are consequences of these judgments: some proportion of graduate students
fails to receive degrees, some manuscripts are never published, and many research proposals
go unfunded. Although individuals make these judgments, they do so in the context of a com-
plex structure of competing yet overlapping intellectual communities defined in numerous,
crosscutting ways, that is, in terms of discipline, subject matter, national and continental iden-
tities, research and funding networks, and diverse understandings of the purposes of research
and scholarship.
These distinct communities have been characterized as epistemic communities (Alston 1989,
Knorr Cetina 1999), and it is particular epistemic communities that are—or should be—the arbi-
ters of research quality for any given study. Despite periodic declarations of the necessity and
superiority of a single episteme (E.O. Wilson 1998, Laitin 2003)^2 , multiple epistemic communi-
ties define the landscape of the social sciences,^3 and a diversity of approaches, rather than a
unitarian hegemony, is arguably a more appropriate measure of progress or maturity in the social
sciences (Dryzek 1986, 1990; Rule 1997). To fully understand these communities’ judgments,
one must understand each one’s particular research gestalt—that bundle of shared epistemologi-
cal and ontological presuppositions, theoretical commitments, research goals, evaluative criteria,
and methodological and reading practices. This is not a simple task, for despite explicit method-
ological training in many disciplines, much learning is still craft based; that is, novices often learn
by reading exemplars and by collaborating with senior researchers, in an apprenticeship-type
arrangement similar to that of the craft guilds. In such learning by doing, by immersion, assump-
tions are not always made explicit. Instead, they form a background knowledge that knits prac-
tices together into a coherent whole. Without that background knowledge, a simple listing of
evaluative criteria, although useful in displaying a particular vocabulary, does little to communi-
cate a research gestalt because the terms are a kind of shorthand for the ontological and epistemo-
logical positions informing the community’s practices. But once one is steeped in that gestalt,
once its positions and habits of mind become second nature, then the relevant evaluative criteria
for assessing the quality of a particular study take on a self-evident character.
In what follows, I offer a very brief characterization of the interpretive gestalt as grounding for