past conflicts, interpersonal hatreds, and the like. Arguably, in com-
parison to other contexts of evaluation, it is on grant peer review
panels that evaluators exhibit their best behavior and are most in-
clined to be generous, in part because the decisions made will have
very little effect on their own working conditions and daily environ-
ment. Thus, panelists often report that they find colleagues from
other universities more gregarious than their own immediate col-
leagues—a reaction common, as well, among academics who share a
sabbatical year at a research institute.
Inconsistent criteria. Maintaining consistent criteria for judging
qualitative and quantitative proposals is crucial to panel legitimacy,
given that tensions between the two types of methods are found in
several social science disciplines, and that explanation and interpre-
tation, and positivism and hermeneutics, are fault lines separating
the social sciences from the humanities. A political scientist who says
he tries very hard “to judge the proposals on their own turf,” also
emphasizes,
I try to be consistent in asking the same kinds of questions
whether it was somebody who’s going to build a formal model
about corruption in Russia, or somebody who was going to access
how changes in international legal standing...[modify] the per-
ception[ofcorruption]...WhatIlookfor,first, is a research de-
sign that’s fairly explicit about the nature of the kinds of calls and
claims that are being made...Iwant to know the exact relation-
ships they’re trying to map out. I want to know something about
the alternative explanations, which ones are being considered,
which ones have already been rejected.Consistency is complicated by the mechanics of deliberation. Pan-
elists compare different subsets of proposals (defined by shared top-
ics, comparable relative ranking, or proximity in the alphabet) at dif-
Pragmatic Fairness / 141