works very well,” but then adds, “It’s just hard to articulate what it is.”
This chapter aims to clarify the complicated dynamics of group eval-
uation.
Producing Legitimacy and Belief through
Customary Rules
Max Weber and Emile Durkheim, two of the founding fathers of the
field of sociology, each wrote about the production of belief. Weber
identified various forms of legitimacy and suggested that the pro-
duction of rational-legal legitimacy requires belief in the use of im-
personal, abstract, and consistent rules.^6 This in turn requires the
bracketing of individual interest. Durkheim, in his writings about
the production of religious feelings and the mechanisms by which
people come to invest in the sacred, maintained that the sacred is
defined by its separation from the sphere of the profane through the
use of rituals—rule-bound processes.^7 Weber’s and Durkheim’s in-
sights are directly relevant to the actions of panel members. Funding
organizations provide evaluators with formal rules. These insti-
tutional mandates constrain the kinds of arguments that panelists
make by affecting the likelihood that specific criteria—whether a
proposal is excessively or insufficiently humanistic, comparative,
policy-oriented, and so on—will be invoked against or in favor of a
proposal. Evaluators do sometimes refer to specific guidelines to
bolster their arguments, or to resolve disagreements. But, as noted
in Chapter 2, program officers give panelists full sovereignty and
rarely enforce the mandates. A sociologist recalls that a proposal
that did not have the focus specified by the competition, but which
everyone judged to be excellent, was funded. Another sociologist,
downplaying the influence of the formal guidelines, comments, “I
doubt that most people read them all that carefully.”
As I explain in more detail later in this chapter, the rules that pan-
110 / Pragmatic Fairness