I said to myself, “You win some, you lose some.”... One of the
reasons I pulled back from vetoing is because we were quite sure
that not all of the fellowships would be accepted, and that there
would be an alternative source of funding. We were quite con-
vinced that a person who I supported and ended up on the wait
list would nevertheless be funded. And that trade-off ultimately
made it seem acceptable.In the end, panelists seem to support a pragmatic understanding
of evaluation—one that is at odds with the ideals of using consistent
standards and of ignoring proposals’ contexts to consider only their
intrinsic qualities. Panelists seem acutely aware that scholarly quality
is relationally defined within the universe of the group of propos-
als being evaluated. Which one is used as point of reference shifts
throughout the deliberation. Across all disciplines, contextual rank-
ing is central to the art of evaluation.
Bracketing Self-Interest and Personal Ties
The influence of self-interest and personal ties on the outcome of de-
liberations is viewed as entirely illegitimate in a panel review. This is
consistent with Weber’s view on the production of rational legiti-
macy, which requires the application of impersonal and consistent
rules. When I asked a panel chair how panelists would likely react to
a member saying, “This is a student of a close colleague of mine and
I’d love to see his work funded,” she replied,
It’s just not a consideration. It can’t be a consideration. You prob-
ably noticed that the panel rejected quite roundly a student of [a
panelist], who described him as the best student he’d had in
twenty-five years...Nobodythought about that. There are other
types of biases that other people bring to the meetings, but theyPragmatic Fairness / 125