2/How Panels Work

I see the need for people to make very careful and [informed] judg-

ments for the academic community to function. It’s more as an obli-

gation than an entitlement, and I’m willing to make those judgments

because I, on the one hand, have confidence in my own experience,

and on the other hand, I defer to people’s judgment that would be in

a position to make good decisions... I think it’s experience-based. I

would like to think that people have looked at my work and said,

“This is good work and he’s a very fine scholar himself, and that’s why

we want him to help us evaluate what’s good scholarship.”

Political scientist

T

he institutional framework of evaluation that structures fund-
ing decisions in the academic world is not secret. Nevertheless,
most of this “nuts and bolts” information—ranging from fund-
ing programs’ objectives and formal criteria of evaluation to how
panels are formed and what panelists are asked to do—is not widely
known. In this chapter, then, I describe the objectives of the five
funding agencies studied and the formal evaluation criteria associ-
ated with those objectives; the structure of the evaluation process,
including the role of the program officer, the selection of panelists
and screeners, and the pre-deliberation ranking work; and, lastly, the
mechanics of the deliberations.
Drawing on the work of Karin Knorr-Cetina, I understand these

22