roles of an evaluator and a chair. This switching made it impossible
for him to seem neutral when coordinating the deliberations; conse-
quently, his credibility in both capacities was undermined. Another
chair was strongly criticized for his patronizing and time-wasting
managerial style. A panelist describes him as repeatedly saying, “‘This
is what we’re doing,’ when everyone knew what we were doing. It was
time to move on. So there was a language that was employed that
was exhausting and repetitious and maybe unnecessary. And I think
it probably could have been cut off.”
During deliberations, a panel typically works its way from the top
of the list of applicants (those receiving the largest number of high
marks) to the bottom. It is generally understood that applicants who
received the highest total rating prior to the meeting will receive
funding, so those cases may be noted at the start of deliberations, but
they are not usually discussed. Likewise, applications that received
low ratings often are not discussed. The bulk of the deliberations
concern the applicants whose proposals have received mixed ratings,
a group whose size depends on what proportion of the proposals will
be funded. As an economist explained:
The applications that were ranked six hundred and seven hun-
dred, you could tell in ten seconds [that they were no good]. It
just gets harder the closer you get to the cutoff and that’s where
we spent more time. So I think there’s a time constraint in all our
lives and we have to focus attention on some things. I don’t think
people got short shrift in terms of the amount of time. I think it
was reasonable. You’re not going to do any better than this.The “middle of the pack” proposals, those that receive mixed rat-
ings, are flawed, but their faults differ. The greatest challenge for
panelists is often the comparison of incommensurate flaws. These
middle-rank proposals consume the most time because individual
How Panels Work / 47