380 CATALYZING INQUIRY
10.3.5.3 The Review Process
Within the U.S. government, there are two styles of review. In the approach relying mainly on peer
review (used primarily by NIH and NSF), a proposal is evaluated by a review panel that judges its
merits, and the consensus of the review panel is the primary factor that influencing a decision that a
proposal does or does not merit funding. When program budgets are limited, as they usually are, the
program officer decides on actual awards from the pool of proposals designated as merit-worthy. In the
approach relying on program officer judgment (used primarily by DARPA), a proposal is generally
reviewed by a group of experts, but decisions about funding are made primarily by the program officer.
The dominant style of review mechanism in agencies that support life sciences research is peer
review. Peer review is intended as a method of ensuring the soundness of the science underlying a
proposal, and yet it has disadvantages. To quote an NRC report,^81
The current peer-review mechanism for extramural investigator-initiated projects has served biomedical
science well for many decades and will continue to serve the interests of science and health in the decades
to come. NIH is justifiably proud of the peer review mechanism it has put in place and improved over the
years, which allows detailed independent consideration of proposal quality and provides accountability
for the use of funds. However, any system that focuses on accountability and high success rates in
research outcomes may also be open to criticism for discriminating against novel, high-risk proposals that
are not backed up with extensive preliminary data and whose outcomes are highly uncertain. The prob-
lem is that high-risk proposals, which may have the potential to produce quantum leaps in discovery, do
not fare well in a review system that is driven toward conservatism by a desire to maximize results in the
face of limited funding resources, large numbers of competing investigators, and considerations of ac-
countability and equity. In addition, conservatism inevitably places a premium on investing in scientists
who are known; thus there can be a bias against young investigators.Almost by definition, peer review panels are also not particularly well suited to considering areas of
research outside their foci. That is, peer review panels include the individuals that they do precisely
because those individuals are highly regarded as experts within their specialties. Thus, an interdiscipli-
nary proposal that draws on two or more fields is likely to contain components that a review panel in a
single field is not able to evaluate as well as those components that do fall into the panel’s field.
A number of proposals have been advanced to support a track of scientific review outside the
standard peer review panels. For example, the NRC report recommended that NIH establish a special
projects program located in the office of the NIH director, funded at a level of $100 million initially to
increase over a period of 10 years to $1 billion a year, whose goal would be to foster the conduct of
innovative, high-risk research. Most importantly, the proposal calls for a set of program managers to
select and manage the projects supported under this program. These program managers would be
characterized primarily by an outstanding ability to develop or recognize unusual concepts and ap-
proaches to scientific problems. Review panels constituted outside the standard peer review mecha-
nisms and specifically charged with the selection of high-risk, high-payoff projects would provide
advice and input to program managers, but decisions would remain with the program managers.
Research initially funded through the special projects program that generated useful results would be
handed off after 3-5 years for further development and funding through standard NIH peer review
mechanisms. Whether this proposal, or a similar one, will be adopted remains to be seen.
Different agencies also have different approaches to the proposals they seek. For example, agencies
differ in the amount of detail that they insist potential grantees provide in these proposals. Depending
on the nature of the grant or contract sought, one agency might require only a short proposal of a few
pages and minimal documentation, whereas another agency might require many more pages, insisting
on substantial preliminary results and extensive documentation. An individual familiar with one kind
(^81) National Research Council, Enhancing the Vitality of the National Institutes of Health: Organizational Change to Meet New Chal-
lenges, The National Academies Press, Washington, DC, 2003, p. 93.