March 2020, ScientificAmerican.com 81Naomi Oreskes is a professor of the history of science
at Harvard University. She is author of Why Trust Science?
(Princeton University Press, 2019) and co-author
of Discerning Experts (University of Chicago, 2019).OBSERVATORY
KEEPING AN EYE ON SCIENCEIllustration by Martin Gee
One of the challenges of writing about science is that impor
tant concepts are not always exciting, and it is no small feat to
make a dull subject shine. Recent events, however, have high
lighted a topic that is both deadly dull and deadly serious: in
stru ment calibration.
Calibration is the process of making sure an instrument is
working accurately. Usually this involves testing against a known
standard (or set of standards). Every scientist who works with an
instrument learns to calibrate it; organizations that make many
measurements, such as the U.S. Geological Survey, have protocols
for ensuring that it is done regularly and accurately. Calibration
services are a major part of the work of the National Institute of
Standards and Technology, on which both industry and other fed
eral agencies rely.
But it is not something that you do once and forget about;
instruments drift, so they have to be checked regularly. Without
calibration, the measurements we make may be meaningless. And
this, it turns out, is what has happened with a highly consequen
tial instrument used every day across America: the Breathalyzer,
used to determine whether a driver has had too much to drink.
In 2017 a Massachusetts judge threw out thousands of drunk
driving convictions on the grounds that the kind of Breathalyzer
used was not reliable. This ruling followed an earlier agreement
among the state’s district attorneys and lawyers re presenting
alleged drunk drivers that data from breath tests would not be
used at trial (except for serious offenses), after evidence emerged
that the results were questionable. By one account, the decision
affected 35,000 outstanding cases from 2011 to 2017. Another
account suggests that the total number of affected cases could
exceed 58,000. Other states have also questioned convictions
based on Breathalyzer results: in New Jersey more than 20,000
drunkdriving convictions have been called into question.
Drunk driving is a huge problem. Every year more than a mil
lion Americans are arrested for it and, according to the Centers
for Disease Control and Prevention, more than 10,000 deaths can
be attributed to it. Society has a compelling interest in identifying
drunk drivers and getting them off the road. But if the tools used
to identify them are unreliable, then innocent people can lose
their licenses and, in some cases, be wrongly convicted and incar
cerated. The reverse is also true: with tens of thousands of cases
where police had reason to suspect that the driver was impaired
now thrown out, it is likely that many guilty parties will go free.
Why did this happen? 0ne pervasive problem is instrument cal
ibration. It turns out the Breathalyzer, like all scientific instru
ments, needs to be regularly calibrated, and police officers often
are neither trained nor equipped to do this work. In some cases, it
appears that police precincts did not even realize calibration was
needed. Here is a pretty simple solution: police need to bring in
technicians to check their instruments. As one company that
offers instrument calibration puts it, just as motor vehicles require
regular maintenance, so do the Breathalyzers used to test their
drivers. It would add a bit to policing costs, of course, but surely
that outcome is preferable to losing years of prosecutorial work or
to sending innocent people to jail while guilty parties walk free.
There is a deeper lesson here about science and technology:
scientific instruments do not perform magic tricks. You cannot
just blow air in a machine and get a good result; accurate data
are the product of sustained attention. It takes good work to get
good numbers. For decades science teachers have been admon
ished to teach not just facts but processes, including the process
of doing scientific research. I recall my own children raising
tomatoes, performing a census of marine life and constructing
volcano simulations. But what state science standards include a
unit on calibration? Maybe it is time they add one.
The miscarriage of justice caused by the misuse of scientific
instruments underscores why we need to understand not just the
findings of science but also the processes by which scientific evi
dence is obtained, as boring as they seem.JOIN THE CONVERSATION ONLINE
Visit Scientific American on Facebook and Twitter
or send a letter to the editor: [email protected]Boring
but Crucial
Instrument calibration is essential
for science—and justice
By Naomi Oreskes
© 2020 Scientific American