48 | New Scientist | 9 April 2022These problems collectively led to the rise and
fall of an entire field of biology called candidate
gene association studies. From the late 1990s,
there were headline-grabbing claims that
specific versions of certain genes active in the
brain cause various mental health conditions.
The field spawned hundreds of media headlines
announcing that scientists had found the gene
“for” depression, schizophrenia, aggression
and so on. But then technology improved
and researchers started doing more laborious
studies involving hundreds of thousands
of people, analysing not just individual genes
but all the participants’ DNA. These showed
that common conditions like depression are
affected by hundreds of gene variants, each
with a tiny effect. None of the results from
candidate gene studies stood up. “It was a
lot of bunkum,” says neurogeneticist Kevin
Mitchell at Trinity College Dublin, Ireland.Cycle of dysfunction
The more I researched this article, the more
I came to despair. The problems are driven
by three sectors that seem locked in a self-
perpetuating cycle of dysfunction. Journals
want flashy results to boost their impact
factors and gain prestige. Funders award grants
to people who have published in the most
prestigious journals. And universities hire
those who bring in the big grant money.And yet there are efforts under way to
break the cycle. They aren’t being led, as you
might expect, by the ancient and venerable
institutions of science. In fact, change has been
driven mainly by junior researchers trying to
fix things from the bottom up. “These have
been people working in their bedrooms,
creating websites, running Twitter campaigns,”
says Orben, who does such work herself.
One approach is “preregistering”, where
researchers describe their hypothesis and their
intended study on a website before they begin.
It means a negative result can’t just be ignored.
With enough detail about the methods, it can
also prevent cherry-picking. Some journals
now let papers display a symbol to show that
the study was preregistered. The Center for
Open Science, a US grassroots organisation
that tackles bad science, has created such a
badge. It also offers two more badges indicating
that researchers will share all their data and
information on methods and materials,
allowing others to check or replicate the study.
In a more radical step, a preregistered study
can be sent to a journal before it is carried out,
a system called registered reports. If peer
reviewers OK it, the research gets published
no matter what the results, further reducing
the risk that negative studies get airbrushed
from history. “The journal accepting the paper
on the basis of its method rather than its
results is transformative,” says psychologistStuart Ritchie at King’s College London.
“This is the best way to do science.”
Currently, only a small fraction of papers
are preregistered or submitted as registered
reports. Perhaps unsurprisingly, progress seems
fastest in the field of psychology. Nevertheless,
I am excited about these practices, known
collectively as open science. If they catch on
more widely, it would be a simple way for
anyone to judge which claims are most reliable.
Slowly, creakingly, there are signs that
funding bodies are trying to address the
problem too. The National Institutes of Health,
the biggest funder of biomedical research
in the US, says that from next year all grant
recipients must share their data. UK Research
and Innovation (UKRI), the umbrella body
for the seven research councils, which fund
most science in UK universities, has similar
ambitions, although no start date yet. Speaking
in February at an inquiry into reproducibility
held by the House of Commons science and
technology committee, UKRI chief executive
Ottoline Leyser really seemed to get the
problems – and she is hugely influential.
Meanwhile, an international agreement
called the San Francisco Declaration on
Research Assessment asks universities and
funders to pledge not to use journal impact
factors in hiring, promotion or grant decisions.
Many heads of universities have signed up,
but it remains to be seen whether this will
influence middle managers who make hiring
decisions, says Orben. Another signatory
is Cancer Research UK, the world’s largest
charitable funder of cancer research. So
maybe those fun runs are worth it after all.
I asked everyone I interviewed for this article
how much progress has been made in fixing
bad science since those alarm bells started
ringing more than a decade ago. Nearly all
say that a good start has been made – but there
is a long way to go. Munafò puts it slightly
differently. “There’s a saying that revolutions
happen very slowly and then all at once,” he
says. “It feels like we’re getting to the all-at-
once stage where real change is actually
happening.” Let’s hope he is right. ❚Clare Wilson is a biomedical reporter
at New Scientist. Follow her on
Twitter @ClareWilsonMedKO
RH
IL^6
5 /G
ET
TY
IM
AG
ES
“ These problems collectively led to the rise
and fall of an entire field of biology”
Cancer Research UK,
which organises
fundraising events like
this one, has signed
an agreement to help
improve standards in
cancer research