22 August 2020 | New Scientist | 37p-hacking – to find the result they want.
Drop a few participants here, change
a number there and convince yourself
that it’s the right thing to do.
The third cause is unforced errors, like
a typo in a spreadsheet that knocks the
whole result off track. That is negligence.
The last category is hype, where scientists
are pushed towards writing up their results
as if they are much more exciting than they
are. The way papers are written now often
makes it sound as if the findings are going
to revolutionise the way we think about
things. But we’re not making more
groundbreaking discoveries than we used
to. It’s just that there’s pressure on how
we’re supposed to write up science, and
lots of media excitement.
If you put all those four together, it looks
as if something is very rotten.How did we let the rot set in?
It crept in due to a system of perverse
incentives. There’s huge pressure to publish
papers and huge pressure to bring in grants –
which is an incentive to publish papersW
HEN Stuart Ritchie was a graduate
student in Edinburgh, UK, in 2011,
he was involved in an incident
that shook his faith in science. With two
colleagues, he tried and failed to replicate a
famous experiment on precognition, the
ability to see the future. They sent their
results to the journal that published the
original research and received an immediate
rejection on the grounds that the journal
didn’t accept studies that repeated previous
experiments.
Ritchie remained a scientist – he is a
psychologist at King’s College London with a
focus on studying human intelligence – but
ever since that rejection, he has been on a
crusade to air science’s dirty laundry. His
latest book is Science Fictions, in which he
shows how, all too often, we can’t rely on the
facts that science provides.Graham Lawton: The grand and scary claim
of your book is that something is rotten in the
kingdom of science.
Stuart Ritchie: Absolutely. We think of science
as being this objective thing that tells us factsabout the world and produces all these
scientific papers, which are almost sacred
things. But a lot of people don’t see how
the sausage is made. I think if they had
more of an idea of how the process happens,
they would question the truth status of
those papers much more. In a lot of cases,
the science is useless, not worth the paper
it is written on.You identify four main causes of rot.
First there’s fraud, when people deliberately
alter or make up results to try to get a paper
published. That’s rare, but not as rare as
we would like to think. If you ask scientists
whether they’ve committed fraud, only
a very small number say yes. But something
like 14 per cent will tell you that they
think they know of an instance where
a colleague has committed fraud.
That blurs into the second thing, which is
bias. Everyone wants to change the world, so
people are biased towards finding significant
results. And that can make them see things
that aren’t there. They might make arbitrary
RO changes to their statistical analysis – called >
CIO
M
ON
TO
YA