12 | New Scientist | 12 October 2019
AFTER an embarrassing decade
in which the results of many
classic psychology and social
science studies couldn’t be
repeated, now there seems to be a
simple solution: common sense.
The “replication crisis” has seen
many eye-catching studies fail
to be verified. We now know, for
example, that children who can
resist marshmallows and hold
out for a bigger reward later don’t
reliably become successful adults,
despite what the classic Stanford
marshmallow experiment found.
To see if we should have been
able to guess as much, Alexandra
Sarafoglu at the University of
Amsterdam, the Netherlands,
and her colleagues recruited
233 volunteers. About half of
them were first-year psychology
students, but none were experts
in the field. After reading a quick
explanation of what replicating an
experiment means, participants
were shown 27 short descriptions
of well-known findings from
social science and psychology.
Of these, 14 had recently been
replicated successfully, including
a 2008 study that found thatstudents think they are more
likely to be questioned in class if
they arrive unprepared. Attempts
to replicate the other 13 studies
hadn’t worked, including a 2013
study that suggested people are
better at recognising emotions
after reading some literary fiction.
After reading about each study,
participants were asked to predict
whether repeating it was likely to
validate its findings. When shownjust the description, participants
guessed correctly 58 per cent of
the time. When also shown a
simple statistical measure of
the strength of the study’s
evidence, this rose to 67 per cent.
Pooling these predictions
made the results even clearer.
When most of the group
confidently guessed the same,
their prediction was almost
always right. Among the 10 most
confident predictions that a
study wouldn’t replicate and the
10 most confident that it would,
the group was wrong just twice.The finding implies that
scientists shouldn’t let surprising
findings override their common
sense. “We are in this crisis for a
reason,” Sarafoglu says. “There is
a strong incentive in science ingeneral to publish sexy findings.
So implicitly, people get pushed
towards finding effects that are
counter-intuitive.”
“I find it very interesting that
laypeople are pretty successful
in predicting replication
outcomes,” says Anna Dreber
Almenberg at the Stockholm
School of Economics in Sweden.
Sarafoglu accepts that the
picture might change if her study
were repeated with a different
selection of studies, or with
differently written descriptions,
but she predicts that people will
be able to replicate this research.
However, the team hasn’t asked a
group of laypeople whether they
agree. What do you reckon? ❚“ We are in this crisis
for a reason. There is
a strong incentive to
publish sexy findings”PE
TE
R^ CAD
E/GET
TYNews
The results of the famous
“marshmallow test”
haven’t been replicatedMicrobiomeA few days in
hospital changes
your gut bacteriaHARMFUL bacteria have been
found to rapidly colonise the guts
of people treated for several days in
an intensive care unit. Heavy use of
broad-spectrum antibiotics, feeding
people through a tube and using
a ventilator could all contribute to
changes in the microbiome, says
the team that carried out the study.
“It’s quite disconcerting,” says
Mark Pallen at the Quadram
Institute, UK, who led the research.“I suspected that something like this
was going on, but I was quite taken
aback at the scale of the changes.”
Pallen and his colleagues tracked
24 people admitted for trauma,
heart attacks, cancer and other
emergencies to Queen Elizabeth
Hospital Birmingham, UK. Many
were unconscious or sedated. After
getting permission from family
members, the scientists took stool
samples from these individuals.
The team then sequenced DNA
from the samples to identify which
microbes were present – and how
that changed during treatment.
Two-thirds of the patientsshowed a marked reduction
in microbial diversity at some
stage. The biggest changes were
associated with intravenous use
of the antibiotic meropenem.
Many of the patients’ guts
became dominated by Enterococcus
faecium, a common pathogenthat can be fatal if it reaches the
bloodstream (Microbial Genomics,
doi.org/dcc9).
It is a small number of people
to draw conclusions from, but the
results align with other studies, says
Joost Wiersinga at the Academic
Medical Centre in Amsterdam,
the Netherlands. While the priority
in intensive care is to save lives,
Wiersinga says medics could do
more to minimise disruption to
gut bacteria. “If you need to give
antibiotics then give them. But think
about which antibiotic, for how
long and if you can stop earlier.” ❚
David AdamEnterococcus
faecium can
spread from
the gut to other
parts of the bodyEY
E^ OF^ SCIENCE/SPLPsychologyLeo BenedictusThe power of common sense
It is easy to guess whether a psychology study will stand up to further testing