3 November 2018 | NewScientist | 31holes. This endgame is modelled in an add-on
calculation in which researchers tweak the
parameters to fit the results of the initial
analytic solution.
This use of precalculated templates is a
problem, Cornish concedes. “With a template
search, you can only ever find what you’re
looking for.” What’s more, there are some
templates, such as those representing the
waves created by certain types of supernovae
explosions, that LIGO researchers can’t create.
That’s why Cornish prefers the third
method, which he helped develop. It involves
building a model from what he calls wavelets.
These are like tiny parts of a wave signal that
can be assembled in various ways. You vary
the number and shape of the parts until you
find a combination that removes the signal
from the noise. Because wavelet analysis
makes no assumptions about what created
the gravitational wave, it can make the most
profound discoveries. The wavelets “allow us
to detect the unknown unknowns”, says
Cornish. The downside is that they tell us
nothing about the physical attributes of the
detected source. For that, we have to compare
the constructed signal against the templates
or the numerical analysis.
The challenge with all three methods is that
accurately removing the signal from the data
requires you to know when to stop. In other
words, you have to understand what the
residual noise should look like. That is
exceedingly tricky. You can forget running
the detector in the absence of gravitational
waves to get a background reading. The noise
changes so much that there is no reliable
background. Instead, LIGO relies on
characterising the noise in the detectors, so
they know what it should look like at any given
time. “A lot of what we do is modelling and
studying the noise,” says Cornish.
Jackson is suspicious of LIGO’s noise
analysis. One of the problems is that there is
no independent check on the collaboration’s
results. That wasn’t so with the other standout
physics discovery of recent years, the Higgs
boson. The particle’s existence was confirmed
by analysing multiple, well-controlled particle
collisions in two different detectors at CERN
near Geneva, Switzerland. Both detector teams
kept their results from each other until the
analysis was complete.
By contrast, LIGO must work with single,
uncontrollable, unrepeatable events.
Although there are three detectors, theywork almost as one instrument. And despite
there being four data-analysis teams, they
cannot work entirely separately, because part
of the detection process involves checking that
all the instruments saw the signal. It creates a
situation in which each positive observation
is an uncheckable conclusion. Outsiders have
to trust that LIGO is doing its job properly.Purely illustrative
And there are legitimate questions about that
trust. New Scientist has learned, for instance,
that the collaboration decided to publish
data plots that were not derived from actual
analysis. The paper on the first detection in
Physical Review Letters used a data plot that
was more “illustrative” than precise, says
Cornish. Some of the results presented in
that paper were not found using analysis
algorithms, but were done “by eye”.
Brown, part of the LIGO collaboration at the
time, explains this as an attempt to provide a
visual aid. “It was hand-tuned for pedagogical
purposes.” He says he regrets that the figure
wasn’t labelled to point this out.
This presentation of “hand-tuned” data in
a peer-reviewed, scientific report like this is
certainly unusual. New Scientist asked the
editor who handled the paper, Robert Garisto,
whether he was aware that the published data
plots weren’t derived directly from LIGO’s
data, but were “pedagogical” and done “by
eye”, and whether the journal generally
accepts illustrative figures. Garisto declined
to comment.
There were also questionable shortcuts
in the data LIGO released for public use. The
collaboration approximated the subtraction
of the Livingston signal from the Hanford
one, leaving correlations in the data – the very
correlations Jackson noticed. There is now a
note on the data release web page stating that
the publicly available waveform “was not
tuned to precisely remove the signal”.
Whatever the shortcomings of the
reporting and data release, Cornish insists
that the actual analysis was done with
processing tools that took years to develop
and significant computing power to
implement – and it worked perfectly.
However, anyone outside the collaboration
has to take his word for that. “It’s problematic:
there’s not enough data to do the analysis
independently,” says Jackson. “It looks like
they’re being open, without being open at all.”
Brown agrees there is a problem. “LIGO has
taken great strides, and are moving towards
open data and reproducible science,” heTo spy gravitational waves, LIGO’s detectors
(pictured, page 28) need a quiet environment>DAVID RYDER/BLOOMBERG VIA GETTY IMAGES“ THE PAPER
ON THE FIRST
DETECTION
USED A DATA
PLOT THAT
WAS MORE
‘ILLUSTRATIVE’
THAN PRECISE”