Generally, we can trace problems with a forecast back to either the
model itself or the data that goes into it. A good rule of thumb is that a
mathematical model should be designed around the data available. If
we don’t have data about the different transmission routes, for
example, we should instead try to make simple but plausible
assumptions about the overall spread. As well as making models
easier to interpret, this approach also makes it easier to communicate
what is unknown. Rather than grappling with a complex model full of
hidden assumptions, people will be able to concentrate on the main
processes, even if they’re not so familiar with modelling.
Outside my field, I’ve found that people generally respond to
mathematical analysis in one of two ways. The first is with suspicion.
This is understandable: if something is opaque and unfamiliar, our
instinct can be to not trust it. As a result, the analysis will probably be
ignored. The second kind of response is at the other extreme. Rather
than ignore results, people may have too much faith in them. Opaque
and difficult is seen as a good thing. I’ve often heard people suggest
that a piece of maths is brilliant because nobody can understand it. In
their view, complicated means clever. According to statistician
George Box, it’s not just observers who can be seduced by
mathematical analysis. ‘Statisticians, like artists, have the bad habit of
falling in love with their models,’ he supposedly once said.[72]
We also need to think about the data we put into our analysis.
Unlike scientific experiments, outbreaks are rarely designed: data can
be messy and missing. In retrospect, we may be able to plot neat
graphs with cases rising and falling, but in the middle of an outbreak
we rarely have this sort of information. In December 2017, for
example, our team worked with MSF to analyse an outbreak of
diphtheria in refugee camps in Cox’s Bazar, Bangladesh. We
received a new dataset each day. Because it took time for new cases
to be reported, there were fewer recent cases in each of these
datasets: if someone fell ill on a Monday, they generally wouldn’t
show up in the data until Wednesday or Thursday. The epidemic was
still going, but these delays made it look like it was almost over.[73]