VOICES
Opinion
PROFILEPeople see stats (both accurate and inaccurate) more
than ever before. Adam Lee asks whether that has
changed the way we discuss insight
We’re approaching a tipping point in
terms of understanding data, which
is awesome. Data practitioners have
worked hard to demonstrate that data and
insight aren’t just mountains of intangible
numbers or impenetrable reports. We’ve
fought to take the data to the people!
We’ve shown the power that can exist in
insight, when the data story is told in an
engaging and relevant way.
We’re also seeing public awareness of data
grow. Using NOW Corpus, the online news
keyword analysis tool, we can see that the use
of the term ‘data’ in the news has grown 70
per cent since 2010 (from 251 times per million
words to 429 times per million). With that
growth of awareness can come fear and
nervousness of the pervasiveness of data and
its uses. A nd some of that fear is wel l-founded.
So we have to be data advocates in a manner
that is both engaging and ethical.
What does that have to do with data and
insight in experimentation? Well,
explanations of experimentation run the risk
of being focused on statistical rigour and data-driven logic, especially if you’re just telling a
client about an outcome. This can make
experimentation appear harder to understand
than it needs to be and therefore harder to
demonstrate the commercial value of.
In turn, it can become eas y for stakeholders
to view experimentation as a nice-to-have
but not as a critical part in organisational
evolution. And then... we’re asked to tell
clients how much that’s worth. Which means
we’ve now got to devise and present a story
that is not only engaging and ethical but also
commercially convincing.
Well, why shouldn’t we be asked to? The
client has been asked to pay a lot of money
for these services. How much that’s worth is
an incredibly tough question to answer.
Sometimes a rate card figure just doesn’t give
the context needed.TRANSPARENCY IS KEY
It’s time to show the workings. Be clear and
ethical about what significances and
likelihoods mean, model any potential gains
responsibly and ensure that any effectivedecay is always in place. Deliver a true sense
of what success and risk could mean. (A 90
per cent likelihood of one outcome occurring
does not mean the other outcome will never
happen. Say this over and over. Print banners.)
Clarity is critical.
Language like ‘win’, ‘lose’, ‘success’ and
‘failure’ makes it immediately difficult to
show wider value and make it very easy to slip
into the ‘tell trap’. You’ll find it near-
impossible to talk about a ‘losing’ test in
positive terms. This shouldn’t be the case.
Experimentation delivers knowledge you can
show. Winning and losing can be a closed loop
but knowledge grows. Knowledge expands.
Showing your thinking around the situation
will enable you to expand on those tests that
didn’t perform as expected and highlight any
positives that have arisen as a result. For
e x ample: “T he change on this par t of the site
may not have resulted in the outcome
expected but the behaviour we’re seeing
suggests that if we considered this as part of
a change to your media strategy, that may
have a powerful impact.”USE THE DATA
This isn’t spinning a bad result. It’s setting
expectations and showing benefit in a way
that enables clients to grasp the value of what
you’re sharing – which goes for any
knowledge-generating discipline.
Use data and insight (from all of your
connected disciplines) to show what the
potential opportunity looks like. Evidence
your success with previous clients and show
how you use data and insight in order to make
things happen.
This approach means you develop the value
of knowledge (no matter if you’ve ‘won’ or
‘lost’). Discussions become more collaborative
and wide-ranging. By showing this, you
become more successful at gaining
stakeholder trust. Showing the most powerful
things possible – the ‘so what’ and the ‘what
next’ – gains the belief of the client in the
ability to deliver on an increasingly broad
scale. In 2020, that’s our challenge: to show,
not tell, exactly how insight and optimisation
can reach outside of the boundaries of
experimentation.Lee is head of analytics is at User Conversion, an
optimisation agency advocating the value and
democratisation of data for clients and the industry.
w: userconversion.comSHOW, DON’T TELL
DATA