New Scientist 2018 sep

10 | NewScientist | 8 September 2018

NEWS & TECHNOLOGY

Timothy Revell

DO COMPUTERS know what we

are thinking? To find out, a team

at the University of California,

Berkeley, has created a set of

gruelling tests that probe AI’s

progress in understanding the

world. None has passed the tests

yet, but one got very close.

The tests examine theory of

mind – the ability to reason about

another’s beliefs – and are inspired

by experiments in psychology.

Each consists of a short paragraph

describing a scenario involving

people – for example, Sally hides

a marble and then Anne moves it

to a new location.

Passing the test revolves around

identifying first and second-order

beliefs. A first-order belief is

working out what someone else

thinks, answering a question like

“where does Sally think the marble

is?”, for example. A second-order

belief is what someone thinks

someone else is thinking, for

example, “where does Anne think

Sally thinks the marble is?”.

Children are normally able

to correctly identify first-order

beliefs by around age 3, but it isn’t

until age 6 or 7 that they can do

the same for second-order beliefs.

Aida Nematzadeh, who led the

work and is now at Google’s AI lab

at DeepMind, and her colleagues

generated 10,000 scenarios and

associated questions that tested

theory of mind via first and

second-order beliefs. They then

put them to four state-of-the-art

AIs. None managed to achieve a

passing score, which was set at

95 per cent. Most humans should

be able to score 100 per cent.

The highest achiever was an AI

called RelNet, produced by Adam

Santoro and his colleagues at

DeepMind, which scored 94.3 per

cent. The other three AIs managed

scores of between 82 and 94 per

cent (arxiv.org/abs/1808.09352).

However, the results were fairly

fickle. Inserting an unrelated

sentence with no bearing on the

tested situations was enough to

bamboozle the AIs. All of them

dropped their scores by between

5 and 20 per cent, suggesting

they weren’t properly grasping

the meaning of the text.

Though no AI has yet displayed

theory of mind, they are

improving at an impressive rate.

Only two years ago, Facebook

produced a series of tests that

would examine AI’s ability to

answer questions about the world.

None tested could handle the task

at the time, but some can now

pass with few mistakes.

A machine with theory of mind

could prove useful, says Alan

Wagner at Georgia Tech Research

Institute in Atlanta. “An AI agent

teaching assistant might be able

to reason about a student’s false

beliefs related to a course, for

example, that the final exam

is next week when in fact it is

tomorrow, and in doing so be

better suited to help.”

Another example is self-driving

cars, says Johannes Bjerva at

the University of Copenhagen,

Denmark. An AI-driven car with

theory of mind might realise that

the driver of another car hasn’t

seen a person in the road, and honk

its horn to warn them, he says.

However, even if an AI can pass

these tests, it may still not have

theory of mind, says Wagner. ■

AI takes ‘marble’

theory-of-mind test

EYEEM/ALAMY STOCK PHOTO

“ If we take action now,

some coral reefs may still

exist a hundred years in

the future”

REPEATING an experiment 47 years

after it was originally carried out has

revealed some rare good news about

corals: some species appear to have

become significantly better at

surviving temperature increases.

In 1970, marine zoologist Steve

Coles collected three species of coral

from a reef in Kane‘ohe Bay, Hawaii.

When he put them in a chamber and

Some corals are

beginning to

beat the heat

gradually heated the water, he found

that they couldn’t tolerate increases

of 1 to 2°C. They started to bleach,

ejecting the colourful algae that live

within them. Without these, coral die.

Overall, no corals from one species

survived, and survival rates were 40

per cent and 5 per cent for the others.

But Coles, at the Hawaii Institute of

Marine Biology, got a different result

when he repeated the experiment

using the same species of coral from

the same area decades later. When

he raised temperatures by the same

amount, it took a few days longer for

the corals to begin bleaching, and more

than half survived (PeerJ, doi.org/cthn).

“That was a big surprise,” says Coles.

The finding is in line with anecdotal

observations from reefs that some

corals seem to be surviving bleaching.

Coles doesn’t know how they do it,

but other research suggests some

corals adapt to higher temperatures

by associating with species of algae

that are more tolerant of heat stress.

It isn't clear whether corals in

Hawaii are individually acclimatising

to rising temperatures, or whether

they have genetically evolved a heat

tolerance that they will pass onto

future generations, says Mikhail Matz

at the University of Texas at Austin.

Either way, it is good news, he says,

although genetic adaptation may

have more lasting benefits.

Coles doesn’t think the change will

be enough to counteract the effects

of global warming on coral. But it

suggests that, if we take action on

climate change now, some coral reefs

may still exist in a hundred years’

time, he says. Katarina Zimmer ■

Tracking marbles can test the

ability to reason what others think–