25 January 2020 | New Scientist | 11Chemistry Artificial intelligenceAlice Klein Donna LuA MOLECULE that looks like a tiny
bow tie and changes colour under
different conditions could be used
to monitor air for toxic substances.
Huan Cong at the Technical
Institute of Physics and Chemistry
in China and his colleagues made
the substance, which they named
BowtieArene, using a pair of
pentagon-shaped molecules called
pillararenes. These were joined
together to create the final product
using a fluorescent molecule called
tetraphenylethylene. This formed
the “knot” of the bow tie.When the team mixed individual
bow-tie molecules together, they
stacked neatly on top of each
other. The resulting orderly
crystals interacted with light
to produce blue fluorescence.
In contrast, when the researchers
broke these ordered structures
apart, by mechanically grinding
or scratching the crystals, the
separated bow ties emitted green
or yellow fluorescence.
Next, they showed they could
make the bow ties blue again by
exposing them to the vapour of a
toxic chemical called xylene, which
pulled the individual molecules
back into an orderly arrangement
(Angewandte Chemie International
Edition, doi.org/djt9).
This colour-switching property
could be used to make sensors
that detect various harmful
chemicals in the air, says Cong.
His team is currently designing
variants of the bow-tie molecule
that are capable of identifying the
presence of different substances. ❚Bow tie-shaped
molecules change
colour in toxic airDEVELOPMENTS in artificial
intelligence often draw
inspiration from how humans
think, but now AI has turned
the tables to hint at a way
our brains learn.
Will Dabney at tech firm
DeepMind in London and his
colleagues have found that a
recent innovation in machine
learning called distributional
reinforcement learning also
provides a new explanation
for how the reward pathways
in a brain work. These pathways
govern our response to
pleasurable events and are
mediated by neurons that
release the chemical dopamine.
“Dopamine in the brain is
a type of surprise signal,” says
Dabney. “When things turn
out better than expected, more
dopamine gets released.”
It was previously thought
that these dopamine neurons
all responded identically,
“kind of like a choir but where
everyone’s singing the exact
same note”, says Dabney.
But the team found that
individual dopamine neurons
actually seem to vary. Each oneis tuned to a different level
of optimism or pessimism.
“They all end up signalling
at different levels of surprise,”
says Dabney. “More like a
choir all singing different
notes, harmonising together.”
The process that inspired
the revelation, distributional
reinforcement learning, is one
of the techniques that AI has
used to master games such as
Go and Starcraft II. This is the
idea that a reward reinforces
the behaviour that led to its
acquisition. It requires an
understanding about how
a current action leads to a
future reward. For example,
a dog may learn the command
“sit” because it is rewarded
with a treat when it does so.
Previously, models of
reinforcement learning in
both AI and neuroscience
focused on learning to predict
an “average” future reward.
“But this doesn’t reflect reality
as we experience it,” says Dabney.
“When someone plays the
lottery, for example, they
expect to win or they expect
to lose, but they don’t expectthis halfway average outcome
that doesn’t necessarily really
occur,” says Dabney.
When the future is uncertain,
the possible outcomes can
instead be represented as
a probability distribution:
some are positive, others
negative. AIs that usedistributional reinforcement
learning algorithms are able
to predict the full spectrum
of possible rewards.
To test whether a brain’s
dopamine reward pathways also
work via such a distribution, the
team recorded responses from
individual dopamine neurons
in mice trained to perform a
task. During training, the mice
were given rewards of varying
and unpredictable sizes.
Different dopamine cells
showed reliably different
levels of surprise (Nature,
doi.org/ggh5kw).
“Associating rewards to
certain stimuli or actions
is of critical importance for
survival,” says Raul Vicente
at the University of Tartu in
Estonia. “The brain cannot
afford to throw away any
valuable information about
rewards. It’s a nice example of
how computational algorithms
can guide us in what to look for
in neural response.” However,
more research is needed to
demonstrate whether the
results apply to other species
or to certain regions of the
brain, he says. ❚An AI learning technique
also works in brains
TE
K^ IMAGE/SCIENCE^ PHOTOLIBRARYArtificial intelligence
may teach us about how
human brains work“ It’s a nice example
of how computational
algorithms can guide
us in what to look for”HU
AN
CO
NG
,^ TE
CH
NIC
AL
IN
ST
ITU
TE
OF
PH
YS
ICS
AN
D^ C
HE
MIS
TR
Y,^ C
HIN
A
BowtieArene contains a
fluorescent molecule