Science 14Feb2020

M

odeling immensely complex natu-

ral phenomena such as how sub-

atomic particles interact or how

atmospheric haze affects climate

can take many hours on even the

fastest supercomputers. Emulators,

algorithms that quickly approximate these

detailed simulations, offer a shortcut. Now,

work posted online shows how artificial in-

telligence (AI) can easily produce accurate

emulators that can accelerate simulations

across all of science by billions of times.

“This is a big deal,” says Donald Lucas,

who runs climate simulations at Lawrence

Livermore National Laboratory and was not

involved in the work. He says the new system

automatically creates emulators that work

better and faster than those his team designs

and trains, usually by hand. The new emu-

lators could be used to improve the models

they mimic and help scientists make the best

of their time at experimental facilities. If the

work stands up to peer review, Lucas says, “It

would change things in a big way.”

A typical computer simulation might cal-

culate, at each time step, how physical forces

affect atoms, clouds, galaxies—whatever is

being modeled. Emulators, based on a form

of AI called machine learning, skip the la-

borious reproduction of nature. Fed with

the inputs and outputs of the full simula-

tion, emulators look for patterns and learn

to guess what the simulation would do with

new inputs. But creating training data for

them requires running the full simulation

many times—the very thing the emulator is

meant to avoid.

The new emulators are based on neural

networks—machine learning systems in-

spired by the brain’s wiring—and need far

less training. Neural networks consist of

simple computing elements that link into

circuitries particular for different tasks.

Normally the connection strengths evolve

through training. But with a technique

called neural architecture search, the most

data-efficient wiring pattern for a given

task can be identified.

The technique, called Deep Emulator Net-

work Search (DENSE), relies on a general

neural architecture search co-developed by

Melody Guan, a computer scientist at Stan-

ford University. It randomly inserts layers of

computation between the networks’ input

and output, and tests and trains the result-

ing wiring with the limited data. If an added

layer enhances performance, it’s more likely

to be included in future variations. Repeating

the process improves the emulator. Guan says

it’s “exciting” to see her work used “toward

scientific discovery.” Muhammad Kasim, a

physicist at the University of Oxford who led

the study, which was posted on the preprint

server arXiv in January, says his team built

on Guan’s work because it balanced accuracy

and efficiency.

The researchers used DENSE to develop

emulators for 10 simulations—in physics, as-

tronomy, geology, and climate science. One

simulation, for example, models the way

soot and other atmospheric aerosols reflect

and absorb sunlight, affecting the global

climate. It can take a thousand of computer-

hours to run, so Duncan Watson-Parris, an

atmospheric physicist at Oxford and study

co-author, sometimes uses a machine learn-

ing emulator. But, he says, it’s tricky to set up,

and it can’t produce high-resolution outputs,

no matter how many data you give it.

The emulators that DENSE created, in

contrast, excelled despite the lack of data.

When they were turbocharged with special-

ized graphical processing chips, they were

between about 100,000 and 2 billion times

faster than their simulations. That speedup

isn’t unusual for an emulator, but these

were highly accurate: In one comparison,

an astronomy emulator’s results were more

than 99.9% identical to the results of the

full simulation, and across the 10 simula-

tions the neural network emulators were

far better than conventional ones. Kasim

says he thought DENSE would need tens of

thousands of training examples per simula-

tion to achieve these levels of accuracy. In

most cases, it used a few thousand, and in

the aerosol case only a few dozen.

“It’s a really cool result,” said Laurence

Perreault-Levasseur, an astrophysicist at the

University of Montreal who simulates galax-

ies whose light has been lensed by the grav-

ity of other galaxies. “It’s very impressive

that this same methodology can be applied

for these different problems, and that they

can manage to train it with so few examples.”

Lucas says the DENSE emulators, on top

of being fast and accurate, have another

powerful application. They can solve “in-

verse problems”—using the emulator to iden-

tify the best model parameters for correctly

predicting outputs. These parameters could

then be used to improve full simulations.

Kasim says DENSE could even enable re-

searchers to interpret data on the fly. His

team studies the behavior of plasma pushed

to extreme conditions by a giant x-ray laser

at Stanford, where time is precious. Ana-

lyzing their data in real time—modeling,

for instance, a plasma’s temperature and

density—is impossible, because the needed

simulations can take days to run, longer

than the time the researchers have on the

laser. But a DENSE emulator could inter-

pret the data fast enough to modify the ex-

periment, he says. “Hopefully in the future

we can do on-the-spot analysis.” j

Matthew Hutson is a journalist in New York City.

728 14 FEBRUARY 2020 • VOL 367 ISSUE 6479 sciencemag.org SCIENCE

IMAGE: NASA

NEWS | IN DEPTH

Emulators speed up simulations, such as this NASA aerosol model that shows soot from fires in Australia.

By Matthew Hutson

COMPUTER SCIENCE

With little training, neural networks create accurate

emulators for physics, astronomy, and earth science

AI shortcuts speed up

simulations by billions of times

Published by AAAS