ADVANCES
14 Scientific American, May 2022Sony Interactive Entertainmentnents, but they also have to preserve their
own driving lines and make sure that they
don’t just give way.”
To teach the program the ropes, Sony
AI researchers used a technique called
deep reinforcement learning. They
“rewarded” the AI algorithm for certain
behaviors such as staying on the track,
remaining in control of the vehicle and
respecting racing etiquette—avoiding col-
lisions, for example. Then they turned the
program loose to try to achieve those
goals. Multiple versions of the AI, dubbed
Gran Turismo Sophy (GT Sophy), each
learned to drive one specific car on one
particular track. Next, the researchers pit-
ted several versions of the program against
human Gran Turismo champions.
In the first test, conducted last July,
humans achieved the highest overall team
score. On the second run, in October 2021,
the AI versions broke through. They
achieved the fastest lap times and beat
their human foes as a team—plus, an AI
won every race.
The human players seem to have taken
their losses in stride, and some enjoyed pit-
ting their wits against the AI. “Some of the
things that we also heard from the drivers
was that they learned new things from
Sophy’s maneuvers as well,” says Erica
Kato Marcus, director of strategies and
partnerships at Sony AI.
“The lines the AI was using were so tricky.
I could probably do them once, but it was so,
so difficult—I would never attempt it in a
race,” says Emily Jones, who was a world
finalist at the FIA-Certified Gran Turismo
Championships 2020 and later raced against
GT Sophy. Although Jones says competing
against the AI made her feel a little power-
less, she describes the experience as impres-
sive. “Racing, like a lot of sports, is all about
getting as close to the perfect lap as possi-
ble, but you can never actually get there,”
Jones says. “With Sophy, it was crazy to see
something that was the perfect lap. There
was no way to go any faster.”
The Sony team is now developing the
AI further, including aiming for one version
that can race any car on any track in the
game. They also hope to work with Gran
Turismo’s developer to incorporate a less
invincible version of the AI into a future
game update.
Because Gran Turismo provides a real-
istic approximation of specific real-world
cars and racetracks—and of the unique
physics parameters that govern each—
this research might also have applications
outside of video games. “I think one of
the pieces that’s interesting, which doesdifferentiate this from the Dota game, is to
be in a physics-based environment,” says
Brooke Chan, a software engineer at the AI
research company OpenAI and co-author
of the OpenAI Five project, which beat
humans at Dota 2. An AI training on Gran
Turismo is learning to understand more
about the physical world, adds Chan, who
was not involved with the GT Sophy study.
“Gran Turismo is a very good simula-
tor—it’s gamified in a few ways, but it
really does faithfully represent a lot of the
differences that you would get with differ-
ent cars and different tracks,” says Stan-
ford University mechanical engineer
J. Christian Gerdes, who was not involved
in the new study. “This is, in my mind, the
closest thing out there to anybody publish-
ing a paper that says AI can go toe to toe
with humans in a racing environment.”
Big differences remain when it comes
to physical roadways, though. “In the real
world, you have to deal with things like
bicyclists, pedestrians, animals, things that
fall off trucks and drop in the road that you
have to be able to avoid, bad weather
[and] vehicle breakdowns,” says Steven
Shladover, a vehicle-automation researcher
at the University of California, Berkeley,
who was also not involved with the study.
“None of that stuff shows up in the
gaming world.”
Gerdes says that because GT Sophy
orchestrates the fastest possible path
while interacting smoothly with often
unpredictable humans, its achievement
could have lessons for other fields in which
humans and automated systems work
together. Beyond automated driving, this
capability might one day aid interactions
such as robot-assisted surgery or machines
helping out around the home.
GT Sophy’s success also upends certain
assumptions about the way self-driving
cars must be programmed, Gerdes adds.
Many automated vehicles optimize their
movements—such as taking turns quickly
without spinning out—based on pro-
grammed-in physics, but GT Sophy opti-
mized through AI training.
“I think the lesson for automated-car
developers is there’s a data point here that
maybe some of our preconceived notions—
that certain parts of this problem are best
done in physics—need to be revisited,”
Gerdes says. “AI might be able to play
there as well.” — Sophie BushwickCars in Gran Turismo Sport