4 | THURSDAY, AUGUST 1, 2019 THE NEW YORK TIMES INTERNATIONAL EDITION
world
Toby Walsh, a professor at the Univer-
sity of New South Wales in Sydney, is
one of the leading experts on artificial
intelligence in Australia. He and other
experts have released a report outlin-
ing the promises, and ethical pitfalls, of
the country’s embrace of A.I.
Recently, Dr. Walsh, 55, has been
working with the Campaign to Stop
Killer Robots, a coalition of scientists
and human rights leaders seeking to
halt the development of autonomous
robotic weapons.
We spoke briefly at the annual meet-
ing of the American Association for the
Advancement of Science, where he
was making a presentation, and then
for two hours via telephone. Below is
an edited version of those conversa-
tions.You are a scientist and an inventor.
How did you become an activist in the
fight against "killer robots"?
It happened incrementally, beginning
around 2013. I had been doing a lot of
reading about robotic weaponry. I
realized how few of my artificial intelli-
gence colleagues were thinking about
the dangers of this new class of weap-
ons. If people thought about them at
all, they dismissed killer robots as
something far in the future.
From what I could see, the future
was already here. Drone bombers were
flying over the skies of Afghanistan.
Though humans on the ground con-
trolled the drones, it’s a small technical
step to render them autonomous.So in 2015, at a scientific conference,
I organized a debate on this new class
of weaponry. Not long afterward, Max
Tegmark, who runs M.I.T.’s Future of
Life Institute, asked if I’d help him
circulate a letter calling for the interna-
tional community to pass a pre-emp-
tive ban on all autonomous robotic
weapons.
I signed, and at the next big A.I.
conference, I circulated it. By the end
of that meeting, we had over 5,
signatures — including people like
Elon Musk, Daniel Dennett, Steve
Wozniak.What was your argument?
That you can’t have machines deciding
whether humans live or die. It crosses
new territory. Machines don’t have our
moral compass, our compassion and
our emotions. Machines are not moral
beings.
The technical argument is that these
are potentially weapons of mass de-
struction, and the international com-
munity has thus far banned all other
weapons of mass destruction.
What makes these different from
previously banned weaponry is their
potential to discriminate. You could
say, “Only kill children,” and then add
facial recognition software to the sys-
tem.
Moreover, if these weapons are
produced, they would unbalance the
world’s geopolitics. Autonomous ro-
botic weapons would be cheap and
easy to produce. Some can be made
with a 3-D printer, and they could
easily fall into the hands of terrorists.
Another thing that makes them
terribly destabilizing is that with such
weapons, it would be difficult to know
the source of an attack. This has al-
ready happened in the current conflict
in Syria. Just last year, there was adrone attack on a Russian-Syrian base,
and we don’t know who was actually
behind it.Why ban a weapon before it is
produced?
The best time to ban such weapons is
before they’re available. It’s much
harder once they are falling into the
wrong hands or becoming an accepted
part of the military tool kit. The 1995
blinding laser treaty is perhaps the
best example of a successful pre-emp-
tive ban.
Sadly, with almost every other weap-
on that has been regulated, we didn’t
have the foresight to do so in advance
of it being used. But with blinding
lasers, we did. Two arms companies,
one Chinese and one American, had
announced their intention to sell blind-
ing lasers shortly before the ban came
into place. Neither company went on to
do so.Your petition — who was it ad-
dressed to?
The United Nations. Whenever I go
there, people seem willing to hear from
us. I never in my wildest dreams ex-
pected to be sitting down with the
under secretary general of the U.N.
and briefing him about the technology.
One high U.N. official told me, “We
rarely get scientists speaking with one
voice. So when we do, we listen.”
So far, 28 member countries have
indicated their support. The European
Parliament has called for it. The Ger-
man foreign minister has called for it.
Still, 28 countries out of 200! That’s not
a majority.Who opposes the treaty?
The obvious candidates are the U.S.,
the U.K., Russia, Israel, South Korea.
China has called for a pre-emptive banon deployment, but not on develop-
ment of the weapons.
It’s worth pointing out there is going
to be a huge amount of money being
made by companies selling these
weapons and the defenses to them.Proponents of robotic weapons argue
that by limiting the number of human
combatants, the machines might
make warfare less deadly.
I’ve heard those arguments, too. Some
say that machines might be more
ethical because people in warfare get
frightened and do terrible things. Some
supporters of the technology hope that
this wouldn’t happen if we had robots
fighting wars, because they can be
programmed to abide by internationalhumanitarian law. The problem with
that argument is that we don’t have
any way to program for something as
subtle as international humanitarian
law.
Now, there are some things that the
military can use robotics for — clear-
ing a minefield is an example. If a
robot goes in, gets blown up, you get
another robot.Robotic warfare has long been the
subject of science fiction films. Do
you have a favorite?
No, most A.I. researchers — myself
included — dislike how Hollywood has
dealt with the technology. Kubrick’s
“2001” is way off, because it is based on
the idea that there will be machinesthat will have the desire for self-pres-
ervation, and that will result in malev-
olence toward humans.
It’s wrong to assume they’ll want to
take over, or even preserve them-
selves. The intelligence we build is
going to be quite different from what
humans have, and it won’t necessarily
have the same character flaws.
These machines don’t have any
conscience, and they don’t have any
desire to preserve themselves. They’ll
do exactly what we tell them to do.
They are the most literal devices ever
built. They’ll follow those instructions,
however perverse they may be.
I dislike “The Terminator,” too. That
technology is far, far away. There are
more mundane technologies we should
be worried about now, like the drones I
mentioned earlier.
Now, I do like “Her,” because it is
about the relationships we’ll have in a
future when we’ll be increasingly
interacting with machines. It will be
possible, as in the movie, that we will
develop feelings for them.
That movie is about how A.I. is going
to be a pervasive part of our existence
in every room, every car. They will be
things that listen to us, answer our
questions, and “understand” us.Since 2013, you’ve been spending as
much time on your activism as on
scientific research. Any regrets?
No. This is important to be doing right
now. Twenty years ago, like many of
my colleagues, I felt that what we were
doing in A.I. was so far from practice
that we didn’t have to worry about
moral consequences. That’s no longer
true.
I have a 10-year-old daughter. When
she’s grown, I don’t want her to ask,
“Dad, you had a platform and author-
ity — why didn’t you try to stop this?”He may be your best hope against the killer robots
Toby Walsh is an expert on artificial intelligence at the University of New South Wales
in Australia. “You can’t have machines deciding whether humans live or die,” he said.DEAN SEWELL FOR THE NEW YORK TIMESA.I. expert in Australia
is on a campaign to ban
autonomous weaponryBY CLAUDIA DREIFUStricity never came to many of these vil-
lages, nor did running water. A five-year
drought has meant long-running bouts
of hunger.
The leader of Parenstu, Celinda Van-
grieken, whose family has lived in Co-
lombia for a century, looked at the refu-
gees from Venezuela — dozens of new-
comers among the hundreds of people
living here. She said she had watched
with sympathy as they arrived, haggard
and desperate.
But while they might be her people,
she said, they were not her blood.
“They said, ‘We’re Wayuu. We’re from
here like you,’” she said. “But this is not
their land.”
On a recent day, an infant with a rash
across her forehead screamed. The girl
had been vomiting blood, her mother
said, and had lost two pounds in recent
weeks.
“She doesn’t want to eat,” said the
mother, Andreina Paz, a 20-year-old
Venezuelan Wayuu who crossed the bor-
der this year after watching her neigh-
bor’s daughters die from malnutrition.
She feared her own daughter would now
die in Colombia.Celia Epinayu buried her own son,
Eduardo, in February. She is not a mi-
grant, but a Colombian Wayuu living on
the lands where her parents raised her.
But as Ms. Paz and others from Vene-
zuela kept arriving, the food for Ms.
Epinayu’s clan grew scarce and she
could not feed her five children, includ-
ing 10-month-old Eduardo.
“Your boy is dead. You have to let go,”
said Betty Ipuana, the local school-
teacher, who was visiting Ms. Epinayu
at her adobe house. “The ones you must
look over now are the ones that are
alive.”
Ms. Epinayu didn’t respond, looking
at the dusty ground.
The strain is evident on the faces of
the putchipu’u, the Wayuu authorities
who mediate disputes and deliver mes-
sages between clans. They sat under a
thatched roof beside the highway, dis-
cussing dozens of new conflicts over
land, and the fear they might widen into
blood feuds between families. On the
northern coast, Colombian Wayuu re-
cently torched the tents of newly arrived
Venezuelans.
“It’s the fear we all have, that this land
can’t support us all,” began Guillermo
Ojeda, speaking to the other mediators
at the table. But he said the Venezuelans
had to be accepted, even if it meant risks
for everyone.
José Manuel Pana, another
putchipu’u, laid down his cane, straight-
ened his hat and said he was not con-
vinced.
“They come to Colombia, and it’s all a
struggle for land to them: They build
their home, and they build a problem for
another family,” said Mr. Pana. “What
have they brought here from Venezue-
la? They’ve brought an infection.”
Ms. Vangrieken, the 72-year-old
leader in Parenstu, recalls the day when
a group of Venezuelan Wayuu first came
to her land with a small box containing
the bones of their relatives.
Under Wayuu tradition, two decades
after a person dies, the family membersreturn to the cemetery for what’s called
the “second wake.” They break open the
tomb, clean the bones and rebury them
at a site they believe their ancestors
came from. But the custom dictates
something else as well: Relatives of the
deceased may also claim the land where
the remains are reburied and build
homes nearby.
Ms. Vangrieken said the possibility of
a mass migration hadn’t even crossed
her mind at the time. It was 2009 and
Venezuela was still prosperous. Only a
few people were asking to build homes
near the remains, and it seemed absurd
to her that anyone would claim Parenstu
as their own.But last year, newcomers began to
show up, as hyperinflation cut off count-
less Venezuelans from food and Mr. Ma-
duro sealed his grip over the nation in a
widely condemned re-election vote.
In January, the political crisis intensi-
fied as the United States and dozens of
other countries recognized the leader of
Venezuela’s opposition as the legitimate
president. American sanctions on Vene-
zuela’s most vital industry — oil — fol-
lowed, taking a serious toll on the nation,
including the Wayuu areas, which are
outside one of the country’s main oil
hubs.
The Wayuu in Parenstu sometimes
found it hard to recognize the new arriv-als as kin. Some had come from cities
and didn’t speak Wayuunaiki, the native
language. They constructed improvised
houses out of poles and plastic, rather
than out of adobe, like the homes in
Parenstu.
But there was one Wayuu tradition
the Venezuelans seemed to know well:
The cemetery gave them the right to
stay.
“My mother always said we should
give them space, that they might even-
tually leave,” said Yomeilia Vangrieken,
one of Ms. Vangrieken’s daughters. “She
made a big mistake.”
Not long after the newcomers began
settling on Ms. Vangrieken’s land, herfamily woke up to an angry crowd of sev-
eral hundred people. They were from
another Colombian Wayuu clan and said
they had come to avenge the beating of a
young man who had been struck by a
Venezuelan Wayuu on their land.
They had followed his footprints in
the sand back to Parenstu, Ms. Van-
grieken recalled, and now demanded a
payment of around $1,500, holding her
responsible as the traditional leader.
“I didn’t have that amount, and nei-
ther did the Venezuelans,” said Ms. Van-
grieken, who offered the clan 10 of her
own sheep instead, nearly all the live-
stock she owned.
Even as tensions grew, Ms. Van-grieken continued to ask for patience
from her clan, arguing that, as Wayuu,
the new arrivals had to be treated as
equals. Malnourished Venezuelan chil-
dren were enrolled in school and offered
meals, though it meant less for Co-
lombians. In one case, a Venezuelan
Wayuu woman arrived sick and was giv-
en an identification card of a Colombian
Wayuu woman so she could be treated at
a public hospital. But the sick woman
died in the hospital; Ms. Vangrieken
fears that the Colombian Wayuu who of-
fered the ID card is now registered as
dead.
Milcidi Palmar, a 32-year-old Vene-
zuelan Wayuu refugee, said shortages of
medicine had led to the deaths of four
members of her extended family.
Then last year, her youngest child,
Mayerli, fell ill. Ms. Palmar wasted the
little money she had on bus rides to a
Venezuelan hospital, which sent her
away with nothing to control the fever.
Mayerli died.
Shortly after, Ms. Palmar’s other
daughter, Wendy, got sick, too. Ms. Pal-
mar said she returned to the hospital, in-
sisting on treatment. She said Wendy
got an injection, but her skin turned pur-
ple in the days that followed. Wendy
stopped breathing.
“There was nothing I could do but
watch them both die,” she said of her
daughters.
The story weighs heavily on Yadira
Martínez, the daughter of Parenstu’s
leader, who often takes over some of the
responsibilities running the village.
She was with Ms. Palmar at the three-
acre plot in Parenstu that Ms. Palmar
and her husband had encircled with a
wooden fence after they occupied it last
year. Ms. Palmar had been making char-
coal to sell — using trees considered sa-
cred to the Wayuu in Parenstu.
Yadira Martínez recalled how she had
played as a child among the trees Ms.
Palmar was now cutting down. Yet she
was split between nostalgia for that time
and sympathy for a fellow mother with
mouths to feed.
“Want to buy some charcoal?” Ms.
Palmar asked, cracking a joke to break
the tension. The two women shared an
uneasy laugh.
Between the drought and the new ar-
rivals, there is little water in Parenstu. A
reservoir has run nearly dry, only a
silted gray pool in a vast desert.
Toward sundown, Celinda Van-
grieken and Yadira Martínez walked to
the edge of town, to check a well hidden
away under scrub brush. It was dry, but
it reminded Celinda of a girl who had
fallen inside and drowned decades be-
fore.
“How could anyone say this land is
not ours?” Yadira said after her mother
finished the story. “This is where we
shed our blood.”
Their clan had settled in Parenstu, ac-
cording to family legend, after a dispute
that began over a pot of necklaces and
quickly escalated into a blood feud over
territory — the kind everyone here
seemed eager to avoid.
A short ride from the well stood the
cemetery of the Venezuelan Wayuu,
bordered by fence of cactuses. Celinda
Vangrieken thought for a moment. The
fate of her village seemed tied to Vene-
zuela in a way she felt she could not con-
trol.
“I just don’t want a war with them,”
she said.Refugee crisis spills out of Venezuela
COLOMBIA,FROM PAGEClockwise from above: A family performing a dance in the Colombian settlement of Parenstu; Milcidi Palmar, a Venezuelan Wayuu refugee in Parenstu; Wayuu girls in the settle-
ment. Members of the Wayuu Indigenous group are seeking refuge in Colombia’s Guajira Desert, where the original inhabitants and new arrivals are both trying to survive.PHOTOGRAPHS BY ADRIANA LOUREIRO FERNANDEZ FOR THE NEW YORK TIMES“It’s all a struggle for land to
them: They build their home,
and they build a problem for
another family.”Maria Iguarán contributed to this article
from Parenstu, Colombia.РРtrolled the drones, it’s a small technicaltrolled the drones, it’s a small technical
Е
Though humans on the ground con-
Е
Though humans on the ground con-
trolled the drones, it’s a small technical
Е
trolled the drones, it’s a small technical
Л
flying over the skies of Afghanistan.
Лflying over the skies of Afghanistan.
Though humans on the ground con-Though humans on the ground con-Лflying over the skies of Afghanistan.flying over the skies of Afghanistan.ИИ
Though humans on the ground con-
И
Though humans on the ground con-З
was already here. Drone bombers were
Зwas already here. Drone bombers were
flying over the skies of Afghanistan.flying over the skies of Afghanistan.Зwas already here. Drone bombers werewas already here. Drone bombers wereПП
ОFrom what I could see, the future
ОFrom what I could see, the future
was already here. Drone bombers werewas already here. Drone bombers wereОД
something far in the future.
Дsomething far in the future.
From what I could see, the futureFrom what I could see, the futureДГsomething far in the future.
Гsomething far in the future.
From what I could see, the futureFrom what I could see, the futureГО
all, they dismissed killer robots as
Оall, they dismissed killer robots as
something far in the future.something far in the future.ОТall, they dismissed killer robots as
Тall, they dismissed killer robots as
something far in the future.something far in the future.ТО
ons. If people thought about them at
Оons. If people thought about them at
all, they dismissed killer robots asall, they dismissed killer robots asОВons. If people thought about them at
Вons. If people thought about them at
all, they dismissed killer robots asall, they dismissed killer robots asВИ
the dangers of this new class of weap-
Иthe dangers of this new class of weap-
ons. If people thought about them atons. If people thought about them atИЛthe dangers of this new class of weap-
Лthe dangers of this new class of weap-
ons. If people thought about them atons. If people thought about them atЛА
gence colleagues were thinking about
Аgence colleagues were thinking about
the dangers of this new class of weap-the dangers of this new class of weap-Аgence colleagues were thinking aboutgence colleagues were thinking aboutГГ
the dangers of this new class of weap-Г
the dangers of this new class of weap-Р
realized how few of my artificial intelli-
Рrealized how few of my artificial intelli-
gence colleagues were thinking aboutgence colleagues were thinking aboutРУrealized how few of my artificial intelli-
Уrealized how few of my artificial intelli-
gence colleagues were thinking aboutgence colleagues were thinking aboutУП
reading about robotic weaponry. I
Пreading about robotic weaponry. I
realized how few of my artificial intelli-realized how few of my artificial intelli-ППreading about robotic weaponry. I
П
reading about robotic weaponry. I
realized how few of my artificial intelli-realized how few of my artificial intelli-ПА
around 2013. I had been doing a lot of
Аaround 2013. I had been doing a lot of
reading about robotic weaponry. Ireading about robotic weaponry. IА"What's
something far in the future."What's
something far in the future.
From what I could see, the future"What's
From what I could see, the future
was already here. Drone bombers were
"What'swas already here. Drone bombers were
flying over the skies of Afghanistan.flying over the skies of Afghanistan."What'sNews"
ons. If people thought about them atNews"
ons. If people thought about them at
all, they dismissed killer robots as
News"all, they dismissed killer robots as
something far in the future.something far in the future.News"VK.COM/WSNWS
ons. If people thought about them atVK.COM/WSNWS
ons. If people thought about them at
all, they dismissed killer robots asVK.COM/WSNWS
all, they dismissed killer robots as
something far in the future.VK.COM/WSNWS
something far in the future.
From what I could see, the futureVK.COM/WSNWS
From what I could see, the future
was already here. Drone bombers wereVK.COM/WSNWS
was already here. Drone bombers were
flying over the skies of Afghanistan.
VK.COM/WSNWSflying over the skies of Afghanistan.
Though humans on the ground con-
VK.COM/WSNWSThough humans on the ground con-
trolled the drones, it’s a small technicaltrolled the drones, it’s a small technicalVK.COM/WSNWSРЕЛИЗ ПОДГОТОВИЛА ГРУППА "What's News" VK.COM/WSNWS