D6 N THE NEW YORK TIMES, TUESDAY, JULY 30, 2019
Toby Walsh, a professor at the University of
New South Wales in Sydney, is one of Aus-
tralia’s leading experts on artificial intelli-
gence. He and other experts have released
a report outlining the promises, and ethical
pitfalls, of the country’s embrace of A.I.
Recently, Dr. Walsh, 55, has been working
with the Campaign to Stop Killer Robots, a
coalition of scientists and human rights
leaders seeking to halt the development of
autonomous robotic weapons.
We spoke briefly at the annual meeting of
the American Association for the Advance-
ment of Science, then for two hours via tele-
phone. Below is an edited version of those
conversations.
You are a scientist and an inventor. How did
you become an activist in the fight against
‘killer robots’?
It happened incrementally, beginning
around 2013. I had been doing a lot of read-
ing about robotic weaponry. I realized how
few of my artificial intelligence colleagues
were thinking about the dangers of this new
class of weapons. If people thought about
them at all, they dismissed killer robots as
something far in the future.
From what I could see, the future was al-
ready here. Drone bombers were flying
over the skies of Afghanistan. Though hu-
mans on the ground controlled the drones,
it’s a small technical step to render them
autonomous.
So in 2015, at a scientific conference, I or-
ganized a debate on this new class of weap-
onry. Not long afterward, Max Tegmark,
who runs M.I.T.’s Future of Life Institute,
asked if I’d help him circulate a letter calling
for the international community to pass a
pre-emptive ban on all autonomous robotic
weapons.
I signed, and at the next big A.I. confer-
ence, I circulated it. By the end, we had over
5,000 signatures — including Elon Musk,
Daniel Dennett, Steve Wozniak.
What was your argument?
That you can’t have machines deciding
whether humans live or die. It crosses new
territory. Machines don’t have our moral
compass, our compassion and our emo-
tions. Machines are not moral beings.
The technical argument is that these are
potentially weapons of mass destruction,
and the international community has thus
far banned all other weapons of mass
destruction.
What makes these different from previ-
ously banned weaponry is their potential to
discriminate. You could say, “Only kill chil-
dren,” and then add facial recognition soft-
ware to the system.
Moreover, if these weapons are produced,
they would unbalance the world’s geopoli-
tics. Autonomous robotic weapons would be
cheap and easy to produce. Some can be
made with a 3-D printer, and they could eas-
ily fall into the hands of terrorists.
Another thing that makes them terribly
destabilizing is that with such weapons, itwould be difficult to know the source of an
attack. This has already happened in the
current conflict in Syria. Just last year,
there was a drone attack on a Russian-
Syrian base, and we don’t know who was ac-
tually behind it.Why ban a weapon before it is produced?The best time to ban such weapons is before
they’re available. It’s much harder once
they are falling into the wrong hands or be-
coming an accepted part of the military tool
kit. The 1995 blinding laser treaty is per-
haps the best example of a successful pre-
emptive ban. Sadly, with almost every other
weapon that has been regulated, we didn’t
have the foresight to do so in advance of it
being used.Your petition — whom was it addressed to?The United Nations. Whenever I go there,
people seem willing to hear from us. I never
in my wildest dreams expected to be sitting
down with the under secretary general of
the U.N. and briefing him about the technol-
ogy. One high U.N. official told me, “We
rarely get scientists speaking with one
voice. So when we do, we listen.”
So far, 28 member countries have indi-
cated their support. The European Parlia-
ment has called for it. The German foreign
minister has called for it. Still, 28 countries
out of 200! That’s not a majority.Who opposes the treaty?The obvious candidates are the U.S., the
U.K., Russia, Israel, South Korea. China has
called for a pre-emptive ban on deployment,
but not on development of the weapons.
It’s worth pointing out there is going to be
a huge amount of money being made by
companies selling these weapons and the
defenses to them.Proponents of robotic weapons argue that
by limiting the number of human
combatants, the machines might make
warfare less deadly.I’ve heard those arguments, too. Some say
that machines might be more ethical be-
cause people in warfare get frightened and
do terrible things. Some supporters of the
technology hope that this wouldn’t happen
if we had robots fighting wars, because they
can be programmed to abide by interna-tional humanitarian law.
The problem with that argument is that
we don’t have any way to program for some-
thing as subtle as international humanitar-
ian law. Now, there are some things that the
military can use robotics for — clearing a
minefield is an example. If a robot goes in,
gets blown up, you get another robot.Robotic warfare has long been the subject
of science fiction films. Do you have a
favorite?No, most A.I. researchers — myself includ-
ed — dislike how Hollywood has dealt with
the technology. Kubrick’s “2001” is way off,
because it is based on the idea that there
will be machines that will have the desire
for self-preservation, and that will result in
malevolence toward humans.
It’s wrong to assume they’ll want to take
over, or even preserve themselves. The in-
telligence we build is going to be quite dif-
ferent from what humans have, and it won’t
necessarily have the same character flaws.
These machines don’t have any con-
science, and they don’t have any desire to
preserve themselves. They’ll do exactly
what we tell them to do. They are the most
literal devices ever built.
I dislike “The Terminator,” too. That tech-
nology is far, far away. There are more mun-
dane technologies we should be worried
about now, like the drones I mentioned.
Now, I do like “Her,” because it is about
the relationships we’ll have in a future when
we’ll be increasingly interacting with ma-
chines. It will be possible, as in the movie,
that we will develop feelings for them.
That movie is about how A.I. is going to be
a pervasive part of our existence in every
room, every car. They will be things that lis-
ten to us, answer our questions and “under-
stand” us.Since 2013, you’ve been spending as much
time on your activism as you have on
scientific research. Any regrets?No. This is important to be doing right now.
Twenty years ago, like many of my col-
leagues, I felt that what we were doing in
A.I. was so far from practice that we didn’t
have to worry about moral consequences.
That’s no longer true. I have a 10-year-old
daughter. When she’s grown, I don’t want
her to ask, “Dad, you had a platform and au-
thority — why didn’t you try to stop this?”Autonomous weapons are
closer than we think, this
leading A.I. expert believes.
By CLAUDIA DREIFUS‘You can’t have machines
deciding whether
humans live or die.’
He May Be Your Best Hope Against the Killer Robots
Toby Walsh, an authority in
artificial intelligence at the
University of New South Wales
in Sydney, Australia. He is
working with a coalition of
scientists and human rights
leaders seeking to halt the
development of autonomous
robotic weapons.DEAN SEWELL FOR THE NEW YORK TIMESA CONVERSATION WITH
TOBY WALSH
tute of Technology in Lausanne; Ms. Martin
had visited his lab two weeks earlier. “We’re
heading toward her with more mathemati-
cal and scientific methods.”
Mr. Solly, Ms. Martin’s partner for the
workshop, is a director at Format Engi-
neers in Bath, England. The firm is perhaps
best known for its work with Arthur
Mamou-Mani, the French architect, and his
design for the 2018 Burning Man temple,
“Galaxia.”
What Mr. Solly and Ms. Martin both ap-
preciate about woven structures is that
there are no nuts and bolts, and few fixings.
For the most part, woven bamboo holds it-
self in place through the friction of the over-
under-over-under intersections. And it’s a
“form-finding” process. As Ms. Martin ex-
plained to her students, “It’s about letting
the bamboo do what it wants to.”
“What’s fun about Alison’s work is how
beautiful it is, and it just comes from her
head,” Mr. Solly said. “I could spend ages
trying to work out on a computer what she
does quickly in a tactile fashion.”
The two first met in person last fall at a
conference on Advances in Architectural
Geometry, and with the São Paulo work-
shop they found an opportunity to collabo-
rate. As an engineer, Mr. Solly envisioned
translating Ms. Martin’s impressive vocab-
ulary of shapes onto bigger structures.
Ms. Martin studied graphic design in the
1970s in London at what is now Central
Saint Martins, and began weaving a decade
later, when life took a detour. In 1985, her
husband, Mauro Cuomo, an Italian comput-
er scientist, left his job at Apple, and they
moved back to Italy, eventually settling in
the remote Tuscan hill town of Fivizzano.
She focused on her family of five, and her
“one-woman mission impossible” to make
them self-sufficient on their small holding.
The property had come cheap, thanks to a
large stand of invasive bamboo.
“We had to chop the bamboo down every
year to stop it from getting into the olives
and vines and other things I was trying to
grow,” she said. She realized that the best
way to deal with bamboo was to treat it as a
resource.
She made practical garden structures to
support climbing plants — peas and borlotti
beans, cucumbers, pumpkins and melons.
She built chicken coops, raised beds, covers
for hay and wood piles as well as shade
structures. But she became frustrated with
the bamboo construction techniques she
found online.
She made models with strips of paper,
grew curious about the difference between
biaxial and triaxial weaves (with two or
three straight strips) and studied how non-
Euclidean geometry could be applied to
weaving.
For instance, a basket maker might start
with a woven tessellation of hexagons.
Swapping out one hexagon for a polygon
with fewer sides — a pentagon, say — intro-
duces a singularity and generates positive
curvature, like the outer curve of a dough-
nut. Swapping in a polygon with more sides,
such as an octagon, generates negative cur-
vature, like a doughnut’s interior.
Ms. Martin searched the internet for im-
ages of hexagonal mesh structures resem-
bling the objects she was creating. There
she encountered Alan Mackay, a crystallog-
rapher who predicted the existence of qua-
sicrystals, and Eiji Osawa, a chemist who
predicted the structure of the buckminster-
fullerene, a soccer-ball-shaped molecule
made of 60 carbon atoms. These scientists
made use of the same geometric rules, and
often gave a nod to patterns they had ob-
served in the weave of traditional bamboo
vases and baskets.
In 2011, she met Kenneth Snelson, a
sculptor, at a seminar in Rome. His motto —
“weaving, mother of tensegrity” — made an
impression, as did Anni Albers, a weaver,
and Ruth Asawa, a sculptor who once said:
“Sculpture is like farming. If you just keep
at it, you can get quite a lot done.”
“That’s the way I see things,” said Ms.
Martin. “I’m not dedicated all day every day
to my oeuvre. A little bit every day adds up
to something.”
In 2015, she won first prize at the Future
Vision contest organized by the Interna-
tional Association for Shell and Spatial
Structures. Her entry, a 13-foot-wide hyper-
bolic Lycra-patchwork mobile, won out over
27 competing entries from M.I.T., Cam-
bridge University and the engineering of-
fice ARUP, among others.
More recently, she began collaborating
with Phil Ayres, an architect in Copenhag-
en. “The key insight that I got from Alison is
how you control double curvature in the
weave system,” he said.
Last year they published a paper, “Be-
yond the Basket Case,” investigating how to
translate traditional weaving knowledge
into computational design tools. Now they
are exploring how to make the rules of
weaving attainable at an architectural scale
(possibly using robotics).
The plan for that Tuesday on the Min-
hocão was to build a dome from 30 strips of
bamboo, harvested the weekend before
from the hillside garden of James Elkis, a
pioneer of the medium, who lives southwest
of São Paulo. (Ms. Martin had seen his bam-
boo constructions online about 15 years ago,
and when Mr. Elkis mentioned that he now
makes bicycle frames from bamboo, Ms.
Martin began twisting a strip into a wheel.)
The group — 27 aspiring young architects
and urban and landscape designers — had
done a practice run with their tutors on the
weekend, with limited success. Their dome,
woven upright, was skewed and pointy at
the top, rather than round.
A crucial part of the workshop curricu-
lum was “digital parameterization” — simu-
lating structures on the computer and
tweaking design parameters from one iter-
ation to the next. For example, a key soft-
ware tool, K2Engineering, designed by Cec-
ilie Brandt-Olsen, predicts a material’s in-
ternal “bending stress” based on the ap-
plied force.
Mr. Solly, having crunched the bamboo’s
numbers and consulted the paper model,
proposed a solution for the skewing: the
dome could instead be woven flat on the
ground and then “popped up.”
Ms. Martin thought that bending the
strips all at once might cause breakage. But
they were keen to get a proof of concept, oneway or another. Mr. Solly said, “We’ll carry a
pile of bamboo up there and see if we get
arrested!”
Setting out from the workshop’s base at
Escola Da Cidade, a private college for ar-
chitecture and urbanism, the bamboo cara-
van wound its way toward the freeway, oc-
casionally breaking into song — “Believe,”
by Cher, and “Evidências,” by the Brazilian
duo Chitãozinho & Xororó. The group
walked up an exit ramp, found a favorable
site and began marking the dome’s 50-foot
circumference in chalk on the roadway.
Visible on the skyline was Oscar Niemey-
er’s sinuous residential high rise, the Edifi-
cio Copan. “It’s not the right angle that at-
tracts me, nor the straight line, hard, inflexi-
ble,” Niemeyer once said. “What attracts
me is the free and sensual curve — the
curve that I find in the mountains of my
country.”
Once all the strips were woven on the
roadway — combining weaving principles
with a fivefold symmetry pattern typical of
Islamic geometry — the students moved
swiftly, lifting the spray of bamboo and
bending down the verticals. The dome
popped into shape nicely, as Mr. Solly had
predicted. “James deserves all the credit,”
said Ms. Martin. Still, she told the students,
“the computer is not your only tool. There’s
a lot of information in the paper model.”
The enterprise drew a crowd, although
the police, cruising by on motorbikes,
hardly took notice. Felipe Rodrigues, an ar-
chitect and a member of the Parque Min-
hocão Association, who was walking a full
lap of the freeway while listening to NPR,
stopped by to discuss the complexity of the
space. “It’s alchemy,” he said: precious pub-
lic space, in a city where shopping malls are
known as the “Paulista beach.”
“The park already exists,” he said. “It’s
already here. The park is the people who
use it. I don’t see it anymore as an elevated
highway. This is a platform for activities on
which anything can happen.”
For the remainder of the week, the stu-
dents assembled in design teams and envis-
aged their own structures for the park. One
team went for a decorative, 50-foot Möbius
loop. Another produced a rolling swoosh
that became an irresistible tunnel for skate-
boarders when installed on the Minhocão
during the workshop’s final day.
“It was wild fun,” said Camila Calegari
Marques, an architect and a Martin groupie
of sorts, having participated in a 2017 work-
shop in Barcelona, Spain, that involved
weaving with wooden strips.
And the dome, it turned out, not only as-
sembled well but also disassembled and re-
assembled efficiently — it went up again
during the finale. Ms. Martin said, “We had
even more useful structural properties than
I had envisaged: deployability, nice struc-
tural stability and highly portable.”
At one point, the group stood back from
their dome, admired it and then looked to
Mr. Lee: Where to next?
“Na curva!” he said, meaning, “At the
curve!” They hoisted the dome overhead —
with Mr. Lee’s 6-year-old daughter under-
neath, seated on Mr. Solly’s shoulders —
and, singing again, walked it down the Min-
hocão to the desired bend in the road.The Old Bamboo, Seen Anew
CONTINUED FROM PAGE D1
Above, Alison Grace
Martin with students in
São Paulo. Right,
James Solly, a director
at Format Engineers in
Bath, England. Below,
Camila Calegari
Marques, a São Paulo
architect, weaving
bamboo strips together.PHOTOGRAPHS BY GABRIELA PORTILHO FOR THE NEW YORK TIMES