november11–24, 2019 | newyork 51
AIWillMakePrejudiceMuchWorse
IFYOU’REFEMALE,themachinesmaynot
recognizeyouashuman.Theymay notseeyou
if you’re transora personofcolor, nor, possibly,
if youhavepoordentalhygieneorcarry a cane
orarediminutiveinstatureorextraordinarily
tall.Themachinesunderstandtheworldbased
ontheinformationthey’vebeengiven,andif
youaren’t wellrepresentedinthedata—ifthe
white-maleprejudiceofhistoryitselfhasdisen-
franchisedyoutodate—thenchancesare tothe
machine you don’t exist. A dark-skinned
womanintheU.K. couldn’t renewherpassport
onlinebecausethedigitalformlookedat her
photoanddidn’t recognizeit asa properface.
Tr anspeopleconfoundairport bodyscanners
andareregularlyhauledoutofsecuritylinesto
befriskedasifthey were terrorist suspects.
Worst-casescenariosare notsofar-fetched.A
self-drivingcarknowstobrake inthecrosswalk
whenit seesa person.Butwhatdoesit under-
standa persontolooklike?
If youthinkstructuralbiasis badnow, inother
words,just waituntilthemachinestake over.
“Bias,”warnsKateCrawford,co-founderofthe
AINowinstituteat NYU, ina lectureshegave
lastyear, “ismore ofa featurethana bugofhow
AIworks.”Andtheworst ofit is that youmay
neverknowhowthemachineshavejudgedyou,
orwhythey havedisqualifiedyoufromthat
opportunity, that career,that scholarshiporcol-
lege. Youneverseetheadonyoursocial-media
feedforyourdreamjobasa plumberorrooferor
softwareengineerbecausetheAIknowsyou’re
female, anditperpetuates thestatus quo.
(Instead,youonlyseeadsforwaitressesorhome
health-careworkers—lowerpayingandwithless
opportunity foradvancement.)Theseare real-
lifeexamples,bytheway.
Thereasonrecruitingenginesdowngrade
candidateswithnameslike Latanyais that peo-
plenamedLatanyahavealwayshada harder
timefindinga job,accordingtoresearchcon-
ductedbyHarvard’sLatanyaSweeney, whoused
herownnameasa sample.(Andif youdohap-
pentobesearchingforLatanyaonline,youwill
findadsalongsideyoursearchforcriminal-back-
groundchecks.)Onerecentexperimentshowed
thatAIs gavespecialpreferencetotherésumés
ofjobcandidatesnamedJaredwhoplayed
lacrosse,whichcorroborateseveryoneofyour
worstfearsabouttheworld.Butthere it is,repli-
catingadinfinitumandwithoutoversight.
Thedataare onlypart oftheproblem.There
arealsothemen(mostlymen:AIresearchers
are 88 percentmale)whobuildthealgorithms
thattellthemachineswhat todo.Sometimes
thefaultydesignisunintentional,aswhen
AmazondecidedtocreateanAIthat sorted
résuméstofindoptimalemployees.Themenin
the Amazon AI lab built their algorithm around
a question—what kinds of people get hired to
work at Amazon—and then loaded generations
of résumés into the machine to teach it the attri-
butes of a successful Amazon employee. And
what did they find? That maleness was a pre-
requisite for getting hired at Amazon because
for as long as Amazon has been in business it
has promoted and rewarded men. Ashamed of
themselves, the AI geniuses scrubbed their pro-
gram. They tried to make the AI neutral, but
they couldn’t guarantee it would ever unlearn
itsbiasedbeginningsandwoundupkilling the
project dead. —LISA MILLER
We Won’t Need
Tribute Bands.
We’ll Have
Hologram Tours.
Frank Zappa and Roy Orbison
tour the country. Whitney Houston
and Amy Winehouse are next.
When all of history's classic rock
acts start selling tickets to
hologram tours, you'll never
wonder again what to buy your
dad for his birthday.
Advertisers Will Know More
About You Than Your Doctor
IN MAY, Airbnb announced a new feature in
partnership with the at-home genetic-testing
company 23andMe: “dedicated pages that cor-
respond with 23andMe’s genetic populations,”
so that users interested in “DNA travel” can
book Airbnbs in whatever region their 23andMe
test results suggest their ancestors came from.
To tech paranoiacs like myself, it was hard not
to notice, in the Airbnb press materials, the
future of internet advertising—and therefore
the future of the internet itself. As the at-home
consumer genetic-testing market expands, and
as the demand for targeted-advertising profiles
increases, what could the outcome possibly be
besides DNA-based ad targeting? Data brokers
would be able to identify you not just as “white”
but as “34 percent Irish” on any major website
you visit. You don’t have to be particularly race-
obsessed to see the opportunities: Airbnb, lan-
guage schools, and doctors offering treatments
for genetic diseases might all like to target their
advertisements based on specific genetic pro-
files. If nation-states wither away in the face of
MAY 21
23andMe
andAirbnb
teamup.
MAY 1 7
UNESCO
report says
AI voice
assistants
like Alexa
affirm
gender
stereotypes.
MAY 20
Whitney
Houston's estate
announces
hologram tour.
Hologram
Whitney
PHOTOGRAPH: BASE (HOUSTON)
