Bloomberg Businessweek - USA (2020-12-21)

◼ TECHNOLOGY Bloomberg Businessweek December 21, 2020

15

PHOTOGRAPH

BY

MAMADI

DOUMBOUYA

FOR

BLOOMBERG

BUSINESSWEEK

operationshighlightsthetensioninherentin

thesearrangements.

JoséHernández­Orallo, a professor at the

Valencian Research Institute for Artificial

Intelligence in Spain, says a lot of internal

research probably doesn’t get pursued or pub­

lishedbecauseofconflictsofinterestora desire

toprotecttradesecrets.Hecompareslettingtech

companies assess their own products to letting

students grade their own homework. “Having

independent organizations or academia doing this

would be desirable,” he says.

Gebru’s most famous work is an example of

how research can push companies to change AI

practices,evenif someinitiallyresist.Whileserv­

ingasa postdoctoralresearcheratMicrosoftin

2018,sheco­authored a paper with MIT researcher

JoyBuolamwinithatshowedhowthecompa­

ny’sfacialrecognitionprogrammisidentifiedthe

genderofdark­skinned women 21% of the time but

worked nearly perfectly on White men. It revealed

similar flaws in programs from companies such

as IBM. The paper prompted the companies to fix

their products.

Google has established ethical research as a

permanent aspect of its operations, treating the

riskofbiasescreepingintoitsproductsina simi­

larwaytocybersecuritythreats.“Fairnessis not

somethingthat’sone­and­done,” said X. Eyee,

who leads a team at Google dedicated to root­

ing out algorithmic bias, in an interview that took

place before Gebru’s exit. “It has to be continu­

ously prioritized, because as society changes, so

will those fairness needs.”

Eyee declined to speak about Gebru’s departure,

through a company spokesperson. But the incident

has clearly strained an already frayed relationship

between Google management and restive factions

of its workforce. Jeff Dean, Google’s head of AI,

addressed the Black Googler Network, an employee

group, soon after Gebru’s departure but didn’t take

questionsdirectly,leavingsomeemployeesfeel­

ingfurtheralienated.Helatercanceledwhatwas

supposedtobeanall­hands meeting with Google

Research scheduled for Dec. 15.

“Therightthingistoacknowledgethatwhat

happenedto Timnitwasn’tright,”wroteone

employeeina question­and­answer session, add­

ing that the executives at the event had failed to

do so. “Without this it is hard for me as a Black

Woman to trust Google.”

Asked what she plans to do next, Gebru says, “I

really want to be in an environment where I don’t

have to constantly be drained by fighting and just

proving my humanity.”

THEBOTTOMLINE Googlehasinvestedheavilyin researching

theethicalimplicationsofitsproducts,butitsattemptstoundercut

criticalresearchexposetensionsin itsapproach.

One solution might be to operate outside the

industry altogether. But there are drawbacks to

thatmodelofauditingAIsystems,too.Academic

institutionsdon’thavethesameresourcesas

corporate research labs and may not have access to

allthedatatheyneed.Also,theclosera researcher

istothepeoplewhoarebuildingtheproducts,

themorelikelysheistoactuallyinfluencetheir

shape,Gebrusays.“It’simportant for people to

be in these corporations, along with the people

who are thinking about the next product,” she

says.“Buttheresearchersneedtohavesome

amountofpower.Theyshouldn’tjustbethere

torubber­stamp or as a fig leaf.” �Dina Bass,

Gerrit de Vynck, Shelly Banjo, and Mark Bergen

Whilemanagingherconveniencestoreinthecen­

tralChinesecityofChangsha,ChenShishunalways

hasaneyeonherphone.Inpart,she’sjusttalking

onWeChatandsharingphotoswithherimprobably

largenetworkoffriendsandneighbors.ButChen

alsomonitorsgroceryappsforbulkdealsonfruits

andvegetables,thengathersordersfrompeopleshe

knowsandhasthefooddeliveredtoherstore.

Theroleofneighborhoode­commercemiddle­

man,ortuanzhang, hasbecomeanincreasingly

vitaloneinChinesecitiessincethefirstCovid­ 19

lockdowns. Chen, the kind of local merchant who

does things like help customers carry purchases

home, took charge of placing online grocery orders

for the neighborhood when it was locked down this

February. Word of her services spread quickly, and

she now has almost a thousand people seeking to

take advantage of steep discounts and cheaper

shipping fees she gets by placing big orders—on the

busiest days, she and an assistant handle 800—and

centralizing deliveries.

There are hundreds of thousands of such opera­

tions across China. People form community buying

groups based on shared neighborhoods, districts, or

even apartment blocks. While some version of this

has existed since rural farmers banded together

● Chinese tech companies

look to partner with local

leaders who help coordinate

neighborhood purchases

China’s New

E-Commerce

Middlemen

● Expected revenue

from community

buying in 2020

$13.6b

“I really want

to be in an

environment

where I don’t

have to be

constantly

drained by

fighting”