on a statue in front of its headquarters.
The Wall Street Journal has reported that
a ByteDance office in Beijing includes
a room for a cybersecurity team of the
Chinese police, which the company in-
forms when it “finds criminal content
like terrorism or pedophilia” on its apps.
Last year, ByteDance was ordered to
suspend Toutiao and to shut down
a meme-centric social app called Nei-
han Duanzi—the name means some-
thing like “implied jokes”—because the
content had become too vulgar, too dis-
orderly, for the state. Zhang issued an
apology, written in the language of gov-
ernment control. ByteDance had allowed
content to appear that was “incommen-
surate with socialist core values, that did
not properly implement public opinion
guidance,” he said.
Three days later, the Times reported
that the Chinese government had de-
ployed facial-recognition technology to
identify Uighurs, a Muslim minority in
the country, through its nationwide net-
work of surveillance cameras. China has
imprisoned more than a million Uighurs
in reëducation camps, in Xinjiang, and
has subjected them to a surge in arrests,
trials, and prison sentences. In August,
I asked a ByteDance spokesperson about
the fear that the massive trove of facial
closeups accumulated on its various prod-
ucts could be misused. Even if people
trusted ByteDance not to do anything
sinister, I said, what if a third party got
hold of the company’s data? The spokes-
person told me that the data of Ameri-
can users was stored in-country—Tik-
Tok’s data is now kept in the U.S. and
Singapore, the rep said—and noted, non-
chalantly, that people made their faces
available to other platforms, too. Of
course, U.S. tech companies often don’t
seem answerable enough to the govern-
ment—or, rather, to the public. The
American system has its own weaknesses.
Dinesh Raman, an A.I.-alignment re-
searcher in Tokyo, who has studied Byte-
Dance as a consultant for some of its in-
vestors, spoke with a mixture of alarm
and admiration about the company’s A.I.
capabilities. “The system is doing bil-
lions of calculations per second,” he said.
“It’s data being transmitted at a scale I’ve
never seen before.” Raman insisted that
TikTok had kept its platform tightly po-
liced in part through its algorithm, which,
he said, is able to identify videos with
dangerous content. (TikTok’s modera-
tors are trained to apply different stan-
dards to every market, the company told
me.) He pointed me to the “Gaga Dance”
challenge, a meme on Indonesian Tik-
Tok that asked users to mirror the poses
of cheerful yellow stick figures that floated
across the screen. The A.I., he suggested,
was training itself in pose estimation, a
deep-learning capability with major sur-
veillance implications. OpenPose, a pro-
gram developed at Carnegie Mellon, has
been used by a Japanese telecom com-
pany to alert shopkeepers to customers
whose movements supposedly signal that
they are likely to steal something.
The Chinese government is more
interested in surveilling and controlling
its own citizens than it is in monitor-
ing foreign nationals; one of the rea-
sons that ByteDance launched TikTok
as a separate entity from Douyin was
to establish a firewall between the Chi-
nese state and users outside China. But
state interference can cross borders. In
August, Facebook and Twitter revealed
that they’d found evidence of a Chi-
nese-government campaign to spread
disinformation about the protests in
Hong Kong, which began in June by
calling for the withdrawal of an extra-
dition bill and have since widened in
scope, demanding democratic reforms.
If you pull up the hashtag #Hong-
Kong on TikTok, you’ll find plenty of
videos, but few, if any, about the protests.
The hashtag #protest elicits demonstra-
tions from around the globe—London,
Melbourne, South Africa, and, especially,
India—but almost none from Hong
Kong. (On Instagram, both #HongKong
and #protest call up plenty of such im-
ages.) Meanwhile, a search for one of the
primary Chinese-language hashtags that
Hong Kong protesters have used on other
platforms yields a small handful of vid-
eos, with a total of a hundred and ten
thousand views. (As the Washington Post
noted, in a piece investigating the rela-
tive absence of the Hong Kong protests
on TikTok, videos hashtagged #snails
have more than six and a half million
views.) It’s true that the Hong Kong user
base is not large, relatively speaking—
TikTok told me that the app had fewer
than a hundred and fifty thousand daily
active users there—though that is the
case for Twitter, too, and videos from the
protests have gone viral on that platform.TikTok is generally thought of as a place
for goofing off rather than for engaging
in political discourse, and a TikTok ex-
ecutive dismissed the idea that the com-
pany was manually or algorithmically
suppressing Hong Kong-related content.
But one of the risks of giving our atten-
tion to entertainment governed by pri-
vately controlled algorithms is that those
who own the algorithms will always be
able to say that they are merely deliver-
ing what we want to see.
A platform designed for viral com-
munication will never naturally be pol-
itics-free. In August, a new sort of video
started appearing on Douyin. Uighurs
in China were using the app’s editing
suite to place themselves against a back-
drop of loved ones who have disap-
peared, as sad string music plays. In one,
a tearful young woman wearing a yel-
low shirt holds up four fingers, one for
each person in the photo behind her. It
may be a double signal: “four” and “death”
are pronounced similarly in Mandarin.
Douyin has deleted many of these vid-
eos, although, like everything that goes
viral on TikTok, they have found an au-
dience on Instagram and Twitter.T
ikTok is not the first social-media
app to begin its life with an air of
freewheeling fun. The darker and more
complicated parts of life never stay away
forever. A college student from Phila-
delphia recently went viral with a mul-
tipart video account of her relationship
with the rapper and onetime Vine star
Riff Raff, which began when she was
seventeen. A Miami student was ar-
rested after his videos were interpreted
as threats to shoot up schools. TikTok
may figure out how to maintain or en-
force a jovial vibe more effectively than
its predecessors have—but, even if it
does, the kids who made it popular may
get bored and move on to the next thing.
Whatever comes along will likely owe
something to TikTok. Facebook has al-
ready released a TikTok clone, called
Lasso, which flopped, and the app re-
searcher Jane Manchun Wong recently
discovered that Instagram has been test-
ing TikTok-like features. A.I.-powered
algorithms are becoming central to the
ways that we process our everyday exis-
tence. Someday, other companies could
use ByteDance’s A.I. systems the way
they now use Google’s cloud-computing40 THENEWYORKER,SEPTEMBER30, 2019