April 9, 2020 19
the kind of surveillance and spying on
citizens practiced in the Soviet bloc.
Capitalism, the theory went, venerated
privacy and protected against surveil-
lance through its embrace of property
as value. That seemed most obvious in
the form of houses with thick walls and
individual bedrooms, but also in semi-
private spaces, like bars and motels. If
private spaces for every individual were
once (say, in the sixteenth century) only
something the rich had, the spread of
wealth to a propertied middle class and
the building of homes with separate
rooms (the invention of “upstairs”) is
what made it plausible for legal think-
ers like Louis Brandeis to speak of the
masses enjoying a r ight to pr ivacy, to be
unwatched—a right to be “let alone.”
It is not surprising that we don’t begin
to see the legal idea of privacy form
until the eighteenth century, with the
spread of private spaces in which one
could conceal oneself from “the un-
wanted gaze,” whether it belonged to
neighbors or government.
Consider the ways that, by the 1960s,
the rise of a propertied middle class
had put each man in his castle, each
drinker in his saloon, each employee
in his own office. Consider the ways
in which private physical spaces (like
bedrooms), along with semiprivate
spaces like motels, bathhouses, and
dance clubs, created their own expec-
tations of privacy. (It is very possible
that various examples of countercul-
ture—the rejections of Victorian mo-
rality, the gay rights movement—came
about when private space permitted
individuals to do forbidden things un-
watched.) The same happened with the
first private virtual spaces like personal
computers and hard drives. Capital-
ism, which called all of these things
types of value, pressed for more private
spaces.
But what we’re learning is that the
symbiosis between capitalism and pri-
vacy was maybe just a phase, a four-
hundred-year fad. For capitalism is an
adaptive creature, a perfect chame-
leon; it has no disabling convictions but
seeks only profit. If privacy pays, great,
but if totalizing control pays more, then
so be it.
In a capitalist system, the expected
level of privacy can actually be cap-
tured by one single equation. Is there
more money to be made through sur-
veillance or through the building of
walls? For a long time, the answer was
walls, because walls made up houses
and other forms of private property.
Meanwhile, if you asked someone
about the size of the “surveillance
industry” in, say, 1990, they’d prob-
ably have looked at you funny. The
conversation would have been about
the hiring of private detectives, or the
hidden microphones popularized by
the Watergate break-in. To speak of
surveillance as a source of economic
value would have been nothing short of
ridiculous.
Today, the balance has shifted.
There is still money in building walls,
but the surveillance industries must be
counted as among the most significant
parts of the economy. Surveillance is
at the center of the business models of
firms like Google and Facebook, and a
part of Amazon, Uber, Lyft, and oth-
ers. Surveillance capitalism is expand-
ing to other industries: Admiral, a
British insurance firm, uses Facebook
data to help price its products differ-
ently to different prospective custom-
ers. (It seems that people who write
in short, concrete sentences and use
lists are safer drivers; excessive use of
exclamation points suggests reckless-
ness behind the wheel.) Life insurance
firms like John Hancock offer dis-
counts tied to an agreement to monitor
the customer’s Fitbit usage. And these
are just examples that happen to fit the
journalistic imperative of being easy to
describe.
Zuboff is right to argue that some-
thing transformational happened in
the early twenty-first century in the
relationship between capitalism, pri-
vacy, and, by extension, human au-
tonomy. What emerged, she thinks, is
a new form of power, which she terms
“instrumentarianism” (not her best
coinage). This form of power, accord-
ing to her, does not depend on coercion
or terror, as under a dictatorial system,
but “ownership of the means of be-
havioral modification.” It other words,
she thinks that the future belongs
to whoever is running the Skinner
boxes.
But her description of the emergence
of this new form of power is not, as I’ve
already suggested, the book’s strong
point. Zuboff tells a story of an evil
and mysterious Google that makes
the awesome discovery of the power
of “behavioral surplus” and hides it
from the world while piling up riches,
like some kind of Victor Von Doom
hiding somewhere south of Palo Alto.
Her narrative will please confirmed
Google haters, and her distance from
the industry does save her from being
anything like an apologist for the
power of the companies, but it risks
caricature.
In Zuboff’s account, the purported
idealism of Google’s founders, Larry
Page and Sergey Brin, always hid
darker motives. Their dedication to
open platforms, like their “don’t be
evil” motto, was just a kind of smoke
screen. But in seeing such sinister in-
tent, Zuboff overcredits Google rela-
tive to lesser-known figures like Lou
Montulli, an engineer who, while
at Netscape, invented the browser
“cookie”—the Web’s first and most im-
portant surveillance tool.
In my view, the history of Google is
a bit less Doctor Doomian and more
Faustian. It’s a tale of a somewhat
idealistic and outlandishly ambitious
company whose mission became cor-
rupted by good old-fashioned revenue-
demanding capitalism. I see its IPO
as the turning point: while claiming
it wanted to be different, the firm
adopted a corporate structure that ulti-
mately had only superficial distinctions
from any other Delaware-incorporated
company. Its role in the rise of surveil-
lance capitalism is therefore a story
of a different set of human failings:
a certain blindness to consequence,
coupled with a dangerous desire to
have it all.
Though we may quibble over the
narrative, Zuboff isn’t wrong about
the result. Google’s success with a
surveillance- driven advertising model
did inspire others, most especially
Facebook and Amazon but also the
cable industry and “share economy”
firms like Uber, to engage in a race
to see who can collect the most infor-
mation about its users, leading us into
what is indeed the age of surveillance
capitalism.All of this leaves one hard question:
Just how much does any of this mat-
ter? Do Google and Facebook, viewed
as agents of behavioral modification,
really have a greater influence on us
than either traditional advertisers or
other sources of influence? The Marl-
boro Man, who debuted in 1954, was
credited with a 3,000 percent increase
in sales of a cigarette that had once
been marketed as a woman’s brand
(original slogan: “Mild as May”). And
how might we measure the influence
of Google against that of an outlet
like Fox News, which follows a more
traditional propaganda formula? Can
platform influence really be compared
to the power of earlier forms of propa-
ganda, like the broadcasts that united
Germany behind Hitler?
Zuboff, anticipating these objec-
tions, warns us not to be blind to new
forms of power. (As the twentieth-
century French philosopher and Chris-
tian anarchist Jacques Ellul pointed
out, it is those who think themselves
immune to propaganda who are the
easiest to manipulate.) But what makes
it hard to answer the question is the fact
that it is entwined with a different phe-
nomenon: the return, to center stage,
of the dark arts of disinformation. No
one can deny the present influence
of social media (see Donald Trump,
election of). Yet much of that power
seems to derive from traditional pro-
paganda techniques: false or slanted
information, scapegoatism, and, above
all, total saturation with repetitive
messages.
The tools of surveillance capitalism
clearly aid and abet propaganda tech-
niques, but they are not the same thing.
To be sure, the microtargeting made
possible by data collection has made it
easier for the Russian government to
reach the right American voters with
fake news and divisive information.
But I’m not certain that this is what
Zuboff has in mind when she depicts
“instrumentarianism” as a new form
of power.
Where she is right is in asserting that
state power and platform surveillance
will combine in terrifying ways. In fact,
where Zuboff operates at exactly the
right pitch of darkness is her discussion
of surveillance capitalism’s marriage
with the state. Here it is not Google or
Russia, but the Chinese government
that is pointing the way.
For some years, the Chinese state
has been trying hard to establish a
social credit system ( )
to keep a running tally of each citi-
zen’s reputation. With the stated goal
of increasing public trust, the idea,
while only partially implemented at
this point, is to create a general so-
ciability score that can be increased
by “good” behavior, such as donating
blood or volunteering, and decreased
by antisocial behavior, such as fail-
ing to sort litter or defaulting on debt.
Losing social credit has already led
in unpleasant directions for some: it
was reported in 2019 that, owing to
“untrustworthy” conduct, 26.82 mil-
lion Chinese citizens were barred
from buying airplane tickets and
5.96 million from traveling on Chi-
na’s high-speed rail network. And
China goes even further in its cou-
pling of military-style surveillance
technologies with big-data analytics
to track and control the Muslim Ui-
ghurs in Xinjiang, using checkpoints,
cameras, and constantly updated fileson virtually every citizen of Uighur
descent.It is hard to imagine anything more
Skinneresque than the engineering
of social trust through rewards and
punishments. There might be a few—
true believers, indeed—who can see
in China’s social credit system a good
model for our future. But for the rest
of us, the urgent question is: How can
we stop it, or something like it, from
happening here? What, if anything,
can be done to avoid the dystopia
we will face when the last remaining
gaps are filled in, when our behavior
is better modeled and even easier to
control?
Reading Zuboff leads to an im-
portant answer to this question. The
protection of human freedom can no
longer be thought of merely as a matter
of traditional civil rights, the rights to
speech, assembly, and voting that we’ve
usually taken as the bedrocks of a free
society. What we most urgently need
is something else: protection against
widespread behavioral control and ad-
vanced propaganda techniques. And
that begins with completely rethink-
ing how we control the collection of
data.
That will require not a privacy
statute, as some might imagine, but a
codified antisurveillance regime. We
need, in other words, laws that pre-
vent the mass collection of behavioral
data. Most people think that privacy
laws are in place to do this, but exist-
ing privacy laws, including the Euro-
pean privacy law, have done little to
actually slow down the collection of
data. Instead, they supposedly give
us more control over when data is
collected and how it is used, which in
practice just means pop-up notices
and the placing of some limits on how
data is used. None of this is bad, but it
doesn’t actually prevent surveillance.
There is a reason that Facebook says
it welcomes European-style privacy
regulation.
A real antisurveillance law would
accomplish something different: it
would stop the gratuitous surveillance
and the reckless accumulation of per-
sonalized data. It would do that by
allowing only the collection of data
necessary to the task at hand: an app
designed to help you mix cocktails
would not, for example, be allowed to
collect location data. Gratuitous sur-
veillance would be banned—and after
collecting data, firms would be forced,
by default, to get rid of it, or fully ano-
nymize the rest of it.
What we have learned, what Skinner
and secret police alike have realized, is
this: to know everything about some-
one is to create the power to control
that person. We may not be there yet,
but there is a theoretical point—call
it the Skinnerlarity—where enough
data will be gathered about humanity
to predict, with some reasonable reli-
ability, what everyone on earth will do
at any moment. That accomplishment
would change the very structure of ex-
perience. As the legal scholar Jonathan
Zittrain has said, it would make life
“a highly realistic but completely tai-
lored video game where nothing hap-
pens by chance.”
That’s why we must dare to say what
would sound like blasphemy in another
age. It may be that a little less knowl-
edge is what will keep us free. Q