<nettime> John Naughton on Shoshana Zuboff: 'The goal is to automate us': welcome to the age of surveillance capitalism

[Patrice Riemens <patrice@xs4all.nl>]

bwo Kiran Jonnalagadda & Frederick Noronha, with thanks


original to:
https://www.theguardian.com/technology/2019/jan/20/shoshana-zuboff-age-of-surveillance-capitalism-google-facebook

'The goal is to automate us': welcome to the age of surveillance 
capitalism

Shoshana Zuboff’s new book is a chilling exposé of the business model 
that underpins the digital world. Observer tech columnist John Naughton 
explains the importance of Zuboff’s work and asks the author 10 key 
questions

John Naughton
The Observer/Guardian, Sun 20 Jan 2019

We’re living through the most profound transformation in our information 
environment since Johannes Gutenberg’s invention of printing in circa 
1439. And the problem with living through a revolution is that it’s 
impossible to take the long view of what’s happening. Hindsight is the 
only exact science in this business, and in that long run we’re all 
dead. Printing shaped and transformed societies over the next four 
centuries, but nobody in Mainz (Gutenberg’s home town) in, say, 1495 
could have known that his technology would (among other things): fuel 
the Reformation and undermine the authority of the mighty Catholic 
church; enable the rise of what we now recognise as modern science; 
create unheard-of professions and industries; change the shape of our 
brains; and even recalibrate our conceptions of childhood. And yet 
printing did all this and more.

Why choose 1495? Because we’re about the same distance into our 
revolution, the one kicked off by digital technology and networking. And 
although it’s now gradually dawning on us that this really is a big deal 
and that epochal social and economic changes are under way, we’re as 
clueless about where it’s heading and what’s driving it as the citizens 
of Mainz were in 1495.

That’s not for want of trying, mind. Library shelves groan under the 
weight of books about what digital technology is doing to us and our 
world. Lots of scholars are thinking, researching and writing about this 
stuff. But they’re like the blind men trying to describe the elephant in 
the old fable: everyone has only a partial view, and nobody has the 
whole picture. So our contemporary state of awareness is – as Manuel 
Castells, the great scholar of cyberspace once put it – one of “informed 
bewilderment”.

Which is why the arrival of Shoshana Zuboff’s new book is such a big 
event. Many years ago – in 1988, to be precise – as one of the first 
female professors at Harvard Business School to hold an endowed chair 
she published a landmark book, The Age of the Smart Machine: The Future 
of Work and Power, which changed the way we thought about the impact of 
computerisation on organisations and on work. It provided the most 
insightful account up to that time of how digital technology was 
changing the work of both managers and workers. And then Zuboff appeared 
to go quiet, though she was clearly incubating something bigger. The 
first hint of what was to come was a pair of startling essays – one in 
an academic journal in 2015, the other in a German newspaper in 2016. 
What these revealed was that she had come up with a new lens through 
which to view what Google, Facebook et al were doing – nothing less than 
spawning a new variant of capitalism. Those essays promised a more 
comprehensive expansion of this Big Idea.

And now it has arrived – the most ambitious attempt yet to paint the 
bigger picture and to explain how the effects of digitisation that we 
are now experiencing as individuals and citizens have come about.

The headline story is that it’s not so much about the nature of digital 
technology as about a new mutant form of capitalism that has found a way 
to use tech for its purposes. The name Zuboff has given to the new 
variant is “surveillance capitalism”. It works by providing free 
services that billions of people cheerfully use, enabling the providers 
of those services to monitor the behaviour of those users in astonishing 
detail – often without their explicit consent.

“Surveillance capitalism,” she writes, “unilaterally claims human 
experience as free raw material for translation into behavioural data. 
Although some of these data are applied to service improvement, the rest 
are declared as a proprietary behavioural surplus, fed into advanced 
manufacturing processes known as ‘machine intelligence’, and fabricated 
into prediction products that anticipate what you will do now, soon, and 
later. Finally, these prediction products are traded in a new kind of 
marketplace that I call behavioural futures markets. Surveillance 
capitalists have grown immensely wealthy from these trading operations, 
for many companies are willing to lay bets on our future behaviour.”

While the general modus operandi of Google, Facebook et al has been 
known and understood (at least by some people) for a while, what has 
been missing – and what Zuboff provides – is the insight and scholarship 
to situate them in a wider context. She points out that while most of us 
think that we are dealing merely with algorithmic inscrutability, in 
fact what confronts us is the latest phase in capitalism’s long 
evolution – from the making of products, to mass production, to 
managerial capitalism, to services, to financial capitalism, and now to 
the exploitation of behavioural predictions covertly derived from the 
surveillance of users. In that sense, her vast (660-page) book is a 
continuation of a tradition that includes Adam Smith, Max Weber, Karl 
Polanyi and – dare I say it – Karl Marx.

Digital technology is separating the citizens in all societies into two 
groups: the watchers and the watched

Viewed from this perspective, the behaviour of the digital giants looks 
rather different from the roseate hallucinations of Wired magazine. What 
one sees instead is a colonising ruthlessness of which John D 
Rockefeller would have been proud. First of all there was the arrogant 
appropriation of users’ behavioural data – viewed as a free resource, 
there for the taking. Then the use of patented methods to extract or 
infer data even when users had explicitly denied permission, followed by 
the use of technologies that were opaque by design and fostered user 
ignorance.

And, of course, there is also the fact that the entire project was 
conducted in what was effectively lawless – or at any rate law-free – 
territory. Thus Google decided that it would digitise and store every 
book ever printed, regardless of copyright issues. Or that it would 
photograph every street and house on the planet without asking anyone’s 
permission. Facebook launched its infamous “beacons”, which reported a 
user’s online activities and published them to others’ news feeds 
without the knowledge of the user. And so on, in accordance with the 
disrupter’s mantra that “it is easier to ask for forgiveness than for 
permission”.

When the security expert Bruce Schneier wrote that “surveillance is the 
business model of the internet” he was really only hinting at the 
reality that Zuboff has now illuminated. The combination of state 
surveillance and its capitalist counterpart means that digital 
technology is separating the citizens in all societies into two groups: 
the watchers (invisible, unknown and unaccountable) and the watched. 
This has profound consequences for democracy because asymmetry of 
knowledge translates into asymmetries of power. But whereas most 
democratic societies have at least some degree of oversight of state 
surveillance, we currently have almost no regulatory oversight of its 
privatised counterpart. This is intolerable.

And it won’t be easy to fix because it requires us to tackle the essence 
of the problem – the logic of accumulation implicit in surveillance 
capitalism. That means that self-regulation is a nonstarter. “Demanding 
privacy from surveillance capitalists,” says Zuboff, “or lobbying for an 
end to commercial surveillance on the internet is like asking old Henry 
Ford to make each Model T by hand. It’s like asking a giraffe to shorten 
its neck, or a cow to give up chewing. These demands are existential 
threats that violate the basic mechanisms of the entity’s survival.”

The Age of Surveillance Capital is a striking and illuminating book. A 
fellow reader remarked to me that it reminded him of Thomas Piketty’s 
magnum opus, Capital in the Twenty-First Century, in that it opens one’s 
eyes to things we ought to have noticed, but hadn’t. And if we fail to 
tame the new capitalist mutant rampaging through our societies then we 
will only have ourselves to blame, for we can no longer plead ignorance.

...................

Ten questions for Shoshana Zuboff: ‘Larry Page saw that human experience 
could be Google’s virgin wood’
Continuing a tradition that includes Adam Smith, Max Weber, Karl 
Polanyi, Marx… Shoshana Zuboff.


John Naughton: At the moment, the world is obsessed with Facebook. But 
as you tell it, Google was the prime mover.

Shoshana Zuboff: Surveillance capitalism is a human creation. It lives 
in history, not in technological inevitability. It was pioneered and 
elaborated through trial and error at Google in much the same way that 
the Ford Motor Company discovered the new economics of mass production 
or General Motors discovered the logic of managerial capitalism.

Surveillance capitalism was invented around 2001 as the solution to 
financial emergency in the teeth of the dotcom bust when the fledgling 
company faced the loss of investor confidence. As investor pressure 
mounted, Google’s leaders abandoned their declared antipathy toward 
advertising. Instead they decided to boost ad revenue by using their 
exclusive access to user data logs (once known as “data exhaust”) in 
combination with their already substantial analytical capabilities and 
computational power, to generate predictions of user click-through 
rates, taken as a signal of an ad’s relevance.

Operationally this meant that Google would both repurpose its growing 
cache of behavioural data, now put to work as a behavioural data 
surplus, and develop methods to aggressively seek new sources of this 
surplus.

The company developed new methods of secret surplus capture that could 
uncover data that users intentionally opted to keep private, as well as 
to infer extensive personal information that users did not or would not 
provide. And this surplus would then be analysed for hidden meanings 
that could predict click-through behaviour. The surplus data became the 
basis for new predictions markets called targeted advertising.

Here was the origin of surveillance capitalism in an unprecedented and 
lucrative brew: behavioural surplus, data science, material 
infrastructure, computational power, algorithmic systems, and automated 
platforms. As click-through rates skyrocketed, advertising quickly 
became as important as search. Eventually it became the cornerstone of a 
new kind of commerce that depended upon online surveillance at scale.

The success of these new mechanisms only became visible when Google went 
public in 2004. That’s when it finally revealed that between 2001 and 
its 2004 IPO, revenues increased by 3,590%.

JN: So surveillance capitalism started with advertising, but then became 
more general?

SZ: Surveillance capitalism is no more limited to advertising than mass 
production was limited to the fabrication of the Ford Model T. It 
quickly became the default model for capital accumulation in Silicon 
Valley, embraced by nearly every startup and app. And it was a Google 
executive – Sheryl Sandberg – who played the role of Typhoid Mary, 
bringing surveillance capitalism from Google to Facebook, when she 
signed on as Mark Zuckerberg’s number two in 2008. By now it’s no longer 
restricted to individual companies or even to the internet sector. It 
has spread across a wide range of products, services, and economic 
sectors, including insurance, retail, healthcare, finance, 
entertainment, education, transportation, and more, birthing whole new 
ecosystems of suppliers, producers, customers, market-makers, and market 
players. Nearly every product or service that begins with the word 
“smart” or “personalised”, every internet-enabled device, every “digital 
assistant”, is simply a supply-chain interface for the unobstructed flow 
of behavioural data on its way to predicting our futures in a 
surveillance economy.

JN: In this story of conquest and appropriation, the term “digital 
natives” takes on a new meaning…

SZ: Yes, “digital natives” is a tragically ironic phrase. I am 
fascinated by the structure of colonial conquest, especially the first 
Spaniards who stumbled into the Caribbean islands. Historians call it 
the “conquest pattern”, which unfolds in three phases: legalistic 
measures to provide the invasion with a gloss of justification, a 
declaration of territorial claims, and the founding of a town to 
legitimate the declaration. Back then Columbus simply declared the 
islands as the territory of the Spanish monarchy and the pope.

The sailors could not have imagined that they were writing the first 
draft of a pattern that would echo across space and time to a digital 
21st century. The first surveillance capitalists also conquered by 
declaration. They simply declared our private experience to be theirs 
for the taking, for translation into data for their private ownership 
and their proprietary knowledge. They relied on misdirection and 
rhetorical camouflage, with secret declarations that we could neither 
understand nor contest.

Google began by unilaterally declaring that the world wide web was its 
to take for its search engine. Surveillance capitalism originated in a 
second declaration that claimed our private experience for its revenues 
that flow from telling and selling our fortunes to other businesses. In 
both cases, it took without asking. Page [Larry, Google co-founder] 
foresaw that surplus operations would move beyond the online milieu to 
the real world, where data on human experience would be free for the 
taking. As it turns out his vision perfectly reflected the history of 
capitalism, marked by taking things that live outside the market sphere 
and declaring their new life as market commodities.

We were caught off guard by surveillance capitalism because there was no 
way that we could have imagined its action, any more than the early 
peoples of the Caribbean could have foreseen the rivers of blood that 
would flow from their hospitality toward the sailors who appeared out of 
thin air waving the banner of the Spanish monarchs. Like the Caribbean 
people, we faced something truly unprecedented.

Once we searched Google, but now Google searches us. Once we thought of 
digital services as free, but now surveillance capitalists think of us 
as free.

JN: Then there’s the “inevitability” narrative – technological 
determinism on steroids.

SZ: In my early fieldwork in the computerising offices and factories of 
the late 1970s and 80s, I discovered the duality of information 
technology: its capacity to automate but also to “informate”, which I 
use to mean to translate things, processes, behaviours, and so forth 
into information. This duality set information technology apart from 
earlier generations of technology: information technology produces new 
knowledge territories by virtue of its informating capability, always 
turning the world into information. The result is that these new 
knowledge territories become the subject of political conflict. The 
first conflict is over the distribution of knowledge: “Who knows?” The 
second is about authority: “Who decides who knows?” The third is about 
power: “Who decides who decides who knows?”

Now the same dilemmas of knowledge, authority and power have surged over 
the walls of our offices, shops and factories to flood each one of us… 
and our societies. Surveillance capitalists were the first movers in 
this new world. They declared their right to know, to decide who knows, 
and to decide who decides. In this way they have come to dominate what I 
call “the division of learning in society”, which is now the central 
organising principle of the 21st-century social order, just as the 
division of labour was the key organising principle of society in the 
industrial age.

JN: So the big story is not really the technology per se but the fact 
that it has spawned a new variant of capitalism that is enabled by the 
technology?

SZ: Larry Page grasped that human experience could be Google’s virgin 
wood, that it could be extracted at no extra cost online and at very low 
cost out in the real world. For today’s owners of surveillance capital 
the experiential realities of bodies, thoughts and feelings are as 
virgin and blameless as nature’s once-plentiful meadows, rivers, oceans 
and forests before they fell to the market dynamic. We have no formal 
control over these processes because we are not essential to the new 
market action. Instead we are exiles from our own behaviour, denied 
access to or control over knowledge derived from its dispossession by 
others for others. Knowledge, authority and power rest with surveillance 
capital, for which we are merely “human natural resources”. We are the 
native peoples now whose claims to self-determination have vanished from 
the maps of our own experience.

While it is impossible to imagine surveillance capitalism without the 
digital, it is easy to imagine the digital without surveillance 
capitalism. The point cannot be emphasised enough: surveillance 
capitalism is not technology. Digital technologies can take many forms 
and have many effects, depending upon the social and economic logics 
that bring them to life. Surveillance capitalism relies on algorithms 
and sensors, machine intelligence and platforms, but it is not the same 
as any of those.

JN: Where does surveillance capitalism go from here?

SZ: Surveillance capitalism moves from a focus on individual users to a 
focus on populations, like cities, and eventually on society as a whole. 
Think of the capital that can be attracted to futures markets in which 
population predictions evolve to approximate certainty.

This has been a learning curve for surveillance capitalists, driven by 
competition over prediction products. First they learned that the more 
surplus the better the prediction, which led to economies of scale in 
supply efforts. Then they learned that the more varied the surplus the 
higher its predictive value. This new drive toward economies of scope 
sent them from the desktop to mobile, out into the world: your drive, 
run, shopping, search for a parking space, your blood and face, and 
always… location, location, location.

The evolution did not stop there. Ultimately they understood that the 
most predictive behavioural data comes from what I call “economies of 
action”, as systems are designed to intervene in the state of play and 
actually modify behaviour, shaping it toward desired commercial 
outcomes. We saw the experimental development of this new “means of 
behavioural modification” in Facebook’s contagion experiments and the 
Google-incubated augmented reality game Pokémon Go.

    Democracy has slept, while surveillance capitalists amassed 
unprecedented concentrations of knowledge and power
    Shoshana Zuboff

It is no longer enough to automate information flows about us; the goal 
now is to automate us. These processes are meticulously designed to 
produce ignorance by circumventing individual awareness and thus 
eliminate any possibility of self-determination. As one data scientist 
explained to me, “We can engineer the context around a particular 
behaviour and force change that way… We are learning how to write the 
music, and then we let the music make them dance.”

This power to shape behaviour for others’ profit or power is entirely 
self-authorising. It has no foundation in democratic or moral 
legitimacy, as it usurps decision rights and erodes the processes of 
individual autonomy that are essential to the function of a democratic 
society. The message here is simple: Once I was mine. Now I am theirs.

JN: What are the implications for democracy?

SZ: During the past two decades surveillance capitalists have had a 
pretty free run, with hardly any interference from laws and regulations. 
Democracy has slept while surveillance capitalists amassed unprecedented 
concentrations of knowledge and power. These dangerous asymmetries are 
institutionalised in their monopolies of data science, their dominance 
of machine intelligence, which is surveillance capitalism’s “means of 
production”, their ecosystems of suppliers and customers, their 
lucrative prediction markets, their ability to shape the behaviour of 
individuals and populations, their ownership and control of our channels 
for social participation, and their vast capital reserves. We enter the 
21st century marked by this stark inequality in the division of 
learning: they know more about us than we know about ourselves or than 
we know about them. These new forms of social inequality are inherently 
antidemocratic.

At the same time, surveillance capitalism diverges from the history of 
market capitalism in key ways, and this has inhibited democracy’s normal 
response mechanisms. One of these is that surveillance capitalism 
abandons the organic reciprocities with people that in the past have 
helped to embed capitalism in society and tether it, however 
imperfectly, to society’s interests. First, surveillance capitalists no 
longer rely on people as consumers. Instead, supply and demand orients 
the surveillance capitalist firm to businesses intent on anticipating 
the behaviour of populations, groups and individuals. Second, by 
historical standards the large surveillance capitalists employ 
relatively few people compared with their unprecedented computational 
resources. General Motors employed more people during the height of the 
Great Depression than either Google or Facebook employs at their heights 
of market capitalisation. Finally, surveillance capitalism depends upon 
undermining individual self-determination, autonomy and decision rights 
for the sake of an unobstructed flow of behavioural data to feed markets 
that are about us but not for us.

This antidemocratic and anti-egalitarian juggernaut is best described as 
a market-driven coup from above: an overthrow of the people concealed as 
the technological Trojan horse of digital technology. On the strength of 
its annexation of human experience, this coup achieves exclusive 
concentrations of knowledge and power that sustain privileged influence 
over the division of learning in society. It is a form of tyranny that 
feeds on people but is not of the people. Paradoxically, this coup is 
celebrated as “personalisation”, although it defiles, ignores, 
overrides, and displaces everything about you and me that is personal.

‘The power to shape behaviour for others’ profit or power is entirely 
self-authorising,’ says Zuboff. ‘It has no foundation in democratic or 
moral legitimacy.’

JN: Our societies seem transfixed by all this: we are like rabbits 
paralysed in the headlights of an oncoming car.

SZ: Despite surveillance capitalism’s domination of the digital milieu 
and its illegitimate power to take private experience and to shape human 
behaviour, most people find it difficult to withdraw, and many ponder if 
it is even possible. This does not mean, however, that we are foolish, 
lazy, or hapless. On the contrary, in my book I explore numerous reasons 
that explain how surveillance capitalists got away with creating the 
strategies that keep us paralysed. These include the historical, 
political and economic conditions that allowed them to succeed. And 
we’ve already discussed some of the other key reasons, including the 
nature of the unprecedented, conquest by declaration. Other significant 
reasons are the need for inclusion, identification with tech leaders and 
their projects, social persuasion dynamics, and a sense of 
inevitability, helplessness and resignation.

We are trapped in an involuntary merger of personal necessity and 
economic extraction, as the same channels that we rely upon for daily 
logistics, social interaction, work, education, healthcare, access to 
products and services, and much more, now double as supply chain 
operations for surveillance capitalism’s surplus flows. The result is 
that the choice mechanisms we have traditionally associated with the 
private realm are eroded or vitiated. There can be no exit from 
processes that are intentionally designed to bypass individual awareness 
and produce ignorance, especially when these are the very same processes 
upon which we must depend for effective daily life. So our participation 
is best explained in terms of necessity, dependency, the foreclosure of 
alternatives, and enforced ignorance.

JN: Doesn’t all this mean that regulation that just focuses on the 
technology is misguided and doomed to fail? What should we be doing to 
get a grip on this before it’s too late?

SZ: The tech leaders desperately want us to believe that technology is 
the inevitable force here, and their hands are tied. But there is a rich 
history of digital applications before surveillance capitalism that 
really were empowering and consistent with democratic values. Technology 
is the puppet, but surveillance capitalism is the puppet master.

Surveillance capitalism is a human-made phenomenon and it is in the 
realm of politics that it must be confronted. The resources of our 
democratic institutions must be mobilised, including our elected 
officials. GDPR [a recent EU law on data protection and privacy for all 
individuals within the EU] is a good start, and time will tell if we can 
build on that sufficiently to help found and enforce a new paradigm of 
information capitalism. Our societies have tamed the dangerous excesses 
of raw capitalism before, and we must do it again.

While there is no simple five-year action plan, much as we yearn for 
that, there are some things we know. Despite existing economic, legal 
and collective-action models such as antitrust, privacy laws and trade 
unions, surveillance capitalism has had a relatively unimpeded two 
decades to root and flourish. We need new paradigms born of a close 
understanding of surveillance capitalism’s economic imperatives and 
foundational mechanisms.”

For example, the idea of “data ownership” is often championed as a 
solution. But what is the point of owning data that should not exist in 
the first place? All that does is further institutionalise and 
legitimate data capture. It’s like negotiating how many hours a day a 
seven-year-old should be allowed to work, rather than contesting the 
fundamental legitimacy of child labour. Data ownership also fails to 
reckon with the realities of behavioural surplus. Surveillance 
capitalists extract predictive value from the exclamation points in your 
post, not merely the content of what you write, or from how you walk and 
not merely where you walk. Users might get “ownership” of the data that 
they give to surveillance capitalists in the first place, but they will 
not get ownership of the surplus or the predictions gleaned from it – 
not without new legal concepts built on an understanding of these 
operations.

Another example: there may be sound antitrust reasons to break up the 
largest tech firms, but this alone will not eliminate surveillance 
capitalism. Instead it will produce smaller surveillance capitalist 
firms and open the field for more surveillance capitalist competitors.

So what is to be done? In any confrontation with the unprecedented, the 
first work begins with naming. Speaking for myself, this is why I’ve 
devoted the past seven years to this work… to move forward the project 
of naming as the first necessary step toward taming. My hope is that 
careful naming will give us all a better understanding of the true 
nature of this rogue mutation of capitalism and contribute to a sea 
change in public opinion, most of all among the young.

• The Age of Surveillance Capitalism by Shoshana Zuboff is published by 
Profile (£25)
#  distributed via <nettime>: no commercial use without permission
#  <nettime>  is a moderated mailing list for net criticism,
#  collaborative text filtering and cultural politics of the nets
#  more info: http://mx.kein.org/mailman/listinfo/nettime-l
#  archive: http://www.nettime.org contact: nettime@kein.org
#  @nettime_bot tweets mail w/ sender unless #ANON is in Subject: