x Abu Dhabi, UAEFriday 19 January 2018

Tweet tweet, who's there? Big Data

A former CIA computer technician is on the run after blowing the whistle on a vast, secret US surveillance operation. But in an age where we all share secrets on social media and companies boast of their ability to harvest that data, why is anyone surprised? Jonathan Gornall reports

The Google data centre in Oregon, where the internet giant stores emails, photos, video and other information shared by its millions of users. Connie Zhou / AP Photo
The Google data centre in Oregon, where the internet giant stores emails, photos, video and other information shared by its millions of users. Connie Zhou / AP Photo

An academic article in the Columbia Law Review captures the concern over secret government data harvesting convulsing commentators on both sides of the Atlantic.

"American society," wrote Alan F Westin, professor of public law and government at Columbia University, "is in the midst of a great debate over privacy, precipitated by the development and use of new surveillance devices and processes by both public and private authorities."

The concern about "these new means of augmented surveillance over individuals and groups", continued Westin, "now spans the ideological spectrum from extreme left to hard right. Worried protests against 'Big Brother' tendencies have become a staple item in the press, government proceedings, law reviews, and social science journals."

In a week filled by revelations about the extent of information gathering by America's National Security Agency, Westin's paper, The Current Impact of Surveillance on Privacy, appears to be a timely analysis.

In fact, it appeared in June 1966.

There is nothing new about the surveillance of individuals by the agents of state and commerce. All that has changed in the past five decades is the technology - aided by our ever-increasing willingness to share every facet of our private lives with the world.

Westin, a lawyer and political scientist celebrated in his New York Times obituary as the man who "defined the modern right to privacy in the incipient computer age", died this year, aged 83.

He missed by a few months the current revelations about America's Prism programme, spilled to the media by a disgruntled former CIA technician. But in his last years, according to his obituary writer, he had "turned his attention to the Niagara of personal data loosed by Google, Facebook and their ilk [though] trying to stem this tide was a hopeless task, and he knew it."

As Westin was coming to understand, privacy as we have imagined it is as good as dead - and every one of us has our fingerprints on the smoking gun.

In this era of almost universal digital connectivity, Big Data is the prize sought by organisations, public and private, whether tracking our shopping habits to sell us more groceries, or mapping our telephone contacts to discover if we are part of a criminal or terrorist conspiracy.

And while we say we prize our privacy, we are our own worst enemies, seduced into becoming willing collaborators in our own surveillance by a subtle blend of vanity, greed and wonder at the irresistible marvels of modern technology.

Take the "Terms of service" agreement you tick without reading whenever you register for a website or download an app. That, says the Zero Knowledge Privacy Foundation, is you agreeing to the organisation's privacy policy and surrendering control of your personal data.

"Just because they're called privacy policies," the organisation warns, "doesn't guarantee that your information will be kept private."

Craigslist, which operates in 70 countries and is visited by 60 million users every month in America alone, states in its small print: "We don't share your information with third parties for marketing purposes" but "may disclose information about its users to law enforcement officers or others, in the good faith belief that such disclosure is reasonably necessary [to] protect the rights, property, or personal safety of Craigslist, its users or the general public."

According to a survey last year by PrivacyChoice, almost a fifth of the top 2,500 websites do not reveal whether they pass on your information to other organisations. And only 2 per cent of sites promise users they will be notified if their details are sought by law enforcement or other government agencies.

In the fast-moving discourse about digital privacy, the debate about the wisdom of the voluntary disclosures we make about our lives on social media sites is already old hat, yet regardless we continue to engage with technology that captures every trace of our existence.

We all know, for instance, that CCTV cameras can follow our progress through a city centre or track our car from one end of a country to the other; that credit cards and customer loyalty schemes record when and where we shop and what we buy; that internet browsers record what websites we visit; that search engines record our searches and that smart phones not only reveal which numbers we call but can also pinpoint our exact position.

And yet still we clamour for the next opportunity to erode even more of the privacy we supposedly hold so dear.

Take Google's Glass project. Spectacles that can record and transmit everything the wearer sees and hears are almost laughably corrosive of privacy, and yet the demand for them is huge. As one contributor put it this week in the online debate at the Guardian over the Prism revelations, "Orwell never predicted that people would be so enthusiastic for surveillance that they would pre-order the tools for $1,500 a pop".

In thrall to technology as it gets smarter, we seem to grow dumber about its implications. Part of the problem is that most of us know so little about it that we are unable to make informed assessments of its value versus its cost. The result is that while we can be stampeded easily by fear of something we do not really understand (remember the Millennium Bug?), we can just as easily persuade ourselves that a technology we find fun and useful is harmless.

That last sentence, for example, I did not type, but spoke on to my Pages document using my Apple iMac's built-in voice-recognition system. The software is not on the computer; instead, the words I speak travel instantly to Apple's servers in the US, where they are converted into type and shot back to me.

And, for all I know, every word - and every calendar entry, article, email and iMessage I send - is stored in Cupertino, California and, perhaps, being pored over by some piece of NSA analytical software.

Every one of the multiple connections we make daily to the wired world contributes to the deluge of Big Data, a vast sea of information growing deeper and broader by the second and which, if managed and analysed correctly, is priceless to governments and corporations alike.

IBM is one of many companies taking part in the digital gold rush which is the development of analytics platforms for the management of Big Data. "Every day," enthuses the company, "we create 2.5 quintillion bytes of data - so much that 90 per cent of the data in the world today has been created in the last two years alone."

Until recently, "there was no practical way to harvest this opportunity". Now, however, IBM invites its customers to "turn 12 terabytes of Tweets created each day into improved product sentiment analysis", to "exploit the 80 per cent data growth in images, video and documents to improve customer satisfaction", or to "monitor 100s of live video feeds from surveillance cameras to target points of interest".

Our confusion over the morality and ethics of Big Data means that while we see terms such as "sentiment analysis" and "social reach metrics" as representative of relatively harmless attempts to sell us soap powder, we perceive as invasions of our privacy identical analytical tactics in the hands of governments attempting to save us from having our legs blown off by terrorists.

Take the Guardian newspaper, which revealed Edward Snowden's disclosures about the NSA. In 2011, during the UK's summer of disorder, the paper "compiled a unique database of more than 2.5m tweets related to the riots".

Isn't that exactly the sort of thing for which the NSA is now being castigated?

As James Clapper, the US director of national intelligence, said last week, the "targeted counterterrorism programme" in the spotlight does not allow the government to listen in on anyone's calls and does not acquire the content of any communications or the identity of any subscriber". Indeed, "the only type of information acquired … is telephony metadata, such as telephone numbers dialled and length of calls".

This, of course, is how they tracked down Osama Bin Laden, but civil-rights organisations say such outcomes do not justify the widespread application of the technique.

"What they are trying to say is that disclosure of metadata - the details about phone calls, without the actual voice - isn't a big deal, not something for Americans to get upset about if the government knows," said the Electronic Frontier Foundation. But, argues the EFF, look what metadata can disclose: "They know you called the suicide-prevention hotline from the Golden Gate Bridge, but the topic of the call remains a secret. They know you received a call from the local NRA office while it was having a campaign against gun legislation, and then called your senators and congressional representatives immediately after. But the content of those calls remains safe from government intrusion."

And so on. But it takes an extremely high dose of delusional paranoia even to begin to think that anyone in authority cares, in a sinister sense, that you oppose or support gun laws, or have considered suicide.

In reality, says Chris James, a science-fiction author whose 2010 book Class Action imagines a society riven by a technology that allows authorities to examine anyone's memories and thoughts, such concerns are "more important to public figures, who may be compromised, than to ordinary citizens. Millions of people already share their videos and photos publicly, and tweet about the most banal aspects of their lives in which very, very few people could be interested. Can these same people then complain that their privacy is invaded if the state monitors their phone calls and emails?"

There is an easy solution for anyone who really wants to avoid the prying eyes of governments and corporations: cut up your credit cards, throw away your iPad and your smartphone and cease broadcasting the minutiae of your doubtless fascinating life via Twitter and Facebook. Live your life unplugged and cease collaborating in your own surveillance.

Not that life in a 19th-century cave would necessarily be without its own problems.

"Perhaps we need to resort to pigeon post," wrote one concerned Guardian reader this week, in an entertaining online debate shot through with the readership's hallmark blend of paranoia and humour. But, replied another, could you trust them? "Could be some stool pigeons."