>>>BAE Systems Detica head of data services, Stephen Shelton, will be discussing Big Data in more depth at the ad:tech London conference taking place on 19-20 September 2012.
A common trend since the emergence of big data has been to draw comparison with the character from Orwell's 1984 whose name now graces a brain-mushingly poor TV show.
The 'Big Data = Big Brother' equation is an easy one to make, because the word 'big' is in both terms -- but there is some validity in the scaremongers' analogy.Big Data privacy concerns?
With cheaper storage, the rise of social media and the all-encompassing nature of cloud computing, data is being collected by the bucket full. This data from disparate sources does not just sit gathering dust either, at least not any more. No, instead the companies and agencies who are leading the way with big data are using all this data which we, the average punter on the street, generate, to create accurate profiles of us, our interests and our life habits. Enter the Big Brother comparisons.
If we take the average Joe or Josephine Bloggs on the street, they will most likely have multiple social media accounts -- Facebook, Twitter, LinkedIn or Bebo and MySpace for those lagging behind the cool kids.
Then you have your email, YouTube accounts, perhaps a Google+ profile, online file sharing, subscriptions to all sorts of websites from which we receive endless streams of newsletters and email alerts, and you may bank online. And that's just when you're on your laptop.
There is of course information about you held by the bank, the local and national governments, not to mention all the data collected by the cash machines you use, the goods you buy with the use of a shop's loyalty card, the petrol stations you pay through the teeth at and the CCTV cameras that can use face recognition technology to track your movements.
This result is customer profiling. With so much data available on those of us who engulf ourselves in the digital world, albeit not all made available for any secondary use, companies can map very detailed dossiers on individuals and many facets of their life.Tracking with cookies
Moreover, online retailers such as eBay and Amazon are getting yet more intelligent in the way they collect and use data. Leading online retailers are able to map a user's exact journey through their site, including which links you click through, how long you hover over certain images, which adverts interest you and what things you search for. In much the same way, by using IP addresses and cookies, Google creates customised profiles for the searcher and is able to cater its results to the individual and their interests. It is worth noting that such profiles are not created on a person, more on an online identity.
It is easy to see, therefore, how in a data driven world our sense of personal identity becomes subordinate to a database. Long gone are the Mad Men days when alcoholic chauvinists ruled the marketing and retail world. Nowadays we are walking data sets, rows in spreadsheets, that companies use to determine how to best target us.
In essence big data is the removal of the human side of business.
Take Proctor & Gamble, for example. The manufacturer of consumer goods uses big data in what it calls 'Business Spheres' of which it has 50 around the world. These Business Spheres, the company's CIO Filippo Passerini says, "helps us make fast, [informed] decisions by combining expert analysis of real-time information and data visualization within an immersive environment. This patent-pending system is being deployed in various formats throughout P&G's network of sites. The system utilises a set of business intelligence capabilities that integrate complex, real-time global data, analytic models, advanced visualisation and IT-analyst facilitation".
The thought of a huge global business monitoring the proletariat from a room with wall-to-wall monitors, something akin to the Architect in The Matrix, does bring up some similarities with Orwell's Thought Police.
This is not just a problem for the consumer either; the same problem exists within the business too. Toby Moore, CTO of Mind Candy, the company behind the children's craze Moshi Monsters, was keen to stress that it is important to keep an element of gut instinct in business decisions. You can't let everything be determined purely by data; the human must control the data, not the other way round.Sharing public data
Initiatives to share more data in the public domain, encouraged by the government to benefit both the public and private sector, may well exacerbate this problem too. The sinister undertones may not be present but there is certainly an element of 1984 about the emergence of big data and the way companies in a range of sectors are using it to monitor us, the consumers, and modify the information we receive accordingly.
Another danger of this is that objectivity is lost. Google collates information on a user's browsing habits -- discovering their interests and surfing patterns, and this determines the results we receive when we use its search engine. The fear here is that we will begin to see what we want to see (Eli Pariser's so-called "Filter Bubble") and not receive an accurate view of the world around us.
But besides feeling as though the large conglomerates have you mapped down to the finer details, Big Data's Big Brother similarity has a lot of benefits. Retailers sending you customised offers on what interests you should be seen as a positive development. Furthermore, more reliable systems for deciding which applicants should be considered as eligible for loans is also a healthy advancement, especially at a time when opaque credit ratings can land everyone is a spot of bother.Bobbies on the Tweet
Big Data is already being cited as making vast improvements in the security and policing. Take the example of the New York Police Department (NYPD). The Big Apple's boys in blue use CCTV with face recognition technology to lower crime in the city. When a crime is committed they can identify the culprit and then by combining this with archived data about the individual as well as things like their transport habits -- which subway stations they travel from and to -- they can ensure they are on site to make the arrest.
In much the same way, police in the UK are using Big Data products from i2 Intelligence Analysis (now owned by IBM) for better, faster decision making. The capabilities of the tool include social network scanning, visualisation tools, multi-source simultaneous search, scalable computing, real time reporting and the functionality to sort unstructured data.
This lets the police force make the most of the mass of information that they collect to discover and disseminate actionable intelligence that can help identify, investigate, predict, prevent, and disrupt criminal, terrorist, and fraudulent activities.Doctors' social rounds
Healthcare is another sector which is reaping the benefits of big data. Health 2.0 -- a community based organisation which runs conferences and support services for users of health technology -- organised a recent US 'code-a-thon', focussed on the health industry. The winning team chose to focus their efforts on demonstrating how medical conditions caused poor sleeping patterns as well as things like how obesity can cause road accidents. The team created a website -- nosleepkills.org -- which allows users to enter personal information and details of their sleep patterns to determine whether they are currently achieving a healthy level of sleep.
Big Data is set to play a major role in the Cancer Knowledge Action Network -- a centralised system to allow doctors to use mobile devices to help people with specific cancer conditions.
These are just a few cases where advancing tools and technologies, combined with massive data sets about individuals, are being used as a force of good.
It may be easy to feel as though there are monitors and sensors tracking your every move or every mouse-click but in reality it would seem that the benefits promise to far outweigh the negatives.
The critics may be quick to jump on the 'Big Brother' bandwagon with a perhaps overly simplistic view that freedom of choice and personal identity is being replaced by binary code, but in reality Big Data can revolutionise the world around us for the better. Improvements in healthcare, national and local security and even a more personalised browsing experience are just the tip of the iceberg as the faster, cheaper and easier analysis of massive and complex data sets leads us to greener pastures.Dominic Pollard is editor at Big Data Insight Group