As more and more people use digital mobile devices, the trail of data spinning around the ether is ever-growing. Now companies are tackling this big data and looking at using it to gain an edge.
Big data may give companies the edge over our daily lives
Every time you switch on your phone, make a call, search for a location on your phone's map app or purchase a product online, you are leaving behind valuable bits of data.
All of this information that determines your habits is floating around on servers across the world, part of a new trend called big data. The two words seem simple enough, but few understand its implications.
From social networks, mobile devices and even embedded sensor chips in fridges and cars, we have so far produced about 2.7 zettabytes (the seventh power of 1000 bytes - units of digital information) of data, the equivalent of 36 million HD videos, and that is set to grow 40-45 per cent on an annual basis.
According to research from Ericsson, people will be using 15 times more data by 2017 than they do today. There will be 3 billion smartphone subscriptions, 5 billion mobile broadband subscriptions and 9 billion mobile subscriptions. By 2020 there will be 50 billion connected devices worldwide.
Each of these devices will be generating data.
So far, deciphering all this data has not been an easy task. There are data scientists whose job it is to create the tools to mine through the deposits of information. These tools are now becoming sophisticated enough they not only track your habits but can predict them as well.
"We are experiencing a very interesting era now, where we see that data is coming upon us from various sources. It is no longer just customer data stored in databases within enterprises generated through business systems," says Ahmed Auda, the business unit executive for IBM's software group in the Middle East.
Our smartphones are now containing more and more personal information. From banking transactions to medical records, our daily lives are being condensed on to a single devices.
While we may think it belongs to us, there are others who have access to it and this data is valuable to many companies who are trying to better understand consumer habits.
"Knowing a customer's location, background, the places they like to go to, the times they go to work or lunch will help organisations personalise their services in a better way," says Mr Auda.
More and more of this data is considered unstructured. It is coming from pdf files, emails, audio files and "likes" on Facebook, shares on YouTube and so on. There is a growing demand and an awareness among clients and organisations, both big and small, of the importance of data and being able to benefit from it to generate stronger business insight and ultimately to transform that into a competitive advantage.
Based on your choices and habits, the algorithms in big data tools aim to predict what choices you will make and what you will do.
They have the ability to completely redefine the way business is conducted. Why should a store offer half-price sales in order to attract as many customers when it can reliably offer the discounts to customers it knows will only come in when there is an offer? Other customers, who historically have been willing to pay full price can continue to do so.
But it is not just in the field of retail where big data can play a role.
One insurance company in Florida is using social media during major disasters to determine which customers to address first.
Security First Insurance is using a software from IBM that analyses messages sent via email and social media using text mining tools, analytics and natural language processing to pick up on words that express distress or convey significant damage to a client's property.
When the bird flu epidemic took off a couple of years ago, countries across the world took what precautions they could to prevent the disease from entering their borders. But monitoring the spread of disease and preventing it could become far more accurate in the future.
"You have tweets like, 'I just landed in New Delhi,' the thinly veiled show-off tweets where people are trying to impress us with where they're travelling to, I call it rich person vanity," said Jer Thorp, the co-founder of the Office for Creative Research, while addressing an audience at the EMC conference in Las Vegas recently.
"I know where they're going to because they're tweeting it and I know where they're from because it is in their profile. Maybe if we watch Twitter closely we can build an almost live model of human travel, which can be used if there is an outbreak of an epidemic human disease."
Creating such a tool is simple enough and with tweets being in the public domain, there are no barriers to using and extracting this data.
Imagine how much more accurate such a tool would be if it were based on a user's mobile phone with access to their medical records. From there you could essentially pinpoint the one person who caused the spread of a disease from one location to another.
The use of this data and prediction of consumer behaviour, however, sits uncomfortably with many, particularly privacy advocates, while some suggest it could take away free choice.
Others are less sceptical and maintain it is convenient and consumers will ultimately benefit by getting tailored services, even if it is intrusive.
Whether the majority of consumers are likely to opt into this new world of business-customer relationship remains to be seen and while corporate attitudes are shifting towards big data, consumers will have to think differently about their data as well.
"Data ownership will be the most important discussion. The problem is lots of people are using our data with our permission, but I don't think people have a good idea of what is being done with that data," said Mr Thorp.
"Going forward, we need to give people a personal relationship with their data."
One major problem, according to Mr Thorp, is the complexity of this data and the way it is presented.
"It doesn't look like anything of value so people are willing it give it up, but we need to think about data in a broader way."
Few are aware of it or even bothered about it. How many people actually check the permissions they grant to mobile apps on their smartphones once they have downloaded them? More and more apps are increasingly asking for permission to access more information and data stored on our phones, from our pictures and cameras to contacts lists, files databases and emails.
"There are different segments in society and their acceptance and readiness to give away this information may vary," says Mr Auda.
"Among the younger generation, there is acceptance to share what they do, where they are, what they eat, whether they are in good health or not. But regulation and governance of data is an important component of the overall big-data story and they are slowly being realised."
Mr Thorp points out three main factors in the future of big data: ethics; ownership; and possibility.
"Ethics is one of the things we will be forced to negotiate with over the next couple of years," he said.
"Companies that are able to understand data ownership and can broker fair negotiation over data ownership with consumers will have the advantage."
Finally in terms of data possibility, Mr Thorp says the technology will change public understanding of data.
Until then, it may be wise to remain wary of putting too much personal data in the hands of those with commercial purposes in mind only.