BIG Data

You are a regular online shopper and every time you see an offer, you are tempted to click the ad. So you switched on your machine, logged into your Facebook account and while browsing through the latest feeds, you observe something. There is a consistency in the advertisements you have been shown. Most of the ads you see are shopping offers. And it doesn't happen to you alone. Neither it happens only on Facebook. All the 565 million internet users (as per 2012 data) are contributing every second to unstructured information through email, videos, tweets, comments, posts, call-center conversations, mobile chats etc. In fact, according to a research by Autonomy, an HP company, 90% of the data in the world was created in last two years, which means 2.2 million terabytes of data is created everyday. Such magnanimous data sets of structured and unstructured data that are so large that it's difficult to process them using traditional database and software techniques are collectively termed as Big Data.

Five years ago, most companies collected data that were a part of their daily transactions and stored them in a database. This data was used primarily to keep track of operations or forecast needs. Today, both the sources and volume of data collected have exploded. It is now possible to collect click-stream data about every potential customer interaction with your web site. Marketers can also collect information about every conversation people are having about their brand. In other words, big data can be used to perform sophisticated analytics, uncover hidden patterns, unknown correlations and other useful information, thereby, enabling organizations to make better business decisions from the data that may be left untapped by conventional business intelligenceprograms. MGI and McKinsey's Business Technology Office suggests that big data will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus.  The manner in which different organizations across different sectors are using big data is beyond imagination.

• Research (CERN)

According to CTO at CERN, one year’s worth of Large Hadron Collider (LHC) data is stored on 83,000 physical disks. And it has 150 million sensors delivering data 40 million times per second. One gigabyte of data is sent per second, and that's after throwing away 99 per cent of what is recorded by the LHC detectors. The data scientists at CERN synthesize and put structure to this huge amount of unstructured data as soon as it is out to extract some value from it. To quote CERN CTO, "It’s like looking for gold; most of what you dig up is just dirt, but you have to filter through it quickly. Maybe it’s the same in some of your businesses. You have a lot of data but it is only the glimmer that you are after." This is a sensitive task because if even a single portion of valuable data is dropped, it will be dropped forever.

• Automobile (Ford)

Ford actually opened a lab in Silicon Valley to improve its cars with big data. In order to progress their cars regarding fuel consumption, safety, quality and emissions, Ford gathers data from over four million cars with in-car sensors and remote application management software. All data is analysed in real-time giving engineers valuable information to notice and solve issues in real-time, know how the car responds in different road and weather conditions and any other forces that could affect the car.Ford is also installing numerous sensors in their cars to monitor behaviour. They install over 74 sensors in cars including sonar, cameras, radar, accelerometers, temperature sensors and rain sensors. As a result, it Energi line of plug-in hybrid cars generate over 25 gigabytes of data every hour. This data is returned back to the factory for real-time analysis and returned to the driver via a mobile app. The cars in its testing facility even generate up to 250 gigabytes of data per hour from smart cameras and sensors.
Big data is also used to find out how people wanted their cars to be improved. Nowadays, Ford listens carefully to what their customers are saying online, on social networks or in the blogosphere, to find out whether for example the Ford Escape sport-utility vehicle should receive a standard liftgate or a powerliftgate. In addition, Ford performs sentiment analysis on all sort of content online and uses Google Trends to predict future sales.

• Financial Services (DemystData)

DemystData, uses big data to link financial institutions to individuals typically excluded from financial systems. The Hong Kong-based company mines social media and other online sources to target the "unbanked", who might otherwise steer clear of financial markets. DemystData then uses this information to help financial institutions improve service delivery to underserved customers.

Such applications of Big Data show how this is becoming a foundational technology across multiple industries.