Big Data – Letting the Data speak for itself

“Big Data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.” – Dan Ariely

Big Data. The term was added to the Oxford English Dictionary in 2013, and MIT offered its first official course on big data in 2015. Hardly an intellectual conversation on data processing or marketing occurs without the term being dropped (often inappropriately). Big Data is all the rage – everyone wants to use it and everyone wants to talk about it. But what exactly is it and why is it important?

What is it?

Intuition, (or indeed the name) would suggest that big data simply refers to very large, almost unmanageable datasets. Datasets that wouldn’t fit on your average computer. However, the truth is slightly different. Especially in our day and age, big has become a relative term, and it is not so much the size of the data that counts, but the tools being used to analyse it or the insights that can be drawn from it.

Thus big data simply refers to data that requires a new or unique approach. Sometimes this involves significantly large amounts of complicated software processing in order to find correlations, patterns, structure or anomalies amidst otherwise chaotic or complex data. So it doesn’t matter how big your data is, it matters how you use it.

13340636_10208396725608112_194919366_o

Big data encapsulates the 3 V’s, Volume, Variety and Velocity [Doug Laney, 2001]. There’s lots of it, it comes in different and incompatible forms, and it’s updating unbelievably quickly. Big data is a tornado that’s turned the business world on its head and it’s here to stay.

In the last decade, there has been a massive explosion of data all over the world. Everything we know today haemorrhages data in some form or the other. By simply logging onto Facebook and liking a page that pops up on your newsfeed, you are adding to the large dataset that Facebook accumulates on your preferences and activities (see: ‘Ten ways in which Facebook has used your data‘) every second. Businesses all over the world want to use this multitude of data to solve problems, learn something about their customers or produce a product, etc. This means that companies are required to set aside time, expertise and creativity, (or hire other companies with time, expertise and creativity) to craft a solution to the problem that leverages the data without simply cutting it down in size.

Who is using Big Data?

 “In God we trust. All others must bring data.” – W. Edwards Deming

Data can be used to help create solutions to all sorts of challenging problems.

1) Lady Gaga’s business manager, Troy Carter, is a big data fan, as reported by The South China Morning Post. Carter created his own little Gaga social network, called Littlemonsters.com, by mining the singer’s 31 million plus fans on Twitter and 51 million plus on Facebook. The aim of this project was to entice as many Gaga fans as possible, thus bypassing facebook and twitter and keeping 100% of future revenues for himself. This man’s a genius.

Lady_Gaga

Image credit: Adele Wiki.

2) Amazon has an unrivalled storehouse of data  on online consumer purchasing behaviour that it can mine from its 200 million customer accounts. Amazon uses that data to build recommender systems that suggest items for you based on what they infer from your previous shopping purchases. They also generate coupons at the point of sale based on your buying habits.

Pro Tip: While searching for that crucial dress online, click the checkout button but don’t complete the checkout process. Most online stores will send you coupons codes/give you better deals to encourage sales.

According to a new report from EKN Research, 80% of e-commerce giants say that they lag behind Amazon in analytics maturity.

3) Sports statistics are used to analyse performance by everyone from fans and betting folk to coaches and managers. Previously a domain for geeks, sports statistics have become increasingly manageable due to big data techniques, with the data now calling the shots.  As these examples from the recent MIT Sloan Sports Analytics Conference demonstrate, analytics are changing the face of pro sports.

4) Detecting fraudulent behaviour before it affects the organisation (VISA, healthcare). Visa says using Big Data they identify billions of dollars in fraud each year. Read the full article here.

Big data is changing the face of companies by allowing them to analyse detailed data that models reality better than anything we have had before, allowing institutions to stay one step ahead of the game at all times. No longer is the expert knowledge of professionals needed to provide insights, it’s time to let the actual data generated speak for itself.

Big Data is still news. It’s still relevant. For most organisations, big data is the reality of doing business.

 

Writing an essay or want to learn more?
1. Read the original 2001 paper by Doug Laney on the 3Vs of big data here.

2. For an overview of the techniques used for big data, click here.

3 Comments Add yours

  1. Atul Garg says:

    Dear Arsha

    The article on big data is very well written, in particular I enjoyed reading the real life applications of big data. This is a fascinating area and most senior executives will be very interested in learning about how they can use big data to improve performance

    Best
    Atul

Leave a Reply

Your email address will not be published.