Big Data - undressed and with no makeup

There is no such thing as Big Data. It is the same data we had before which grew much less than we think. The so-called Big Data is the only data we have. There is no such thing as Small Data.

All graphs below are  extracted from a 2013 report by Rexer Analytics

What we think is not what we get

Figure 1: Typical data sets structure is fairly constant over the years

But our perception (not reality) shows this data structure

Figure 2: Compare the imaginary 26% versus the real 8% in more than 100 million records 

I say: Wow!

Where the data comes from?

Figure 3: Sources of Large Datasets

Contrary to our beliefs, social media data, times series via sensors are not the bulk of data. Big Data is about crunching numbers to discover our habits as customers, maybe being supervised by institutions funded with public money, etc.

How Big Data "doctors" describes themselves

That is a good one. There are probably more titles, than substance. All job descriptions below involve practically the same occupational skills. These are marketing names, in essence.

Figure 4: How Scientists working in data call themselves

I love the "Predictive Modeler" title. Sounds like a knighthood.

How data analysis  people work?

They use mainly an open source statistical package called R and which has over two million users. Commercial packages are loosing steam. The new start up data companies will not tell you that, but their modelling is nothing but statistics. Which is great by the way. 

The R usage has exploded. Learn R and create ten startups, if you want.

Figure 5: R usage is exploding

Do they implement recommendations based on big data analysis?

Figure 6: Only 16% over all implement changes following Big Data analysis

Best way to have something applied in real life is to employ a consultant and then use corporate governance to implement it. 

To me, Figure 6 shows we have a long way to see real changes, based on analysis on data which is not as big as we think, it is still focused in customers and how to make money out of people, use R which in essence is the same Statistics since Gauss discovered it's bell.

What is new is the formidable power crunching of clouds and supercomputing. Just google BoscoR.