Hope vs. Motivation: Why Big Data needs empathy and emotion

Because, - says Om Malik, one of the most extraordinary thinkers on Silicon Valley and the  founder of GIGAOM -

The problem with data is that the way it is used today, it lacks empathy and emotion. Data is used like a blunt instrument, a scythe trying to cut and tailor a cashmere sweater...

He concludes

The idea of combining data, emotion and empathy as part of a narrative is something every company — old, new, young and mature — has to internalize. If they don’t, they will find themselves on the wrong side of history.

The first reaction most people trained to follow procedures, is disappointment. Derrick Harris writes;

"Some people say big data is wallowing in the trough of disillusionment, but that’s a limited worldview

(but) there are small pockets of technologists who are letting their imaginations lead the way. In a suddenly cliché way of saying it, they’re aiming for 10x improvement rather than 10 percent improvement. They can do that because they now have a base set of analytic technologies and techniques"

Usually the big complains in social enterprise are for loss of privacy. But this is the surface not the reason

It is the unintelligent way of using data, with no empathy and no deep insights. Absolutely no one will get any business from me by filling my  mailing box (the snail mail box)  with junk offers. And like me, there are many people sick of telemarketers phone calls every evening at 8:00 p.m.. Or broadcast prerecorded message reminders from schools and doctors.

There is no difference in scientific data. The amount of data collected at the ATLAS detector from the  Large Hadron Collider (LHC) in CERN, Geneva  is described like this:

If all the data from ATLAS would be recorded, this would fill 100,000 CDs per second. This would create a stack of CDs 450 feet high every second, which would reach to the moon and back twice each year. The data rate is also equivalent to 50 billion telephone calls at the same time. ATLAS actually only records a fraction of the data (those that may show signs of new physics) and that rate is equivalent to 27 CDs per minute.

Sure, this is a technological advance to find the Higgs particle inside this massive data. Somebody must have a lot a Hope and Motivation

Figure 2: The shape of the curve Hope versus Motivation can  predicts the success

Higgs particle discovery needs to correlate to the humans feelings behind. the endeavor. I call them Hope and Motivation. Compare Higgs curve to medieval failed philosopher's stone

Paraphrasing  Malik Om insight. "The idea of combining data, emotion and empathy as part of a narrative is something every science research team has to internalize."