Big Data is reshaping the marketing profession as it is increasingly used to mediate the consumer-brand relationship. But does the technology-driven approach to data analytics mean that progress is stalled?

We need to look to the human side of data for Big Data investments to be realized.

At times, the debate in marketing about the use of data seems very polarized. On the one hand are fierce critics such as advertising industry luminary John Hegarty, co-founder of advertising agency Bartle Bogle Hegarty. He argues strongly about the dangers of being "slave to data"  and how "data in itself creates nothing. It's creativity which makes things."

On the other side of the argument are a wide range of marketers, technologists, and consultants promoting the value of data as a means of driving impressive levels of growth. McKinsey, for example, has estimated that a retailer using Big Data can potentially increase its margin by more than 60%.

My belief is that it is too easy to take a polarized position. The answer is neither purely about data driving the agenda nor simply about ignoring this amazing resource. So, how do you walk a middle path through this?

Overwhelmed by So Much Data

Brands are just not sure how to manage the vast expanse of data they now have available to them.

A recent survey with business decision makers found that only 27% considered their company able to turn their customer data into useful information and act on it.

My view is that marketers have been placing too much faith on the "data" side of the equation with huge amounts of expectation being placed on data scientists to navigate a way through the challenge. This profession reflects the skill set of Silicon Valley, namely the "numeric" disciplines such as statistics, computer science, applied mathematics, and economics.

These skills are necessary but not sufficient to get value from data.

Data Scientists and Social Scientists

Behind the data points, for marketers at least, sit people. And where people are involved, we move from linear systems that align to identifiable "laws" to complex system non-linear in nature.

We need social science to guide us through Big Data, to provide the analytical frameworks that help us to explain the patterns in the data and to create hypotheses to test.

The reliance on data scientists at the expense of social scientists might explain why we are coming to a point at which the Big Data bandwagon may be seen to be faltering.

Some authors suggest that perhaps we are now in the "trough of disillusionment" of Big Data, the hype around it having surpassed the reality of what it can deliver. Gartner suggested that the "gravitational pull of Big Data is now so strong that even people who haven't a clue as to what it's all about report that they're running Big Data projects."

I set out in my book Humanizing Big Data why brands need to adopt hybrid approaches in a fresh approach to Big Data. This is where we take the positives from different disciplines and meld them together.

A Three-Pronged Approach

I believe there are three key parts to the puzzle:

Social science

We need new ways to navigate the data available to us and that help us to test new hypotheses to create differentiation, which is hard for others to replicate.

Brands need to get more comfortable with using theoretical frameworks to help explain consumer behavior.

The increasing popularity of behavioral economics and related disciplines suggests that an increasing willingness to engage with new ways of thinking. The challenge is to find a new breed of consumer researchers and marketers that can translate these explanatory frameworks into a language that can be understood in a business environment and more importantly, deploy it in a way that improves business performance.

I am certainly not alone in thinking this. The UK's Campaign for Social Science recently suggested that "without a better grasp of people and their motives, technological advances may fail to realize their potential and may be frustrated or blocked."

Consumer research

We can only disentangle the "signal from the noise" by understanding the nuances of consumer behavior. Otherwise, simply using statistical significance rather than "colloquial significance" will lead to all manner of erroneous conclusions. This understanding comes through market research but also through ensuring that the task of understanding consumers is not just the mandate of a centralized team of data scientists.

As economist and philosopher Friedrich Hayek long ago pointed out, "organizations think too highly of centralized knowledge, while ignoring knowledge of the particular circumstances of time and place."

Huge amounts of valuable, nuanced understanding resides throughout the structure of an organization. Better use needs to be made of this.

Data analytics

While it is tempting to think that technology will allow the "data will speak for itself" the reality is that this falls a long way short of what marketers need from Big Data.

We are starting to see a call for a greater understanding of the human side of Big Data analytics. Academics Donald Marchand and Joe Peppard suggested that "IT fumbles analytics" because of this very issue. They believe that alongside the IT professionals, there should be specialists in cognitive and behavioral sciences who understand how people perceive problems, use information, and analyze data in developing solutions, ideas, and knowledge. As they point out, analytical techniques are tools for thinking, it is people who do the actual thinking and learning.

* * *

Investment in big data is huge and set to grow further. IDC forecasts that global spending on business analytics will rise from $51.6 billion in 2014 to $89.6 billion in 2018.

Those huge sums are part of a "Big Data arms race" by organizations looking for new ways to drive growth, profitability and market share. However, without understanding the essentially human nature of Big Data, these investments will surely fail to be fully realized.