You can hardly open a business or a technology publication these days without finding an article about the promise of Big Data. After all, the idea of having unlimited customer insights, in real time, guiding every decision, sounds quite attractive.
New technologies (Hadoop, Mongo, NoSQL, and others) are offering businesses the opportunity to analyze very large amounts of unstructured data at a scale and speed that was simply not possible just a few years ago.
And the predictions are quite bold: "enterprises adopting this technology to outperform competitors by 20% in every available financial metric," according to Gartner.
True, data is a key resource for any marketer, or for any businessperson. The success stories of businesses using data to improve the business are well documented. Also true is that the promise of having infinite insights is quite attractive. However, you will hit bumps on the journey to the Promised Land. Today, Big Data is more a piece of the technology than a complete solution with a clear path to business value.
Before you embark on a Big Data project, you have to separate the buzz from the reality. I can think of five challenges that limit the ability of marketers to turn the Big Data hype into insight, business value, and ultimately, increased revenue.
1. Data Sources
Most companies struggle to get "good" data, "OK" data, and sometimes even "some" data about their customers. Without an ability to extract data from your systems, you won't have much data to analyze. I have been at organizations where customer information was scattered across 80 databases. The dynamic nature of business today, with mergers and acquisitions, only makes the problem worse. Data integration remains a technology problem that is not solved by Big Data.
2. Deriving Insights
Just because you have all the data in the world does not mean you can find diamonds in it. Without the right ideas, any data analysis can prove right the wrong assumption. Remember the words of the U2 song: "We thought that we had the answers, it was the questions we had wrong." To generate insights from large amounts of data you must have the right kind of questions. If you think of Big Data as a customer crystal ball, what would you ask it?
3. Acting on Insights
There is a Big Gap between insight and action based on that insight. Most companies don't take time to read (or act) on the customer feedback that is readily available. Many marketers don't look at basic Web analytics to act on the insights that have been at their fingertips for years. Is your organization willing and able to make changes to act based on the insights you could generate from a Big Data initiative? Do you have a plan?
4. Tools
We are in the early days. Big Data requires new tools to query and report on the data. The bright future for Big Data probably will require new data integration tools, new querying tools, new reporting tools, and new dashboards. This issue will likely be solved soon because of the investment being made by software companies in this space.
5. The Trap of Data
Customers are human; we make emotional decisions. Being too focused on data is a risk. If Steve Jobs acted on data alone, he would have never built an iPhone. Will you give up your instincts?
As the Harvard Business Review notes, "Human behavior is nuanced and complex, and no matter how robust it is, data can provide only part of the story. Desire and motivation are influenced by psychological, social, and cultural factors that require context and conversation in order to decode. Data...reveals what people do, but not why they do it."
So Now What...
So should businesses ignore Big Data? A Big No would be my answer. The technology available today brings exciting possibilities to businesses. It would be a big mistake to not try to take advantage of it.
What I am recommending is two things: Don't get blinded by all the buzz of the Big Data shiny object; and, as with any initiative, if you go at it with an awareness of the challenges ahead, you are more likely to find success. Maybe Big Success. Who knows?
If you are looking for a place to start, consider any large unstructured data source—a data set that does not require integration because the data is already in one location: transactions, Web logs, multiyear sales data, etc. Look for a resource that has experience not only with the technology but also with the business process of extracting insights from Big Data and can help build an action plan around them.
However, if you are looking at big data as a way to get customer insights, there may be an easier path that does not require technology investments, consultants, or project management: If you want to learn about your customers, pick up the phone and talk to them. No amount of computer analysis will be able to express in a spreadsheet or a report the emotions, problems and dreams in the mind of a customer that you can get in a personal conversation.
Looking at Big Data is a scientific way to get customer insights, sitting at the other side of the spectrum of becoming a true social business. No reason you can't do both.
