If you were running for president and a Gallup poll directly asking voters their voting intentions said you would win yet a prominent data guru's predictive model said you'd lose—would you be confident on election night?
Mitt Romney faced this situation in 2012, and he believed the favorable Gallup poll results. After all, the results came right out of voters' mouths. But as we know, the poll was wrong, and Obama won the night.
Interestingly, Romney is the former CEO of Bain & Company, the elite consulting firm that, along with Satmetrix and Fred Reichheld, co-created a simple survey and metric widely used in business to gauge customer satisfaction: Net Promoter Score (NPS). Like the Gallup poll, NPS delivers a simple, digestible metric, which gives senior corporate management the same misleading type of measure the Gallup poll gave Romney.
Let's dive into the debate surrounding NPS and examine the existing and emerging tools aiming to disrupt its dominant position.
The Popularity of NPS
Why is NPS so widely adopted?
- NPS is simple and based on the answer to a single, clear question: How likely are you to recommend this product to a friend? A respondent noting 0-6 is considered a detractor, 7-8 a passive, and 9-10 a promoter. NPS is the percent of total promoters minus the percent of total detractors.
- Likelihood to recommend a friend may not be precise, but it is at least a general gauge of customer sentiment.
- An elite consultancy, Bain introduced NPS through an elite publication, Harvard Business Review, then a few brand-name early adopters jumped on the bandwagon.
What NPS Gets Wrong
NPS is simple and popular... but NPS has several inherent flaws that make it an inconsistent, unreliable indicator of the overall state of customer sentiment and satisfaction. Flaws include...
- It's not representative. CMO found NPS only has a 10-15% response rate, with more than 85% of total customers not represented.
- Detractors are miscategorized. "Detractors" are people who are only unlikely to recommend a product; however, there is no indication they would discourage it. Consider a different metric, Word of Mouth Index (WoMI), which only considers someone to be a detractor if, when asked, he or she specifically indicates that he or she would discourage use of the product.
- Bias abounds. People tend to respond to satisfaction surveys when they are extremely unhappy or extremely happy, resulting in an over-representation of promoters and detractors.
- It's easily gamed. Environmental factors can easily influence responses, such as when and where the question is asked (e.g., the answers given immediately after the purchase could differ from the answers that would be given months later).
- It varies widely by industry. Even the creators of NPS acknowledge there's little correlation to revenue growth in some industries.
The American Marketing Association (AMA) published a longitudinal study to corroborate claims made by NPS' creators, specifically that NPS is "the single best predictor of growth" and, "the only number you need to grow." Further, the creators denigrated the rival American Customer Satisfaction Index (ACSI) metric as having zero correlation with annual revenue growth. Bain brazenly claimed, "There is none."
The AMA found both NPS and ACSI models independently showed clear correlation with revenue growth, with ACSI showing a higher correlation in two of them. Not only did this completely undermine the creators' assertion, it indicated that NPS may be worse than what they considered a useless metric. With that sort of discredited hyperbole, promotion of NPS often resembles a slickly packaged marketing effort than a mission-critical enterprise tool.
Alternatives
Those issues aren't only an indictment of NPS; customer surveys in general have limitations that have frustrated companies for years, due to their focus on the numeric results and limited feedback that don't tell a deeper story.
Customer experience expert Jeanne Bliss notes that what's missing is the second and most important part of the equation.
She explains, "Companies have lost sight of the reason for improving their customers' experience—it's to 'earn the right' to their continued relationship and over time, their recommendation.
"Growth of the value of a customer base is the true test of business success. Unfortunately earning great survey scores have become the motivation instead of 'earning the right to business.' NPS, too, has fallen into this unfortunate trap in some companies.
"The value of NPS is to understand and rescue customers at risk and to be rigorous about improvements. But like other survey metrics, the motivation is the score and not changing the operation to deliver improved experiences to the customer."
Fortunately, companies can get the necessary tools to analyze feedback to improve customer experience and even predict future loyalty based on customers' actions themselves.
Predictive analytics, for example, ingests all streams of customer data that a company holds (usage, demographics, customer support, natural language, social media sentiment, etc.) and looks for patterns that indicate the customer's likelihood to remain a customer. For instance, would you rather know that 3 out of 10 people are detractors, or 3 out of 10 people became detractors after using your FAQ page or logging in more than two weeks after registration?
* * *
Compared with NPS, analyzing more complex data is a simpler solution for numerous reasons:
- Customers need not respond to any survey whatsoever.
- It's more accurate because it tests actions and behaviors, rather than intent.
- It delivers both individual and collective results.
- It can easily be consolidated to a single numeric metric.
- Insights are actionable because data analysis shows which actions, behaviors, or contributing factors create positive or negative states.
Ultimately, by using customer feedback to gain clearer insight and drive change and results throughout your organization, companies can keep their customers happy and even woo back some detractors, overstated or not, that are slipping away.
As a parting thought on Net Promoter Score, I've never seen a NPS on the NPS. If you pooled all companies using this system, what would the resulting score be?
My guess is it would have its fair share of detractors.
