Statistics have been called “an engine of knowledge” by one risk management expert. And while it’s true that some business managers don’t have a fundamental grasp of statistical concepts, we also know there is opportunity for misuse of mathematics. Is statistics the “new grammar” or are efforts to attach certainty to life’s events doing more harm than good?

In May 2010’s issue of Wired Magazine, author Clive Thompson laments the poor mathematical literacy of his fellow citizens. For example, he cites people laughing at the concept of global warming as they face some of the harsher winters on record, or the extra-vocal debate on vaccines and possible links to autism. Mr. Thompson would tell us that it’s the trend lines that matter, and we too often look at the trees and miss the forest.

The problem, he says, is that “statistics is hard” and an overall understanding of this important discipline is severely lacking. He says, “If you don’t understand statistics, you don’t know what’s going on, and you can’t tell when you’re being lied to.”

Thompson is correct that statistics are difficult for most of us, and that thinking by the numbers takes training and much effort. It’s also true that one must understand statistical concepts, especially when percentages, populations, and probabilities are bandied about in business and technical press. However, broader acceptance of the power of statistics should be tempered with limitations of this mathematical science.

Before accepting any statistic, study or experiment as gospel, the following should be considered (there may be more…):

1. Assumptions: What are the assumptions underpinning the research? As seen from recent debate on CBO numbers for the U.S. health reform package, assumptions matter tremendously.
2. History: How much historical data was used in the study? What was the time scale? As seen from the 2008 financial crisis, the models used by Wall Street mavens often only took into account 10 years of data in judging the volatility and probability of failure of complex financial instruments.
3. Samples: Are the samples selected randomly? From what populations? Is there enough data for statistical significance?
4. Data Quality: The output is only going to be as good as the quality of data feeding the analysis. Garbage in, garbage out.
5. Survivorship Bias: Author Nassim Taleb points out that “losers are often not in the sample.” Does the analysis include a population of survivors and those who also failed?
6. Falsification and Omission: Yes, in an era of IPCC’s Climate Gate, one needs to ascertain if data are hidden, missing or outliers ignored.
7. Association equals causation fallacy: Correlation does not equate to causation (a common mistake made by marketing and finance executives alike).
8. Proper Application of Statistics: The effective use of statistics by insurance actuaries, scientists, and even casino managers is well-documented. However, real danger results when mathematical concepts are used to denote certainty indecision-making and divining behavior of markets.

Now, please don’t get me wrong. Statistical analysis is very important for many industries (e.g., health care, transportation, and manufacturing). Statistics, however, can give us an illusion of control in a world that’s much more complex than our models suggest. Nassim Taleb, author of the Black Swan likes to remind us that “(real) life isn’t a casino.”

Statistical analysis is definitely a powerful gadget in the business manager’s decision-making toolkit. But one needs to understand the limitations of this science.

After all, Taleb points out that many of today’s statistical models work as though we have “full knowledge of the probability of future outcomes.” And this just isn’t so, especially when it comes to fat tails, or the “ten sigma” event. Indeed, sometimes those rare events have extremely large impacts. Were he alive today, the former captain of the Titanic, E.J. Smith would wholeheartedly agree.

Questions:
• Clive Thompson calls statistics "the language of data." How important is it for marketers to understand and apply statistical concepts?
• "Lies, damned lies, and statistics" is a phrase popularized by Mark Twain in the context of using statistics to unduly persuade, obfuscate or even swindle. Can statistics get its reputation back? If so, how?