At a recent neighborhood function, I had a discussion with someone who worked in the finance area for one of the Big 4 accounting firms. Since he specialized in the valuation of companies, I asked him about the models he used and he brushed me aside to tell me it was all about the assumptions, not the models per se.

I couldn't agree more with his statement. I have often termed the rise of measuring everything to be the deification of numbers. It's not that I disagree with the well-known axiom that you can't manage what you don't measure—it's just that I have to ask how, and why, you're measuring it.

That premise is the basis of this article. Although putting each of your metrics through the paces may not seem exciting, it is those measures that are the basis for any business case you construct or objective you set.

Each metric or measure—marketing or otherwise—should be vetted in light of its reliability, validity, sensitivity, responsiveness, cost/benefit, comprehension, and balance.

Reliability

In research, reliability means repeatability or consistency. A measure is considered reliable if it would give you the same results over and over again, given that what you are measuring is not changing.

If you weigh yourself on a scale that is always 10 pounds off (and your weight is stable), you would have a reliable measure, it's just not valid for the intended purpose. But I'm getting ahead of myself. Each measure in your company should be tested to determine if it's measured in a reliable and tenable fashion.

In other words, are people measuring the area using the same methods, processes, decision models, assumptions, and so on?

If the way you measure something is not clearly outlined, leaving room for little interpretation, it will have more “noise” or error and is less likely to be considered reliable. The fundamental basis for standardized processes is the push for reliability, which is a necessary but insufficient condition in determining whether a measure is valid.

For example, in sales, pipeline analysis (e.g., I'm in the Qualify or Develop stage) is meaningless without a clearly defined sales process and stage gate criteria.

Test: What evidence is there that the measure is reliable and is being measured in a consistent, repeatable, and defendable manner?

Validity

Although validity is often thought to mean accurate or reasonable, it must be put in context to be meaningful. A measure is valid for a particular purpose, and it must be reliable in order to be valid. Something may be valid for determining market share, but not for the valuation of the business. Within research, there are a number of ways to test the validity of a particular measure.

Two tests that are pertinent to business would include content validity, which looks at whether other content or subject matter (finance or marketing personnel) experts would agree on the use of the measure for its intended purpose.

The other, predictive validity, a form of criterion validity, looks at whether the measure is predictive of the construct it's intended to provide guidance about. The latter test would be more applicable to a key performance indicator (KPI) as predictive of another macro measure, such as revenue.

Test: Assuming the measure is reliable, what evidence is there that it is measuring what it's intended to measure?

Sensitivity

This criterion focuses on the sensitivity of the measure to changes in the underlying construct. For example, how much does the underlying construct (like campaign costs) need to change (1%, 10%, and so on) before it registers or is detected?

Test: How sensitive is the measure to changes in the underlying construct?

Responsiveness

This criterion focuses on whether the value of the measure changes immediately when the underlying construct changes. Today's lingo would link this criterion to the real-time (or zero-latency) enterprise (RTE) concept.

Many executives would like to be able to pull up a dashboard of metrics on their business and have it be updated in real-time, with each metric being highly responsive to actual changes in their operating reality. How long does it take for your “check engine” light to come on?

Subscribe today...it's free!

MarketingProfs provides thousands of marketing resources, entirely free!

Simply subscribe to our newsletter and get instant access to how-to articles, guides, webinars and more for nada, nothing, zip, zilch, on the house...delivered right to your inbox! MarketingProfs is the largest marketing community in the world, and we are here to help you be a better marketer.

Already a member? Sign in now.

Sign in with your preferred account, below.

Did you like this article?
Know someone who would enjoy it too? Share with your friends, free of charge, no sign up required! Simply share this link, and they will get instant access…
  • Copy Link

  • Email

  • Twitter

  • Facebook

  • Pinterest

  • Linkedin


ABOUT THE AUTHOR

image of Michael Perla

Michael Perla is Sr. director & life sciences people leader, Business Value Services, at Salesforce.

LinkedIn: Michael Perla