In an increasingly cost-focused business environment, marketing is coming under more and more pressure to justify spending and achieve ROI hurdle rates laid down by management.
ROI has been a sensitive issue in marketing because of its inherent intangibility: For most marketing vehicles, it is difficult to directly estimate the proportion of revenues in the financial year that are the result of marketing.
Direct mail, on the other hand, seemingly makes it very easy to measure ROI. With direct mail, you have an upfront and direct measure of sales that came as a result of the campaign, since each application is tracked from mailing to response to closing.
An ANA-sponsored study conducted by WPP's Lightspeed Research in 2003 cited direct mail as the best vehicle for measuring ROI, and pointed to media advertising as the worst vehicle for measuring ROI. But contemporary methods of measuring direct mail ROI may not always be as accurate as one might like to believe.
Direct mail accounts for up to 20% of total advertising spend, and it might be well worth the effort to compare standard direct mail ROI measurement to methods used in measuring ROI for other vehicles.
How do DM campaigns typically measure ROI?
Typical DM campaigns use control groups to evaluate performance of campaigns. A control group is a subset of the total population that is to receive mail in a campaign. This control group is set aside and not sent any mail as part of that campaign, which gives marketers a measure of what sales would have been in the absence of direct mail.
The population that receives mail is called the test group. Revenue per customer is calculated for both groups over a time period that includes the promotion period and a sufficient amount of time afterward to rule out purchase acceleration impact (revenue per customer = Total Revenues in group / Number of people in the group). The higher the number of responders in a group (control or test) the greater the revenue per customer.
Incremental revenue per customer is calculated as the difference in revenue per customer for the test group and revenue per customer for the control group. Incremental revenue per customer is a measure of revenue that is above and beyond revenue that could be expected in the absence of the direct mail campaign.
Dividing this number by the campaign cost per customer (calculated as campaign cost / number of people mailed) yields the ROI of the campaign.
How is ROI typically measured for other vehicles?
Mass media ROI, on the other hand, is usually calculated using econometric techniques like marketing-mix modeling, because it is virtually impossible to directly measure what proportion of total sales is due to a particular mass media channel (except in the case of direct response advertising).
These models de-compose total sales into sales due to each marketing activity, while controlling for environmental, seasonal, and competitive effects. You can calculate ROI for each marketing activity by dividing revenues (or profits) from each marketing activity by the cost of that activity.
So can these two approaches yield different results (and why)?
If direct mail ROI is measured using mass media techniques like marketing-mix models, you could end up getting different results, and depending on circumstances either one of them could be closer to the correct ROI.
There could be two potential issues with measuring direct mail ROI using control groups. First of all, you could have a very small control group in comparison with the population being mailed, which could affect the robustness of the results. This is especially true if the mailed population is geographically or demographically diverse, as this would increase the probability that the control group is not truly representative of the mailed population.
The control group might also be exposed to different environmental and marketing stimuli (apart from direct mail) than the mailed population. This might cause differences in revenue per customer in control group to be different from the test group for reasons other than direct mail... but erroneously attributed to direct mail.
In marketing-mix models, this problem doesn't occur because there is no control group per se, and all substantial variables that impact sales are typically included in the model.
Direct mail ROI measured from marketing-mix model can be erroneous if significant variables that drive sales are excluded from the model. But the drawback of this approach is that it needs a lot of effort in terms of time, data collection, and modeling (conversely, it has the benefit of measuring ROI for all vehicles in one go).
In conclusion, both approaches can yield similar results if the limitations of each is understood and factored into the final estimate. If the mail population is relatively homogenous, it may not be necessary to use marketing-mix models to measure direct mail ROI, but for a diverse target population it may be better to use marketing-mix models to estimate ROI.
