In my last article, I defined multivariate testing and how it can optimize your Web marketing, as well as five common errors to avoid. Let's now look at what to measure in your tests and how to define your criteria for success.
Before you start formulating a test hypothesis, or begin running your tests, the first and most important step is to ensure that you have clearly defined objectives for your Web site. You'll want to examine your marketing goals in order to determine the appropriate success factors that all of your organization's stakeholders can agree upon.
Let's start with typical measureable Web site goals:
- Make money: sell product, generate leads and advertising or promotional click-throughs.
- Save money: enable users to adopt self-service features and answer product and service questions on their own (such as through online FAQs and documentation).
- Create brand awareness and industry visibility.
It's important for all stakeholders to agree on the goals of your Web site, because when a decision is made to adjust or optimize something on the site, everyone's needs should be addressed. You'll want to make sure you are testing the things that truly matter for your organization and balancing performance across all stakeholders.
Once you determine the goals, the next step is sifting through potential key performance indicators (KPIs) to decide which will accurately measure progress toward your specific marketing goals and benchmarks. For example:
- If your goal is to make money, you'll want to track those pages and areas of the site that users click on in the conversion funnel, such as the Buy Now button and resulting Thank You pages.
- If your goal is to save money, track the interactions with both the self-service areas (like FAQs and help content) as well as non-self-service areas (like contact and help ticket generation) of your site.
- Brand awareness goals can be more difficult to track, but certain KPIs can be proxies for customer loyalty, such as recency and frequency of visits and time spent on site, and the percentage of return visitors. Other behaviors to track could include "send to a friend" or "print page" features that indicate a visitor's interest in sharing your site with others.
Beware of scenarios where an increase in one desirable KPI can cause a decrease in another (perhaps more valuable) KPI. This cannibalization can sometimes be a Catch-22, so the best practice is simply to track both KPIs to provide increased visibility of user behavior.
Once you've agreed upon your Web site goals and KPIs, determine the goals of your multivariate testing strategy. These are often related to your Web site goals but can be much more granular.
Let's look at a typical retail/e-commerce Web site as an example. The owners of this site will obviously want their site to make money; here are some of the things they could test and measure in support of that goal:
- Account or newsletter registration: Which navigation path successfully led to the most completions of a new account registration, newsletter signup, or other lead-generation form?
- E-commerce: What elements of the Web site led to the most "Add to Cart" clicks, followed by successful order completion pages (e.g., "Thank you for your order")? Which combination of product information such as graphics, descriptions, layout, and color increased average order value?
- Promotions/offers: Did the visitor click on a promotion, and then take the additional step(s) necessary to complete the conversion process? Bear in mind that it is not enough to simply track clicks from the promotion; if you do so, you risk optimizing for what is most persuasive in getting users to "click" one step further, but not necessary all the way to the conversion. What you need to do is measure both clicks and conversions, and in doing so you will determine the best way to get people to convert. You may end up with fewer people clicking on the promotion, but a higher percentage of them will convert, leading to a higher absolute number of conversions.
The Value of Composite Metrics
Often what we see in measuring Web site success are composite metrics. That's when two or more KPIs are formulaically combined to yield a score for measuring engagement and other advanced metric.
In other scientific and marketing disciplines, this is variously referred to as a rating, fitness function, or calculated metric, and it's useful when there are several meaningful KPIs that can and should be measured.
Using a composite metric provides a simplified way to measure a group of visitor interactions and behaviors. Typically, certain individual KPIs are weighted before factoring them into the composite goal, depending on their importance.
Here is an example of a weighted composite metric that comprises four different KPIs, shown as c, a, s, p:
- c = of clicks from promo into product page, weight = 1
- a = of products added to cart, weight = 1
- s = of entrances that led to a checkout process, weight = 5
- p = of purchases, weight = 15
Let's now call this the "visit score", and calculate it by adding up the weighted values of each individual KPI: (c * 1) + (a * 1) + (s * 5) + (p * 15). With this visit score, you now have a sophisticated way of determining how valuable each visit was toward your marketing goals. Within the context of a multivariate test, this will enable us to determine how changes to the site cause an increase or decrease in average visit score.
Next time, I'll talk more about how to determine your site factors and which ones to consider for multivariate testing. If you have any questions or comments in the meantime, feel free to reach out to me at ehansen@sitespect.com.
