Can we identify benchmarks for Web success? What metrics can justify Web-based projects? How do you confirm the most valuable customers? How do you calculate the ROI of personalization? How do you calculate the value of Web intelligence?
To answer these questions related to measuring the success of a Web site, Web executives, marketers, academics, software and service vendors, and members of the press gathered in two countries, over six days, with 22 speakers and 24 discussion group sessions for the Emetrics Summit of 2004.
The smartest, most experienced people in the field—including experts from Amazon, HP, MSN and SAP—presented their best thinking on the complete range of issues. In this two-part article, I will summarize the most valuable nuggets that sparked my attention for clarity and insightfulness, or just made that light bulb go on over my head.
As the producer of the conference, and author of a book on the subject, I know that producing more data is not the issue. Rather, knowing how to focus on the mission-critical information and how to turn it into insight to make decisions about the Web site is key. Toward this end, the conference delivered useful guidance.
We began in London.
I opened the conference by covering the various ways data is collected and revealed a handful of success-critical observations that can be found on the pages of my other articles here at MarketingProfs.
Dan O'Sullivan from University College Cork took the stage next and presented the results of the CMO Council study on Marketing Performance Management (MPM). The Council surveyed over 250 Chief Marketing Officers and interviewed 17 of them to find out how well they were measuring marketing in general.
The consensus? Not well.
While over 90% of the surveyed companies rated MPM a high or moderate priority, few had a formal MPM system and less than a quarter were satisfied with their ability to measure marketing performance.
The scariest factoid was the discovery that these CMOs felt their Web site was one of the easiest marketing activities to measure, right after direct mail and just before telemarketing and contact management. It may be easier to collect data from a Web site, but putting that information to use is still a hard nut to crack.
Neil Mason of Dot Biz Consulting delved deeper into the dark mysteries of extra-server measurements in order to shine some light on the various services that assess the Internet as a whole. His astute premise was that your visits and transaction may be growing steadily, but if they are not keeping pace with the growth of the Internet, you're losing ground.
Neil divided the world of extra-server metrics services into four groups:
- Panel-based services
- Survey-based services
- Aggregators
- Analysts
The panel-based services, such as Nielsen NetRatings and Comscore, recruit a panel of respondents "who are selected to be representative of the total Internet population. Data is continuously collected via software downloaded onto the respondent's computer. Surfing activity is collected and transmitted back to central processing center for data aggregation and reporting."
Think consensual, opt-in spyware… and you've got the right idea.
Survey-based systems collect data via surveys online, face-to-face or over the telephone, and record claimed online behavior ("Yes, I'm doing more research and less music downloading this year"). Surveys and panel service have the advantage of knowing a great deal about the surfer: demographics, personal interests, awareness of and attitude about different sites, etc.
Aggregators don't know what the individual is thinking or feeling, but they watch a great many of them. Hitwise, for example, watches traffic flow across the Internet by piggy-backing onto some of the largest ISPs.
Analysts focus on given sectors. There are myriad analysts whose job entails studying and reporting about a given marketplace. Neil recommends buying their knowledge to keep abreast. Some can even help you choose a Web analytics tool or service. If you want to do that yourself, then you'll want to heed the advice of the person who spoke next.
Terry Lund was an independent consultant when he presented Evaluating Vendors. He wrote a wonderful buyer's guide for those on the horns of this particular dilemma, and then went off to join one of those vendors.
His advice was solid and he broke it out into five steps:
- Involving the right people
- Understanding Web analytics basics
- Goals and budgets
- Vendor-selection process
- Final selection and negotiation
That may all seem obvious and basic, but I'm constantly surprised by my clients who skip steps 1 and 3. That's why the world will always need consultants.
One comment that Terry made was picked up and became a bit of an Emetrics Summit mantra: "Do a little, learn a lot." Instead of trying to boil the ocean—redesign the whole Web site or implement an all-encompassing Web analytics program—bite off a little bit and chew on it for a while. Walk before you run. Buy an inexpensive tool, play with it a while and learn enough to make a much better decision when it comes to choosing a serious toolset.
When focusing on Web site success, the first item to measure is how well you are attracting people to your site. For a look at Measuring Search Engine Marketing, I asked Mike Grehan, author of the Search Engine Marketing Book to come and enlighten us. Instead, he frustrated and confounded us by explaining just how nebulous search engine results really are.
Mike described how something as simple and straightforward as tracking a pay-per-click click from point A to point B can be tricky. In fact, Mike had trouble deciding what to call his presentation:
- Dealing With Nonky Numbers
- Dealing With Dodgy Data
- Dealing With Murky Metrics
We've got organic search results that are impossible to control, pay-for-consideration models that include no guarantees, pay-for-inclusion options that won't help with ranking and pay-per-clicks that may well include fraudulent traffic from the likes of www.i-faker.com, or housewives in India who are paid a handful of Rupees per click. No wonder Mike is still wondering what to call his presentation.
After a well-deserved lunch break, we gathered into groups to discuss the Top 10 Nastiest Web Analytics Problems. The final list included some 50 problems, but popular vote determined ten that were the most pernicious: "Today's Top 10 Web Analytics Problems," an earlier article that lists and describes them.
The next speaker was a bit of a shape-shifter. Matthew Berk went from Web site builder to industry analyst at Jupiter Research and then back to the world of the hands-on Web manager at Tommy Hilfiger.
Mathew looked back on the 10 years we've been building sites under the influence of the power of "guru-based design" (people who claim to know these things were revered) and the lure of personalization (if we can know everything about a customer, we should!). Surrounding all of this excitement, everyone became an expert. Your mother, your CFO, your dentist—everybody knows how to make your Web site better. We're only just now coming into possession of best practices.
Mostly, Mathew lamented the "Fantasy of Perfect Measurement." Expectations are enormous, especially in the upper reaches of any organization, where they've been told we can measure everything. They really do believe, as the CMO Council's study revealed, that Web site activities are among the easiest to capture and report.
Mathew offered up some Web Site Mantras that belong on a poster:
- Site analytics is about optimization.
- In lieu of reporting, seek discovery.
- Drive measurable improvements through hypothesis-based testing and tweaking
- Take comfort in outsourcing site measurement technology.
- Adopt higher-order Key Performance Indicators for all Web sites.
Richard Fone of ABC Electronic (ABCE) described this "industry owned, tri-partite, not-for-profit organization that works with and for media owners, advertisers and media buyers to help them better understand and gain confidence in the data they use." The ABC is the UK's publishing industry's audit bureau—the Audit Bureau of Circulations. The ABCE is the online version, chartered with auditing Web sites that sell advertising space and auditing their measurement methods. This means they are tasked with coming up with advertising metrics and standards. Turns out, there's a great deal of work in this area across multiple associations. Tune into the ABCE for the alphabet soup of organizations working on advertising standards.
Our next speaker earned a gasp from the audience when he said he was in charge of an 18 TB (yes, that's terabyte) customer data warehouse. With the exception of the people from Amazon, we were all very impressed. Seth Romanow is the Director of Customer Knowledge Management at Hewlett-Packard.
Seth took a leaf from Terry Lund's "do a little, learn a lot" by telling us: train your team to use the tools, let them use the tools, and then teach them how to go deep; don't expect them to learn all of it at once. He then went on to describe the complexity involved in bringing together all of the data you collect about customers at all touchpoints. Difficult, challenging and frustrating, but the economies of scale and the potential for insight sifted out of the data warehouse made up for the bruising and the scars. The one critical factor? Solid support from upper management.
If that sounds like something we've already covered, then you are beginning to understand the value of a conference. So many voices repeating the same thing tend to drive the points home.
Louise Brown is the Portal Business Manager at MSN.co.uk. Hers was the only voice that spoke of a dozen people whose sole responsibility is the home page. They know the value of each click to each channel. They update the MSN home page three times a day, modifying ads, subscriptions and news headlines, measuring the immediate impact of each change.
Because their visitors are members, they know the age, gender and geographical location of each one. Makes for some very targeted promotions. They remember which headlines are too provocative: they entice clicks but insufficient reading time after the click. They note which images of female celebrities women click on. They watch in wonder as more men that women click on the "Seven Signs He's Cheating" banner.
The result? Automated optimization of content to audience segments, which leads to more clicks and more advertising revenue with a targeted response rate 60% more effective than a random rotation of the propositions.
Jorge Gutierrez was having a tough time convincing people at Avaya that Web analytics was a valuable tool. So he decided to stop working for the Web Analytics department. His new title is Senior Manager Web Intelligence & Optimization Management. Nobody wants metrics—metrics are scary. But everybody wants intelligence and optimization. Good move.
Jorge also wanted to build a great team, so he went out and found people who had pictures on their desks of their first Web sites, right next to the pictures of the kids and spouse. "I want people who are passionate about this stuff!"
We'll wrap up the first part of this review with Georgie Camfield, the Business Development Analysis Manager at Argos Limited. Georgie's life is consumed with making sense of some truly multichannel commerce. Argos accepts orders in person, on the phone, online and via SMS text messages. You can direct them to deliver your toaster or lamp or desk chair to your home or office, or have it set aside for pick up in the store. Tying all those bits together is quite a challenge, as is getting a large company to embrace Web analytics.
Georgie started off with two critical questions that really should be part of every Web manager's day:
- What do we really want to know?
- What will we do with it?
Given that there is so much that can be collected, why should we? Most important: How do we convince the Powers That Be that this Web analytics stuff is a good idea? Her solution was to pitch different benefits to different internal organizations.
Marketing wants to identify high-value customers and effective partners while optimizing email targeting. Content developers want to optimize the sales navigation path through the site, to know where the exit rates are the highest and get a handle on search effectiveness. The merchandising team wants to track conversion by product type, assess their promotional efforts and monitor up-selling and cross-selling. As always, IT needs to keep a finger on the pulse of capacity planning.
Georgie reminded us all to validate and revalidate the benefits you bring to the table. As you go through the process of making your business case, getting consensus, finding tools, testing tools, training people, etc., etc., several things will happen:
- Senior executives will change roles.
- Corporate rules will change.
- High-level initiatives will go out of fashion.
- You will lose the will to live.
She then presented us with the Web Analytics Project Emotion Chart. A classic. The uplifts and downbeats of getting a project from concept through proof of value. It's a hoot.
Her wrap up was simple:
- Do know exactly what you want.
- Do be clear with requirements.
- Do persevere.
Stay tuned for the next installment. We'll wrap up the UK version of the 2004 Emetrics Summit and report on the other (US) side of the Pond.
