Five Keys to a Successful Usability Test

Let's suppose that the "usability" mantra you've heard over the past few years has finally convinced you that conducting user research with your target audience is an essential aspect of your company's Web site development efforts.

We'll assume that you want to uncover major usability issues before a new site launch or you need detailed customer feedback regarding a live site that just isn't making the grade.

What are your next steps? How do you manage a lab-based usability project from the early planning stages through delivery of a great findings report for the development team?

Here are five things to consider that will help you plan a successful test, set realistic expectations with your colleagues, stick to a budget, and wind up with actionable results.

1. Evaluate your objectives

Lab-based usability testing is about keenly observing and listening to participants as they attempt to use an interface. It's qualitative research. Expect the moderator to elicit lots of think-aloud feedback and straightforward opinions.

If you're objectives are to quantify user preferences or present statistical data, a lab-based usability exercise is not the appropriate research solution for your needs.

Your overall objectives should include these:

Gathering in-depth feedback on whether participants can navigate your site and accomplish common tasks such as finding key information, using a search function, completing shopping cart checkout and contacting customer service
Determining whether participants clearly understand the value proposition of your interface
Ascertaining whether your site content (and possibly a competitor's) is relevant and easy to comprehend
Learning whether participants are helped or hindered by the site's visual design

Establishing unambiguous objectives for all stakeholders to approve at the beginning of the project will set expectations properly and will pave the way for a successful test.

2. Get to know some vendors

Start your vendor search with a set of criteria that will enable you to compare apples to apples. Here are some good questions to consider:

Does the vendor employ an in-house moderator, or does it hire an outside resource? Many of the biggest consultancies and ad agencies hire "boutique" usability firms to do the work, because they can't maintain the cost of permanent usability researchers or just don't have the expertise. Don't be afraid to go directly to a specialist. You'll probably save money and bypass unnecessary layers of bureaucracy.
Can the vendor travel? If your company is building a Web site to sell ski jackets, Florida isn't the ideal testing location. A quality usability vendor will have experience testing and recruiting in any geographic location.
Does the vendor have experience testing an interface similar to yours? It's always good to learn whether a vendor has a broad range of experience testing Web sites, intranets and extranets. A vendor who has covered a wide variety of industry verticals is also a plus.
Can the vendor manage everything end to end? Since you may not have internal staff, experience or time to tackle important parts of a usability project, find out whether your vendor can provide added value by putting all the pieces of the puzzle together. There's plenty to accomplish, such as screener development, recruiting, facility selection, video equipment set up, writing a moderator's guide, scheduling participants, catering, travel planning and participant incentives.

If you outline your needs in an RFP and ask vendors for itemized costs, you'll be able to make an informed comparison.

3. Review a sample summary report

One of the concerns that most decision makers have about conducting qualitative research is the uncertainty of what's going to be delivered at the end of the project. Before signing on the dotted line with a vendor, ask for a sample of the final deliverable.

The most helpful reports include a methodology, executive summary and a detailed findings and recommendations section for addressing the specific issues that have been uncovered. In addition, the report should be written in a format that assumes that some readers were not able to attend any of the sessions from behind the one-way mirror.

You should also find out whether the moderator is available to present the findings report in person to expand on important ideas.

4. Rally the team

Get your team excited about the project by involving it early on. Designers, information architects, producers, researchers, marketing managers and advertising people have diverse agendas and visions for a Web site. Tap into your usability vendor to address questions from key stakeholders, even if they seem trivial.

It's a good idea to hold the testing near your corporate offices or to allocate a suitable travel budget so that all key stakeholders can attend. There is no substitute for watching the live sessions especially if there have been differences of opinion on how to best implement certain aspects of an interface. In addition, the discussion that occurs between all the different players almost always results in fresh ideas, a new perspective or a renewed commitment to addressing the needs of the customer.

5. Set a realistic schedule

A typical lab-based usability test includes 5-7 one-on-one interviews per day (usually 60-90 minutes each) at a professional focus group facility or quiet office environment over 1-3 days.

Before the actual testing occurs, you will need about two weeks to work with your usability vendor to accomplish the following:

Hold a kickoff meeting to agree on the test objectives
Secure a quality testing facility or controlled environment
Determine where the testing will take place and make travel plans
Identify your target audience and approve a screener for recruitment
Review technology issues and video taping requirements
Develop a moderator's guide to outline all the questions and tasks that will be presented to participants
Complete a dry run to address timing limitations and to help prioritize tasks in the moderators guide

Once all the interviews are finished you should expect a final report within 7-10 days. When you add it all up, expect the entire project to take 3-4 weeks. Your vendor should provide a project plan with milestone dates.

* * *

By following those lab-based usability success steps, you'll be able to stay on track, avoid major headaches and add real value to your Web site development process.

Subscribe today...it's free!

MarketingProfs provides thousands of marketing resources, entirely free!

Simply subscribe to our newsletter and get instant access to how-to articles, guides, webinars and more for nada, nothing, zip, zilch, on the house...delivered right to your inbox! MarketingProfs is the largest marketing community in the world, and we are here to help you be a better marketer.

Sign in with your preferred account, below.

ABOUT THE AUTHOR

Lon Taylor is a principal usability consultant with First Insights and cochair of the New York Research Special Interest Group. He can be reached via www.firstinsights.com.