Let's suppose that the "usability" mantra you've heard over the past few years has finally convinced you that conducting user research with your target audience is an essential aspect of your company's Web site development efforts.
We'll assume that you want to uncover major usability issues before a new site launch or you need detailed customer feedback regarding a live site that just isn't making the grade.
What are your next steps? How do you manage a lab-based usability project from the early planning stages through delivery of a great findings report for the development team?
Here are five things to consider that will help you plan a successful test, set realistic expectations with your colleagues, stick to a budget, and wind up with actionable results.
1. Evaluate your objectives
Lab-based usability testing is about keenly observing and listening to participants as they attempt to use an interface. It's qualitative research. Expect the moderator to elicit lots of think-aloud feedback and straightforward opinions.
If you're objectives are to quantify user preferences or present statistical data, a lab-based usability exercise is not the appropriate research solution for your needs.
Your overall objectives should include these:
- Gathering in-depth feedback on whether participants can navigate your site and accomplish common tasks such as finding key information, using a search function, completing shopping cart checkout and contacting customer service
- Determining whether participants clearly understand the value proposition of your interface
- Ascertaining whether your site content (and possibly a competitor's) is relevant and easy to comprehend
- Learning whether participants are helped or hindered by the site's visual design
Establishing unambiguous objectives for all stakeholders to approve at the beginning of the project will set expectations properly and will pave the way for a successful test.
2. Get to know some vendors
Start your vendor search with a set of criteria that will enable you to compare apples to apples. Here are some good questions to consider:
- Does the vendor employ an in-house moderator, or does it hire an outside resource? Many of the biggest consultancies and ad agencies hire "boutique" usability firms to do the work, because they can't maintain the cost of permanent usability researchers or just don't have the expertise. Don't be afraid to go directly to a specialist. You'll probably save money and bypass unnecessary layers of bureaucracy.
- Can the vendor travel? If your company is building a Web site to sell ski jackets, Florida isn't the ideal testing location. A quality usability vendor will have experience testing and recruiting in any geographic location.
- Does the vendor have experience testing an interface similar to yours? It's always good to learn whether a vendor has a broad range of experience testing Web sites, intranets and extranets. A vendor who has covered a wide variety of industry verticals is also a plus.
- Can the vendor manage everything end to end? Since you may not have internal staff, experience or time to tackle important parts of a usability project, find out whether your vendor can provide added value by putting all the pieces of the puzzle together. There's plenty to accomplish, such as screener development, recruiting, facility selection, video equipment set up, writing a moderator's guide, scheduling participants, catering, travel planning and participant incentives.
If you outline your needs in an RFP and ask vendors for itemized costs, you'll be able to make an informed comparison.
3. Review a sample summary report
One of the concerns that most decision makers have about conducting qualitative research is the uncertainty of what's going to be delivered at the end of the project. Before signing on the dotted line with a vendor, ask for a sample of the final deliverable.
The most helpful reports include a methodology, executive summary and a detailed findings and recommendations section for addressing the specific issues that have been uncovered. In addition, the report should be written in a format that assumes that some readers were not able to attend any of the sessions from behind the one-way mirror.
You should also find out whether the moderator is available to present the findings report in person to expand on important ideas.
4. Rally the team
Get your team excited about the project by involving it early on. Designers, information architects, producers, researchers, marketing managers and advertising people have diverse agendas and visions for a Web site. Tap into your usability vendor to address questions from key stakeholders, even if they seem trivial.
It's a good idea to hold the testing near your corporate offices or to allocate a suitable travel budget so that all key stakeholders can attend. There is no substitute for watching the live sessions especially if there have been differences of opinion on how to best implement certain aspects of an interface. In addition, the discussion that occurs between all the different players almost always results in fresh ideas, a new perspective or a renewed commitment to addressing the needs of the customer.
5. Set a realistic schedule
A typical lab-based usability test includes 5-7 one-on-one interviews per day (usually 60-90 minutes each) at a professional focus group facility or quiet office environment over 1-3 days.
Before the actual testing occurs, you will need about two weeks to work with your usability vendor to accomplish the following:
- Hold a kickoff meeting to agree on the test objectives
- Secure a quality testing facility or controlled environment
- Determine where the testing will take place and make travel plans
- Identify your target audience and approve a screener for recruitment
- Review technology issues and video taping requirements
- Develop a moderator's guide to outline all the questions and tasks that will be presented to participants
- Complete a dry run to address timing limitations and to help prioritize tasks in the moderators guide
Once all the interviews are finished you should expect a final report within 7-10 days. When you add it all up, expect the entire project to take 3-4 weeks. Your vendor should provide a project plan with milestone dates.
* * *
By following those lab-based usability success steps, you'll be able to stay on track, avoid major headaches and add real value to your Web site development process.
