If you want to better understand your customers, you may find yourself thinking about putting together a short questionnaire. You might, for example, want to know how satisfied your customers are with your service, or their reactions to your web site. You might want to sent them an email message as the questionnaire, or perhaps post a questionnaire on your web site.
If you don't know the basics of questionnaire design, you'll find yourself quickly confused. But this short tutorial may help. In it, we lay out some simple general rules about questionnaire design and explain the basics of measurement. As always, this is a quick overview, and we have books in our book section that go into far more depth.
To start you off, here are some general rules you need to think about when putting together any type of questionnaire:
GENERAL RULES
- Efficiently ask all the questions that are important. Avoid questions that seem off topic. This requires that you first clearly identify what it is you want to know.
- Shorter questionnaires are better than longer ones, but more important than length is how easy it is to complete the questionnaire. So, you need to make your instructions very clear and make sure the wording of all questions is unambiguous.
- Pre-test a questionnaire to make sure of the above. Do this with real respondents, or at least co-workers. This is very important! Pre-testing ensures that the questionnaire is easy to fill out.
- Open-ended questions, while often necessary, are the least likely to be answered. You may need to use them, but you'll increase your response rate by translating them into a scale.
SCALES
With these general rules in mind, let's spend a few minutes understanding the idea of scales, since this is what you are likely to use to measure your customers.
First of all, there are many types of scales, but three are often the most used in the types of marketing research you're likely to find. We'll explain these briefly below.
MULTIPLE POINT SCALES
Undoubtedly you have seen the 7-point Likert scale. While often derided, this is in fact a highly legitimate means of collecting information about consumer's perceptions, attitudes, beliefs, and other responses. We gather data from consumers by stating a specific response and asking a respondent their degree of agreement with the statement.
Note that there is nothing magical about a 7 point scale. You can use a 5 point scale instead, but always remember that the more points on the scale, the more possible information you can acquire.
Finally, an odd number of points allows a respondent to have a neutral position on the scale. If for some reason you don't think that respondents have a neutral position on a question (which is not likely), you can use an even point scale (say 4-points).
So, a typical scale item might be as follows:
"I found this web site to be very easy to navigate."
| Strongly Disagree | Strongly Agree | |||||
| 1 | 2 | 3 | 4 | 5 | 6 | 7 |
Notice the wording of the statement. Were we to say "I found this web site to be easy", rather than "very easy", we would not give people a strong enough statement to either agree or disagree with, thus forcing people more toward the middle (4) of the scale. You want to word the statement to get maximum variation on the scale, and by making the question strongly worded we achieve that.
Another way of doing this is by using a scale such as follows:
"How easy was it to navigate the web site?"
| Very Difficult | Very Easy | |||||
| -3 | -2 | -1 | 0 | +1 | +2 | +3 |
This provides a clear neutral point (0) and makes it a bit more clear that when someone ticks of the, say, -3 that they are really saying the web site is very difficult. The point is to think through how you respondents will not only react to a scale, but what do you want to measure.
CHECKBOX SCALES
This is a scale that measures an identity (e.g., are you a man or woman?). Marketing research uses these types of scales to gather demographic and other similar information. For example, you might have a series of check boxes and ask a respondent to check the box that closest indicates their income.
A key thing to think about when using checkbox scales is whether the respondent can be identified in multiple ways. For example, you might ask them which magazines they read. Since they may read several, you need to let them check off all that apply.
TRUE ZERO SCALES
Any time you want to measure something that has a true zero point (like income or age) you are using this type of scale (known as a "ratio scale"). For example, you would be using this types of scale if you ask someone how many years they have been using web browsers.
Note that often researchers will translate true zero scales into identify scales. This is what happens when, say income is broken up into several categories and respondents are asked to identify which category best describes them. The key issue here is to make sure the categories are meaningful. For example, does it really make sense to have an age category of, say, 25-38? By doing do, you are implicitly assuming that someone who is 39 is in a completely different category. Think hard before blindly translating things like age into checkbox scales.
EXAMPLE - REACTIONS TO MY NEW WEB SITE
Now, let's put this all together with a simple example. Assume you had some people beta testing a new web site, and now you want to know people's reactions to it. You might send them an email asking for their responses to several questions.
But what do you want to know? That's the first question you need to answer.
Let's say, you decide that you want to know a) how easy the site is to navigate, b) how easy it is to search for things on the web site, c) how interesting is the content, d) how relevant is the content, e) how appealing is the layout, etc., etc.
Now that you have the questions you're most interested in, you might also think about demographic information that might help you understand why people might have different types of reactions to your web site. For example, maybe people with more prior experience with web browsing might find it easier to search your web site than people with less prior experience. If that is possible, you certainly want to ask respondents their level of prior experience.
Other demographic information, even if not related to your central question, might be useful in understanding your target market, so you might ask these questions as well.
START OFF EASY
Start off the questionnaire with questions that are interesting and very easy to answer. If respondents find they cannot answer the first questions easily or fund them threatening in any way, they may refuse to continue filling out the questionnaire.
For example, you might start off asking them "How long have you been using the web site?" which you might have them answer either with a specific number or as part of an checkbox scale (as described above), or "Approximately how many times have they visited your web site?", etc.
THE MIDDLE
After asking the opening questions, you need to start focusing the respondent on the important questions. Two rules of thumb can be followed. First, make sure the order to the questions is logical. This means avoiding sudden changes in topics when possible. Second, always move from broader questions to narrower questions. This is called funneling.
Continuing our example, you might first ask a general question about how they liked your site (using a multiple point scale, as above). Then, you could move on to specific issues, such as their level of interest in your web site, the degree to which your web site is appealing, etc.
THE END
You could end your questionnaire with some further general questions. These are typically classification questions, such as demographic information using checkbox scales.
ORDER
One thing you want to keep in mind when designing a questionnaire is the order of questions near each other. There are situations where the order might bias a respondent. That is, by answering one question, they are more likely to answer a subsequent question a certain way.
For example, consider the following two questions:
How much money do you make?
How much are you willing to pay for my product?
Here, by asking the first question, you are biasing the respondent to answer the second question. If they don't make much money, the first question will remind them of that and bias their answer to the second question (lower).
The point is that you need to also make sure that the questions don't force respondents to answer in certain ways. With care, some clear thinking, and pre-testing you can avoid some of the pitfalls in questionnaire development.
