Despite the many desirable aspects of survey research, you also find limitations. It’s difficult to get good data when the subjects are people, no matter how you go about it. Even scientific researchers, who make every effort to conduct controlled studies, cannot control experimental conditions with human subjects as they do with lab animals.
Reaching the right respondents for your survey isn’t always easy. Some people are hard to reach; others are reluctant to participate. People who are available and willing to respond might or might not have the same behavior and attitudes as those who are not.
After you have a satisfactory pool of respondents, don’t think that your troubles are over. You may not get answers to all your questions. People don’t always know the answers. A question that seems simple to you may not seem simple to the respondent.
Perhaps you’ve asked about the respondent’s income. Did you mean that one person’s income or the household’s? Would that include the children’s income? Nontaxable income as well as taxable? What about losses? What if the income varies? The respondent might wonder whether the right answer would be the level of income she’s had recently, expects soon, or typically earns, and those might be three different things.
Survey researchers sometimes seem to forget that respondents are less interested in the survey topic than they are. You may be very interested in ketchup, ketchup purchasing habits, ketchup flavor and texture preferences, and all things ketchup, but most people aren’t.
So even a willing respondent may be unable to answer all your in-depth ketchup questions. He may buy ketchup, but not recall when or how often, or what price he paid or what brand he bought, let alone how that brand compares in flavor and texture with each of the major competing ketchup brands. So don’t kid yourself about the level of depth you can expect in survey responses.
And then the worst survey problem of all is this: not asking the right questions. You can ask every question you can think of and still miss the point. The most important issue in a customer’s (or patient’s, constituent’s, or member’s) mind could be an issue that you just haven’t given any thought to.
That’s why many surveys end with an open-ended question, such as “Is there anything else we can do to improve your experience?” While it is a good idea to ask such questions, you have no guarantee that you’ll get all the information you need.