Planning and Designing Surveys
Surveys are everywhere, and their quality can range from good to bad to ugly. The ability to critically evaluate surveys is important when you’re working with statistics.
You’re interested in the willingness of adult drivers (age 18 and over) in a metropolitan area to pay a toll to travel on less-congested roads. You draw a sample of 100 adult drivers and administer a survey on this topic to them.
If you were to select your sample by drawing numbers at random from the published phone directory and calling during daytime hours on weekdays, how could these actions bias the results?
A. Not everyone has a phone or a listed phone number.
B. Not everyone is at home during the day on weekdays.
C. Not everyone is willing to participate in telephone polls.
D. Choices (A) and (B)
E. Choices (A), (B), and (C)
Answer: Choices (A), (B), and (C) (Not everyone has a phone or a listed phone number; not everyone is at home during the day on weekdays; not everyone is willing to participate in telephone polls.)
Using published phone directories as a sampling frame and scheduling calls during only one part of the day can introduce several types of bias to a study. First, people without a phone or a published phone number have no possibility of being selected for the sample. Second, people who aren’t at home during the scheduled time can’t supply data.
And third, phone surveys in general are subject to non-response bias because a high proportion of people contacted may refuse to participate in the survey. All these factors may bias the sample and the data, so your results don’t represent the target population.
What is the main problem with the survey question “Don’t you agree that drivers should be willing to pay more for less-congested roads?”
Answer: It indicates that one answer is preferred and may introduce bias.
In responsible polling, questions should be stated in a neutral manner.
Of the 100 people in your sample, 20 choose not to participate. You later discover that they’re in a lower income bracket than those who did participate. What problem does this introduce into your study?
Answer: non-response bias
Non-response bias occurs when those among the sample who defer to take part in a study differ in some significant way from those who do take part.
One of your questions asks whether the respondent voted in the last election. You find a much higher proportion of individuals claiming to have voted than is indicated by public records. What is this an example of?
Answer: response bias
In this case, it’s likely that some of the respondents weren’t being truthful. This is when response bias occurs. For example, most people believe that voting in elections is a positive characteristic, so they’re more likely to report having voted, even if they didn’t.
If you need more practice on this and other topics from your statistics course, visit 1,001 Statistics Practice Problems For Dummies to purchase online access to 1,001 statistics practice problems! We can help you track your performance, see where you need to study, and create customized problem sets to master your stats skills.