You remember the old joke: When does an online study represent the population well?
Answer: When you get lucky.
I recently spoke to a client who swore that online research works extremely well for him. He spoke boldly, “I have never had a train wreck!”
I suggested to him that perhaps he was extremely lucky or he was missing something. He was quite offended.
Then with a lump in my throat I explained. Online research works well when the variables being studied are not influenced by the factors that affect bias in the data set.
The truth is that some product categories are not impacted by the online respondent variables that cause some respondents to stick with online panels and others to attrite. For example there is no obvious reason why a respondent who enjoys completing a few hundred questionnaires a year should prefer Pepsi over Coke. There is a reason why those who refuse to complete surveys at all might choose one cola over another, we know that there are demographic differences between cola choice. However, as long as the panel you choose always sends you those respondents most willing to complete questionnaires you remain safe.
You just gotta be a bit lucky. That’s all.

Comments are closed.