What does it take to know what people actually think? What I mean is: with all of the surveys and polls out there, what do we really know?
I understand that, in theory, polls and surveys tell us what people are thinking because they have a sample size. But it seems like there are a lot of other factors besides just how big your sample is–right? For instance, you’d have to worry about people’s ages and people’s races and political beliefs, too, if you wanted to make sure your sample was truly representative. Is this even possible?
You are absolutely right: creating an accurate sample for a survey or poll is indeed more involved than simply grabbing enough participants! As you astutely point out, there are plenty of ways for a survey or poll to go off the rails by simply including too many of a particular type of person, whether that’s the elderly, the young, men, women, or people of certain professions or from certain regions.
To make matters worse, the very act of collecting samples for polls and surveys can create sample size bias. Imagine, for instance, that you conducted a survey using phones. That’s a tried-and-true method that is becoming tougher and tougher to get right, because more and more young people don’t have landlines. And even calling mobile numbers might not help, because you could then risk excluding too many folks with lower incomes. And what if younger people are simply less likely to pick up the phone?
The act of constructing an accurate survey or poll is quite difficult. Experts work carefully to get the proper sample sizes, and they take pains to present their numbers in a way that allows researchers and others to see what’s really happening within the numbers. Well-designed surveys will have cross tabulation, for instance. A crosstab will break down results and variables in a way that can help readers spot patterns and avoid the pitfalls that come with not reading surveys and polls properly or closely enough. It’s very easy to misinterpret polls and surveys. If there is a public distrust of polling numbers, it no doubt stems in part from our widespread inability to make proper use of polling information in the media and among ourselves!
The important thing to know, though, is that the difficult task of accounting for the many variables in sample sizes is nevertheless a task that our polling experts are a match for. While some simple online surveys and minor polls may be untrustworthy, you can rest assured that studies have proven our major political polls and other surveys to be quite accurate. Just remember to look for polls from reputable pollsters, and to use proper survey tools when conducting surveys of your own for any important purposes.
“Just because it’s common sense doesn’t mean it’s common practice.” — Will Rogers