Today’s Washington Post contains the following headline: Mitt Romney Fares Better Than Gingrich Against Obama in Florida Poll.
Yesterday’s State Column headline says: Ron Paul Scrambles for Top Tier Finish in Minnesota, based on the Public Policy Polling company’s survey of Minnesota voters.
But what does it all mean? Personally—not speaking for the BBB here—I am not a fan of polls. When I was growing up in Idaho, during fire season, our mealtimes were often interrupted by a ringing phone. Sometimes it was a pollster, wanting to know how my parents would vote.
My mom always said the same thing: “I am a registered voter in the State of Idaho, and I consider voting to be my civic duty.” She would not tell the caller if she was a Republican or a Democrat or anything else. Her point was that our country’s ballots are cast in secret for a reason. It’s nobody’s business how you’re going to vote. Nobody should be able to influence your vote—not even Oprah. Or to intimidate it. You should vote your conscience and yours alone.
But polls are a fact of life, and if you’re going to read report after report about them, you should know how they work. As a former reporter, I took a look at the three news stories mentioned above.
The Washington Post story correctly mentioned who conducted the poll–Quinnipiac University. (You also want to know in some cases who paid for it. Stear clear of polls paid for by candidates or other non-objective parties.)
The Post story said when the poll was conducted–Jan 19-23, ending the day before President Obama’s State of the Union address. This is important because opinions can change quickly, especially after significant events.
The story also mentioned another poll–”A Quinnipiac poll of likely primary voters taken over the same period showed a virtual dead heat between the two top Republicans.”
See how they compared two polls from the same organization? Since different techniques can cause different results, comparing polls from two different organizations can result in an apples-to-watermelons type of comparison.
I also noticed the word “virtual”–a race should never be described as a “dead heat” unless the candidates’ numbers match exactly. In this case, it is almost a dead heat because one has 36 percent and one has 34.
Let’s look at the State Column. They correctly report the number of people involved in the survey they’re quoting–in this case, 303–and who they are–people likely to vote Republican.
“Former House Speaker Newt Gingrich,” the article says, “is leading a number of Sunshine State polls.”
When you read that a candidate is “leading,” it should mean that the number of theoretical votes he’s received are at least twice the error margin of the poll. In this case, plus or minus 5.6 percentage points. So to lead, Gingrich must be ahead by at least 11.2.
If the story says the race is “close,” the difference should be less than the sampling error margin.
What about CNN? I was pleased to see that their word choices were precise. The story did not suggest that any particular candidate “would” win, but rather that the poll “indicated” a particular direction. Polls can be wrong, and voters sometimes change their minds before Election Day.
However, none of the stories (unless I missed something) explained how the various polls were conducted. Telephone surveys by robocall, for example, can’t exclude children from adult samples, so may not be as valid.
You should also consider the location of the survey group. Urban voters in Florida probably have different opinions people in rural Minnesota. Be wary of polls claiming results to the first decimal place (error margins are ok). This isn’t really believable from a sample.
And finally, remember that the wording of survey questions matters. Small question differences can cause big result differences. Interviewer skill and refusal to participate by randomly selected respondants—skewing the randomness—are among potential sources of error in surveys.
For an in depth look at the proper reporting of polls and surveys I recommend the Associated Press Stylebook 2011, the news industry Bible. The section on how to interpret and report on polls and surveys takes up two pages!