Political Polling Common Errors

Political polling in the United States can be a dicey subject. While there are proven methods for getting fair and unbiased results in polling, in the United States citizens are almost never given these results freely. As most of the ‘infotainment’ networks out there fund, produce, and advertise their own polls that serve the agendas of the parties they support, we as Americans are forced to do our own research and come up with our own methods for finding unbiased polling. While it’s not a terrible prospect for Americans to do their own research on candidates and issues (in fact it is ideal for a true working democracy), I think we all know that the sad truth is that most Americans will not be doing this research any time soon, and instead will be relying on the information provided to them on cable news channels. But I digress… In this blog I will be discussing common polling errors and biases, including those involving sampling, survey construction, and even campaign “push polling,” and hopefully showing some good examples of all of these errors in practice.

An easy glance at the president’s job approval rating polls on pollster.com (http://www.pollster.com/polls/us/jobapproval-obama.php) reveals that two of the leading polling organizations, Gallup and Rasmussen, have overwhelmingly incompatible numbers when it comes to President Obama’s approval rating, with Gallup showing a 52% approval rating against only a 38% disapproval rating, and Rasmussen showing a 47% approval rating against a staggering 52% disapproval rating. Let’s say for a second that it would be unfair to assume that either one of these polling organizations is affiliated with or supports one political party or the other. It’s a fair assumption, as neither one has ever had visible problems with push polling or reports of campaign financing. This being the case, the problem with the discrepancy between these numbers could have been in the sampling of the polls. Researchers try to get a sample size that is substantial enough to get a sampling error of plus or minus 4% or less (Hollihan, 174). Since the difference between these numbers is much higher than that, perhaps one or the other of these organizations did not get a sufficient sample size. However, this seems unlikely since both organizations are professional pollsters and as such are well-versed in the mathematics of sample sizes. A more likely explanation is that the sample sizes that were obtained by one, the other, or both groups were not accurate representatives of the American public. It is possible that despite precautions and efforts to give everyone in the population an equal chance of being polled, that the sample was skewed by large numbers of certain groups (ibid, 174). Perhaps for example, Rasmussen received a disproportionately large number of rural respondents, leading to a more unfavorable viewing of the president. Conversely, perhaps Gallup got a lot of respondents from urbanized neighborhoods or a lot of African-American respondents, making their viewing more favorable of the president.

According to Hollihan, common problems with survey questions include oversimplification of issues; loaded, biased and unclear questions; and making inferences of the results of a poll for questions that were never asked (ibid, 175). Recently in Virginia, a poll was conducted to see if Virginians were in favor of closing the ‘Gun Show Loophole,’ with the pro-gun state falling surprisingly on the side of closing the so-called loophole (http://twitter.com/ProtestEasyGuns/status/4968881627). Unfortunately, the question in the poll said very little about what this loophole actually was. The problem was that people see the word ‘loophole’ and automatically assume that there is something illegal or shady going on. The ‘loophole’ in this case involves a buyer buying from an ‘unlicensed dealer.’ This is perfectly legal under state law, and selling guns at a gun show is one of the safest ways to sell firearms, but the word ‘unlicensed’ in this case once again leads those being polled that something illegal is going on, or that the dealer is in some way shady or untrustworthy. The survey question in this way has a large amount of loaded language, and one could argue also that despite the appearance of a clear question, what the inquiry leaves out makes the question’s language more opaque than what it ought to be. Hollihan uses two similar examples of the Nixon and Clinton impeachments, in both cases showing how those being polled often did not understand what they were being asked. In the case of Nixon, the word ‘impeachment’ did not agree with people, but when asked about the trial of Nixon without the operative word in the question, they were in favor of it (Hollihan, 175-176). In Clinton’s case, some members of Congress voted for impeachment, but didn’t necessarily want him removed from office, further muddying the issue for voters (ibid, 176).

Political “push polling” is another way of getting inaccurate results in polling. With sampling problems and even survey question problems, often times the mistakes made that create inaccurate polls are just that: mistakes. The same is not true for “push polling.” Push polling is the deliberate attempt by a pollster (or campaign posing as a pollster) to influence someone under the guise that they are simply conducting an honest poll (Hollihan, 177). A recent example of this is a poll in Ohio that was conducted over the phone during the 2008 presidential election. The pollster would ask the persona series of ‘agree/disagree’ questions about Barack Obama. Sample questions are as follows: “agree/disagree that Obama is too inexperienced to be President,” “agree/disagree that Obama is too risky of a choice for President,” “agree/disagree that Obama as President would pose a threat to our national security;” and you get the idea from there (http://www.huffingtonpost.com/2008/09/11/nasty-anti-obama-push-pol_n_125607.html). You see, even though the questions gave the person the chance to agree or disagree, every question is framed in a negative way towards the candidate. This particular poll did not care about peoples’ responses at all; rather it was intended to make people who planned on voting for Obama re-think their vote and perhaps go with a different candidate instead. Hollihan recounts a similar example of push polling by the Bush campaign in 2000, one that asked agree/disagree questions framing John McCain’s stances on key issues in a negative light. This campaign took it even a step further though, asking McCain supporters (in the South no less) if they would re-think their vote if they knew McCain had fathered an illegitimate black child (McCain has an adopted Bangladeshi daughter) (Hollihan, 178-179). This inference put the Senator’s moral credibility into question in a key demographic, and who knows if that played a major role in George W. Bush’s victory in the Republican Primaries that year.

As you can see, nothing in politics can ever be easy and cut-and-dry, and political polling gets very complicated. Biased pollsters are always looking to sway votes; reckless amateurs can often throw off accuracy through hastily written and poorly thought out questions; and even when we get an accurate and unbiased polling organization, sampling can cause even the most neutral of polls to become skewed if the conditions are at all wrong, resulting in vastly different numbers for the same polls done by different organizations. What we must continue to try and do is work to reduce the number of errors and biases in polls in an effort to keep polls relevant in today’s political landscape. In order to do so, as Americans we must reject the polls fed to us by television and radio infotainment, and instead do our own research and find our own sources polling that we find to be reliable and accurate for all aspects of the political spectrum.

Works Cited

Hollihan, Thomas A. Uncivil Wars: Political Campaigns in a Media Age. 2nd ed. Boston: Bedford/St. Martin's, 2009. Print.

"PE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd" //1){setTimeout(function(){onCondition(D,C,A-1,B)},B)}}};//]] / Protest Easy Guns: 8 out of 10 Virginians wan ..." Twitter. Web. 27 Oct. 2009. .

"Pollster.com: National Job Approval: Pres. Barack Obama." Pollster.com - Political Surveys and Election Polls, Trends, Charts and Analysis. Web. 27 Oct. 2009. .

Stein, Sam. "Nasty Anti-Obama Push Poll Launched In Ohio." Breaking News and Opinion on The Huffington Post. 11 Sept. 2008. Web. 27 Oct. 2009. .