Two recently released surveys of registered voters appear to provide contradictory results regarding the 2012 presidential election. The first, conducted by Pew during the second week of March, shows President Obama handedly beating a generic Republican candidate for president.
However, CNN released a second poll of registered voters yesterday, conducted by Fairleigh University from March 21-28 that indicates Obama is in essentially a dead heat with both Mike Huckabee (46%-46%) and Mitt Romney (Obama leads 44%-43%) in the 2012 race.
As Peter Baumann notes in an e-mail, these results seem surprising; it is more often the case that incumbents do worse against hypothetical generic opponents because survey respondents can fill in the blank with their ideal candidate. When matched up with specific candidates, however, voters are asked to choose between two real people, and incumbents often benefit. If Peter is right, what explains the results in these two surveys which show Obama doing better against the hypothetical Republican? The answer, I suspect, lies in the question wording of the respective surveys and in the smaller sample size of the Dickinson survey.
Pew begins its survey by asking respondents whether they have a favorable or unfavorable opinion of a number of people, beginning with President Obama. They also ask about former presidents Bush and Clinton and House Speaker John Boehner.
Although Obama’s job approval ratings are hovering in the mid-40’s, his personal favorability ratings have consistently been higher. In the Pew poll, 58% of respondents view Obama “very” or “mostly” favorably, compared to 27% who view Boehner this way. (Bush is viewed favorably by 42% and Clinton leads the pack with 67% favorability rating). After leading with the question about Obama’s favorability rating, and implicitly comparing it to that of Boehner and Bush (and Clinton), Pew then goes on to ask about the 2012 hypothetical matchup with a generic Republican. So respondents, in thinking about that generic “Republican”, do so in the context of being asked about two unpopular Republicans: Bush and Boehner.
Now look at how Fairleigh Dickinson begins its survey.
By beginning the survey this way, respondents are “primed” to answer the head-to-head matchups question primarily in terms of their views on whether the country is heading in the right direction, and on Obama’s job approval – not his favorability ratings compared to other Republicans. As you can see, a substantial percentage of Americans believe the country is going in the wrong direction, and a near majority disapprove of the job Obama is doing. It’s not surprising, given the way the question wording primes them, that respondents in the Fairleigh Dickinson poll see Obama doing less well in head-to-head matchups than Pew finds him doing against a generic Republican opponent.
There is a second difference between the polls as well. Although the Dickinson survey of 800 respondents has a margin of error of +/- 3.5%, which is quite similar to Pew’s margin of error (+/- 3%) this does not hold for all the survey questions in the Dickinson poll. In fact, the numbers for these subsamples of respondents comparing Obama to Romney and Huckabee are much smaller (see table above) meaning the margin of error for both groups is substantially larger than the poll’s overall margin of error. So, we need to view these results indicating that Obama is in a dead heat with both candidates with the requisite dose of salt.
My point here is not to say that one poll is more accurate than the other. It is to remind you that, as we enter the 2012 presidential campaign, you will be inundated with polling data, much of which will be spun by politicians and pundits according to their political persuasions and which will be hyped by news media outlets that views poll results as “hard” news. As consumers, you need to cut through the spin and hype to examine the details of the polls. Pay attention to both question wording and order, and to the margin of error associated with survey subsamples. All polls are not alike.
Addendum (2:32 p.m.) Three days ago Quinnipiac released a nationwide poll taken March 21-28 that received quite a bit of national play for its headline “Obama Gets Lowest Approval, Reelect Score Ever”. His approval/disapproval rating was 42/48%, and 50% of respondents said he did not deserve reelection. Of greater relevance to this post, however, is that in the hypothetical 2012 matchup between Obama and a generic Republican, the poll found that Obama got 36% of the vote compared to 37% against the unnamed Republican. In contrast to the Pew poll, however, the Quinnipiac poll used different question wording; the generic matchup question was the first one Quinnipiac asked respondents, so they were not primed to respond in terms of a specific Republican, as they were in the Pew poll. Again, it is a reminder that question order can influence polling results, even when identical questions are asked.