Monthly Archives: November 2008

Some Data on Undecideds

There’s a nice summary by Charles Franklin at Pollster.com  (see here) of the National Election Studies analyses dating back to 1948 regarding how the undecideds break in presidential elections.  As I noted in an earlier post, the basic rule of thumb is that challengers do slightly better than incumbents, but that doesn’t apply in this election since there’s no incumbent (although Obama would have you believe that McBush is the current officeholder!)  Franklin notes that there’s rarely a lopsided split toward one candidate or the other. The most recent one-sided split occurred in 2000, when about 7 in 10 (66%-23%) of the undecideds broke for Bush over Gore.  But usually the break is closer to 55-45, which is how Pew allocated their votes for McCain and Obama in the poll I cited yesterday, and which is consistent with how leaners break when pushed by pollsters.

The other point Franklin makes is one that Jesse Gubb brought up yesterday: a substantial number of undecideds just don’t vote.  Based on the graph, a rough “guestimate” is that 10-15% of undecideds don’t bother voting. Of course, given the small subsample sizes, there’s a lot of uncertainty in these estimates.  And Franklin doesn’t bother telling how undecideds are defined either – is it voters who have not made up their mind in the last day?  The last week?

What does this suggest for 2008?  As Bert and I have both noted, there’s no clear pattern across elections that seems to explain how undecideds break, except perhaps for a slight partisan effect that Bert picked up in his analysis. Based on the demographics of the undecideds,  and the results in those races that have tightened in recent days (e.g., in Pennsylvania where McCain is winning 5-1 among late deciders), I continue to believe that the majority of the undecideds will go to McCain, but not in great enough proportions to win the election for him.  The exit polls will give us an early indication regarding how those who come to a decision in the last two weeks voted. Interestingly, if you look at the tracking polls so far (and I’ll recheck this data today) the number of undecideds is holding steady at about 6%.  That’s why Pew, in their final poll, had to make a decision to allocate them.

I just want to be clear here: given current polling figures, even if the undecideds break substantially for McCain, it won’t be enough to give him a victory in the popular vote. It may be enough to put him over the top in key battleground states, however.  I’m doing a longer analysis of that now.

Considering the Final Pew Poll: Some Reminders

The Pew Research Center is normally one of the most reliable pollsters – they use sound sampling techniques, generally have a robust sample size, and in 2004 came very close to nailing the final election results. Thus it is with some interest that I looked at their final election survey, which was released today.  Its headline read: “Obama leads McCain 52-46 in campaign’s final days.”  However, if you go beyond the headline to read the actual survey, you find that in fact the survey indicates that Obama is up 49%-42% over McCain, with 7% undecided, and another 2% voting for Nader or Barr. How did they arrive at the figures trumpeted in the headline?  By deciding to allocate the undecideds between the two candidates. In this case, they assigned McCain 53% of the undecideds, and Obama 47%.  Presumably they did so by looking at the same demographics that I noted in an earlier post  – that undecideds are slightly more white, less affluent, less educated and more likely to be women – and therefore were more likely to vote for McCain. We will know in less than 72 hours whether that allocation was consistent with the final vote.

In the meantime, what does the Pew survey tell us if the undecideds remain unallocated?  The most important finding is that Obama is in the lead by 7%, more than the margin of error in this poll (2.5%).  That’s the good news for Obama. The good news for McCain is that this represents a net gain for McCain of 8% since the last Pew poll from 5 days ago – a pretty substantial gain in that time period (keeping in mind that this may simply reflect random variation across the two polls).  Interestingly, that gain was due to a combination of slippage in Obama’s support and increases in McCain’s.  More interesting still, the number of undecideds did not go down in this period.

Where did McCain make his gains, if not among undecideds?  Primarily among independents, especially middle-income voters and women.  Most notably, he gained 16% among independent women voters, and about 8% among young (18-29) independents since the previous poll.

Keep in mind that this is only one poll (I will deal with the latest tracking polls in a separate post). Moreover, even with McCain’s latest gains, Obama remains comfortably ahead with less than 72 hours until election day.  Unless other polls indicate similar trends, Obama has little to worry about. For McCain to close this gap he would need to win all the undecideds while keeping Obama below the magic 50% mark.  But from McCain’s perspective, it is better to be behind 49-42, than to be behind 52-46. even though the latter figures make the race appear to be closer.  As long as Obama hasn’t broken the 50% threshold, there is a glimmer of hope.

The Pew poll reminds us, then, that some polls showing Obama at more than 50% do so by pushing “leaners” or, in Pew’s case, deciding to allocate the undecideds.  As always, read the fine print.

A Bradley Effect, or a Sally Field Effect?

In analyzing the election to this point, I have relied heavily on polling data. But what if the polling data is wrong?  Of course, all polling is based on random sampling so there is invariably some uncertainty due to statistical error, as well as “house” effects based on differences in how pollsters project turnout, weight their samples by partisan i.d., etc.

But is it possible that a different bias is at play, one that systematically influences all the polls in ways that may lead them to overstate support for Obama? That’s the premise behind the much-discussed “Bradley” effect (also known as the “Wilder”, or “Gantt” or “name of black politician here” effect). As most of you know, the Bradley effect is the idea that when polled, a portion of white respondents deceive the pollsters by stating their intention to vote for an African-American political candidate who in fact they have no intention of supporting. As a result, polls systematically overstate the likely vote for the African-American candidate.

By now, most of you are familiar with this argument.  Scholars and pundits have debated whether the effect ever existed, and whether it exists now.  Pundits such as Nate Silver, citing among other data a research paper by Harvard lecturer Dan Hopkins, argue that there has been no sign of a Bradley effect in American politics since the 1990s.  Indeed, Silver argues that support for Obama was often understated in the polls during the primary, particularly in the South.

I want to suggest that Silver and other pundits are missing an alternative and more pervasive source of polling bias.  Our concern should not be for a “Bradley” effect – it should be for the “Sally Field” effect.  Many of you will recall that when Sally won her second Oscar for Places in the Heart, in her acceptance speech she uttered the memorable lines: “I can’t deny the fact that you like me, right now you like me!”  And that’s my worry about a possible polling bias – not that a subset of Americans are racist, but that, like Sally, they want to be liked.

Political scientists have ample evidence that Americans often answer pollsters’ questions in ways that make them appear to conform to socially acceptable norms of behavior.  They do so because, much like Sally Field, they want to be accepted by their peers. For example, National Election Studies indicate that in past presidential elections some 12% of Americans report that they voted when in fact the turnout numbers prove they did not.  A similar number of Americans claim to be registered voters when they are not. This is particularly true for blacks living in the Deep South who, due to the civil rights movement, may feel the social pressure to say they have registered to vote more acutely than whites. The point is that Americans of all races are not immune to the social pressures that drive them to tell pollsters what they think pollsters want to hear.

How would this Sally Field effect bias the current presidential polls?  Assume that a large number of Americans actually would like to see an African-American elected president in the belief that it would signify that we have clearly moved beyond the racist legacy that has so tainted our country’s history.  At the same time, however, polling evidence indicates that many individuals have doubts about Obama’s candidacy. They worry about Obama’s Senate record which, according to research by by Jeff Lewis and Keith Poole, makes Obama the 10th most liberal Senator during his time in Congress. That voting record places him to the left of John Kerry, who lost the 2004 election in part because many voters feared he was too far left of the American ideological mainstream. Indeed, Hillary Clinton, who outpolled Obama among primary supporters in the 2008 nominating campaign, was most effective when she contrasted her more centrist voting record with Obama’s more leftist one.  McCain, in attacking Obama’s “socialist” tendencies, is taking a page out of the Clinton playbook.

Other voters worry about Obama’s lack of national experience – he has spent only two years in national politics, compared to McCain’s more than two decades working in Congress.

In short, many Americans oppose Obama not because of race, but because of his ideology and lack of experience. In a head-to-head matchup with McCain, they would not vote for Obama even if he was white.

When polled, then, these voters face a dilemma:  they understand that a vote for an African-American candidate is in a larger sense a vote signaling that this nation has irrevocably turned a corner on race relations.  Obama’s election would demonstrate that Americans have taken a giant step toward being a truly color-blind society – an ideal shared by almost all voters. They see this hope on a daily basis, in media reports of record registration especially among young voters,  long polling lines for early voting, and predictions of huge turnout on Election Day, most of it presumably fueled by the voter interest in Obama’s candidacy.  This election, they are told again and again, will finally demonstrate whether America has shed its racist past.

But this same subset of conflicted voters doesn’t support Obama – not as a presidential candidate.  They support John McCain. Not because they are racist, but because McCain is closer to their political views, because he has more national experience, because they admire his war record.  In short, they are “natural” McCain supporters.

My concern is that this subset of voters wants to do the “right thing”; when asked, they feel the need to respond in way that signals they are decent people who appreciate the historical import of the Obama candidacy.  This perfectly human need to be part of a larger movement leads them to tell pollsters they are undecided, or even that they will support Obama, when in fact they have no intention of doing so.

That’s how the polls could be overstating Obama’s support. It’s not the Bradley effect. It’s the Sally Field effect.  Most Americans aren’t racist. But they are human.

Now, do I see any evidence that this is occurring, and in what proportions?  One indicator might be if there were many more undecideds at this point in the election than at a similar time in 2004.  A quick comparison of the polls in both years indicates that the number of undecideds is no higher this year. On the other hand, if the Sally Field effect is in play in a way that leads voters to overstate their support for Obama, it won’t be obvious until Tuesday’s results are in.  If Obama loses, the media narrative will undoubtedly center on latent racism skewing polls.  In the absence of compelling evidence, you should view that claim with skepticism.  We all want to be liked.

How many Democrats? How many Republicans?

 

Many posts ago I told you that one indication of where an election is likely to go is revealed by the underlying partisan predispositions of the voting public. Rasmussen is one of the few pollsters that periodically polls the public regarding its partisan identification and weights their surveys accordingly. This is important because it allows us to see if the underlying partisan identification among voters is changing across a campaign. Today Rasmussen released their final partisan weighting before the election, and it remains almost unchanged from its previous weighting in October: 39.9% Democrat, 33.4% Republican and 26.76% unaffiliated. This represents an almost miniscule shift in the Republican direction in the last month, but does not eliminate the gains the Democrats made in partisan backing since the fiscal meltdown beginning in late September.  In mid-September, prior to the meltdown, Rasmussen’s surveys indicated 38.7% Democrat, 33.6% Republican and 27.7%. So since the fiscal meltdown, Democrats have gained 1%, Republicans have gained nothing and independents have lost a 1%.  In short, Republicans have lost ground within the electorate during the current campaign. The current partisan distribution is in stark contrast to 2004, when Rasmussen’s November weighting showed an almost evenly divided electorate, with 37.1% Republicans, 38.6% Democrats, and 24.3% independents.  This demonstrates the steep hill McCain has had to climb during this campaign.

So what explains the difference in the Rasmussen and Gallup tracking polls? Rasmussen gives Obama a 5% lead, but Gallup’s traditional likely voter model gives Obama a 10% lead.  What happened is that Gallup changed their estimate of the likely voter turnout on Election Day from 60% to 64%, in response to the early voting, and they are estimating that most of this increased turnout will come from Obama voters.  So it’s not that they are picking up any movement toward Obama – they have simply changed their model’s assumptions.  Rasmussen has not – so it appears they are contradicting one another.  But in fact both Rasmussen and Gallup have Obama getting 51-52% of the vote – they only differ regarding how much McCain will get.

When we look at the daily tracking polls, all of them now have Obama at 50% or higher except for IBD (Obama at 48%) and Zogby (Obama at 49%).  The average for Obama among the daily trackers, including Zogby and IBD, is 50%.  Where the polls differ is in the support for McCain, whose support ranges from 42 to 46%.  This means that McCain is not likely to win this even if he takes the undecideds 5 to 1 – he has to count on some slippage in Obama’s support.

To summarize, everything points to an Obama victory in the popular vote on Tuesday – unless there is some systematic bias in the polls that is overstating Obama’s support and/or understating McCain’s.  Tomorrow I’ll address the much discussed but little understood Bradley effect.  Are polls systematically overstating Obama’s support?

Answering Viewer Mail (and It’s About Time!)

It’s been a while since I’ve had an opportunity to react to the many excellent comments and questions you have been sending my way, so let me take the opportunity to do so now, before the storm hits.  I’ll work somewhat in reverse chronological order…

Conor Shaw suggests that Obama’s ability to turn so many traditionally red states into pink, tossup or even blue states vindicates his decision to run a 50-state campaign.  It is true that states that have not voted Democratic in several elections, such as Virginia, Indiana, North Carolina and possibly even Georgia, are now in play and several of these (Virginia the most obvious) may in fact go Democratic.  But as I noted in my post regarding Arizona, Obama is leaving himself open to criticism if somehow McCain holds on by a narrow margin in enough battleground states to eke out an Electoral College vote win. Then the critics will be wondering why Obama didn’t concentrate his resources to seal the deal.  Andrew Piccirillo also weighed in on this topic, arguing that Obama believes his election prospects improved by expanding the playing field, in part because it forced McCain to spread his more limited resources across a larger area than he might like.  I guess we will be able to judge the effectiveness of this strategy on Tuesday.

Conor also suggests that the race would be closer if Clinton had been the Democratic nominee.  Again, we will never know. But I suspect that she would have fared better in the big ticket rust-belt states of Ohio, Michigan and Pennsylvania and in the Appalachian areas including West Virginia, as well as Florida.  She likely would have done worse in North Carolina, Georgia and perhaps Virginia.  Keep in mind that she outpolled Obama in the primary states during the nominating phase (depending on how you count Florida and Michigan).  So I’m not sure I agree she would have been the weaker candidate.  In fact, I could make a strong case that she would have done better among the very voters that are now undecided.

Jane asks what the minimum gain in the popular vote McCain needs in order to plausibly have a shot in the Electoral College.  Note that at this point he could hold onto North Carolina, Florida, Indiana and Georgia without much increase in his current level of popular support. But to keep Ohio and Virginia, and flip Pennsylvania and then win either Colorado or New Hampshire, he needs to net a minimum of 5% nationally in the next 72 hours in my view.  I don’t see it happening.

Marshall suggests that the undecideds may consist of a large portion of voters who are actually going to vote for McCain, but who are hiding their preference by characterizing themselves as undecideds in order not to be labeled racist.  This gets to the heart of the discussion on the recent posts regarding how the undecideds will go.  I think the majority will go to McCain, and that may indeed swing several battleground states to him.  At this point, however, I don’t think it will be enough to give McCain the election.  Tomorrow I hope to devote an extended post to this idea.

Olivier and Rob both think I underestimate the impact of the Powell endorsement because even if it did not persuade voters to vote for Obama, it certainly dominated the news coverage for 48 hours, or more, thus preventing more favorable (to McCain) stories from getting aired.  I don’t disagree that, all things being equal, McCain would rather not have had the Powell endorsement dominating the news at that point. My objection, however, was to the media characterization that this endorsement “dealt a body blow” to McCain’s candidacy.  The general thrust of the media coverage was to suggest that this endorsement might seal the deal for Obama.  I should add here that Olivier in particular has pushed me to more explicitly recognize the importance of a candidate’s winning the media cycle in terms of the opportunity costs it imposes on the opponent.  In part, our differences are ones of perspective driven by our professions – I’m more interested in the larger picture, while Olivier reacts more acutely to the daily narrative.  I think we agree on more than we disagree here, but I hope readers of this blog appreciate the exchange between us as much as I do.

Bhima argues that because the undecideds are less engaged and informed about the election, the fact that they may swing the election as a group indicates a weakness in how we choose the president. Andrew disagrees, arguing that undecided voters are not necessarily ignorant voters (and I apologize to Andrew if his post was somehow deleted – I remember reading it but can’t find it now).  I would note that the number of undecideds has dwindled to about 5-6%, and I expect it to be about 3% on election day.  I’m not sure if Bhima would find this reassuring or not, but I guess I lean toward Andrew on this point.  Again, I hope to do an extended post on this question tomorrow.

Finally, Jack Goodman wonders whether the relatively “smoother” polling graphs for many of the red states (and the occasional blue state like Vermont) suggests that voters in these states are simply more closeminded.  I would suggest not.  Instead, I think it simply reflects the fact that neither candidate has bothered to campaign in strongly red or blue states, and thus voters aren’t really given a choice between the two.  How many times have we seen Obama, McCain, Biden or Palin in Vermont?  Without visible signs of debate, media coverage of competing arguments, etc., opinions are less likely to change.  That’s why these states are less frequently polled as well.

As always, these are great questions and comments, and I only wish I had time to answer them all at the time they are asked.  Keep them coming, and I’ll do my best.