Can We Trust the Gallup Generic Ballot Results?

At this stage in the electoral cycle, the fundamentals that drive the midterm vote – unemployment, real disposable income, attitudes toward Obama – are for the most part fixed, barring a major event. The key question now becomes who will show up at the polls?  As the latest Gallup generic ballot results indicate, turnout is the key to whether on Nov. 2 Democrats lose an average number of seats for the post-World War II era – say about 30 – but retain control of the House, or lose 40 seats or more and are swept out of power in a tidal wave of voter anger.

Let’s look at the latest generic ballot results from Gallup.

If we look only at the generic ballot results of registered voters, we see Republicans ahead by 5%.  That translates into a projected Republican gain of perhaps 40-45 seats (depending on the forecast model), enough to regain the House if everything goes right.  But now look at the results of a survey of likely voters, as opposed to registered voters.  Assuming 40% turnout, which is about par for the course for midterm elections in the post-World War II era, Republicans have a whopping 17% lead on the generic ballot.  That translates into a wave election come Nov. 2, with Republicans picking up 65 or more seats, according to some forecast models. This would be a stunning repudiation of Democratic control of Congress. (Note that Gallup estimates whether one is a “likely voter” based on answers to seven questions that have proved reliable predictors in the past.) If Gallup’s likely voter numbers are correct, it’s 1994 all over again – or worse – for Democrats.  The question is: can we believe Gallup’s numbers? Alan Abramowitz, for one, does not. To begin, Abramowitz believes Gallup’s proportion of Republican to Democrats (55% to 40%, including leaners) in their likely voter, low turnout model, overstates likely Republican turnout. He is equally critical of the predicted proportion of turnout among those age 65 or older (27%) and the proportion of young voters under age 30 (only 7%).  He also questions Gallup’s results for nonwhites: “Among nonwhites other than blacks, a group that comprises about 13% of likely voters, a generic Republican is leading a generic Democrat by 10 points, 52% to 42%. That’s a group that voted Democratic by a 2-1 margin in the 2006 midterm election. Moreover, it’s a group that has never given a majority of its vote to Republican candidates for Congress in any election since the advent of exit polling. According to the 2006 exit poll results, about two-thirds of these “other nonwhite” voters are Latinos. How plausible is it that at a time when the Republican Party is closely associated with stridently anti-immigrant policies that Latino voters are moving in droves toward Republican candidates? Not plausible at all… .”

Abramowitz implies that the only value of putting out these numbers is to change the campaign narrative and, presumably, dampen Democratic turnout – a serious charge.  In his words: “But what is the value of putting out results that defy logic but which can influence perceptions of the current electoral climate among political elites as well as the public?”

What are we to make of Abramowitz’ criticisms?  To begin, Gallup is not suggesting Latinos are moving in droves to Republicans – only that turnout will be heavier among Republican-leaning voters.  More generally, note that political scientists (including myself) are an inherently conservative lot – we tend to believe that future events will unfold largely as past events did.  When they don’t – and 1994 is a case in point – we tend to get caught flat-footed.  No political scientist that I know of predicted the Republican sweep in 1994.  In this vein, Abramowitz discounts Gallup’s results because they are inconsistent with turnout among subgroups in previous elections.  But what if this election is in fact a “wave” election, in which turnout disproportionately favors one group?  Gallup, relying on its likely voter model, is suggesting precisely that – the enthusiasm gap between Republican and Democrats is going to skew turnout in ways that will produce unusual voting patterns.

Note also that the margin of error for Gallup’s results increases when we focus on subgroups within the sample.  If we keep this in mind, Gallup’s likely voter model for subgroups – while historically unusual – is not as outlandish as Abramowitz suggests.  For example, the demographic profile of the 2010 electorate as posited by Gallup is not much different from previous midterm profiles; it is slightly less white, slightly older and slightly more educated than a comparable profile of the 1994 electorate which was the last “wave” election.  The proportion of Republicans, Democrats, and Independents in the 2010 survey is almost identical to 1994’s partisan breakdown.  The big difference is that Gallup sees today’s likely voters as more conservative than in 1994 (54-40%) but also – and Abramowitz doesn’t mention this – more liberal than in 1994 (18% to 12%).  In other words, the big difference between 1994 and today is a drop in the number of likely voters who call themselves “moderates” from 48% to only 27%.

I understand Abramowitz’s concern with Gallup’s number.  But I do not believe they “defy logic” – in fact, they are plausible if we are on the brink of another wave election in which turnout is disproportionately greater among older, better educated and conservative voters (Tea Party anyone?), even accounting for the growth of nonwhites in the electorate as a whole since 1994.  Gallup’s likely voter model could be wrong.  But we need to be careful about jettisoning a methodology that has proved reliable in the past just because this time around it is provided results that seem historically atypical (or because we don’t like the outcome!)

A final thought. Gallup’s generic ballot numbers have been stable for three weeks, but that does not mean they can’t change in the remaining two weeks. Gallup estimates that there is still a block of voters – about 5% – who remain undecided. If we look at previous midterm election cycles dating back to 1994, there have been instances in which the results changed considerably in the last two weeks.  For example, in 2002, Republicans were trailing the Democrats in the Oct. 20 generic ballot results by 3%, but in the final poll they led by 6%, a net pick up of 9% points. They ended up gaining 6 seats in the House that fall.  In 2006, Democrats’ lead in the poll dropped 6%, from 13% to 7%, in the same time period, but they still gained 30 House seats.

In the end, it’s going to come down to turnout.  If Gallup’s likely voter numbers hold, it will be 1994 all over again.


  1. I agree that polling has proved to be quite accurate in past election cycles (your pre-election analysis of Scott Brown’s improbable victory comes to mind). However, Jess and I have long been interested in the question of whether pollsters miss young people who do not own landlines? The Economist just ran a piece to this effect, claiming that pollsters were nervous about their predictive accuracy due to the fact that “cell phone onlys” (CPOs) now make up 25% of the electorate. As we know, this younger demographic leans heavily Democratic. Do you think there is any chance that this skews the data? Would this sampling bias also carry over to impact the inclusion of minorities?

  2. Orion – Great question. The short answer is that yes, there’s a real chance the cell phone only users can skew the polling data. Pew put out a study recently showing that there is a definite political bias among cell phone users – they tend to be younger, better educated and more likely, as you indicate, to vote Democrat. Note that automated pollsters, like Rasmussen, don’t sample cell phone users at all. Gallup, in contrast, does include cell phone users in their surveys. Whether they are getting the correct weighting is hard to know. Interestingly, however, it is possible to get a reliable survey sample even without including cell phone users. How, you ask? Since this question comes up a lot, I should probably devote a separate post to it.

Leave a Reply

Your email address will not be published.