Another nice teaching moment, this one courtesy of Jay Cost at the Horserace blog and Mark Blumenthal at Pollster.com. Gallup released their latest generic ballot results under the headlines “Democrats Jump Into Six Point Lead on the Generic Ballot”.
As the following table shows, this jump comes after many months of polling data in which Democrats either trailed or were tied (within the poll’s margin of errors) with Republicans in the generic question.
As I’ve blogged about before, the generic ballot question is a useful predictor of the number of House seats won by each party in midterm elections, so people pay attention to these results. (Note, however, that Gallup is not yet using their likely voter screen – when they do, history suggests Republicans will gain about 5% over the results of their polling of registered voters.)
Not surprisingly, progressive bloggers like Tom Schaller at Fivethirtyeight.com and Andrew Sullivan – while issuing the usual caveats (it’s only one data point, it could be an outlier, etc.) – jumped on these latest results as a possible sign that Democrats were regaining support among voters. Sullivan opined: “It is unwise to discount the intelligence of the American people – a trait more endemic among liberals than conservatives. The latest Gallup generic poll is striking – because it suggests that voters in the end may vote on substance not spin and ideology.”
But what substantive issue explains the reversal in voter sentiment– was it the financial legislation that just passed Congress? Voter backlash to the Republican handling (see Joe Barton) of the Gulf oil spill? Belated recognition that the health care legislation is a good thing? Growing disgust with Republicans as the “party of no”?
How about none of the above? Yes, the six-point Democrat advantage lies outside the poll’s margin of error. But as Blumenthal reminds us, Gallup’s results are based on probability sampling – remember, each Gallup poll is followed by some version of this methodological blurb: “one can say with 95% confidence that the maximum margin of sampling error is ±3 percentage points”.
What does this mean? Essentially, if Gallup repeatedly sampled the population of registered voters, approximately 95% of the time the sample results would fall within 3 percentage points (plus or minus) of the actual proportion of Democrat and Republicans supporters in the generic ballot question. Put another way, 5% of the time the Gallup poll results might fall more than 3 percentage points from the actual proportion of Republican and Democrat supporters. That means we can expect, very rarely, a result that lies outside the margin of error even if there’s been no actual opinion change.
Now, we can’t know what that actual number of Republican and Democrat supporters is – Gallup can only estimate it by taking a random sample of all registered voters. Let’s assume, however, that the actual support for the Democrat on the generic ballot question is pretty close to the average Democratic support among registered voters, as polled by Gallup for the last 20 weeks. That average number is 45.6%. In that 20-week period, Blumenthal finds only one poll that lies more than 3% (plus or minus) from this average – and that is the most recent poll showing 49% support for Democrats in the generic ballot. Note that this most recent result lies is just barely (.4%) outside the margin of error.
Bottom line? I can understand why Sullivan equates wisdom with support for liberal policies, and why his world view might color his interpretation of the polling results. But, as always, we need to separate our personal preferences from our analysis of the facts. It’s possible Democrats are picking up support among voters as we move closer to the midterm. It’s also possible, however, that this is a perfectly predictable statistical fluctuation associated with polling based on random sampling, and that there’s been no real change in voter sentiment at all. We won’t know for certain until several more polls are in. But I would be cautious about discarding 20 weeks of results on the basis of a single poll.
In the meantime, skepticism rather than certitude should be your watchword.
Addendum: I’ve tweaked the wording of my original post in order to better describe the meaning of the phrase “margin of error” in the context of probability sampling. The substance of the post – that we can’t be sure this most recent result indicates a real shift in voter sentiment – hasn’t changed.
Matthew – It looks like you were right. See Charlie Cook’s column –
http://www.nationaljournal.com/njonline/po_20100803_8293.php
Perhaps you should have waited until Gallup came out with their polling numbers on August 2. That poll revealed the fact that Republicans jumped out to a 48-43 lead and that the “enthusiasm gap” between Republican and Democratic voters has widened to an all time high. Read and weep! Already, while Democratic voters hold an advantage in shear numbers over Republican voters, Republican voters typically vote in much large numbers. That being said, the same Gallup poll showed that 44% of Republican voters stated that they were “very enthusiastic” about voting this November, versus only 22% of the Democratic voters who said the same. That my friend, represents an incredible 2 to 1 margin; of which is the largest gap between Republican and Democratic voters ever recorded by Gallup – and by a rather significant amount. Pew research, did a similar poll on July 1, with numbers largely duplicating those being recorded by Gallup; and was also an all time record for Pew. You can cut your numbers as you may, but the results are nonetheless going to be what they are. It’s becoming more and more clear by the day that (1) the Republicans will take both the House and Senate in November and (2) Barack Obama will most definitely be BADLY beaten in 2012 – irrespective of who the Republican candidate is. Even an improved economy won’t help. Obama will ultimately go down in history as one of the worst, if not THE worst President that this country has ever had. Mark my word!!!
Al – Without passing judgment on your claim that “Obama will ultimately go down in history as one of the worst if not THE worst President”, I suggest you go back and carefully read my entire post. If you do, you will see that I suggested that the recent polls that some interpreted as indicating a Democratic surge in fact suggested no such thing. Instead, I pointed out that they were far more likely to be a function of the variability built into random sampling rather than reflecting any actual change in voter sentiments.
I do appreciate your enthusiasm for the Republicans’ chances come November. In a future post I will assess whether the data indicates that your enthusiasm is grounded in fact, or is simply the longings of a Republican-leaning voter.