Tag Archives: generic ballot

Predictions, Predictions: Congress and the Courts

How reliable is the generic ballot survey question in helping forecast the size of the Democratic losses that will occur in November?  As we move closer to November 2, a growing proportion of my blog posts will undoubtedly focus on the midterm elections.    Many of these posts will likely mention the generic ballot question that is asked by a number of polling firms.  As most of you know, this question typically take some version of the following form: “Looking ahead to the Congressional elections in November, which party do you plan to vote for if the election were being held today?”  The survey results, as many of you have heard me say, are actually a useful indicator of the likely outcome of the November midterm election.

But how useful? In a recent blog post, Nate Silver at fivethirtyeight.com raised important questions about the utility of the generic ballot question results. Silver writes: “It might be the case that the generic ballot is fairly stable, but that doesn’t necessarily mean it’s all that useful an indicator. In addition to the fact that the consensus of polls (however careful we are about calibrating it) might be off in one or the other direction, there’s also the fact that the thing which the generic ballot is ostensibly trying to predict — the national House popular vote — is relatively irrelevant to the disposition of the chamber, or the number of seats that each party earns.”

Political scientist Alan Abramowitz, (via Brendan Nyhan) takes issue with Silver’s comments, calling portions of them “pretty silly.”  I wouldn’t characterize Silver’s comments as silly, but I certainly disagree with portions of his post, particularly his claim that the national House popular vote is “relatively irrelevant to the …number of seats each party earns.”  Although it is not a perfectly precise indicator of how many seats a particular party will win (or lose) come November, it does provide a useful approximation.  Similarly, the generic ballot question does help us predict, within a margin of error, what the likely popular vote will be come November. To be sure, in some midterms it has proved more useful than others.  But it is not irrelevant.

The bottom line, then, is that the generic ballot results, properly understood, helps us predict the November midterm results. To see why, it helps to understand how political scientists factor the results of this question into their midterm forecast model.

When a generic ballot survey comes back with results showing, as the most recent Gallup survey poll does, that Republicans are favored over Democrats by 7%,  50%-43%, that doesn’t necessarily mean Republicans will win 7% more congressional seats.

Indeed, as Silver correctly points out, the generic ballot result can’t even tell us how many overall votes each party will get in the midterm, never mind how they will do in each of the 435 districts.  But political scientists understand this.  As it turns out, they don’t necessarily have to perform a district-by-district level analysis to get a decent handle on the total number of seats each party will win.  Instead political scientists can use the generic ballot results as one factor in statistical models that also take into account the existing political context as well as “structural” attributes associated with a midterm election.  So, for example, Abramowitz’s midterm forecast model uses only four variables to predict the midterm outcome, as measured by seats lost/gained:  the results of the generic ballot question, presidential popularity (as measured by the Gallup poll), the number of seats held by Republicans going into the midterm election, and a simple indicator variable that signals this is a midterm election, rather than a presidential one.  (This last variable is important because the president’s party typically loses seats in the midterm election.)

How do we know that the Abramowitz model (or others like it) is reliable?  Because it is constructed using previous midterm election results.  In effect, by looking at previous midterms, Abramowitz can build an overall “generic” statistical model that says, on average, how important each of these four variables is in determining the midterm election results.  Then, by plugging the current values for each variable into the existing model, he comes up with a forecast.  Political scientists have constructed different forecast models, but they typically all include some mix of variables that measure voters’ partisan sentiment, structural attributes associated with the midterm, and some indicator of the environment (political and otherwise) in which the election is being held.

Now, these models aren’t completely foolproof.   They are predicated on the assumption that the factors influencing the current midterm will behave pretty much as they have in the past. Say something unexpected happens – terrorists attack the World Trade Center, for example.  That may unexpectedly distort the relative importance of some variables, thus throwing the forecasts slightly off.  Moreover, even without unexpected events, there is always some uncertainty involved with these forecasts – some unexplained variance for which the model cannot account.

But it would be wrong to suggest, as Silver does, that the generic ballot question is “irrelevant.”  In fact, it is a highly relevant and useful predictor of the midterm outcome – as long as it is evaluated within an overall understanding of the factors that drive midterm elections.  And right now, it favors the Republicans by 7% – not a good sign for Democrats at all.

I will have much more to say about the generic ballot in the next several posts.  Before doing so, however, there’s another prediction that I need to discuss: Elena Kagan’s Senate confirmation vote to the U.S. Supreme Court.  Long-time readers will recall that in an earlier post I set the over/under for the Kagan no votes at 35.  In a sign that my predictive powers may be slipping (remember that I nailed Sotomayor’s exact confirmation vote) Kagan was confirmed with 37 votes in opposition, so I was two votes off.  (If you must know, it was Nelson and Brown). Will Loubier, on the other hand, hit the final vote margin squarely on the head, thus winning a Presidential Power “It’s the Fundamentals, Stupid!” t-shirt.   Here’s Will, looking justly proud of his prognosticative abilities:

For the envious among you, note that I’ll be giving away another t-shirt in the “predict the midterm outcome” sweepstakes.  Stay tuned!

Tuesday’s Results: Good News for Obama and Democrats?

I’ve been on deadline with a conference paper, so am commenting somewhat belatedly on last Tuesday’s election results, focusing on the highly publicized Senate primary races in Colorado and Connecticut.  As you know, in Colorado, incumbent Democrat Michael (“One T”) Bennet defeated challenger Andrew Romanoff by a relatively comfortable 54%-46%, a margin somewhat larger than what many pre-election forecasts suggested.  You will recall that Bennet was appointed to the Senate in 2009 to fill the seat of Ken Salazar, who became Obama’s Secretary of the Interior.  In the Republican primary, local District Attorney Ken (“One Tea Party”) Buck topped former Lieutenant Governor Jane Norton, 52-48.  Buck has Tea Party backing, while Norton was viewed more favorably by the Republican “establishment”.

In Connecticut, meanwhile, former WWE wrestling executive Linda McMahon won the Republican Senate primary over Representative Rob Simmons and the Tea Party candidate Peter Schiff.   McMahon now faces Democratic Attorney General Dick “I might have fought in Vietnam” Blumenthal in November to fill the seat of retiring Senator Chris Dodd.  Despite some biography-related gaffes, Blumenthal is up by double digits over McMahon in early polls, although his lead is shrinking.

For the most part, the mainstream media painted this as a good night for Democrats, and for President Obama – indeed that was the exact Politico headline in their coverage of the results: “Primary Night Yields Good News for President Obama and Democrats”.  A bit more cautiously, the New York Times headline reads, “Incumbent Backed by Obama Wins Colorado Primary” thus associating Obama with the outcome without necessarily crediting him for it.

Each story went on to suggest that the results indicated that the prevailing narrative, which shows Republicans making great gains come November, might need to be reconsidered. In Politico, John Harris writes, “Republicans, meanwhile, were left with several new reasons to wonder whether all the favorable national trends showing in the polls are enough to overcome local candidates who are inspiring little confidence about their readiness for the general election 12 weeks from now.” In the Times’ coverage of the results,  Kirk Johnson writes, “The predictions of doom for incumbents and establishment candidates this campaign season are proving to be more complex in the real world.”

Well, yes and no.  It is true that leading Democrats, most notably President Obama, as well as Vice President Joe Biden and others, campaigned heavily on Bennet’s behalf.  Had Bennet lost, the prevailing media narrative almost certainly would have been that still another candidate backed by Obama went down to defeat.  So, given the alternative, the Colorado outcome certainly was good news for Obama.  However, do the results “complicate” our understanding of midterm elections?  Do they potentially herald a strengthening of Democrat support heading into November?

No. In fact, the Colorado and Connecticut Senate primary outcomes do nothing to change my belief that the underlying electoral fundamentals are still pointing to strong Republican gains come November. Let’s start with Colorado.  Keep in mind that it highly unusual for candidate challenging an incumbent in a primary to win 46% of the vote.  This despite the fact that Bennet spent about $1.9 million on advertising to Romanoff’s roughly $757,000, much of it in a ten-day advertising blitz near the end of the campaign when polls indicated Romanoff might pull off the upset. All told, Bennet spent almost $6 million on the primary campaign, compared to less than $2 million for Romanoff.  Bennet officials admitted that they had not anticipated the need to spend this much on advertising but they took no chances when polling suggested they were in a tight race.  In fact, total spending in these Colorado primaries broke the state record.

Note also that Bennet is not running like a typical incumbent; despite his backing from the Democrat establishment in the primary, in his victory speech he immediately pivoted to portray himself as a political outsider not yet tainted by Washington politics.  When asked, he made no commitment to having the President come to Colorado to campaign for him in the general election. This despite that fact that polling data has Bennet in a dead heat with the Republican Buck.

Note also that although turnout was up in both party primaries, due to the contested races, it was up more in the Republican primary, where 45% of registered Republicans voted compared to about 40% of registered Democrats. According to state records, there are 802,000 registered Democrats in Colorado, 830,000 Republicans, and 730,000 independents.  Bottom line: it’s not clear to me that Bennet has a markedly better chance than did Romanoff to beat Buck.  Polls showed both Democrats in a dead heat with the Republican.

In Connecticut, meanwhile, analysts pointed to the fact that McMahon won less than 50% of the vote in a three-way race as evidence that she’s not a strong candidate.  While Blumenthal must certainly be considered the front-runner in this Democrat-leaning state, I would be cautious about adopting the interpretation that Tuesday’s results signify McMahon’s weakness.  First, in contrast to the Colorado races, turnout in the Connecticut Republican primary was quite low, in part because it was vacation time and also because McMahon already had the backing of the party establishment coming out of the Republican convention earlier this year.  Second, keep in mind that independents don’t vote in the Connecticut primary, and they constitute about 43% of Connecticut voters.  How many of them will support McMahon over Blumenthal?  Third, how likely is it that Simmons’ and Schiff’s supporters are going to vote for Blumenthal over McMahon? Fourth, Democrats are pouncing on McMahon’s wrestling background as a sign that she’s not electable.  Jesse Ventura anyone? Finally, keep in mind that McMahon has very deep pockets.  Given all these factors, I expect this race to tighten come November.

In short, you should not let the headlines from Tuesday’s races fool you into thinking the underlying electoral fundamentals suddenly shifted. I’ve just begun crunching some of the forecast numbers and will have much more to say about them in the coming months.  But the bottom line is that, unless these midterm Senate races are driven by an entirely different set of dynamics than previous midterms, the situation is still not good for Democrats.

As evidence, some of you may remember my warning about the recent results of Gallup’s generic ballot question which asks voters whether they will support the Republican or Democrat congressional candidate come November.  After weeks of favoring the Republicans, the results flipped during a two-week period in mid-July to put Democrats in the lead. Progressive bloggers hoped it heralded a reversal of political fortunes. In contrast, I cautioned (along with others like Mark Blumenthal) that the change might simply reflect the typical random variation associated with public opinion sampling.  Here’s the most recent Gallup results:

As you can see, Republicans are back in the lead, and by the largest margin (tied) held by either party since Gallup began tracking the generic ballot in March.  This does not mean that voters have suddenly switched back to supporting Republicans.  Instead, it is more likely that the underlying voting dynamics have remained remarkably stable and that July’s results were, as I suggested, simply random variation due to sampling.  Since March Republicans have been tied or ahead of Democrats in almost every poll except for that two-week period in July, and they continue to maintain an “enthusiasm” advantage in terms of their likelihood to vote in November.

Bottom line? Tuesday’s “good news” headlines notwithstanding, the election results in Colorado and Connecticut do not complicate or even change the political science narrative at all:  the fundamentals are no more favorable to Democrats or Obama today than they were last week.

The Latest Gallup Poll: Are Democrats Gaining Momentum?

Another nice teaching moment, this one courtesy of Jay Cost at the Horserace blog and Mark Blumenthal at Pollster.com. Gallup released their latest generic ballot results under the headlines “Democrats Jump Into Six Point Lead on the Generic Ballot”.  

As the following table shows, this jump comes after many months of polling data in which Democrats either trailed or were tied (within the poll’s margin of errors) with Republicans in the generic question.

As I’ve blogged about before, the generic ballot question is a useful predictor of the number of House seats won by each party in midterm elections, so people pay attention to these results.   (Note, however, that Gallup is not yet using their likely voter screen – when they do, history suggests Republicans will gain about 5% over the results of their polling of registered voters.)

Not surprisingly, progressive bloggers like Tom Schaller at Fivethirtyeight.com and Andrew Sullivan – while issuing the usual caveats (it’s only one data point, it could be an outlier, etc.)  – jumped on these latest results as a possible sign that Democrats were regaining support among voters. Sullivan opined: “It is unwise to discount the intelligence of the American people – a trait more endemic among liberals than conservatives. The latest Gallup generic poll is striking – because it suggests that voters in the end may vote on substance not spin and ideology.”

But what substantive issue explains the reversal in voter sentiment– was it the financial legislation that just passed Congress?  Voter backlash to the Republican handling (see Joe Barton) of the Gulf oil spill?  Belated recognition that the health care legislation is a good thing? Growing disgust with Republicans as the “party of  no”?

How about none of the above?  Yes, the six-point Democrat advantage lies outside the poll’s margin of error.  But as Blumenthal reminds us, Gallup’s results are based on probability sampling – remember, each Gallup poll is followed by some version of this methodological blurb:  “one can say with 95% confidence that the maximum margin of sampling error is ±3 percentage points”.

What does this mean?  Essentially, if Gallup repeatedly sampled the population of registered voters, approximately 95% of the time the sample results would fall within 3 percentage points (plus or minus)  of the actual proportion of Democrat and Republicans supporters in the generic ballot question.  Put another way, 5% of the time the Gallup poll results might fall more than 3 percentage points from the actual proportion of Republican and Democrat supporters.  That means we can expect, very rarely, a result that lies outside the margin of error even if there’s been no actual opinion change.

Now, we can’t know what that actual  number of Republican and Democrat supporters is  – Gallup can only estimate it by taking a random sample of all registered voters.  Let’s assume, however, that the actual support for the Democrat on the generic ballot question is pretty close to the average Democratic support among registered voters, as polled by Gallup for the last 20 weeks.  That average number is 45.6%.  In that 20-week period, Blumenthal finds only one poll that lies more than 3% (plus or minus) from this average – and that is the most recent poll showing 49% support for Democrats in the generic ballot.   Note that this most recent result lies is just barely (.4%) outside the margin of error.

Bottom line?  I can understand why Sullivan equates wisdom with support for liberal policies, and why his world view might color his interpretation of the polling results.  But, as always, we need to separate our personal preferences from our analysis of the facts.  It’s possible Democrats are picking up support among voters as we move closer to the midterm.  It’s also possible, however, that this is a perfectly predictable statistical fluctuation associated with polling based on random sampling, and that there’s been no real change in voter sentiment at all.  We won’t know for certain until several more polls are in.  But I would be cautious about discarding 20 weeks of results on the basis of a single poll.

In the meantime, skepticism rather than certitude should be your watchword.

Addendum: I’ve tweaked the wording of my original post in order to better describe the meaning of the phrase “margin of error” in the context of probability sampling.  The substance of the post – that we can’t be sure this most recent result indicates a real shift in voter sentiment – hasn’t changed.