On the eve of Election Day, I am a happy man. Why is this, you ask? Because the fundamentals-based forecasts issued by almost a dozen political scientists before Labor Day are – in the aggregate – looking remarkably prescient. The average prediction of those eleven models has Obama winning 50.3% of the two-party vote, while the median gives him 50.6%. So far, these forecasts seem to be holding up quite well, with both the RealClearPolitics and Pollster.com aggregate national poll showing this race, as measured by the popular vote, as essentially a dead heat, one day before the election. Score one for political science!
Of course, that doesn’t tell us who is going to win, which is what most of you want to know. Fear not! We need only consult the state-based forecasts issued by Drew Linzer, Sam Wang, Simon Jackman, and Tom Holbrook and Jay DeSart. (There are others out there, but these are the ones whose methods are most transparent, and with which I am most familiar. If you want a bit of background on their methods, see this article on “the rise of the quants” ). Although these forecast models differ in some of the particulars (whether to compensate for a pollster’s “house effect”, how to weight the state polls, the relative weight place on polls of likely vs. registered voters, etc.), they all operate on the same assumption: that state-based polls, taken in the aggregate, provide a very accurate indicator of who is going to win that state, particularly this late in the game. That, in turn, makes it relatively easy to put together an Electoral College forecast. All of them have done so, and as I’ve discussed in an earlier post, they all see it as more likely that Obama wins the Electoral College vote. This doesn’t mean they believe Romney can’t win – they just see it as less probable than an Obama victory.
The process by which these political scientists (Wang is actually a neuroscientist, but he gets honorary membership) put together their predictions is in stark contrast to the methods used by the traditional pundits. Consider this projection by Jay Cost, a very smart analyst who writes for the Weekly Standard. Cost believes Romney will win this election, and in explaining why, he took a shot at political science forecast models: “Both political science and the political polls too often imply a scientific precision that I no longer think actually exists in American politics. I have slowly learned that politics is a lot more art than science than I once believed. Accordingly, what follows is a prediction based on my interpretation of the lay of the land. I know others see it differently–and they could very well be right, and I could be wrong. I think Mitt Romney is likely to win next Tuesday.” As evidence for his prediction, Cost cites two points: Romney is leading among independents, and most voters think he will do a better job handling the economy.
Cost is not alone in thinking that Romney is going to win – there are some very smart people who have vast experience in electoral politics who agree with him. Here is a list of the most prominent political pundits, and their predictions. However, as I scan the list, I can’t help but notice that the bulk of people who agree with Cost in predicting a Romney victory are conservatives, including Karl Rove, Glenn Beck, Ari Fleischer, Jay Cost, Peggy Noonan and Dick Morris. On the other hand, many of the best-known liberal pundits – Markos Moulitsas, Jamelle Bouie, Jennifer Granholm, Donna Brazile and Cokie Roberts – think Obama will win. Now, all of them claim to be looking at the same data – the same polls, the same candidate strategies, the same advertisements, etc. How, then, can we explain why they end up with dramatically different predictions? More generally, why do liberals think Obama will win, and conservatives think Romney will?
The answer, I think, is that people – liberals, conservatives and everyone else – are very good at seeing patterns in data that suggest outcomes that conform to their preferences. Mind you – these aren’t implausible patterns – indeed, what makes them so seductive is that they are very plausible. Cost, for instance, is correct that most polls indicate that Romney is viewed as better able to handle the economy. But notice what he writes: “Poll after poll, I generally see the same thing. Romney has an edge on the economy. That includes most of the state polls.” At the same time, however, he evidently is discounting those same state polls that, looked at in the aggregate by political scientists, indicate that Obama is more likely to win the Electoral College. So, the question becomes: why value what the state polls say in one area – Romney’s handling of the economy – while discounting their overall projections that say Romney is more likely to lose?
The worry I have when analysts “interpret” data is that it leaves room for personal preferences to sneak in. Taken to an extreme, it leads to far-fetched inferences like this one tweeted earlier today by Peggy Noonan: “I suspect both Romney and Obama have a sense of what’s coming, and it’s part of why Romney looks so peaceful and Obama so roiled.” Really? She can see the election outcome by “reading” their faces? This presumes that both Obama and Romney know “what is coming” – highly unlikely in a 50/50 race- and that she has some method – a sixth sense? – for inferring when facial expressions reveal a person’s inner thoughts. Maybe she can see dead people too.
Ok. That was a cheap shot. Let me be clear. I think Noonan is a very smart person. Her memoir of her years as a Reagan speechwriter is one of the best accounts of life in the White House that I’ve ever read. But I don’t believe she can read faces.
And that leads me to my broader point. When I consider this latest election cycle, the most important development in how it has been covered, in my view, is the growing prominence of analysts whose methods are both more rigorous and more data-driven than what we are used to seeing from traditional “pundits”. I think we are witnessing a sea change in political analysis, one that will leave an indelible mark on future coverage of presidential elections. Increasingly, the traditional seat-of-the-pants, intuition-based method of analyzing elections is giving way to a less impressionistic mode of analysis. To be sure, these new methods are not infallible by any means. But they are a step forward. And political scientists are leading that movement.
To be fair to Cost, and Noonan, and all the rest of the “traditional” pundits, and the new ones too – they at least had the courage to put their professional reputations on the line and make a prediction. So I am going to do the same – tomorrow morning. I can tell you now – my prediction will be entirely atheoretical, and will be based on the latest state-based polling averages. But to make it interesting, I will make a wager: if my prediction regarding the winner tomorrow is incorrect, I will pay the bar bill (alcohol only) for everyone who attends the Election Night at the Grille, which Bert Johnson and I will be hosting. So keep your receipts! The festivities start at 7 p.m. and, as always, I’ll be living blogging the election returns while keeping the crowd at the Karl Rove Crossroads cafe – er, the Grille – entertained as well. For those in the area, I hope to see you tomorrow night. For the rest of you, please join me at this site. We are hoping to break our all time record for participation.
I’ll see you tomorrow.
Professor Dickinson, the internet is a funny thing (Thanks for inventing it, Al Gore!). Apparently this website exists: isnatesilverawitch.com – it should help with results.
Alas, unless I can see the analyst’s methodology, I don’t bother with looking at their predictions. This goes for witches too….
Saying that Cost and Noonan “put their professional reputations on the line” by making predictions implies that anyone in the media will remember what they’ve said, or hold them to it. Do you really think that, if Obama wins and proves her wrong, any media outlet will become less inclined to have Peggy Noonan on as a pundit?
As David Atkins noted yesterday:
“Of course, if the media functioned this way, it would have fired all the people who were wrong about the Iraq War, climate change, supply-side economics, austerity, and the rest. There is no accountability in the punditry business.”
Some others predicting a Romney/Ryan landslide are Newt Gingrich, Geroge Will. Also predicting a Romney win is my personal idol, Charles Krauthammer.
I wish I was in Midd for the free booze.
Professor, I have read through all these polls and I still cant figure one thing out and was hoping you could enlighten me. For Romney to win, the state polls have to be very skewed. And it seems that they could be skewed because they are “reweighted” to reflect a population that a pollster predicts. Everyone seems to gloss over this. My questions are why cant they get a large enough sample for a state poll so that it doesnt need to be reweighed. and secondly, how does each of these major pollsters come up with what their weights are and why are they so different? Maybe I am being simplistic but it appears that the results of the poll matter less than how much each polls reweights the population. If one guy reweights to D+1 and another does D+9, I already know what the results are going to say. Thanks
Sheldon – These are all smart guys. They may yet be right.
And I wouldn’t dream of challenging that wager…still, wish I could be at the election festivities. I’ll tune in from some DC establishment. My roommate and I have been in line for over an hour so far to vote.
Stuart – No, because she has great insights on a number of topics, and can speak from experience about working in the White House, speechwriting, etc. So I don’t think she should lose her job if she gets an election forecast wrong. AFter all, I’ve missed two of the last six (sort of)! On the other hand, I hope that four years from now, when she makes her next prediction, someone asks her how the last one turned out, and what she learned about it that is influencing her next prediction. It’s all about the learning process!
Tarsi – Vote early, vote often! It’s the American way…..look forward to your comments tonight.
Aren’t these conservative pundits making these random predictions to comfort their side.
Would Jay Cost EVER say “Romney is ahead in the polls but Obama is secretly winning.”
As has been said above, they are risking very little as they are not political scientists to begin with.
John – Great question. I can tell you that this issue is not being glossed over – in fact, it is at the heart of the debate between Romney supporters and Obama supporters regarding how best to interpret the polls. Romney supporters believe that many state polls are oversampling Democrats. But keep in mind that most of the major pollsters (but not all) do not fix the partisan composition of their likely voter samples. They do weight their sample to match broad demographic patterns in the U.S. Census (gender, race, etc.) and then select their “likely voters” based on a series of questions that have been reliable in the past. But most don’t adjust the sample to achieve a certain partisan division. The second point I would make is that the state level polls, while not perfect, have been generally reliable in the past. By reliable, they come within 1-2% of the final vote, in the aggregate. There’s no immediate reason I can think of why that should be different this year. Sure, it is possible that their likely voters screens are somehow missing a late surge in enthusiasm among likely Romney voters.
But I can’t simply jump to that conclusion without additional evidence.
Thanks–really appreciate the response. I guess what’s being glossed over is how the pollsters reweight. Why does Rasmussen and Gallup think it will be more even split while the other pollsters think it will be D+7 or higher.
Is that because they have much different samples or much different reweights? And within the reweights, i assume everyone has the same census data so the difference in reweights is just based on likely voters?
Matt,
It is very brave of you to “do the same (make a prediction)– tomorrow morning.” Can I make a Vegas bet that way? Or will you be analyzing the post-election “steal the election” maneuvering of the lawyers both sides have hired?
Dale – Believe it or not, that was written yesterday!
Hi John,
So, my understanding is that Rasmussen does use partisan weighting. I don’t think Gallup does, but I could be wrong. But Gallup may be weighting on demographic variables in a way that has partisan implications – say,if they think turnout among African-Americans will not be as high as some other pollsters do. But yes, likely voter screens can vary as well, leading some pollsters to through out some voters while others keep them in. This is a reminder that none of these pollsters are vary transparent regarding their methods, so it makes it harder to evaluate their results. Which is another reason that I shy away from relying on any single polling outfit.
I thought you might find this interesting… MSNBC’s Chuck Todd (hardly partisan) was on Jim and Margery yesterday morning on 96.9 FM Talk. He was asked what to think of Nate Silver’s 538 blog (and its bullish outlook on Obama’s chances). Todd replied that a lot of the state polls we have seen this year are generally garbage and noted, with reference to Silver’s blog (I paraphrase from here on out): When you take something that comes out of a donkey’s rear, and average that with something else that comes out of an elephant’s behind, the average of all that is still going to smell like you know what.
Anyways, he went on to say that the national polling is a better indicator of where this race is right now (essentially tied) and that the makeup of the turnout (specifically how many old white folks will turn out in Toledo, Ohio) will swing the result one way or another. Bottom line, should be an extremely long night…
It all comes down to whether the assumptions on weighting of D+7, +8,+9 used by most of the polls in the RCP group are right, or if it is D+3 (or less)in which case it will be President Romney on January 20, 2012.
Best Tweet of Day, goes to Tim Tebow, who tweeted “The Democrats will take an early lead, until the Republicans get off work.”
I think Sean Trende’s analysis of the divergence in national vs state polling is a good read too: http://www.realclearpolitics.com/articles/2012/11/06/2012_a_close_race_with_a_high_degree_of_uncertainty.html
What do you think about the problems with likely voter mis-reporting? (I like to call it lying.) I thought that most recent research on voter validation and intentions is that the actual voting sample was much more representative of the registered rather than likely voting population (and drifting slowly towards the entire population demographics). I’ve been thinking a lot about the implications of the Ansolabehere & Hersh and the Rogers & Aida papers…it would seem to suggest that when all is said and done, there will likely be a small but potentially significant systematic bias towards Obama. Probably not enough to flip NC, but probably enough to make a large difference in VA or FL. In fact, Romney more or less “caps out” nationally among registered (not likely) voters at, I’m not kidding, 47%.
More details here.
Matthew,
Thank you for mentioning my piece in your post! A few points of clarification:
I am not a “traditional pundit” insofar as I am quite familiar with the methodologies of the “quants,” and have written at length about why I think their approaches are lacking. Most recently, I wrote a post explaining how the polls of Ohio are non-normally distributed, thus creating potentially intractable difficulties in putting together a probability estimate from the polls (which appear to be bi-modal).
In fact, I started blogging in 2004 by doing a very version of what Silver, Linzer, etc. are doing — confidence intervals off state poll averages. It is quite easy to do, in fact. I do not do it anymore because I no longer think the approach is a good one.
In particular: one could very easily put together a statistically powerful predictive model built around the final Gallup poll of likely voters. It shows a Romney victory. Similarly, various predictive models built on fundamentals — e.g. Hibbs’s Bread & Peace — show a comfortable Romney win.
This gets to my fundamental methodological objection with the “quants,” which as I stated in the column you site, it creates a false sense of precision. One can use different variables to build a model that predicts the past with great accuracy AND points to dramatically different results today.
Regards,
Jay Cost
Jay,
Thanks for the response. I’ve followed you since you were at the RCP and am familiar with and admire your work. You are right to point out that you shouldn’t be grouped with the traditional pundits. If the world is still standing after tonight, I’d like to discuss in greater detail why you have soured on the the statistical analyses that many of my colleagues are increasingly turning to. Nate Silver and I had a good discussion on this topic earlier, and it would be great to get your thoughts.
Meanwhile – good luck tonight!
– Matt
Jay, you should go back to doing the stat-based approach. It seems to work better. Don’t be the Murray Chase of pollsters.