I’ve made this point before, most recently during the 2008 presidential campaign when Silver’s forecast model, with its rapidly changing “win” probabilities, made it appear as if voters were altering their preferences on a weekly basis. This was nonsense, of course, which is why the political science forecast models issued around Labor Day proved generally accurate.
But in light of Silver’s column yesterday, it bears repeating: he’s not a political scientist. He’s an economist by training, but he’s really a weathercaster when it comes to predicting political outcomes. That is, he’s very adept at doing the equivalent of climbing to the top of Mt. Worth (a local skiing area for those not familiar with God’s Green Mountains), looking west toward Lake Champlain to see what the prevailing winds are carrying toward us, and issuing a weather bulletin for tomorrow. Mind you, this isn’t necessarily a knock on Silver’s work – he’s a damn good weathercaster. In 2008, his day—before election estimate came pretty close to nailing the Electoral College vote. More generally, at his best, he digs up intriguing data or uncovers interesting political patterns. At the same time, however, when it comes to his forecast models, he’s susceptible to the “Look Ma! No Hands!” approach in which he suggests the more numerous the variables in his model, the more effective it must be. In truth, as Sam Wang demonstrated in 2008, when his much simpler forecast model proved more accurate than Silver’s, parsimony can be a virtue when it comes to predictions.
Why do I bring this up now? Because, in the face of conflicting data, weathercasters can become unstrung if they are used to simply reporting the weather without possessing much of a grasp of basic meteorology. In yesterday’s column which the more cynical among us (who, moi?) might interpret as a classic CYA move, Silver raises a number of reasons why current forecasts (read: his!) might prove hopelessly wrong. Now, I applaud all efforts to specify the confidence interval surrounding a forecast. But the lack of logic underling Silver’s presentation reveals just how little theory goes into his predictions. For instance, he suggests the incumbent rule – which he has spent two years debunking – might actually come into play tomorrow. (The incumbent rule says, in effect, that in close races, almost all undecideds break for the challenger). Silver has provided data suggesting this rule didn’t apply in 2006 or 2008. You would think, therefore, that he doesn’t believe in the incumbent rule. Not so! He writes, “So, to cite the incumbent rule as a point of fact as wrong. As a theory, however — particularly one that applies to this election and not necessarily to others — perhaps it will turn out to have some legs.” Excuse me? Why, if there’s no factual basis for the incumbent rule, will it turn out to apply in this election?
The rest of the column rests on equally sketchy reasoning. Silver concludes by writing, “What we know, however, is that polls can sometimes miss pretty badly in either direction. Often, this is attributed to voters having made up (or changed) their minds at the last minute — but it’s more likely that the polls were wrong all along. These are some reasons they could be wrong in a way that underestimates how well Republicans will do. There are also, of course, a lot of reasons they could be underestimating Democrats; we’ll cover these in a separate piece.”
Let me get this straight: it’s possible the polls are underestimating the Republican support. Or, they might be underestimating Democrats’ support. I think this means if his forecast model proves incorrect, it’s because the polls “were wrong all along”. Really? Might it instead have something to do with his model? Come on Silver – man up! As it is, you already take the easy way out by issuing a forecast a day before the election, in contrast to the political scientists who put their reputations on the line by Labor Day. Do you believe in your model or not?
The bottom line: if you want to know tomorrow’s weather, a weathercaster is good enough. If you want to know what causes the weather, you might want to look elsewhere.