Nate Silver’s new 538 website opened last week to generally lackluster reviews. “It just isn’t working,” according to Tyler Cowens. Paul Krugman agrees: “I’m sorry to say that I had the same reaction. Here’s hoping that Nate Silver and company up their game, soon.” Ryan Cooper is more blunt: “To summarize: it’s terrible.” What do all these critics find so objectionable?
One major criticism is that the problems and related questions that dominate the news, particularly in the political arena, are not always amenable to the type of unrelenting statistical analysis that Silver and his minions emphasize. As the New Republic’s Leon Wieseltier puts it, “Many of the issues that we debate are not issues of fact but issues of value. There is no numerical answer to the question of whether men should be allowed to marry men, and the question of whether the government should help the weak, and the question of whether we should intervene against genocide.”
Related to this is a fear that Silver’s claim that he just does analysis, and not advocacy, masks the truth that his punditry is no more bias free than that of any other pundit. As Cooper puts it, “In an attempt to focus solely on objective analysis, Silver is ignoring one of the hardest-won journalistic lessons of the last decade — there is no such thing as ideology-free journalism.”
I confess that I find both objections less than compelling. To begin, no political scientist that I know would disagree with Wieseltier’s observation regarding the fact-value distinction. Indeed, in his classic study of decisionmaking in organizations, the late, great Herbert Simon (he won a Nobel Prize for his study of decisionmaking) observed, “It is a fundamental premise of this study that ethical terms are not completely reducible to factual terms.” But, Simon cautioned, to assert that there may be an ethical component to an administrator’s decisions is not to say that those decisions involve only ethical elements. Put another way, political scientists may not be able to tell voters whether electing Barack Obama over Mitt Romney is a better (or worse) outcome for the nation, or the world. But we can say something about what the determinants of the presidential vote are likely to be, and what election outcome it is likely to produce. If I understand Silver correctly, that’s all he and his team are trying to do at this new website. Should government help the weak? I doubt Silver knows the answer. But he might be able to tell us if government can help the weak.
Similarly, I doubt that Silver believes he views his analyses through a value-free ideological lens. Anyone who read his FiveThirtyEight column at the New York Times understands what Silver’s political views are. But rather than compensating for one’s implicit biases, political or otherwise, by – as Cooper advocates – “wear[ing] your ideology on your sleeve” I’d argue instead that there is merit in trying very hard to prevent those biases from contaminating one’s analysis. One way to do so is to be explicit about the theoretical and methodological assumptions built into one’s analysis. This approach differs from, and is more useful, than analysis that is explicitly harnessed to the cause of advocacy. There is something to be said for disciplined thinking designed to discover underlying truths, no matter how inconvenient. And that means not only clarifying the assumptions built into one’s analysis – it also means trying to specify how certain one is about one’s conclusions. How confident am I in my prediction that Obama will beat Romney?
It is on this last point, I think, that I come closest to agreeing with Silver’s critics like Krugman, who worry that without explicit theorizing, Silver’s data-driven research may tell us less than we think. As Krugman writes: “But you can’t be an effective fox just by letting the data speak for itself — because it never does. You use data to inform your analysis, you let it tell you that your pet hypothesis is wrong, but data are never a substitute for hard thinking. If you think the data are speaking for themselves, what you’re really doing is implicit theorizing, which is a really bad idea (because you can’t test your assumptions if you don’t even know what you’re assuming.)”
Krugman echoes a point I’ve made before about Silver’s work: that unlike political scientists, he is not fully transparent about what goes into his analyses, such as his presidential forecast models. Without a glimpse at the moving parts, we can’t be sure what we are learning. It is one thing to say that Obama will win the Electoral College vote, but we need a theory to understand why he won that vote. In Silver’s defense, however, he is not pretending to be a political scientist – and why should he? He has a wider audience and (presumably) earns more money, than any political scientist I know. If he wants to hide the ingredients that go into his “special” forecast brew in order to make it appear more satisfying (and more original!), I say more power to him. As a career move, it certainly has served him well to this point.
I confess that I believe some of the carping by pundits regarding Silver’s website is a reaction to his contrarian and what some perceive to be condescending attitude toward the work of mainstream journalists whose writings grace the major newspaper op-ed pages. In that vein, here Silver is describing how his new site differs from the work of the best known columnists: “Uhhhh, you know … the op-ed columnists at the New York Times, Washington Post, and Wall Street Journal are probably the most hedgehog-like people. They don’t permit a lot of complexity in their thinking. They pull threads together from very weak evidence and draw grand conclusions based on them. They’re ironically very predictable from week to week. If you know the subject that Thomas Friedman or whatever is writing about, you don’t have to read the column. You can kind of auto-script it, basically.”
It is true that Silver has always presented his work with a certain “Look Ma, no hands!” flair that in some cases overstates the novelty, and effectiveness, of what he is doing. (See, for instance, how his model stacks up to political scientists’ when it comes to forecasting the last midterm elections.) He has made a living by accentuating – exaggerating? – the difference between his data-driven analyses and what he sees as the hedgehog-like tendencies of more conventional columnists. I understand why columnists resent Silver’s tone. (I confess that my tone when criticizing pundits has sometimes crossed that line as well!) But I also think that he’s not entirely incorrect in his criticisms of conventional punditry. Too often it does harness data – if it uses data at all – to the cause of advocacy.
The bottom line? If you want a data-based take on the likely outcome of political events, like the upcoming midterm elections, Silver’s site is probably as good as any. (And here I would recommend the work of Harry Enten at Silver’s site). But if you want to understand why those outcomes occur, there are better places to start.
Of course, if you want your political analysis spiced up with a dose of the plucky, in-your-face speak-truth-to-power attitude exemplified by this young analyst, then you’ve come to the right place:
