As the political pundits parse last night’s Republican debate – a topic I will tackle later – I want to return to a story that attracted quite a bit of media play earlier this week. Three days ago New Hampshire television station WMUR in conjunction with CNN released a poll that showed Hillary Clinton and Bernie Sanders in a “statistical tie” in New Hampshire. Not surprisingly, the poll generated quite a bit of media coverage, with The New York Times headline for its story on the poll proclaiming that Clinton and Sanders were in a “dead heat.” Other news outlets, citing the same poll, made similar claims. In fact, the survey, which was in the field during the last week of July, showed 42% percent of likely Democratic primary voters saying they will vote for Clinton, with 36% saying they are backing Sanders. How can the New Hampshire race be a “tied” when the poll shows Clinton with a 6% lead? The answer is that because the two candidates’ numbers fall within the poll’s sampling margin of error (a measure of how confident pollsters are in their results), one can’t discount the possibility that Sanders is actually tied, or perhaps even ahead, of Clinton. Remember, surveys are simply estimates of the sentiments of an underlying population – in this instance, likely Democratic primary voters in New Hampshire – and one’s confidence in the results depends in part on how many people are surveyed and what confidence level we are willing to accept in evaluating the results. In this case, the WMUR poll’s margin of error at the 95% confidence level for the Democratic nominating race is +/-5.9%. In describing the race as a “statistical tie”, then, the WMUR pollsters are acknowledging the possibility that despite Clinton’s 6% lead, Sanders’ actual support might be at the upper end of the margin of error, and Clinton’s at the lower end. (Of course, it’s possible their support lies outside the margin of error, but this is even more unlikely.) Hence, WMUR’s decision to label the race a “statistical tie.”
At the risk of nitpicking, however, I would argue that a “statistical tie” is not the equivalent of a “dead heat”, The Times’ headline notwithstanding. To understand why, one should also ask: what is the probability that a purely random sample of 274 likely Democratic voters (the size of the WMUR poll on the Democratic side) would show Clinton ahead by 6% if in fact there is no difference in polling support between Clinton and Sanders in the underlying population – that is, that they really are tied? It turns out that it is not very likely – in fact, a simple test of the difference in survey sample results suggests there is a less than 10% probability that the race is actually tied, given the survey findings showing Clinton ahead by 6% (and making certain other assumptions about how the WMUR poll was conducted.) So, it is true that we can’t be sure that Clinton is ahead, at least not using the conventional 95% uncertainty level. But it is much more likely, given these poll’s parameters, that she is leading Sanders than that they are in a true dead heat. My quibble with most of the media stories reporting the WMUR poll is that they don’t make the difference between a “statistical tie” and an actual tie very clear.
“Fine,” you respond. “At least I can take comfort in knowing that Bernie is closing the gap with Hillary.” And, in fact, the first line of The Times story notes that “Senator Bernie Sanders of Vermont continues to tighten the race with Hillary Rodham Clinton in New Hampshire, according to a poll released on Tuesday.” As evidence, the author notes that a previous WMUR/CNN poll of likely Democratic voters that was in the field from June 18 to 24 found Clinton leading Sanders by 43%-35% (with a margin of error of +/- 5.2%). Based on these two polls, then, it appears that Sanders has gained 2% on Clinton – evidence that, according to the Times, “Mr. Sanders continues to gain momentum after months of negative publicity about Mrs. Clinton’s use of a private email server as secretary of state.”
Well, maybe. Again, it is useful to put this claim in proper context too. If you parse the polling numbers, the one-month change from a 43%-35% Clinton lead in early July to 42%-36% margin this week comes out to approximately a handful of respondents expressing a preference for Sanders rather than Clinton this month compared to last month. Now, this might reflect an actual change in the sentiments of the underlying population. Or, it might just be the result of picking up a couple more Bernie supporters in the random sampling process, even though there’s been no actual change in voter preferences. The bottom line is that we should be cautious about extrapolating that Sanders is gaining on Clinton based solely on a 2% change in the WMUR polling results across a one-month period.
This doesn’t mean Bernie hasn’t gained ground on Hillary in New Hampshire. As this Pollster.com poll aggregation shows, if we widen our time horizon it’s clear that Bernie has closed the polling gap, particularly when Elizabeth Warren’s name was dropped from the survey options.
But there hasn’t been a lot of recent polls in New Hampshire. A NBC poll in late July had Clinton up by a larger margin, at 47%-34%, while a recently-released Gravis poll has it 43%-39% in Clinton’s favor. (I haven’t looked closely at the internals of either poll.) Right now the aggregate Pollster polling has Clinton up 43.3%-38.8%. RealClearPolitics, which uses a slightly different aggregating algorithm, has Clinton with a more substantial lead over Bernie, at 44.8%-31.6%.
The bottom line is that rather than a “dead heat”, Clinton is probably leading Bernie in New Hampshire, and that it is not even clear, despite an abundance of recent negative news coverage for Clinton and Bernie’s well-attended campaign events, that he has gained all that much ground over the last month. Alas, for a media with a vested interest in seeing a competitive race for the Democratic nomination, that narrative is probably a lot less interesting, even if it is likely to be a bit more accurate.