I want to comment briefly on the recent dustup in the blogging and polling world caused by DailyKos founder Markos Moulitsas’ accusation that “the weekly Research 2000 State of the Nation poll we ran the past year and a half was likely bunk.” Markos links to an independent analysis he sponsored that points to anomalies in the Research 2000 survey results and strongly suggests the results had to be manipulated. He goes on to say, “I hereby renounce any post we’ve written based exclusively on Research 2000 polling.”
Research 2000 had been hired by Kos more than a year ago to provide polling in races that weren’t often surveyed by mainstream pollsters, and to provide national tracking results as well. Some websites, like RealClearPolitics initially avoided using Research 2000 results presumably because they feared the results would be driven by DailyKos’ ideological leanings. Others, like TalkingPointsMemo and fivethirtyeight.com incorporated Research 2000 results into their composite tracking polls and other analyses. Presumably they will now drop Research 2000 from their websites until these charges are resolved.
I don’t know if Markos’ charges are true. We will certainly know more after other analysts begin sifting the polling results and the independent analysis. In any case, this is clearly going to result in lawsuits on both sides. Markos is going to charge Research 2000 with fraud, and the polling firm will likely countersue for libel.
Mark Blumenthal here and Charles Franklin here have good discussions of the controversy from a polling perspective. For my purposes, however, there is a broader point to be made – one that I’ve argued before when discussing opinionated blogging sites such as Markos’ DailyKos. Because they have such a strong world view, these are wonderful sites to visit when you need to commune with like-minded people. But you shouldn’t go there for objective analysis! They are churches that nurture the soul – they are not intended to give you an unvarnished take on the political world.
The problem on these sites becomes separating out spin, or opinion, from fact-based analysis. I don’t mind the opinion – it is often provocative and entertaining. But the Research 2000 polling results were presented as fact, not opinion. In acknowledging that the surveys that figured prominently on the DailyKos website for almost two years were possibly fraudulent, Markos’ defends himself by noting, “I want to feel stupid for being defrauded, but fact is Research 2000 had a good reputation in political circles.” He then lists some of Research 2000 clients.
This is a weak defense. The reality is that it was quite clear to anyone who had an open mind that Research 2000 survey results of individual races were often off the mark in a particular direction, and that their national tracking polls consistently showed higher approval ratings for Obama than did other polls. But there was no indication that this bothered anyone at the DailyKos website, because the results typically slanted toward what they wanted to believe! Now they are shocked – shocked! – that Research 2000 might have been skewing the polling results.
In looking over their results during the last year, I did not have access to the internal sampling data on which Research 2000 based its analyses. But it became clear to me, when I compared its results to multiple others polls, that they were often outliers in one direction and after a while I simply discounted their results. Consider, for example, the recent Arkansas Democrat primary between Blanche Lincoln and Bill Halter. Research 2000 posted the last survey there, and they had Halter up 4%. Here’s what I wrote on this blog on election night: “Finally, one thing to keep in mind when we look at the polling in Arkansas, which shows Lincoln losing to Halter: most of those polls are by Research 2000, a polling firm closely tied to the Daily Kos website which has come out strongly for Halter. Unfortunately, Research 2000 polls have been very inaccurate, in large part because their voter sample over represents younger voters.” Later that night, in analyzing returns, I wrote: “I just took another look at the final Research 2000 Arkansas poll – it had Halter up over Lincoln by 49-45%. With a 4% margin of error, and the additional bias built into the Research 2000 survey, I think that means Lincoln goes into this with a very slight lead.”
Lincoln, as you know, went on to win comfortably. And it appears that I may have been too charitable to Research2000, if Kos is to be believed – it wasn’t simply that they oversampled younger voters.
My point here is not to tout my own forecasting skills (anyone remember my Scott Brown-Martha Coakley prediction one week before that special election?) It is to remind you – particularly my students who frequent websites like Kos, or DailyDish, or Michele Malkin – not to confuse advocacy with analysis. The great danger of lurking on a site with a uniform perspective is the echo chamber effect; you begin to substitute the prevailing world view for fact-based analysis.
Let me be clear: no analyst is bias free. But some at least try to discipline their analysis by sticking with the facts and noting when their analysis strays from the data into the realm of conjecture. Kos and his followers are clear that their views skew Left. That’s fine. But I think that the strength and uniformity of the DailyKos world view made posters and readers there susceptible to accepting the data manipulation that they now accuse Research2000 of engaging in. They wanted to believe, and Research 2000 fed into those preconceptions.
And that’s my worry about the blogosphere: that students will gravitate to those sites that reinforce their ideological or political predispositions, and accept uncritically what passes for fact-based analysis there. It is important to maintain a healthy skepticism when entering these sites. Your role model should be Ronald Reagan.
Reagan, in presenting an arms agreement he had negotiated with Russian leader Mikhail Gorbachev, noted that he acted according to this Russian proverb, which he translated as, “Trust but verify.” Gorbachev responded, “You repeat that at every meeting!”
And with good reason.
I’m not saying you should not visit sites like DailyKos, or their counterparts on the Right. When you do, however, remember Reagan’s “Russian” proverb. Had Kos done so, he might not have found himself in the current predicament.