With less than two weeks remaining before the midterm election, polls continue to indicate that Republicans will make major gains in both the House and Senate. That has led some Democrat supporters (and others as well) to wonder whether the polls might be overstating Republican support. One potential source of error is a failure by pollsters to adequately account for cell phone only (CPO) households. A recent study released by Pew indicates that the CPO households have a distinct partisan bias; as the following table shows, landline-only surveys of likely voters tend to lean Republican by roughly 5% over surveys that include cell phones and landline households.
This has led some people to suggest that polls are overstating Republican support, particularly since some of the most prolific pollsters, such as Rasmussen, rely on automated survey methods that do not survey CPO households at all. (This is because of restrictions imposed under federal law that prohibit automatic dialing of cell phone numbers. By automated surveys, I mean surveys conducted by automated voice, rather than a real person conducting the interview.)
While the Democrat bias in CPO households is certainly very real, I don’t think Democrats should pin their hopes on the belief that surveys are systematically understating Democrat support. To begin, pollsters are very much aware of the CPO problem. Many major pollsters, such as Gallup, already include CPO households in their sample. But even those polling outfits such as Rasmussen who do not sample CPO may nonetheless provide accurate survey results. Consider a Pew study that compared the results of landline only with landline and CPO surveys conducted during the 2008 presidential race. As the following Pew chart shows (source here), the average error in the landline-only polls was no larger than those that included CPO in their surveys. Although the mean error in landline surveys slightly favored McCain, the difference in errors between combined CPO-landline surveys and landline-only surveys is statistically indistinguishable.
How can this be? The answer lies in how well pollsters conducting landline-only surveys weight their sample demographically. Simply put, as long as the pollster compensates by choosing landline voters who share the same demographic variables associated with CPO households that have similar political views, landline-only surveys will not be biased. That is, a properly weighted survey that relies on landline-only households will provide the same results as a survey that includes CPO households. What are these demographic variables? They may include income, marriage status, whether the respondents have children, or education, to name the most prominent.
Of course, with fully 25% of American households CPO, the process of weighting becomes a more difficult task than it was even two years ago. To achieve the proper weighting pollsters relying on landline-only surveys have to choose the right democraphic weights and they may be forced to over sample from some demographic subgroups in order to achieve a statistically significant subsample. This oversampling costs extra money, and some companies may be tempted to take shortcuts to avoid spending the money to oversample from the requisite groups. We won’t know until after the election whether and to what degree landline surveys understate Democratic support. But keep in mind that many of the most prominent pollsters, such as Gallup, are already including CPO’s households in their survey.
The bottom line? I think it is highly unlikely that there is an across-the-board systemic bias in the polling so far that is understating the level of support for Democrat candidates for Congress, even among automated pollsters such as Rasmussen. Democrats are in trouble, and it’s not because pollsters aren’t sampling CPO-only households.