Who Will Win The Senate? A Primer on Midterm Forecasts

With the nominating phase of the congressional campaign just about over and the midterm elections less than four months away, you are going to see an increasing number of predictions, prognostications and more than a few statistically-driven forecast models purporting to tell us how the Republicans and Democrats are going to do in the House and Senate. Accordingly, I thought it might be useful to present a short primer on the types of forecasts you are likely to see, so that you can makes sense of the predictions.

Generally, you will encounter three types of forecasts. The first type are individual race-specific predictions made by the veteran handicappers like Charlie Cook, Stu Rothenberg and their associates. These predictions use a combination of on-the-ground reports, opinion polls and other bits of evidence to divide the field into safe, leaning and tossup (or their equivalent) races. With their descriptively detailed updates focused on the competitive races, these types of horse-race forecasts are in many respects the most interesting to follow, particular when they look at the high-profile Senate races. Right now, for example, Cook is predicting that the Republicans will gain between 4 and 6 Senate seats. Rothenberg puts the number at between 4 and 8. The Republicans, you will recall, need a minimum of 6 to regain a Senate majority so both handicappers see a Republican takeover as well within the realm of possibility. The implicit assumption in these models is that individual races can turn on factors idiosyncratic to that particular race, and thus the most accurate prediction depends on understanding these myriad influences.  In short, if we want to know which party is going to control the Senate, you need to build from the bottom up by aggregating the results of the individual races.

The second type of predictions are those produced by the structural forecast models developed by political scientists. In contrast to the handicappers like Cook and Rothenberg, these models eschew any interest in local detail in favor of macro-level factors, such as national economic growth, the president’s approval ratings and the number of exposed seats, to generate a prediction which is usually measured in terms of how many seats will be lost by the president’s party. The assumption built into such models is that fundamental national tides affect all races and thus all forecasters need to do is to measure those tides to generate an accurate prediction. To the best of my knowledge, Edward Tufte constructed the first such midterm House forecast model back in 1974 that was predicated on only two measures: the president’s approval and the annual growth in real disposal personal income per capita. In effect, he was modeling the outcome of the House midterm races – specifically, the share of the national popular vote the president’s party received – as referendum on the President’s performance.

Since Tufte’s pioneering effort political scientists have generated dozens of such forecast models, with most of them trying to predict the distribution of House seats between the two parties, rather than the overall vote share. Some early models focused solely on economic indicators. But most are predicated on variants of Tufte’s model, with measures for presidential approval included. More recent versions include additional variables. These might include “seat exposure” (which party has more seats on the line); a “surge and decline” variable (the idea is to capture the effect of the decline in midterm turnout based in part on how the parties did in the previous presidential election); and a time in office variable to capture the waning influence of a party that has held the presidency for a long time. The most recent innovation to these models – and one that I will discuss in a moment – is to include a generic vote variable based on national surveys that ask respondents which party’s candidate they plan to vote for in the midterm election.

In assessing these structural forecasts, you should keep two considerations in mind. First, most of the models are based on midterm elections occurring during the post-World War II era. So they are predicated on a very limited numbers of cases – 2014 is only the 17th midterm election in that period – which means that even the most accurate predictions have a very wide margin for error. In assessing the prediction of a particular model, you should always look to see if a prediction interval is provided in addition to the predicted seat outcome. Moreover, as Ben Lauderdale and Drew Linzer have cautioned in their critique of presidential forecast models, there is a tendency with a sample this small for modelers to over-fit their predictions by basing them too closely to the particular elections studied. Nonetheless these models are theoretically the most interesting because, when done well, the modelers are very explicit in explaining why midterms turn out the way they do. So we learn the most from these efforts in terms of understanding the dynamics driving election outcomes. Note that these are one-shot deals – once the numbers are plugged in, a forecast is generated and that is that. There’s no updating based on new data.

The final set of forecast are what might be called mixed models. Typically, these start out with a variant of a structural mode (a prior, to use  Bayesian terminology), but then the prediction generated by that model is updated based on race-specific polling data. By the time Election Day rolls around, most of these mixed models will be mostly poll-based, which means (as Drew Linzer demonstrated so effectively in the 2012 presidential election) they are likely to be very accurate. These are the models featured at the Washington Post’s MonkeyCage’s Election Lab or the New York Times’ Upshot site. The basic idea behind these efforts is that if you want to know how people are likely to vote in the 2014 midterm, you should probably ask them and incorporate their response into your forecast.

You might think these mixed models would all generate basically the same forecast. As of today, however, they are not. The MonkeyCage, for example, is giving Republicans an 86% chance of retaking the Senate.  The Time’s Upshot, on the other hand, gives the Republicans only about a 59% chance  of taking the Senate. Why the difference? As the MonkeyCage’s John Sides explains here,  it is partly because as of today the Upshot is likely weighting the polling data, which is a bit more favorable in some states to the Democrats, more heavily than is the MonkeyCage. So, which forecast is more accurate? I don’t know, and neither do they! But that is probably beside the point since it is likely that the two models’ predictions will converge as we get closer to Election Day.

The more important point is that this combination of structural models and polling data is likely to produce a more accurate forecast than either approach alone. (For the more technically-minded among you, Simon Jackman explains why here.)  The use of mixed forecasting is more prevalent now because of advances in computing power and the greater ease of access to polling data. Still, the forecasts are not foolproof – pay attention to that confidence interval when evaluating predictions! – which means that in a very close election cycle, as this one appears to be, none of these approaches may be precise enough to nail down who will control the Senate with perfect accuracy. (Control of the House appears not to be in play at all this cycle.)

Of one thing I am sure. In the face of forecast uncertainty many of you will cherry pick the model whose outcome you like the best, while damning those you disagree with for their bad data, faulty assumptions and poor methodology. If you are one of those who believe in advocating the equivalent of “unskewing the polls”, I apologize in advance for injecting a dose of reality, no matter how unpleasant, into your political fantasy world in the coming months. I promise to do so as gently as possible.

Now let the forecasting begin!

UPDATE 5 p.m. Sam Wang has waded in with his own Senate forecast that puts the projected final Democratic seat total at either 49 or 50.  He makes the important point that a small swing in the partisan share of the vote is going to determine which party controls the Senate – it’s that close!  See his post on the topic here.

How Do Americans Like Their Policies? In Moderation, Please.

In Monday’s post I took issue with Vox founder Ezra Klein’s claim that, in his words “the idea of the moderate middle is bullsh*t.”  You will recall that Ezra argues that indexes purporting to show the existence of a moderate middle are misleading, because they are often amalgams of ideologically extreme opinions. But we don’t need to rely on indexes as evidence that many Americans have moderate policy views – we can look at their responses to specific survey questions. The American National Election Studies (ANES) researchers have been asking Americans about their opinions on issues for several decades now. Typically the question starts by giving the respondent two policy extremes and then asks her to place herself somewhere on a 7-point scale anchored at either end by the extreme responses. As an example, here is a question asking individuals their views regarding government spending on services: “Some people think the government should provide fewer services, even in areas such as health and education, in order to reduce spending. Other people feel that it is important for the government to provide many more services even if it means an increase in spending. Where would you place yourself on this scale, or haven’t you thought much about this?”

Day Robins tabulated the responses to five of those questions for which we have data across time, including the government spending/services question. In each case the moderate, or middle response (Option 4) is typically the modal response, or close to it, across the entire survey history. (Note: although researchers often group the “don’t know/haven’t thought about it” bottom response with the moderate/middle answer, Day has not done so here. So the charts below likely understate the number of respondents adopting the moderate position.)

Here’s the government spending/services time trend.

gov services.16

As you can see, even without including the “don’t knows” the most frequent response across a quarter century has been the middle, moderate choice. (The second most frequent response in recent years is 5.) For the most part, we see the same general pattern on the other four questions Day has graphed; the moderate position is typically the modal response. Here’s the graph of responses regarding whether to increase military spending.

military spending.16

Here’s one asking about government creation of jobs.

This one addresses aid to minorities. For most of the time period the moderate position holds sway, although in 2008 it is slightly eclipsed by option 7 “Help themselves”.

aid to minorities.16
Finally, here’s one measuring support for government health insurance. Here option 1 – strong support for government health insurance is marginally preferred to the moderate position in the 2008 survey. But note that support for the moderate position has actually been increasing, while option 7 – private coverage – has lost support.

health insurance.16
Now, before Max Kagan jumps all over me, these graphs are not evidence that most Americans are moderates across all issue areas. In fact, it is quite possible that their views vary depending on the issue. But the graphs do suggest that for most of the time on each of these issues, a plurality of Americans stake out a moderate position, with the exception of whether the government should provide health insurance and, in 2008, aid to minorities. And this is without including the “don’t knows/haven’t thought about it” response with the moderate answers. If we lump those in, as Abrams and Fiorina do for the 2012 responses to these questions, the moderate views appears even more popular.

Fiorina 3 moderatesMoreover, on three of these issues across most of the time period  – government services, jobs, and military spending – the responses are normally distributed, with most responses clustered around the center of the ideological spectrum, similar to what the Abrams/Fiorina graph shows for 2012. The government health insurance answers skew slightly liberal, and the aid to minorities responses are weighted more to the conservative side of the response distribution.  If Americans were becoming more polarized, we might expect to see a decrease in the number of Americans adopting a centrist position, and increases in those choosing more extreme positions, on these issues.  On the other hand, if what most pundits describe as ideological polarization is really party sorting, then I think we’d be more likely to see the graphs that Day has compiled.

Yes, at the individual level many Americans hold contradictory and sometimes extreme positions. But many hold moderate views too. And, in the aggregate, at least on these five issues, most respondents across much of the last quarter century cluster at or near the middle of the ideological spectrum. Yes there is some movement across responses categories.  But it is hardly enough to lend credence to the notion that the moderate middle is a myth.  And it is consistent with the notion that there has been no significant growth in ideological polarization.

Why It’s Always Morning in America: The Legacy of Carter’s Malaise Speech

Thirty five years ago this evening, President Jimmy Carter went on nationwide television to give his infamous “malaise” speech. You can watch the speech in its entirety here.

If you listen carefully, you’ll notice that he never actually uses the word “malaise”. Instead, he talks about the “crisis of confidence” that threatened the nation’s “will” and “unity”. It is an emotionally searing speech, noteworthy for the candor and level of introspection Carter displayed to a national audience that has rarely, if ever, been expressed by any elected official again. Unfortunately, through the years, the “malaise” speech has come to symbolize all that went wrong with the Carter presidency. As a result, pundits – and presidents too – have tended to draw the wrong lesson from this event.

Some context is useful in understanding why Carter gave the speech. Three years into his presidency Carter’s approval ratings had dropped below 30% and were at the low point of his presidency to date. While he was preoccupied with two important overseas trips – one to sign the SALT II treaty with the Soviet Union and then a diplomatic visit to Japan and South Korea – Americans back home were waiting in long gas lines as they suffered through another energy crisis triggered in part by the Iranian revolution that overthrew the American-backed Shah. Initially Carter planned on giving still another speech urging Congress to address the energy crisis, but ultimately he decided that it would be waste of his time, since he had already given several talks on this topic. Instead, he retreated to Camp David, and summoned leading public figures, including many prominent religious leaders, to meet with him to discuss the state of the nation. Then, like Moses coming down the mountain, Carter went on television to give his famous address, one that went well beyond discussing the energy crisis, although he discussed that too. But most of the speech showcased Carter, the former Baptist preacher, as the nation’s “minister-in-chief”, beginning with his self-flagellation as he recounted criticism of his leadership, and then addressing what he believed to be a growing loss of confidence in the nation’s leaders and institutions.

As you might expect, the speech generated a mixed reaction. In part this was because Carter undercut the message of unity by which he ended the speech when three days later he asked for the resignations of his entire cabinet and key White House advisers (he accepted three cabinet resignations). Gallup polling data that I can find suggests a modest boost, from 29% to 32% in his already tepid approval ratings, although the polls are taken so far apart (the first on July 10 and the next on July 30) that it’s hard to know how much movement to attribute to the speech and subsequent firings. (Reportedly, internal White House polling showed a much bigger overnight boost in approval, but this was before the cabinet firings.)

The long-term impact of the speech, however, may have been more significant. Like the ebullient FDR running against the dour Hoover during the depths of the recession in 1932, Ronald Reagan, with his sunny optimism, was immensely successful during the 1980 campaign at portraying Carter as the nation’s “scolder-in-chief” who was too willing to blame Americans rather than his own inept leadership for the nation’s ills. Although polling suggested that many Americans’ views on the issues were closer to Carter’s than to Reagan’s, it did not prevent Reagan from winning the election. When Reagan won reelection in a landslide four years later by campaigning on the “It’s morning again in America” theme, the die was cast. But it wasn’t just Reagan who capitalized on Carter’s failed attempt to level with the public – Ted Kennedy later noted that the speech convinced him to challenge the incumbent president for the Democratic nomination.

Mindful of the purported lesson of Carter’s “malaise” speech, no national candidate would ever again make the mistake of speaking so candidly, and in such critical tones, about the American people even if much of that criticism was self-directed and perhaps even true. Instead, candidates on the hustings are much more likely to take a page from the Reagan playbook by emphasizing the indomitable American spirit, can-do work ethic, etc.  And woe to any candidate who slips up and leaves himself open to the charge that Americans might be at fault for anything, as in this Romney reaction to an Obama statement.  That is the lasting legacy of the “malaise” speech.

Meanwhile, Carter went on to have a much more successful post-presidency – indeed, he has arguably been one of the most effective ex-presidents in history due to his philanthropic efforts through the aegis of the Carter Center.  He has never expressed any regret to my knowledge regarding the “malaise” speech – indeed, the only regret I have ever heard from him regarding his presidency was his failure to include one more helicopter in the ill-fated Iran rescue mission. But that is a topic for another day. For now, let’s remember when it was morning, again, in America.


The Myth of the Myth of the Moderate Middle

Vox founder Ezra Klein posted this diatribe last week explaining why he believes “moderate voters are a myth” that was extreme even by Ezra’s standards. Drawing on this draft paper by Douglas Ahler and David Broockman, Klein argues that the notion of moderate voters is a statistical artifact that occurs when researchers aggregate responses to several survey questions into a single ideological scale. So, greatly simplified, the idea is that if an individual gives a very conservative answer to one question, and a very liberal one to another, the “index” based on these responses might classify her as a “moderate” by, in effect, balancing the two extreme responses. But in fact all the survey responses show is that she has diverse but extreme opinions – not moderate ones.

In principle, Klein’s criticism is correct. Indeed, scholars have long been aware of this problem when it comes to creating indexes of ideology. Note, however, that it is not only a problem with indexes that purport to show that voters are mostly moderate – other indexes professing to show that Americans are becoming more polarized also suffer from similar coding issues. (In this vein, see this Fiorina, Abrams and Pope’s critique of this Abramowitz/Saunders article on ideological realignment in the United States. Their essential point is that slight movement in responses to one question can, depending on coding, make it appear that voters are changing positions from moderate to extreme on an ideological index .)

Had Ezra stopped there with a caution to be careful about interpreting indexes that purport to measure ideology, I would have been happy. But he doesn’t.  Instead, Ezra goes on to dismiss the idea of a moderate middle entirely: “The deeper point here is that the idea of the moderate middle is bullsh*t: it’s a rhetorical device meant to marginalize some policy positions at the expense of others. There’s no actual way to measure it, or consistent definition animating it, and it doesn’t spontaneously emerge in any of the data.”

I’ll have more to say about the Ahler/Broockman paper in another post. (By way of preview, I think their findings actually support the notion of a moderate middle.) But the notion that the existence of a moderate middle is “bullsh*t”, to use a scientific term is, well, cow dung. Even if many Americans hold inconsistent views, and thus it is hard to classify them ideologically does not mean that on most issues the preferred position among Americans is an ideologically extreme one. Consider, for example, this analysis by Abrams and Fiorina regarding the distribution of responses from 1 to 7 to five policy questions asked by the American National Election Studies in 2012.  (Typically, the questions lay out two alternative policy positions on an issue, such as whether the government should or should not make an effort to help the social and economic standing of African-Americans, and asks respondents to place themselves on a 7-point scale in terms of where they stand on the issue.  Position 4 is the most moderate one, with 1 and 7 holding down the extremes.)

Fiorina partisan nonchange.emfAs you can see, the modal response to questions asking where the respondent stands is the moderate one on all five issue areas. (Note: the authors code the “don’t know/haven’t thought about it” as a moderate response based in part on comparing responses with other surveys that do not include these response options, and finding little difference in the distributions of responses.) Nor is this unique to 2012. Day Robins went back and looked at responses to these questions through several decades and found that for almost every survey the modal response on these five issues is the middle, or moderate, one.  I’ll show some of this data in graphical form in a separate post.

This is not the same as showing that at the individual level voters are mostly moderate, of course. It could be that different moderate coalitions consisting of different individuals show up on each issue. On the other hand, it doesn’t justify dismissing the idea of a moderate middle either, at least when it comes to choosing policy options in a number of issue areas. As Philip Bump reminds us, we need to be careful when we adopt short-hand language for any concept. But while Klein is right to point out the dangers of creating indexes to measure ideology, when it comes to the American voter I’m not ready to throw out the concept of a moderate middle in its entirety. That would be too extreme.

Sunday Shorts: How A President’s Polyps Led to Scandal and Who Won’t Rock

Here are Sunday’s Shorts:

On this day in 1985, a 74-year old Ronald Reagan underwent surgery to remove a benign polyp from his intestine. While under the knife doctors discovered a cancerous polyp which necessitated a second surgery to remove a portion of his intestine. The event would be of little historical interest (Reagan made a full recovery) were it not for the fact that this also marks the period during which Reagan first approved what became the Iran-contra affair – the ill-fated effort to trade American arms to Iran in return for their help in securing the release of Americans held hostage by Mideast terrorist groups. In his July 17th diary entry, Reagan writes, “Some strange soundings are coming from some Iranians. Bud M. [Robert McFarlane, Reagan’s National Security adviser who initiated the plan to trade arms for hostages] will be here to talk about it. It could be a breakthrough on getting our seven kidnaps victims back. Evidently the Iranian economy is disintegrating fast under the strain of war” [excerpt reprinted in Lou Cannon’s Role of a Lifetime]. McFarlane met with Reagan in the hospital the next day, with only Chief of Staff Don Regan present, and informed the President that it might be possible to open up a communication channel, using Israeli officials as intermediaries, with members of the Iranian government. McFarlane’s goal in initiating contact was largely strategic; he sought to better position the U.S. to address the power vacuum likely to occur in the aftermath of the death of the Iranian leader Ayatollah Khomeini. However, it is clear that Reagan’s primary motive in agreeing to McFarlane’s plan was to secure the release of the hostages, which included the Mideast CIA station chief William Buckley.  Reagan had just come off a meeting with one of the hostage’s families and was visibly moved by their appeals to him to do something. However, there is no evidence that either McFarlane or Regan made it clear to Reagan that by subsequently approving the sale of arms to Iran in order to secure the hostages’ release, Reagan was in effect negotiating with kidnappers, thus violating administration policy. Indeed, there is some question whether Reagan, who was recovering from major surgery, was in the best frame of mind to consider such a momentous decision. When pressed later by members of the Tower Commission to recall when he actually approved sending arms to the Iranians, Reagan gave conflicting accounts and it became clear that he simply didn’t remember when, or even if, he gave the initial approval. It is a reminder that although we quite rightly hold presidents solely accountable for the decisions they make, those decision are based in part on the information, expertise and advice of their advisers. In this case Reagan’s advisers failed the President, with significant consequences.

Here’s Reagan’s national address almost two years later, on March 4, 1987, in the aftermath of the report of the Tower Commission which he appointed to investigate the Iran-contra affair, in which Reagan finally admits he traded arms for hostages:

Meanwhile, in the aftermath of the announcement that the Republicans will hold their 2016 nominating convention in Cleveland, Harry Enten presents the following data regarding the impact of convention siting on election results.  If conventions boost a candidate’s performance in that state, one would “expect to see a party’s candidate improve relative to his national performance as compared with the same party’s candidate four years ago. For example, the GOP convention in 2012 was held in Tampa, so Mitt Romney should have done better in Florida than John McCain did in 2008, once we control for the fact that he did better nationwide.” But that’s not what Enten finds. Consistent with my post on the matter, Harry concludes: “There is no proof that a convention site has much effect on a state’s voting patterns.”

Hope  you had a great Sunday!