Monthly Archives: June 2012

Quack, Quack Mr. President

It seems that no one is really happy with the Supreme Court’s health care ruling.  As Louis Tiemann’s excellent comment to my last post reminds me, court decisions that are politically crafted to appease contending parties often make bad case law. In this instance, as Louis points out, Chief Justice Roberts’ efforts to maintain the individual mandate – but not under the pretense that it is interstate commerce – may have opened up a different can of worms regarding the extent of Congress’ taxing authority.

For political reasons, of course, the Obama administration has always insisted that the health care penalty is not a tax.  Here’s the President in 2009, strenuously denying George Stephanopoulos’ attempts to equate the penalty with a tax:

[youtube  /watch?v=rL7ak__MGyw&feature=player_detailpage]

Despite Boy George’s quoting from the Merriam dictionary to determine the definition of a tax, the President was adamant that the penalty is not a tax: “For us to say that you have to get health insurance is absolutely not a tax increase….You can’t just make up that language and decide that’s called a tax increase!”  And later:  “I absolutely reject that notion [that the penalty is a tax increase]”  Alas, the President’s words were lost on Chief Justice John Roberts and the Supreme Court’s four liberals.  The reality is that because the Supreme Court holds that it is a tax – which Chief Justice Roberts just did – then, legally speaking, the penalty is now a tax.

And yet, the White House continues to argue otherwise! Despite the fact that by calling the penalty a tax, Roberts was able to justify the individual mandate (and with it the perception that the Court is a neutral umpire), White House officials yesterday insisted that the penalty is in fact not a tax.  In this respect, they are siding with the Court’s conservative wing.  In the morning press gaggle, White House press secretary Jay Carney made it clear that the White House would try to distance itself from Roberts’ interpretation of the individual mandate penalty:

MR. CARNEY:  It’s a penalty because you have a choice.  You don’t have a choice to pay your taxes, right?  You have a choice to buy — if you can afford health insurance — and you can, I assume, Jared.  So if you don’t buy it, and you can afford it, it is an irresponsible thing to do to ask the rest of America’s  taxpayers to pay for your care when you go to the emergency room.  So your choice is to purchase health care reform or a penalty will be administered.”

The same argument is being made by Obama’s campaign surrogates, such as Massachusetts’  Governor Deval Patrick, who in a conference call organized by the Obama campaign on Friday  insisted that the penalty is not a tax.  Clearly the Obama campaign is worried that the Republicans  will use the Roberts Court’s ruling as a means to define the Affordable Care Act  as a tax increase. The problem for Republicans, of course, is that as governor of Massachusetts Romney signed into law a law that included a similar provision.  As this Wall Street Journal  article points out “While Republicans wanted the law struck down, they’ve used the court’s legal reasoning to argue that Mr. Obama now is responsible for a huge tax increase. But Mr. Romney approved a near-identical requirement for Massachusetts when he was governor that has since brought in millions in fees, penalties or, as Mr. Roberts would suggest, taxes. The law was signed in April 2006; Mr. Romney left office at the start of 2007, and the requirement to have insurance went into effect Dec. 31, 2007.”

Meanwhile, the public has had a decidedly mixed reaction to the Court’s decision as well. As this Gallup poll shows, there’s a clear partisan divide among Americans, with most Democrats applauding the Court’s ruling, most Republicans opposing it and independents about evenly divided.

So, it appears that neither Obama nor Romney is thrilled by how the Chief Justice sought to thread the needle in a way that protected the Court’s reputation as a non-partisan political actor. Nor, for that matter, is the public.  For all these reasons, I think health care will play a minor role in this fall’s presidential election in large part because opinions regarding the health care law are already baked in, and because economic issues are of far greater concern to most voters. It may energize activists on both sides (already Romney has raised several millions dollars in the aftermath of the Court’s ruling), but it isn’t going to switch many votes.  And money is not going to be the deciding factor in this race.

The bigger impact may occur, as Louis suggests, down the road, as the Court deals with the legal fallout from Roberts’ effort to differentiate a tax from a penalty, as well as from the more restricted interpretation of the interstate commerce clause.  In the meantime, the Affordable Care Act survives thanks to Roberts rejecting the President’s claim that the penalty is really just a penalty.  Quack, quack, Mr. President!  Quack, quack!




Roberts Was Right, Institutionally Speaking

So, what explains the Supreme Court’s decision to uphold the Patient Protection and Affordable Care Act, more popularly known as Obamacare?   As most of you will recall, I thought the justices’ votes would fall largely along ideological lines, as captured by a short-hand measure – whether the justice was nominated by a Republican or Democratic president.  My back-of-the-envelope reasoning was based on more sophisticated models developed by legal scholars regarding how justices make decisions.  Based on my reading of this “attitudinal” model, I anticipated a five-to-four decision rejecting the argument that the mandate fell under the umbrella of the interstate commerce clause.  The government, in fact, cannot make you eat broccoli.  And that is precisely what happened – the Court’s Republican appointees ruled as a bloc against the Democrats to say the mandate was not permissible under the interstate commerce clause.  Here’s the key section, written by Chief Justice John Roberts, from the Supreme Court’s health are ruling:

“The individual mandate, however, does not regulate existing commercial activity. It instead compels individ­uals to become active in commerce by purchasing a product, on the ground that their failure to do so affects interstate commerce. Construing the Commerce Clause to permit Con­gress to regulate individuals precisely because they are doing nothing would open a new and potentially vast do­main to congressional authority. “

This extends a series of Supreme Court rulings dating back to the Rehnquist court in which a majority of the justices have begun to put limits on an expansive reading of the interstate commerce clause.   So far, so good for the attitudinal model.

What I did not anticipate, however, was Roberts’ then joining the four Democratic-nominated justices to uphold the mandate under Congress’ taxing power.  Granted, I’m no legal scholar, and this is a blog about the presidency, but I don’t recall too many legal experts who saw this twist coming either.  Roberts, now aligned with the four Democratic court nominees, writes:

“Sustaining the mandate as a tax depends only on whether Congress has properly exercised its taxing power to encourage purchas­ing health insurance, not whether it can. Upholding the individual mandate under the Taxing Clause thus does not recognize any new federal power. It determines that Congress has used an existing one.”

To justify this ruling and mollify conservatives, Roberts’ claims a distinction between Congress’ power to regulate interstate commerce clause and Congress’ taxing power, namely that the power to tax does not give Congress the same degree of control over individuals as does the power to regulate commerce:

“Third, although the breadth of Congress’s power to tax is greater than its power to regulate commerce, the taxing power does not give Congress the same degree of control over individual behavior. Once we recognize that Con­gress may regulate a particular decision under the Com­merce Clause, the Federal Government can bring its full weight to bear. Congress may simply command individ­uals to do as it directs. An individual who disobeys may be subjected to criminal sanctions. Those sanctions can include not only fines and imprisonment, but all the at­tendant consequences of being branded a criminal: depri­vation of otherwise protected civil rights, such as the right to bear arms or vote in elections; loss of employment op­portunities; social stigma; and severe disabilities in other controversies, such as custody or immigration disputes. By contrast, Congress’s authority under the taxing power is limited to requiring an individual to pay money into the Federal Treasury, no more. If a tax is properly paid, the Government has no power to compel or punish individuals subject to it.”

I will leave it to legal scholars to parse the relative impact of the power to regulate commerce versus the power to tax.  But the practical implications of Roberts’ ruling are the same: the mandate survives and the health care law is essentially intact.  (To be sure, the Court also struck down, by a 7-2 margin, with justices Elena Kagan and Stephen Breyer joining the Republican voting bloc, the Medicaid extension portion of Obamacare.  The majority ruled that the federal government could not take away all of a state’s Medicaid money if the state does not agree to the Medicaid extensions mandated by Obamacare.   Practically speaking, I’m not sure what effect this will have, since it is hard to believe states will turn down federal monies, but it does strike another blow, legally speaking, for federalism.)

So, the question becomes: what motivated Roberts to side with the four Democratic justices and uphold the mandate?  One answer, of course, is that he buys his own argument that the penalty provision of the mandate is really a tax, and thus perfectly permissible under the Constitution.  Never mind that when enacting the law Congress did not call it a tax and that the Obama administration, for the most part, studiously avoided using that term. Roberts ruled, essentially, that if it walks and quacks like a tax, it is a tax.

The problem I have with this argument is that the other Republican justices did not accept Roberts’ reasoning; they argue, in their dissent, that it is a penalty, not a tax, and that the two are not the same.  Note that the dissenters include Justice Kennedy who, according to the attitudinal model, would typically vote similarly to Roberts.  But not this time. In their dissent, the Republican justices write:

“The Government contends, however, as expressed in the caption to Part II of its brief, that “THE MINIMUM COVERAGE PROVISION IS INDEPENDENTLY AUTHORIZED BY CONGRESS’S TAXING POWER.” Petitioners’ Minimum Cov­erage Brief 52. The phrase “independently authorized” suggests the existence of a creature never hitherto seen in the United States Reports: A penalty for constitutional purposes that is also a tax for constitutional purposes. In all our cases the two are mutually exclusive. The provision challenged under the Constitution is either a penalty or else a tax. Of course in many cases what was a regulatory mandate enforced by a penalty could have been imposed as a tax upon permissible action; or what was imposed as a tax upon permissible action could have been a regulatory mandate enforced by a penalty. But we know of no case, and the Government cites none, in which the imposition was, for constitutional purposes, both. The two are mutually exclusive. Thus, what the Government’s caption should have read was “ALTERNATIVELY, THE MINIMUM COVERAGE PROVISION IS NOT A MANDATE-WITH-PENALTY BUT A TAX.” It is important to bear this in mind in evaluating the tax argument of the Government and of those who support it: The issue is not whether Congress had the power to frame the minimum-coverage provision as a tax, but whether it did so.”

What, then, explains Roberts’ ruling, if not the attitudinal model?  As I tweeted yesterday, I think the answer lies in his position as Chief Justice.  Roberts, in my view, was thinking institutionally when he decided to uphold the mandate.  That is, he was careful in making his ruling to protect the Court’s autonomy and future power prospects.  As Chief Justice he has greater stake than his colleagues in maintaining the Court’s reputation for impartiality. In this sense, Roberts was motivated by the same sentiments that guided John Marshall in the celebrated Marbury v. Madison case: the need to protect the Court’s institutional interests.  Remember, the power of the Court rests on the degree to which other actors, and the public view its decisions as legitimate – that is, not motivated by purely partisan reasoning. By upholding the mandate, and thus leaving Obamacare reasonably intact, Roberts protected the Court’s institutional interests. It is also the case, however, that once we dig beneath the obvious implications of this ruling – that Obamacare survives – there is much in the ruling that conservatives can like, beginning with the limits on the extension of Medicaid.  In terms of legal doctrine, Roberts’ reasoning appears, at least in theory, to reduce the scope of Congress’ use of the interstate commerce clause and spending power as a means of enacting social welfare measures, although the practical implications of this can be debated.  And, of course, by labeling the penalty a “tax”, Roberts has handed the Republicans a political sword which they can use in their effort to repeal the bill, and to attack Obama in the general election campaign.  So this was not an unalloyed victory for the President or his party.   On the whole, however, it certainly was better for them than the alternative: striking the bill down in its entirety.

The broader lesson, and one that I did not fully appreciate in thinking about how the Court would rule, is that justices are free to vote their beliefs as long as they are not subject to conflicting constraints.  In this case, I think Roberts’ role as Chief Justice, and his desire to keep the Court from appear overtly partisan, overruled his ideological preference, which was to strike the mandate down in its entirety.  In so doing, however, he has better positioned the Court’s conservative majority to pursue its ideological preferences in future cases.

3:17 p.m. Charles Krauthammer makes essentially the same argument in this op ed piece. (Thanks to George Altshuler for the link to Krauthammer’s piece.)


OK, OK, Enough Already! I’ll Post Something On Health Care

Nothing has happened in the ensuring three months to change my view, expressed in  my original post on the Supreme Court’s health care hearings last March, that that the best predictor of how the Court will rule today is how the lower courts’ have broken down on this case.  That means Republican appointees will vote against the individual mandate, and Democratic ones will support it.  This is not because Supreme Court justices view cases through a purely partisan lens, but because their legal perspective can’t help but be shaped by the same factors that influenced their partisan outlook.  In closely contested legal cases that turn on interpretations of ambiguous words, such as how to define “interstate commerce”, I don’t see a better decision rule.

This is a much guess as anything, of course, but I think it is a reasonable guess given the evidence.   For what it is worth, I think a decision to strike down the mandate, but only the mandate, is consistent with public opinion.  Thus, this recent Pew Research Center poll shows that a plurality of the public disapproves of the health care law in total..

This split has remained relatively consistent since Court’s public hearings on the case last March.  Most notably, the public’s views break down along partisan lines, particularly when it comes to the individual mandate.

This is why I think the electoral implications of today’s decision are likely to be overblown by the media.  Yes, both camps will try to frame the ruling in a way that bolster’s their candidate’s chances heading into November, but as the poll suggests, they will be mostly preaching to their respective choirs.   And while independents are, according to Pew, largely against the mandate, there is some evidence that opinion among this group is more closely divided regarding other components of the law.

Keep in mind as well that surveys consistently show that the economy and jobs are viewed as more pressing concerns by most voters, and that 70% or more of adults surveyed are satisfied with their current health care coverage.  So while I have no doubt that today’s decision is going to generate a torrent of media coverage, we should be careful not to be too swayed by the inevitable partisan spin that will inform much of what is said in the next several days.  Yes, in a close election, one can cite any number of factors as being potentially deciding, but in weighting those factors, health care comes in far below economic concerns.

If the Court renders its ruling on a narrow 5-4 basis, as I’m guessing it will, we will also hear about much damage this will do to the Court’s public standing.   Again, we need to keep this in perspective; public opinion toward the Court, while remaining generally favorable, has nonetheless fluctuated quite a bit in recent years.  I don’t anticipate that a split decision will inflict any lasting damage on the Court any more than its foray in 2000 into election year politics did – and that was viewed as a far more partisan decision.

Finally, the Court ruling today is not the final word on health care reform particularly if it renders a narrow ruling against the mandate, but does not throw out the rest of the law.

So, for now, sit back, grab your morning beverage of choice, and enjoy the fireworks.  Assuming the twitter feed doesn’t collapse, the SCOTUS twitter feed should have the decision almost as soon as it is announced.

10:19  No sooner did I send this than the Court decision came down – but not after CNN misreported the initial results.   In any case, it appears that mandate was upheld under the Congress’ taxing power and not as interstate commerce.  In effect, the Court stepped in and made the argument that the Obama administration should have made.   Interestingly, the Court broke down almost exactly as you might have thought – with the one exception of Roberts, who sided with the majority.   So, the attitudinal model of court decisionmaking gets a pretty big boost here, predicting 8 of the nine justices.  But in the end, it was that 9th that proved decisive.

Now, let the media overreaction begin!

10:29  Ok, I give up.  Again. Now some media outlets are saying that the practical effect of the decision is to undercut the mandate because people who refuse to pay the “tax” will not be penalized by their refusal.  I think I am going to step back and let legal experts actually read the ruling before trying to assess its meaning.

Here’s the actual opinion  by the Supreme Court, fresh from the printing press.

Not surprisingly, there is a LOT of spin going on right now, on both sides – almost all of which is overstating, in my view, the political impact of this decision.  Let’s read the opinion, let the political dust settle, and see where things really stand in 24 hours.  For now, and not surprisingly, everyone on every side sees a silver lining.  Conservatives think the Court just limited Congress’ use of the commerce clause, and strengthened federalism, consistent with recent trends in the Court’s decisions. Liberals are happy the mandate stands, albeit as an illustration of Congress’ taxing power.  Romney is happy to see the ruling take health care off the table, so the election can now be about the economy, where he runs stronger.  Obama is happy the mandate, and thus health care, survived pretty much intact (although the Medicaid restrictions still need to play out).  Roberts is happy because he managed to side with both the conservatives and liberals in one decision.  Even political scientists are happy because the justices ruled pretty much along the lines that their partisan affiliation would have one predict – indeed, on the interstate commerce clause, they ruled exactly as one might predict.

Only CNN is unhappy, because they blew the initial call.



No, Political Scientists Are NOT Lousy Forecasters – In Fact, They Are Pretty Good

First Nate Silver. Now Jacqueline Stevens!  Stevens is the Northwestern political scientist whose op ed piece in Sunday’s New York Times  sparked more than a little debate regarding the role of government efforts through National Science Foundation grants to fund political science research, and the quality of that research more generally.  Stevens leaves no doubt that she considers most NSF funding to have been wasted; as she writes,  “It’s an open secret in my discipline: in terms of accurate political predictions (the field’s benchmark for what counts as science), my colleagues have failed spectacularly and wasted colossal amounts of time and money.”

Not surprisingly, that rather sweeping statement generated a lot of pushback, much of it from political scientists (see here and here and here, for starters.) As the critics point out, her overly-broad generalization and cherry-picked examples misreads much of what political science tries to do, and how well it has done it, and in the process manages to distort Phil Tetlock’s argument in his recent book regarding when and why political “experts” often miss the mark in their predictions.  (Hint: Stevens’ op ed piece show evidence of falling prey to some of the cognitive biases Tetlock cites in his book.)

Rather than belabor these points, I want to focus on another claim she makes as a way of explaining more generally why her criticism misses its mark by such a wide margin.  In pointing out forecasting flaws in the discipline, Stevens’ writes, “Political prognosticators fare just as poorly on domestic politics. In a peer-reviewed journal, the political scientist Morris P. Fiorina wrote that ‘we seem to have settled into a persistent pattern of divided government’ — of Republican presidents and Democratic Congresses. Professor Fiorina’s ideas, which synced nicely with the conventional wisdom at the time, appeared in an article in 1992 — just before the Democrat Bill Clinton’s presidential victory and the Republican 1994 takeover of the House.”

The problem with Stevens’ assertion is that it completely mischaracterizes Fiorina’s published research on divided government.   (Full disclosure: Fiorina sat on my dissertation committee [something he rues to this day, no doubt], and was later a colleague of mine [again, not necessarily by his choice!]).  In fact, Fiorina’s research laid out a model that sought to explain the persistence of divided government in the post-World War II era despite on-going claims by pundits (and some political scientists) that it lead to policy gridlock and political stalemate.  And yet Americans more often than not voted in a way that maintained divided government.  Why did they so frequently act against their own presumed self-interest, and in favor of gridlock and stalemate? For Fiorina, the answer was rooted in part by the (not always fully articulated) desire among a subset of Americans to achieve a modicum of ideological moderation by “balancing” the parties in the face of increasingly partisan polarization.  In short, as the parties become more extreme, voters are more likely to split their tickets. Now, he does not claim this desire for “balance” is the only factor contributing to divided government.  Other structural factors, such as the increasing professionalization of state legislatures, which serve as a crucial breeding ground for individuals who eventually run for Congress, certainly influenced the long period of Democratic control of Congress, since that professionalization disproportionally advantaged Democrats.  That meant voters who sought to balance the two parties at the national level usually focused on voting in a Republican president. He acknowledges as well that his balancing model is but one of several possible ways that voters’ choices might produce divided government.

The crucial point, however, is that contrary to Stevens’ assertion, the transition to a Democrat president and Republican-controlled Congress in 1994 after the Republican midterm wave is perfectly consistent with his explanatory model! Indeed, in looking at state governments, Fiorina explicitly lays out a model that explains when states might choose a Democratic governor and a Republican-controlled legislature.  That model is applicable to the national level.  In her op ed piece, Stevens would have us believe that Fiorina’s model predicted that under divided government Democrats would control Congress and Republicans the presidency indefinitely.  But that is not what he predicted – he laid out a more general model of divided control which encompassed different political permutations that included varying patterns of Republican and Democrat presidents and Congresses.  The key to evaluating his model, then, is whether the change in the composition of divided government from 1990 to 1994 is consistent with his premises.  That’s the proper test of his forecast – one Stevens ignores.

More generally, Stevens’ mischaracterizes the nature of political science forecasting.  Thus, Fiorina’s forecast model is not designed to say that in 1994, because we had a Democratic president, Republicans would reclaim control of the House and the Senate.  (In fact, I know of very few political scientists who saw this happening!)  Instead, as with most forecast models, Fiorina constructed a probabilistic explanatory framework premised on some clearly articulated assumptions.  That is, he suggested that when certain factors were in place – extremely polarized parties, a difference in the perceived strength of the presidency and Congress – many voters would act in a particular way that would often lead to divided government.  The results in 1994 are consistent with his argument.

And this gets back to my debate with Nate Silver, and to the importance of theory more generally.  Anyone, looking back on national elections dating to the Truman presidency, can predict that the odds are good that we are going to see divided government again in 2012.  But without a theory, we don’t know why, which renders the prediction essentially useless.  What if, in fact, Republicans sweep to power and control all three branches come November?   That might not mean our model of divided government is wrong – in fact, it might be fully consistent with that model.  It all depends on the underlying theory – did we actually have the conditions that the theory says must be in place for divided government to occur?  For political scientists, then, it matters why a prediction is right or wrong.

And this leads to my second point: when outcomes do not comport with theory, we learn something that, hopefully, allows us to make the theory better.   (Of course, this doesn’t mean a post hoc readjustment of the theory to fit the latest data point.)  Forecasting is a way to test whether our understanding of an event is correct.  In making that forecast, however, political scientists try as well to signal how specific the likelihood of an outcome is.   We might not be able to state that the Soviet Union will collapse in 1992 anymore than the medical doctor can tell you if and when you will get cancer.  But perhaps we can make some general statements about the conditions under which authoritarian regimes are more likely to collapse, and with what probability, just as the doctor can recommend some dietary and environmental factors that medical science suggests are likely to decrease your chances of getting cancer.

The reality is that because of political science, we know a lot more about politics than would otherwise be the case.  In some areas – such as forecasting presidential elections – we actually know quite a bit (more than Nate Silver’s criticisms about election forecasting indicate, I would argue.)  Moreover, the best political scientists – because they rely on clearly articulated theories and have a method for testing that theory – are less prone to making the types of errors that Tetlock associates with “experts”.  This is not to say we don’t make mistakes.  But even when our forecasts prove wrong, learning can still occur.

For all these reasons, I disagree with Stevens’ assessment of the state of the discipline, at least that portion of the discipline with which I am most familiar.  Given the limits of what can be expected in a probabilistic world, political scientists are, in fact, pretty good forecasters, and they are getting better.

Dickinson and Silver, Take Two

Whether he did so out of frustration or some other emotion, I want to thank Nate Silver for taking time from his busy schedule to respond (twice!) to my critique of poll-based forecasting models similar to his.  This type of exchange is common for academics, and I always find it helpful  in clarifying my understanding of others’ work.  Based on the email and twitter feedback, I think that’s been the case here – my readers (and I hope Nate’s too!) have benefitted by Nate’s willingness to peel back the cover – at least a little! – on the box containing his forecast model, and I urge him to take up the recommendations from others to unveil the full model.  That would go a long way to answering some of the criticisms raised here and elsewhere.

Because of the interest in this topic, I want to take an additional minute here to respond to a few of the specific points Nate made in his comments to my previous post, as well as try to answer others’ comments. As I hope you’ll see, I think we are not actually too far apart in our understanding of what makes for a useful forecast model, at least in principle.  The differences have more to do with the purpose for, and the transparency with which, these forecast models are constructed.  As you will see, much of what passes for disagreement here is because political scientists are used to examining the details of others’ work, and putting it to the test.  That’s how the discipline advances.

To begin, Nate observes, “As a discipline, political science has done fairly poorly at prediction (see Tetlock for more, or the poor out-of-sample forecasting performance of the ‘fundamentals based’ presidential forecasting models.)”  There is a degree of truth here, but as several of my professional colleagues have pointed out Nate’s blanket indictment ignores the fact that some forecast models perform better than others.  A few perform quite well, in fact. More importantly, however, the way to  improve an underperforming forecast model is by making the theory better – not by jettisoning theory altogether.

And this brings me to Nate’s initial point in his last comment: “For instance, I find the whole distinction between theory/explanation and forecasting/prediction to be extremely problematic.”  I’m not quite sure what he means by “problematic”, but this gets to the heart of what political scientists do: we are all about theory and explanation.  Anyone can fit a regression based on a few variables to a series of past election results and call it a forecast model. (Indeed, this is the very critique Nate makes of some political science forecast models!) But for most political scientists, this is a very unsatisfying exercise, and not the purpose for constructing these forecast models in the first place.  Yes, we want to predict the outcome of the election correctly (and most of the best political scientists’ models do that quite consistently, contrary to what Silver’s comment implies), but prediction is best seen as a means for testing how well we understand what caused a particular election outcome.  And we often learn more when it turns out that our forecast model misses the mark, as they did for most scholars in the 2000 presidential  election, and again in the 2010 congressional midterms, when almost every political science forecast model of which I’m aware underestimated the Republican House seat gain (as did Nate’s model).  Those misses make us go back to examine the assumptions built into our forecast models and ask, “What went wrong? What did we miss? Is this an idiosyncratic event, or does it suggest deeper flaws in the underlying model?”

The key point here is you have to have a theory with which to start.  Now, if I’m following Nate correctly, he does start, at least implicitly, with a baseline structural forecast very similar to what political scientists use, so presumably he constructed that according to some notion of how elections work.  However, so far as I know, Nate has never specified the parameters associated with that baseline, nor the basis on which it was constructed. (For instance, on what prior elections, if any, did he test the model?) It is one thing to acknowledge that the fundamentals matter.  It is another to show how you think they matter, and to what degree. This lack of transparency (political scientists are big on transparency!) is problematic for a couple of reasons.  First, it makes it difficult to assess the uncertainty associated with his weekly forecast updates.  Let me be clear (since a couple of commenters raised this issue), I have no principled objection to updating forecast projections based on new information. (Drew Linzer does something similar in this paper, but in a more transparent and theoretically grounded manner.) But I’d like to be confident that these updates are meaningful, given a model’s level of precision.  As of now, it’s hard to determine that looking at Nate’s model.

Second, and more problematic for me, is the point I raised in my previous post.  If I understand Nate correctly, he updates his model by increasingly relying on polling data, until by Election Day his projection is based almost entirely on polls.  If your goal is simply to call the election correctly, there’s nothing wrong with this.  But I’m not sure how abandoning the initial structural model advances our theoretical understanding of election dynamics.  One could, of course, go back and adjust the baseline structural model according to the latest election results, but if it is not grounded on some understanding of election dynamics, this seems rather ad hoc.  Again, it may be that I’m not fair to Nate with this critique – but it’s hard to tell without seeing his model in full.

Lest I sound too critical of Nate’s approach, let me point out that his concluding statement in his last comment points, at least in principle, in the direction of common ground: “Essentially, the model uses these propositions as Bayesian priors. It starts out ‘believing’ in them, but concedes that the theory is probably wrong if, by the time we’ve gotten to Election Day, the polls and the theory are out of line.”   In practice, however, it seems to me that by Election Day Nate has pretty much conceded that the theory is wrong, or at least not very useful.  That’s fine for forecasting purposes, but not as good for what we as political scientists are trying to do, which is to understand why elections in general turn out as they do.  Even Linzer’s Bayesian forecast model, which is updated based on the latest polling, retains its structural component up through election day, at least in those states with minimal polling data (if I’m reading Drew’s paper correctly).  And, as I noted in my previous post, most one-shot structural models assume that as we approach Election Day, opinion polls will move closer to our model’s prediction. There will always be some error, of course, but that’s how we test the model.

(Drew’s work reminds me that one advantage scholars have today and a reason why Bayesian-based forecast models can be so much more accurate than more traditional one-shot structural models is the proliferation of state-based polling. Two decades ago I doubt political scientists could engage in the type of Bayesian updating typically of more recent models simply because there wasn’t a lot of polling data available.  I’ll spend a lot of time during the next few months dissecting the various flaws in the polls, but, used properly, they are really very useful for predicting election outcomes.)

I have other quibbles. For example, I wish he would address Hibbs’ argument that adding attitudinal data to forecast models isn’t theoretically justified. And if you do add them, how are they incorporated into the initial baseline model – what are the underlying assumptions? And I could also make a defense of parsimony when it comes to constructing models. But, rather than repeat myself, I’ll leave it here for now and let others weigh in.