• How Predictive are the Polls?

    by  • September 9, 2012 • 0 Comments

    While we wait to get a sense of Obama’s post-convention bounce, it’s worth taking a look at how predictive the trial-heat polls can be at this (or any) point of the campaign. I’ll use the state-level surveys from 2008 as a guide. Applying my model to these data, I can estimate the daily level of support for Obama vs. McCain in all 50 states, from May through November. We can then compare the candidates’ standing in the polls during the campaign to each state’s election outcome, and see how those differences varied over time.

    To begin, here’s the trend in voter preferences in Florida in 2008. Polls are plotted as circles, for the percent supporting Obama (out of the total for Obama or McCain). The thick line is the model’s estimate of the underlying level of support for Obama. The horizontal line indicates the result of the election, in which Obama received 51.4% of the two-party vote.

    The trend is similar to what happened in most states: Obama polled behind his eventual vote share throughout the summer, then lost more ground following the RNC and selection of Sarah Palin. But once the the financial crisis began, Obama quickly gained in the polls – and by early October, the polls were about (on average) where the election ended up.

    Next, I calculate the difference between each state’s daily estimate of voter opinion and the state’s election result, and average these across all 50 states. This is the mean absolute deviation, or MAD.

    On average, the state polls (fed through my model) were off by 2%-3% through August, increasing to a maximum average error of 4% after the RNC. But by Election Day, the poll-based estimates only differed from the actual outcomes by an average of 1.4%. In states that were most competitive – and therefore polled more frequently – the error was even lower. If we isolate the eight closest states, where Obama finished with between 46% and 54% of the two-party vote, the average error on Election Day was a minuscule (and fairly remarkable) 0.4%.

    So, in 2008, the polls were highly accurate over the last month of the campaign; and somewhat less so prior to that.

    But looking at the polls isn’t the only way to predict the election outcome. In August and September of 2008, political scientists published a series of forecasts of the national-level vote, based upon long-term structural factors such as levels of economic growth, unemployment rates, whether the incumbent is running for re-election, and so forth. How well did these forecasts perform? The median prediction was that Obama would win 52% of the two-party vote. If we assume uniform swing and home-state effects relative to 2004 (as I describe here), this would have translated to an average state-level error of 3.2% – greater than that of the poll-based estimates for almost the entire campaign.

    In fact, the minimum state-level MAD that any national-level election forecast could have produced in 2008 was 2.6%. Here I’ve plotted the state MAD under various assumptions about the national vote outcome (again, using uniform swing with home-state effects). The reason for this lower bound is that although uniform swing is an extremely useful model, it isn’t perfect. From election to election, state vote outcomes still tend to vary by an additional 3%-5% or more beyond the national swing.

    The forecasts that I’m showing on this site are produced by combining long-term factors with estimates of current opinion based on the state polls. This stabilizes the forecasts relative to the polls alone, while also reducing the forecast MAD, as can be seen in my paper (Figure 4). In 2008, state forecasts using my model maintained an average error below 2% for the final two months of the campaign.

    RNC Verdict: No Bounce (yet)

    by  • September 5, 2012 • 0 Comments

    The first batch of state polls are in, following last week’s Republican National Convention. Two surveys were completed 8/30 in North Carolina, and five more finished 9/2 in North Carolina, Michigan, Colorado, and Florida (2). It’s not a whole lot to go on, but so far, all indications are that voters have not swung towards Romney as a result of the RNC. If anything, Romney’s level of support appears to be continuing its slight downward trend from mid-August.

    Is this surprising from a historical standpoint? Absolutely. A week of high-profile, positive convention exposure for the Romney campaign should have made at least a few percentage points of difference. Traditionally, it has. (And indeed, some analysts claim to be seeing a bounce; though others don’t.) But if there was truly no effect of the RNC on voter opinion – even in the short term – then it strongly suggests that either voters didn’t like what they were hearing, had already made up their minds, or simply weren’t paying much attention (all relatively speaking, of course). Considering Obama’s current lead in the polls, none of these scenarios are good for Romney. And if it turns out that Obama does receive a bounce coming out of the DNC, then we can likely conclude that the reason Romney didn’t get a bounce from the RNC isn’t because voter opinion is just fundamentally difficult to move this year.

    As more polls are released this week, estimates of the state trends during both the RNC and DNC will continue to update. Each time a new survey is added to the dataset, the model re-calculates the entire set of trendlines for all 50 states. So we may see some corrections. On the other hand, the states where this latest group of polls were fielded had all been pretty well-polled leading up to the RNC. If Romney gained ground from the convention, it should be showing up somewhere, and right now it’s not.

    New Feature: State Trend Details

    by  • August 31, 2012 • 1 Comment

    I’ve added a new feature to the site to give us a better look at the state-by-state trends in voter preferences during the campaign. These can be found at the State Trend Detail link, for both Obama and Romney (though the two trends will be mirror images of one another). As I noted on Tuesday, the data indicate that Romney has been slowly, but steadily, gaining on Obama across a broad range of states since May.

    Since there’s been so little overall change in public opinion so far, these trends are nearly impossible to see on the original Poll Tracker page. Those plots use a consistent – but very wide – vertical scale, which emphasizes the differences between, rather than within, the states. In the State Trend Detail, the ranges of the vertical axes are much narrower – and vary by state – to focus on where, and to what extent, attitudes are shifting within states. The Poll Tracker page still provides a broader view of the spread of the polls themselves.

    Importantly, it doesn’t seem that there’s been any defining campaign event that could be responsible for these trends. The erosion in support for the president has been extremely incremental. The old standby explanation is that undecided voters are increasingly paying attention to the campaign, and cluing into the mediocre state of the economy. Absent (sadly!) any hard data on state-by-state campaign spending, this seems reasonable enough. I’m eager to see what the next few days of polling indicate about the effects of the RNC. Hopefully there will be some new post-convention polls out by Monday or Tuesday, so look for another update then.

    Finally, a bit of housekeeping. The BEA released their second estimate of Q2 GDP growth on Wednesday: 1.7%, updated from 1.5% in the original report. Results from the next run of the model (which should be posted by tomorrow) will use this number for the baseline projection of Obama’s two-party vote share, which tics up slightly from 52.3% to 52.4%.

    Romney’s Trending… Slowly

    by  • August 28, 2012 • 4 Comments

    With the Republican National Convention starting today, the Romney campaign has got to be hoping for a modest convention bounce to finally push them ahead of President Obama in the polls. Obama’s lead has held steady for months – and the conventional wisdom is that voter preferences have been unusually stable this year, perhaps due to an increasingly polarized electorate, voters’ closer familiarity with the candidates, or even simply that a lot of voters have already made up their minds.

    Whatever the reason, here’s a case where the conventional wisdom is right… well, mostly. It’s true that we haven’t seen anywhere near the volatility that occurred during the Republican presidential primaries, for example. But, Mitt Romney has been slowly cutting into Obama’s lead over the past few months. It’s a bit hard to see on the poll tracker page, because the magnitude of the trend is so small – only about 0.5% in an average state. It’s definitely there, though. And the effect has been largest – between 0.7% and 0.8% – in a set of states you may recognize: New Hampshire, Iowa, Wisconsin, Michigan, Minnesota, Montana, Virginia, and Colorado – swing states where Obama is ahead by only a couple of points.

    We can zoom in on some of these states to see the pattern more clearly. I’ll include Ohio as well, where the size of the trend has been somewhat smaller. Here I’m plotting only the estimated percent supporting Obama, over time. The vertical dashed line corresponds to August 11, when Romney announced Paul Ryan as his vice-presidential nominee. It appears that the movement away from Obama started much earlier than that. And if anything, the decline in support for Obama has leveled off – or even rebounded – following the Ryan pick.

    How am I getting such detailed trends from such limited, state-level only, polling data? Compare my Virginia trendline to the trackers at HuffPost-Pollster or TPM. One trick is that I’m setting aside undecided voters, which are basically just noise. But the more significant feature of my model is that it is designed to simultaneously borrow information from the polls across all 50 states. When there are fewer polls in one state, polls in the other states help fill in the gaps, while still allowing each state to follow its own unique path. The model, in other words, adds a huge amount of statistical power to detect small effects. This is the benefit of an integrated modeling approach, and why (to put it plainly) 50 state series are more informative than one national series.

    It will be interesting to see how things unfold over the next week, and whether Romney can accelerate this trend. This is just speculation, but I’m a bit skeptical that we’ll see a large GOP convention bounce. Hurricane Isaac is expected to hit New Orleans tonight. It’s already shaping up to be a major news event, including many, many comparisons to Hurricane Katrina in 2005. If people remember anything about Katrina, it’s probably going to be their own negative reaction to the government’s response to the disaster under Bush (e.g., here and here). Convention bounces happen because parties are treated to nearly uninterrupted positive coverage for days on end. Not only does Hurricane Isaac threaten to push the RNC out of the headlines, but in a way that’s framed negatively for the GOP. If Isaac turns out to be less destructive than Katrina – which seems likely after New Orleans spent seven years and $14 billion reinforcing its infrastructure – and President Obama is seen as handling the storm competently, then it wouldn’t surprise me if Romney doesn’t see a quick uptick in the polls as the convention wraps up.

    Mitt Romney, Stuck in a Moment

    by  • August 2, 2012 • 0 Comments

    The latest electoral vote forecast is unchanged at Obama 337 – Romney 201, but with today’s update, the uncertainty in those estimates decreases a bit. The main reason is another round of state polls showing Obama maintaining his lead in pivotal states like Florida, Pennsylvania, Ohio, and Wisconsin. But just as importantly, new polls in safe Romney states – including Arizona, Missouri, and North Dakota – also show no consistent up- or down-trends.

    Taken together, these polls let us not only be more confident in current estimates of Obama’s standing, but also in the overall stability of the race up to this point. One of the ways my model projects forward to Election Day is by assessing the past volatility in voter preferences. If and when the polls begin to indicate more instability in voter preferences, more uncertainty will go back into the forecast. As I mentioned the other day, however, an unusually large proportion of voters are already saying they’ve made up their minds. It’s not as if nothing’s happening out on the campaign trail – only, none of it’s moving public opinion.

    The bottom line is that Mitt Romney is facing a very large uphill climb right now. Every major survey aggregator is in agreement that Obama is ahead in current state-level polling – and many see the number of states in the tossup category dwindling. HuffPost has Obama at 290 electoral votes, with only Florida, North Carolina, and Virginia as tossups. Not to be outdone, TPM puts Obama at 310, with just two tossups – Virginia and Colorado. RCP is less aggressive, but still shows Romney needing to (nearly) run the table.

    To put things in perspective, Romney could win Indiana, North Carolina, Colorado, Virginia, New Hampshire, Florida, and Iowa, and still lose the election. For Romney to rebound, something will have to happen over the next three months to either win over decided Obama supporters, or make undecided voters break overwhelmingly away from Obama – which evidence suggests could be a challenging proposition.