Almost the day after the US election, pollsters and election forecasters readily admitted that their models and surveys seemed to have gotten it wrong once again.
Even though votes are still being counted and the data is still being sifted, American analysts have begun reflecting on the entire election forecasting industry, which predicted a much larger win for President-elect Joe Biden than what we saw last week.
Models combine two types of numbers. The first are the “fundamentals” — the factors that shape voter choices. For example, how the economy’s status affects incumbency chances or the fact that a party winning three times in a row has only happened once in the last 70 years.
Andrew Gelman and Merlin Heidemanns of Columbia University, who created a poll aggregation model for the Economist, wrote: “Like most forecasts, our model … applies past patterns of voters’ behaviour to new circumstances … ‘How often have previous candidates in similar positions gone on to win?’ If those historical relationships break down, our forecast will misfire.”
Then, researchers look at the polls (answers from representative samples). The model averages the polls, weights each one according to their sample sizes, and then, corrects for any biases. Nate Silver, a figurehead in the election forecasting community and the editor of established data outlet FiveThirtyEight, specifically distinguishes himself from a pollster, stating that his organisation’s job is to understand how wrong the polls could be to create probabilistic forecasts.
The final model blends the fundamentals with the poll averages. With these two types of information in place, the researchers run simulations a large number of times to find how many times a candidate receives over 270 electoral votes. In 1,000 simulations, if Biden wins 500 times, he has a 50 per cent chance of winning. As Election Day nears, researchers give the polls greater weight over the fundamentals.
Also in Explained | Why US President-elect Joe Biden is unlikely to weigh in on Kashmir
What happened in 2016?
The mathematic oracles of American elections had confidently prophecied a Hillary Clinton victory. Respected mainstream surveyists gave Clinton a lead up to four points. She ended up leading by a 2.1 percentage point in the popular vote. FiveThirtyEight faced the heat for forecasting that Hillary Clinton had a 70% chance of winning the White House. Silver said people were taking election poll results out of context.
The Economist wrote: “Mr Trump’s unlikely triumph in 2016 left many quantitative election forecasters looking silly. Sam Wang, a professor at Princeton, vowed to eat a bug if Mr Trump, whom he said had just a 1% chance of victory in November 2016, came even close to winning. (He chose a cricket.)” 📣 Click to follow Express Explained on Telegram
Post-mortems from institutions such as the Association of Public Opinion Research came to conclude that the polls had underestimated weights for voters without college degrees. The New York Times’ Upshot found that the lack of weights by education status miscalculated Trump support by four points, matching the error. In many ways, it was a simple underestimation of how many voters were white and did not have a college degree. In another error, late-deciders ended up voting for Trump more than predicted, and overall Trump voter turnout exceeded expectations.
Claiming they had fixed the errors, the statisticians stated they had learned their lessons from 2016.
What happened in 2020?
“There’s no question the polls missed (again). But we won’t know by how much until all votes are counted (including estimates of rejected ballots). Then we will reassess. But I think it’s fair to say now that in so many ways, including political polling, Trump is sui generis,” tweeted Director of Monmouth Poll Patrick Murray the day after elections.
Surveys showed Biden leading by at the least eight percentage points in the final stretch of the campaign season. He will most probably end up with a four to five percentage point win. Even both campaigns’ own private polls underestimated Republican candidates.
At the state level, the predictions were even more off. RealClearPolitics and FiveThirtyEight overpredicted for Biden in every swing state except for Arizona. Florida in particular was way off the mark; with almost four points, Trump took the state that polls on average had predicted for Biden by three points. The New York Times and the Washington Post had Biden at a 17 and 11 point lead in Wisconsin. So far, it is at a one percentage point difference. Congressional races were even worse, with Democrats blindsided by their losses.
“Polls (esp. at district-level) have rarely led us more astray & it’s going to take a long time to unpack,” tweeted Cook Political Report editor Dave Wasserman the day after the elections.
Also read | Want a preview of President Biden? Look at the campaign trail
What went wrong?
It’s too early to tell, but the theories have begun to percolate. One theory by Zeynep Tufekci is that there is not enough past data to accurately create fundamentals because factors in elections change so substantively every time.
Other potential answers might lie in the final turnout data. Nate Cohn of the New York Times says either 2020 presented a new set of problems, or the problems of 2016 may have never been fixed. He leans towards the former, mostly because education weighting didn’t change the predictions. Polls found that white voters without a college degree were to vote for Biden at higher rates than Clinton, but the final results showed that they didn’t shift as predicted. Another error was in the calculations of senior voters, who were predicted to vote for Biden by 23 points more than Trump. But in reality, seniors didn’t vote for Biden at any higher rates.
Cohn makes the point that these are not failures in estimating the size of groups, but more so their attitudes. This is related to right-wing claims of a “silent majority” that votes for Trump but hides their political beliefs. After 2016’s failures, polls lost their credibility and perhaps fewer Trump supporters were willing to respond to survey questions.
One obvious potential wrench in the numbers was the pandemic. Polls from before the pandemic hit (between October 2019 and March 2020) were more accurate than as the election neared. One theory suggests that Democrats were more likely to be locked down during this time and were more likely to respond to polls than Republicans. Responses did increase in that time, and hot spots began showing more support for Biden. In other words, this was not increased support for Biden; this was an increase in the likelihood of a Biden supporter to respond.
Is this about substantive problems or presentation?
Some political pundits say the problem is the presentation of the numbers to a mass audience, rather than a problem of the numbers. For example, if Biden is given a 65 per cent of winning the election, that means he has almost a one-in-three chance of losing. However, most voters who hear a 65 per cent chance imagine a large probability.
Contenders argue that political oracles have created such large margins of errors and caveats that they can say they were right no matter the result, rendering them effectively useless. Silver has heavily rebuked the narrative that the polls were wrong, writing that his organisation had rightly predicted that Biden could survive a normal or even slightly larger polling error and still win. “Voters and the media need to recalibrate their expectations around polls — not necessarily because anything’s changed, but because those expectations demanded an unrealistic level of precision — while simultaneously resisting the urge to ‘throw all the polls out.’ … If you want certainty about election outcomes, polls aren’t going to give you that — at least, not most of the time.”
How are institutions and people responding?
While the analysts prefer to present their arguments in contestation to each other, the general reflection across the board seems to be relatively cohesive: lessen the obsession with polls.
“Much of American democracy depends on being able to understand what our fellow citizens think. That has become a more challenging task as Americans sort themselves into ideological bubbles … Public-opinion polling was one of the last ways we had to understand what other Americans actually believe. If polling doesn’t work, then we are flying blind,” wrote David Graham of the Atlantic.
Some have problematised the entire numbers game itself, not just 2016 weights versus 2020 weights. Silver rose to fame in predicting baseball games, “but unlike in baseball … this game doesn’t always have a predictable set of rules that all players abide by. There’s much more noise in the signal that can interfere with an algorithm,” said Slate politics editor Joshua Keating.
“We should take the money we spend on pollsters and we should put to organising on the ground. My understanding is that Trump had people on the ground for a year in Florida. I would like to see us relying less on polling because it’s increasingly becoming less perfect in getting us what we want,” Congresswoman Pramila Jayapal said in a webinar the day after elections.
Don’t miss from Explained | Why Americans disagree over granting statehood to Puerto Rico and Washington, D.C.
News organisations are similarly investing more in internet trend analysis and local news coverage to make up for the polling errors.
Tufekci said: “Instead of refreshing the page to update predictions, people should have done the only thing that actually affects the outcome: vote, donate and organize. As we have found out, everything else is within the margin of error.”
In this Nov. 7, 2020, file photo Vice President-elect Kamala Harris holds hands with President-elect Joe Biden as they celebrate in Wilmington, Del. (AP Photo/Andrew Harnik, File) 
Share
