After the nominations of Hillary Clinton and Donald Trump for the US presidency in late July, it seemed obvious that the US election would be the most contentious in history. This is my fourth article on the US elections.
A very large part of my articles has focused on the margin of victory for Clinton. Well, not unlike T.S. Eliot, I come back to where I started — a double-digit win for Clinton. In large part, this propulsion to 10 per cent (from a 7.3 and 4.2 per cent victory margin of Obama in 2008 and 2012, respectively) is due to differences in support from the two groups that Trump has alienated the most — women and Hispanics. My estimates have diverged significantly from the US polls (see graph for various estimates), and done so for the last three months. Mine have stayed close to a landslide win for Clinton — the opinion polls have moved all over the map but stayed close to the 2-4 per cent victory margin for Clinton. Why this large difference in the forecasts?
One major difference is that I (like most Indian pollsters) look at the historical and demographic (caste, race, sex, region, age) propensities to vote. These micro propensities are then “filtered” by opinion poll estimates to arrive at an estimate of the final vote shares. Thus, this demographic “long-run” estimate, barring huge unexpected events, does not lead to much change in forecasts during the course of a political campaign. I have practised and tested this method in several state and national elections in India, and done so for almost 30 years. [See ssbhalla.org for a (partial) sampling].
In contrast, for American (and British) pollsters, the demographic and/or historical detail is secondary to the “easy” estimate provided by the numerous daily opinion polls. The pollsters look for shifts in the polls, and make a forecast. Hence, the wide variation, over time and across polls, in the estimate of the final vote. This has worked well in the past — and also horribly. Most polls in November 2012 had forecast a tight race, and the most prestigious of them all, Gallup, actually forecast a Romney victory. The polls have also gone wrong in several referendums, as well as in the US Republican primaries.
My latest “filtered” estimate of the presidential vote is a Clinton win, 52-42 per cent. Interestingly, sharp differences in election forecasts have emerged in the US during the course of the last few days. Of course, no one knows (for sure or with 100 per cent probability) which way the US election will go, and almost every pundit has opined. A very large proportion of the pollsters agrees that Clinton will win — the differences are in the estimates of the probability of winning. The pollster who got it all right in 2012, Nate Silver of 538 fame, has broken ranks and has been forecasting a close election — Clinton with only a 67 per cent chance of winning. In addition, on Sunday, November 5, he opined that Clinton was only one state away from losing the electoral college. The US is now very competitive in sophisticated election analysis — compared to 2012 when Silver was close to a lone ranger. Now, snapping at his heels are websites like RealClearPolitics (RCP), Huffington Post, and New York Times Upshot, to name just a few. A few days ago, the chief pollster at Huffington Post, Ryan Grim, accused Silver of putting “his thumb on the scales” in an article titled “Nate Silver Is Unskewing Polls — All Of Them — In Trump’s Direction”.
This divergence captures well the different philosophies of polling. Grim appears to be more like an Indian pollster who does not take each twist and turn in the opinion polls as the gospel forecast; Silver is a classic American pollster who adjusts the polls as the data seems to suggest. He has a right, and authority, to do that because he is the pre-eminent pollster in the US today. If Clinton wins narrowly (or loses), then the Silver version of forecasting will get a push up; equivalently, if Clinton wins easily (with a margin greater than five per cent), Silver’s method will come under more questioning.
The prestigious and experienced Gallup organisation was wrong in the 2012 polling. The dent Gallup received from that wrong forecast has been deep. So watch out for the election results Tuesday night. Apart from Clinton and Trump, it is likely to affect the fortunes of pollsters — and their methods. Election polling, and forecasting, is a risky business.
Talking about risk — let me add my (risky) forecasts on other aspects of the US election. All estimates are based on a historical analysis of US election data since 1948. Vote shares: Most estimates are below 48 per cent for Clinton, with 538 forecast at 48-45 per cent, that is, a narrow win for Clinton. My estimate — a double-digit winning margin for Clinton.
Electoral college (EC) votes: If the winning margin is one per cent, Clinton obtains 303 EC votes and wins 26 states; with three per cent margin, she adds Florida for 332 EC votes; with a 10 per cent margin, she adds North Carolina’s 15 votes to reach 347.
So look out for Florida and North Carolina to get an early “forecast”. According to data reported by NYT’s Upshot, the average estimated probability of seven pollsters is 65 per cent for a Clinton win in Florida and 63 per cent for a Clinton win in North Carolina. Senate: Most estimates converge on 50 to 51 seats for the Democrats. We concur.
Congress: Forecasting the election to the House of Representatives is truly tricky. Very few pollsters attempt this difficult task. We undertake it with some trepidation. Our model, with a 10 per cent victory margin for Clinton, results in “too close to call” result with a slight eight-seat-edge to the Republicans.
That’s it folks! The long wait will soon be over. As everyone has noted, this is not an ordinary election. The future of Western civilisation (literally) depends on the outcome. We are confident of our forecast; if we are wrong as in Trump winning, or Clinton winning by a narrow 2-3 per cent margin, we will be very unhappy, and not just for making a wrong forecast.
The writer is contributing editor, ‘Indian Express’, and senior India analyst at Observatory Group, a New York-based macro policy advisory group.