© The Indian Express Pvt Ltd
Tags:
Journalism of Courage
Elections, in the true sense, are a festival of democracy. There is anticipation for the announcements of dates, for the nomination process to begin and for the election campaign to commence. In a phased election such as the current one, the first and last days of voting also become big events. But for many in India, the most awaited event is the day when exit polls are released.
Exit polls give estimates about how people voted in an election. They are arrived at on the basis of interviews with voters right after they exit the polling stations (hence exit polls), as well as other calculations related to voter data.
A large number of Indians give the same importance to exit polls as they do to the actual results. Normally, exit polls are released on the last day of voting, as agencies conducting such polls are mandated by the Election Commission of India (ECI) to wait until polling has been completed in all phases. This is to avoid influencing voters who are yet to vote.
On the day when exit polls are finally released, pollsters often give varying estimates. It is interesting to see that people generally take interest in those exit polls whose estimates are closer to their political preferences. They have little interest in looking at the vote share estimates, which some polls indicate, leave aside looking at their methodologies. More often than not, the accuracy of exit polls is judged by personal opinions on political parties.
When exit polls for the ongoing Lok Sabha elections are released on the evening of June 1 (Saturday), this phenomenon is likely to be repeated. Many television channels will rush to air the polls. Unfortunately, there is greater competition for being the ‘first’ to show the exit poll numbers, than about having the superior quality of data.
Many exit polls have also proven erratic in recent years, throwing up conflicting results. Last year, many polls estimated incorrect winners in the Legislative Assembly elections for Chhattisgarh and Madhya Pradesh, while some were way off the mark in Rajasthan.
There is no pattern to this – an agency may have predicted the results for Madhya Pradesh correctly, but made a mistake in Rajasthan’s case. There were states where all of the exit polls were correct (Telangana) and others where everyone was wrong (Chhattisgarh). In such a situation, how does one read the exit polls which will come out on Saturday evening? How do you judge their accuracy?
Today, some judge the accuracy of an exit poll by looking at the survey agency that conducted it, or the television channel that commissioned it. Some others look at the sample size — a common notion is that the bigger the sample size, the more reliable the poll. In reality, these should not be the indicators for judging the accuracy of an exit poll, which depends on many different factors.
The science of surveys, which includes exit polls, works on the assumption that data were collected after interviewing a large number of respondents using a structured questionnaire, whether over the telephone or face-to-face.
This method is not new; it began back in 1957 during the second Lok Sabha elections when the Indian Institute of Public Opinion conducted a poll. But not even the best guesswork or estimate can skip the methodology required. Without a structured questionnaire, the data can neither be collected coherently nor be analysed systematically.
Since exit polls began in 1957, there has been enormous improvement in at least one aspect, which is sample size. Gone are the days when a national sample of 20,000 to 30,000 respondents was considered large. Today we have survey agencies conducting exit polls with samples as big as 10 lakhs. Exit polls of a few lakhs samples have become very common today.
Though the Lokniti-Centre for the Study of Developing Societies (CSDS) does not normally conduct exit polls, it did conduct a few — I recall the first exit poll during the 1996 Lok Sabha elections, using a sample size of 17,604. We went on to make a very accurate national projection of both vote share and seats.
The Lokniti-CSDS has continued its voting behaviour study (National Election Study) using post-poll surveys as a tool. Our post-poll survey of the 2019 Lok Sabha election had a sample size of little more than 25,000. Our seat projections may have been off the mark on some occasions, but the vote share estimates have been very close. (Information in public domain).
Yes, a large sample size is important, but based on my experience, I can say that more than the sample size, what matters is how representative it is, if it reflects the various types of voter profiles. But in recent years, the pressure on television channels (which in most cases are the sponsors of these exit polls) to have the largest sample has resulted in exit polls with bigger and bigger samples.
Our (Lokniti-CSDS) predictions about the Chhattisgarh state elections were wrong both in 2018 and 2023, and both were based on post-poll surveys. In the 2017 Uttar Pradesh Assembly elections we predicted the winner correctly, but were way off the mark for the final tallies for different parties. If asked whether a bigger sample would have meant more accurate results, I would not have hesitated to say no. Certainly, something else went wrong with those post-poll surveys; maybe a case of some fake interviews filed by the investigators, which we could not figure out on time.
Technology — call-backs to respondents, images of interviews being conducted and phone calls from the field, WhatsApp groups and similar tools — have helped us in overcoming such shortcomings, yet there is no thumb rule for how to get the prediction correct.
There are other challenges. The prediction of seats is based on a swing model — the poll makes an estimate of vote shares for different parties and alliances by interviewing selected respondents, and the seat forecast is made based on the result of the previous election.
Estimating the vote share is not an easy task, given various diversities in India — of location, caste, religion, language, levels of education, economic class — all of which impact voting behaviour. Over- or under-representation of any of these diverse sections of voters can affect the accuracy of estimates.
If these were not enough, there are other difficulties. Since the swing model is applied on the previous vote shares, a change in alliances, or a split or a merger of parties between two elections poses a difficulty. For example, BJP and JD(U) in Bihar.
Measurement of swing and electoral change is easier when the contest is limited to two parties. The complexity of swings increases as more and more political players are added.
How comprehensive it is
The count method is time-consuming and labour-intensive, as one is expected to make an estimate for each seat. When agencies claim to have made seat-wise estimates, it is presented as the most comprehensive poll. This is when the sample size is as high as several lakhs.
However, some agencies have made innovations in the count method and this results in maximising gains by spending relatively less time and resources.
While an exit poll might claim to have covered all constituencies, in practice the poll is not required in some seats — for instance, why would one waste time and energy in conducting a poll in Varanasi where the Prime Minister is contesting, or in Gandhinagar where the BJP president is contesting? If one looks carefully at constituencies, state-wise, many such seats could be eliminated and one could still make the most accurate estimate.
After this elimination method combined with the count method, the survey is required in a limited number of difficult constituencies (swing constituencies). It is possible for an innovative exit poll to be far more accurate than polls conducted using traditional methodology. But while the polls using traditional methodology estimate vote share and help us analyse voting behaviour on the basis of different socioeconomic backgrounds, the count method can hardly give an estimate of vote shares, and any systematic analysis of voting behaviour could only be a dream.
Time to reflect
Many exit polls just throw a number for seats, no vote share, no methodological details. Should we even consider these as exit polls? I think the time has come when we need to distinguish between a real exit poll and estimate polls.
Vote share estimate is mandatory for any poll. If one is not estimating the vote share, the question that should be asked is: then what were you doing while conducting the polls? And what innovative method did one apply for conducting a poll that does not estimate votes, but predicts seats?
Sanjay Kumar is a Professor at the Centre for the Study of Developing Societies (CSDS)
This is an updated version of an explainer published in 2023.
