Bias exists in survey responses, but also in government’s own telling
The immediate cause for suspecting the genuineness of survey responses is the divergence in the estimates of households with access to toilets.
The data show that in the rural sector, only 54 per cent of the respondents were the heads of the households surveyed.
Differences between survey estimates and comparable data from administrative sources are not surprising. The survey data are believed to present a more realistic view, especially when it relates to access to public goods and services. Generally, the distrust is more on administrative data from implementing agencies. While the National Sample Surveys (NSS) are not meant to evaluate government programmes directly, its estimates, with a given margin of error, help in an independent assessment of outcomes. Impact assessments are, however, mostly done through specially designed studies.
In a recent article (‘Between the lines of a survey’, IE, December 4, 2019), the writers, one of them being the Chief Statistician of India, have claimed that surveys measuring the impact of government programmes have become less reliable. Several factors are stated to support this conclusion. These mostly deal with the tendency to conceal information. They claim that there is no incentive for people to correctly report their well-being in surveys; rather they perceive an incentive in concealing information on access to goods and services provided by the government.
The immediate cause for suspecting the genuineness of survey responses is the divergence in the estimates of households with access to toilets. The NSS survey pegged households reporting access to a toilet at 71.3 per cent. But when compared to claims of achieving complete access for all rural households or the estimates from the National Annual Rural Sanitation Survey, the NSS estimate falls short of official acceptability. The same survey found the percentage of urban homes with access to toilet at 96.2.
OPINION | The new Indian Muslim
The earlier NSS surveys showed very high figures for rural homes without access to toilets, though there were schemes for rural sanitation even then. The authors of the article claim that this was due to the schemes remaining mostly on paper unlike now when these are actually reaching the people. The fact remains that there has been a creditable reduction in households without access to latrine as per the latest survey, evidently due to the Swachh Bharat Mission (SBM).
It is well known that cooperation from survey respondents has been dwindling. This writer was part of the NSSO for many years and had the opportunity to observe respondents from close quarters. In the early years of the NSSO, the villagers genuinely believed that these surveys were meant to inform the government of their living conditions and responded favourably to field investigators, often providing them accommodation and other assistance to conduct interviews.
Experts agree that respondent cooperation cannot be taken as granted and it requires sustained efforts to cajole them to sit for interviews. Interviewing the head of the household is not always possible and data have to be gathered from other members. Non-government survey agencies have the flexibility to recruit local investigators and adopt innovative practices, which are not available to government agencies like the NSSO.
OPINION | Faiz and a dying pluralism
However, the temptation to attribute the divergence between government claims on toilet use and survey estimates to untruthful reporting by respondents to extract more benefits overlooks the survey environment. The assumption that respondents have nothing to gain by admitting to having access to facilities is an oversimplification of the data collection scenario, as is accepting the achievements reported by implementing agencies against given targets as the gospel truth.
The data show that in the rural sector, only 54 per cent of the respondents were the heads of the households surveyed. It was the spouse of the head who responded in 31 per cent cases, while in the rest, the respondents were other members. Usually, information relating to financial aspects of the households is better known to the household head. But, information on access to facilities like toilets can be disclosed by any member accurately and does not suffer from recall or respondent bias.
An important factor in constructing a toilet is the availability of space. The survey shows the average area of a house in rural area was just 46.6 square metres. The data on land possessed shows that around 14 per cent of rural households possessed less than 50 square metres of land, while 23 per cent possessed land between 50 and 200 square metres. Forty per cent of households that reported having no access to a toilet belonged to these two categories.
Sweeping generalisations on respondent’s behaviour is an oversimplification. Bias does exist in survey response just as administrative reporting has an inbuilt tendency to reflect positively on performance.
The writer is former acting head of the National Statistical Commission
This article first appeared in the print edition of January 4, 2020 under the title “Bias Here And There”
Share
