Shamika Ravi is wrong. It is unfair to blame survey methodology when results disagree with a view point

The key argument emerging from this is that survey mechanisms are archaic and fail to assess the rapid changes India is experiencing. The overestimation of rural population in national surveys, including NSS and NFHS, is cited as evidence of this lacuna. The poor response rate by the wealthy is also quoted as an additional reason for the inaccurate reflection of the country’s progress in the surveys.

The national data gathering system has gone to great lengths to make the samples statistically representative. These are always drawn separately for rural and urban areas and the geographical regions left out for reasons beyond control are specified. The population estimated from NSS has always been lower than the Census numbers or its projected figures.

Consequently, the organisation desists from reporting national level estimates in absolute terms, except in establishment surveys. This discrepancy was examined in the past but no specific fault in the survey methodology was identified. In fact, the estimated number of households in NSS is closer to the Census number. It is the household size which is lower compared to that of the Census. This is more so in urban than rural areas. Non-sampling errors have been considered as the possible cause.

The key point highlighted by Ravi is that the share of rural population estimated by NSS/NFHS post 2011 Census is much higher than the projected share of rural population from the Census. Population projection is based on assumptions on fertility, mortality and migration using appropriate statistical models. The urban population is projected using the urban-rural growth differential (URGD) from the period 2001-2011, which implicitly takes into account increases both in statutory and Census towns. Now, as we move away from the Census year, the NSS survey frame will include all the towns of the previous Census (not only the statutory towns, as mentioned in Pronab Sen’s article, ‘Statisticians aren’t stupid’, IE, July 10), along with their notified areal expansion and the newly created statutory towns.

Understandably, the settlements that could have been identified as Census towns in the survey year are not included. The resulting difference in the population figures has been noted as evidence of an archaic sampling frame.
What the NSS or any survey agency does is to generate statistically valid estimates separately for rural and urban areas and mostly report percentage values rather than absolute numbers under different categories. Their procedures are not designed to estimate population size in the survey year.
Ravi is right in suggesting that using a lower weight for the urban component, estimated through a survey would, to an extent, reduce the national figure. She argues that the Census has been conservative in its urban projections as India achieved the urbanisation level projected for 2016 in 2011.

Interestingly, using the similar URGD method, the UN has projected a much higher share of urban population and given three alternate scenarios. It should then be possible to pick up the projected urban population, depending on the narrative of progress we want to project for the country, especially when the last Census data are over 13 years old and the RGI has not made efforts to reconsider its urban projections, despite several representations.

Ravi notes a relationship between response deficits and wealth levels which is well known. In most surveys of the US Bureau of Labour Statistics or the US Census Bureau, the response rates are far lower than in NSS. The substitution method used in NSS for non-cooperating households ensures that the response rate is near cent per cent. In a typical NSS household survey, the head of the household or the spouse responds in almost 95 per cent of the cases. In urban areas, nearly 50 per cent of the respondents are females, this being only around 35 per cent in rural areas. The observation that the response rate of men falls significantly with wealth needs investigation, even if anecdotal evidence may suggest it. While it is universal that household surveys are increasingly becoming difficult due to non-cooperation of households and non-access to richer households in urban areas, no amount of statistical reforms can change this. What is needed is greater awareness regarding its importance among people and high quality training and professionalism of the survey staff, as adopted in NFHS.

Sampling theory and practice is a scientific discipline. This is not to suggest that all surveys conducted by official agencies have uniform quality and little agency bias. Some of the surveys conducted by MOSPI have also been found to be of indifferent quality due to engagement of outside agencies. Also, unlike values estimated from physical experiments in laboratories, statistical estimates have sampling and non-sampling errors. But in case of NSS, NFHS etc., attempts are continuously being made to determine and eradicate these. It is unfair to blame the survey methodology when survey estimates are in disagreement with a certain narrative. We must guard against use of surveys and studies whose methodologies are opaque, sample sizes too small or selection process too purposive, just because they support a particular view point.

Mohanan is former head of the National Statistical Commission and Kundu is Senior Fellow of the World Resources Institute