Follow Us:
Thursday, December 12, 2019

It is time we recognised that survey data cannot be interpreted in the way it used to be

India was a pioneer in data collection some 70 years ago. It should be recalled that GDP data in the first 30- odd years was based on the consumption data collected by the NSSO. Poverty rates, and policies, were based on NSSO data.

Written by Surjit S Bhalla | Updated: July 27, 2019 10:19:11 am
Credibility of government data We do not know a priori whether the NSSO data are a correct reflection of the underlying reality, or the national accounts data produced by the CSO are a better proxy.

Correct analysis of data is imperative for appropriate policy response. This is obvious. The reason I am repeating the obvious is that there are serious problems with the data that we, policymakers and academics, are basing our policy decisions on. Problems with methods and accuracy of data collection. The discussion on data for policy has been across the board — socio-economic (riots, infant mortality rates, sex ratio etc) and economic (GDP growth, exchange rate, poverty etc.). In this article, I would like to point to some of the perils of poor data quality and often erroneous interpretation.

As we all know, and appreciate, India was a pioneer in data collection some 70 years ago. The National Sample Survey Organisation (NSSO) was set up in 1950 and has been providing important information about most of our everyday concerns — consumption, employment and prices. It should be recalled, and remembered, that GDP data in the first 30- odd years was based on the consumption data collected by the NSSO. Poverty rates, and policies, were based on NSSO data.

In my 2002 book, Imagine There’s No Country, I reported that something was amiss with the then recent NSSO data on consumption. The 1999-2000 NSSO consumption survey (S) reported average per capita consumption to be only 56 per cent of consumption as measured by national accounts (NA) — this was termed survey capture or S/NA. In China, close to the same year, the S/NA ratio was as high as 96 per cent; in Korea 85 per cent; in India in 1960, 92 per cent. Only 10 per cent of some 700 consumer expenditure surveys in the world had a S/NA ratio less than India in 1999.

We do not know a priori whether the NSSO data are a correct reflection of the underlying reality, or the national accounts data produced by the CSO are a better proxy. This has to be based on hard empirics (what is now fashionably called Big Data Analysis) and not on ideological or other predilections.

That there might be a problem with the NSSO consumption data reports was recognised by the first National Statistical Commission (under the late Suresh Tendulkar). I was a member of this commission and had numerous discussions with the NSSO authorities in Calcutta. The NSSO survey capture deteriorated further in 2009 and 2011, dropping to an average of 48 per cent in the two years.

The demotion of NSSO data became a hard reality at the time of the GDP revision in 2015. Until this revision, the growth of employment as revealed by the Employment and Unemployment Surveys of the NSSO (now called PLFS) was used to estimate growth in value-added in the wholesale and retail trade sector of the economy (accounting for some 15-20 per cent of the GDP). For example, employment growth in the wholesale sector during 1999-2004 was 3.2 per cent per annum. This rate of growth was assumed to be the same rate — that is 3.2 per cent for every subsequent year between 2004 and 2011. Post the availability of the 2011-12 NSSO employment survey in 2013, the new employment growth “reality” hit the CSO — employment had barely grown, across sectors and in wholesale, during the seven high GDP growth years 2004-11 — only 0.3 per cent a year, rather than the assumed 3.2 per cent.

The CSO, as a responsible statistical outfit, had to come out with a new method to estimate GDP growth in the wholesale trade sector — they came up with the eminently sensible idea of deriving employment growth via the (real) growth in state value-added taxes.

The reason for this extended discussion is to put into perspective the recent debate on the PLFS data — unemployment at 6.2 per cent, a 45-year high, etc. There will be another occasion to discuss the employment aspects of the PLFS data. Today, I want to discuss a little-known, and troublesome, aspect of the PLFS 2017-18 data — its perspective on consumption behaviour.

It is not well-known that NSSO employment surveys have a smaller consumption module — some 30 questions on household consumption rather than the 300-odd questions in the consumption survey. Between 2004 and 2011, this smaller module tracked the growth in consumption reasonably well — the average rate of growth of per capita consumption in the two NSSO surveys (smaller module employment and the exhaustive consumption) were similar to each other and to the growth of consumption as reported by the national accounts.

Results of the 2017-18, NSSO consumption survey have not been released as yet. But details of household consumption in 2017-18 are available via the recently made public unit-level PLFS data. These data are a shocker — it is surprising that those questioning the accuracy of GDP data post 2011-12 have not latched onto these figures as well. Perhaps because they are too outlandish to be believed. PLFS 2017-18 and EUS 2011-12 consumption model show a log growth in average per capita consumption of 24 per cent between the two years. Consumer prices rose by an average of 36 per cent during the two years. Real growth in consumption — minus 12 per cent over six years. (Both 2011-12 and 2017-18 were good weather years) A real consumption decline of anything even close to this magnitude has not been observed at any time in Indian history (not even in pre-historic times!) or to the best of my knowledge in any country in the world (other than Zimbabwe and Venezuela and other hyper-ventilating economies).

How inaccurate is the consumption growth recorded by the employment surveys in 2011/12 and 2017/18? Grotesquely inaccurate. And, the reasons for this ever-declining survey capture is something that the new CSO/NSSO combine should seriously examine. In contrast, note the per capita growth in selected (consumption related) items: Total NA consumption is 34 per cent; agriculture is 11 per cent; oil is 25 per cent; electricity is 29 per cent; mobile users 29 per cent; passenger cars 31 per cent. I am leaving this for discussion at a later date, several items of greater use and declining prices (for example, telecommunications, banking services, insurance) — all point to a significant increase in real consumption and above a 6.5 per cent growth in GDP.

What is going on? Why are sample survey data missing out on underlying trends? I can think of two major explanations. (For regular readers of my columns, this will be repeating myself for the umpteenth time.) Please, please, recognise that the world economic environment has changed, and changed radically over the last 20-odd years. In particular, Indian inflation is now structurally down to the world inflation levels from being significantly above during the period 2004-2015. All of us need to understand the implications of long-term two to three per cent inflation levels. When I went to graduate school in the 1970s, this was considered frictional inflation. Just a year ago, almost to the date, the RBI/MPC raised policy rates to 6.5 per cent, warning of impending higher GDP growth (sustained above 7.5 per cent) and higher (sustained >5 per cent) inflation levels. All of us make mistakes, but the changed economic environment means that we have to stop doing old-fashioned and/or knee-jerk analysis and policy.

The second changed reality is technological change. And, the mother of all changes, of course, is climate change — and in case those caught in the past (the 1970s or even the 1990s) haven’t caught on, climate change is the most deflationary. Within technological change, there are newer methods of gathering information. In addition, modern techniques of computer aided surveys suffer from interview bias — the same question, asked of the same person by two different computer-supervised individuals, can elicit radically different responses.

Let us all embark on a new mission. Collect good quality data and analyse it in a dispassionate manner with one over-riding objective – what policy will enhance and accelerate GDP growth so that we can redistribute better. All else is old-style garam hawa.

This article first appeared in the print edition on July 27, 2019 under the title ‘Let big data analysis begin’. The writer is contributing editor, The Indian Express. Views are personal.  

For all the latest Opinion News, download Indian Express App

More From Surjit S Bhalla
0 Comment(s) *
* The moderation of comments is automated and not cleared manually by indianexpress.com.
Advertisement
Advertisement
Advertisement
Advertisement