Abhijit Banerjee and Esther Duflo are directors of the Abdul Latif Jameel Poverty Action Lab, a research centre at the Massachusetts Institute of Technology. In this session moderated by Deputy Editor Harish Damodaran, they speak about their experiments with social sector schemes in India, why NREGA does a “poor” job of identifying the needy and how RTE has contributed to worsening learning levels in schools.
Why Abhijit Banerjee and Esther Duflo?
MIT economists Abhijit Banerjee and Esther Duflo are known for using ‘randomised controlled trials’ to evaluate the success or failure of government intervention programmes. The Abdul Latif Jameel Poverty Action Lab, a global research network of which they are directors, has undertaken several such “field experiments” to see how MGNREGA, micro-credit and other social welfare schemes have worked on the ground. More recently, they conducted a series of programmes with NGO Pratham to see if educational outcomes can improve by teaching children on the basis of their learning levels rather than their grades
Harish Damodaran: What have your studies using impact evaluation or field control techniques revealed in India? How many programmes have you evaluated?
Duflo: At the moment, we have 35 active projects. The Abdul Latif Jameel Poverty Action Lab (JPAL) has been active in India since 2007 across sectors — education, environment and health. In NREGA, both of us were involved in a project in Bihar. It is a place where there is low implementation of NREGA. Very few people get it, though there is high demand for it. So we proposed to pilot a system that if you want direct benefit, it should not be to the individual but to the panchayat. So now, when the panchayat needs money for something, they don’t have to ask money from the block, which asks the district and then the Centre. Instead, you can write an electronic invoice that goes straight to the central bank of India, which sends the money into the panchayat’s account. The panchayat then gives the money to the postal account of the beneficiary. In Andhra Pradesh, we worked on a direct transfer scheme, where money goes to the beneficiary instead of going to his postal account system. In the course of it, we were dismayed that in places where the programme was put in place, there was a drop in NREGA expenditure. This was not something we were expecting. But another aspect that was much more important was the effect on transparency. Now you cannot get money unless you say I want to give money to Abhijit, so it becomes very easy for someone to go and check if Abhijit exists. We noticed a significant drop in the number of ghost workers and that reduced leakage.
Surabhi: The government has encouraged people to open accounts under the Pradhan Mantri Jan-Dhan Yojana. But the dormancy level in these accounts is very high. What are the behavioural changes that can lead to more savings in the formal sector?
Banerjee: Part of the problem is that people don’t put money in the account, and if they do, they don’t necessarily take it out. It makes sense to route some payments that the government makes to their accounts. Once that happens, maybe we’ll start to see some money staying. If people had to put money in their accounts, they would think, ‘Why bother? I need this much money and I’ll use it’. But if the money started in the account, then he might say, ‘I don’t need all of it right now’. Money coming first to people’s hands and then to their accounts will be different from if money first came in people’s accounts and then to their hands.
Duflo: Banks aren’t poor-people friendly. Yet, there are things we know will help, like cellphones. If people knew they have easy access to money, even if they don’t have it in the form of cash, they’ll start putting it in their accounts.
Anil Sasi: Does technological intervention — like M-PESA in Kenya that uses cellphones as a platform — help in targeting schemes better? Does that work better than your experience with schemes in India, where NREGA and the rest have been criticised for not targeting that well?
Banerjee: M-PESA is not meant to target. It is a universal platform and the middle-class uses it. That’s part of the reason for its success. NREGA targets very poorly, for odd reasons. The idea of the programme was to be responsive. That is, when I need a job, I’ll get it. But everything in the system is against it. Most months of the year, there are no jobs. Second, it is dependent on how interested your panchayat head is in running the programme. The third thing is that the state pools run dry. Two years ago, in Bihar, the state pool ran dry because now the rule is that you have to upload 60 per cent of your muster roll before your next cash inflow comes. If you haven’t done that because some panchayat has been defaulting, you are in trouble. There was a World Bank study in Bihar that found that if you gave uniform transfers to every rural person, it will be targeted better than NREGA.
Harish Damodaran: Would you then advocate scrapping NREGA and replacing it with a universal programme like the one you suggested?
Banerjee: You could do a universal programme where, every week, you go to a place, stand in a line, use biometrics and collect money. Nobody in this room will do it, it is too much of a pain. That generates excellent self-selection. We should have something much more like the old food-for-work programme. This elaborate system of checks and balances has made NREGA much more manipulation proof. But as a result, it has become a pain. So we need something in between so that when push comes to shove, we know that we’ll go somewhere and get money.
Sunil Jain: But isn’t NREGA self-selecting in the sense that only some people would go for these jobs?
Banerjee: NREGA is a very costly way of doing self-selection. With cash transfers, you have to combine them with very low-intensity self-selection. I have to go put my fingerprint in the machine and the machine gives cash with no intermediary — only the ATM and biometric recognition.
Yamini Lohia: We just saw another ASER report that highlighted rather dismal learning outcomes for Indian children. What sort of interventions do you think will improve outcomes?
Duflo: This was the tenth year of ASER and there has been very little progress in the last few years. We are failing the children on a massive scale. There has been improvement in enrolment and in the physical capacity of schools. But learning is not about enrolment, teacher-student ratio, having latrines in school; it’s about if we are serious about learning. But for first time, we have a plus side to this. We have been working with Pratham to evaluate programmes that can be scaled up and we are finally confident about two programmes. You don’t try to teach nuclear physics to someone who does not do subtraction. Instead, you start with subtraction. If someone doesn’t recognise numbers, you start by teaching them that. We organise kids by their level of learning and then move to the next step. You could do it by setting up camps during school hours and sending volunteers to teach. The other way is to embed it in the school system. But the problem is that teachers willy-nilly come to school and their day is focused on trying to teach the curriculum with a big C. We tried both models — teacher model and camp model. In Uttar Pradesh, we tried the camp model — 50 days over the school year, run by volunteers. The level of school is horrendous there, kids learn next to nothing in the course of the year. But the improvement due to the programme was important. We found that only 27 per cent of the children who didn’t do the camp can recognise a paragraph by the end of the school year as compared to 50 per cent for those who attended the camp. This is scalable. In Haryana, we tried to teach it through the government system. That didn’t have such a spectacular effect as in UP, but the results still went up — from 47 to 55 per cent. I hope this is scaled up. I think there is a big stumbling block in acknowledging that you cannot go anywhere unless you try to address this problem of learning first. The Right to Education Act did a wonderful job of making this problem worse because the Act replaced testing with this bizarre Continuous and Comprehensive Evaluation (CCE) which is neither comprehensive nor continuous.
Banerjee: I think this in some sense is a problem of our inherited educational culture. We have a culture that is inherited from a colonial education system, which was explicitly aimed at recruiting an elite class that was going to work for them. We still think that anybody who can’t learn reading by themselves in Class 1 is a little bit of a buddhu. I think in some sense, a fundamental cultural shift is needed. The Right to Education makes it legally binding on the school to teach the syllabus. Parents have bought into that. Everybody thinks the priority of the school system is to cover all this material, instead of the idea that every child should be able to read, every child should be able to do math.
Uma Vishnu: How important are assessments in improving learning levels?
Duflo: The system puts no emphasis on assessments. That’s the problem. The Right to Education removed that. Children aren’t tested at all. Teachers are too busy doing CCE, which is so complicated that they have completely abandoned the idea and just mark everyone present. Even if you want to teach children at the right level, you will not even know where the kids stand. So assessment is a fundamental tool because there is no other way to measure what you have achieved with the kids. Reintroducing some measure of assessment in the classroom is important.
Banerjee: The teacher might very well know which children can’t read. I am not sure if they have to test them to know that. What’s really important is that there should be some way for parents to know how the school is doing on an average. I don’t think it is so important that we, the public, know that Esther is failing her second year, Class II, arithmetic. It is very important to know that Esther’s school is failing every child. I think the key role of assessment is in empowering the community. What is not there in the system — and which now becomes impossible to put under the Right to Education — is knowing where the class stands, and you need to know whether or not this teacher is doing their job. The Right to Education is a right which should be measured by whether the children are learning. Why is it a right if it cannot be monitored by anybody except by the potentially guilty party — the teacher? Any right needs some externalisation, and I think by not having any evaluation, we have removed that one piece of externalisation.
Duflo: It is also possible that the teacher doesn’t know which child reads and which doesn’t. When we asked teachers in UP to rate whether the kids could read that one paragraph, they didn’t know. They misclassified the kids and also completely overestimated the average. So teachers also need that tool of evaluation. I don’t think there is any way to progress without assessment.
Banerjee: What is really important is the essentialisation of children, which is to say you are capable or incapable of doing something. In some sense, that has nothing to do with whether there are tests or not. Japan has an intense assessment system where the child is told, ‘You can do this, you have to be able to do it’. The difference is not in whether you are assessed or not but how the information is sent back to you. It has to come back to you saying, ‘Look, if you are not doing this, that is our failure. We’ll make sure you do it, you’ll get there’.
Amrith Lal: Does teaching in the mother tongue improve standards and quality of education?
Banerjee: This is a very difficult topic, very sensitive. I’ve seen one study — from Mali. In Mali, everyone used to be taught in French, which is nobody’s language and there are lots and lots of local languages. So they switched to teaching in a number of local languages and they saw large gains. But this is a very fraught topic and it isn’t clear what you are trying to achieve. I don’t know if there’s any hard data on this but in my parents’ generation, people would start learning English when they were 10 or so and they learnt just fine. So once you master one language maybe you could learn the others faster. I think there’s some interesting stuff to be done by following children over a long period of time and figuring out what happens.
Amrith lal: While there are obvious concerns with the education system, how do you think that will affect their employability?
Banerjee: Tamil Nadu and Himachal did the PISA (Programme for International Student Assessment) and India was ranked second from the last. So if we are really that far behind, there has to be a problem. I can’t imagine that this has no consequences for our employability. It is not so much employability as productivity.
Archna Shukla: On the one hand, there are these dismal learning figures and, on the other, the Indian economy is touted to be growing rapidly. Do you see some sort of contradiction in our economic model?
Duflo: Right now, I don’t think people are fully aware of the extent to which the children are being cheated. If and when they become aware, there could be a backlash, even a political backlash. Scores and scores of young people will get out after 10 years of education and think, ‘I got nothing out of 10 years of education. Give me a job that corresponds to my 10 years of education’. And then, it could take the form of a pretty nasty political resentment.
Banerjee: We invested heavily in a lot of elite human capital but the economy was not using them very well. Those people are doing very well, they are globally wealthy and well-educated but I don’t know if that has any guarantee for anybody else.
Raj Kamal Jha: It’s been eight months of the new government. As someone who works in the social sector, how do you see there this shift in Indian politics?
Banerjee: Nothing much has changed in the social sector. The fact is that much of the constructive innovation was driven by the states, whatever their politics. So it wasn’t as if the pro-poor Communist government of West Bengal was particularly interested in fixing anything about the education system. I don’t see an obvious correlation that more right-wing governments are less interested in the social sector. In some governments, like Tamil Nadu and Andhra and Telangana, there seems to be a drive from the top to do something about these things. Then there are other states like UP where there is absolutely no interest in these programmes, whatever the politics of the government. Rajasthan seems to be innovating in some ways. It seems to be a mixed bag. I don’t think the politics of the party itself has anything to do with it.
Transcribed by Vandana Kalra and Nikita Puri