Andreas Schleicher, director for the Directorate of Education and Skills at the Organisation for Economic Cooperation and Development (OECD), spoke with Ritika Chopra about India ending its decade-long boycott of the Programme for International Student Assessment (PISA), which tests learning levels of 15-year-olds in reading, mathematics and science. India stayed away from PISA in 2012 and 2015 on account of its dismal performance in 2009, when it was placed 72nd among 74 participating countries. Edited excerpts:
How will you explain the reluctance on part of SAARC nations to participate regularly in PISA, and why do you think they should?
There have been some longstanding participants (from this region). Countries like Indonesia, Malaysia, Singapore have been with us for many years. However, PISA also provides a challenge for countries in this region. If you think about India, much of the instruction is still geared towards having students reproduce subject matter content, and PISA doesn’t place a lot of emphasis on this.
To be successful in PISA you have to be able to creatively use your knowledge and address novel tasks.
What are the most vivid examples of how countries have benefitted by participating in PISA?
The most significant aspect (of participating in PISA) is to be part of a global network of expertise that develop assessments. Many countries have joined the assessment simply to take part in the thinking about the future of education and what it means for teaching practice and assessment.
PISA is also a mirror in which countries can see themselves in the light of what other countries are achieving. For example, before PISA, small nations such as Singapore or Finland were not very much on the radar of education policy, but it has turned them into a kind of reference point.
In the latest iteration in 2015, questions covered collaborative problem solving, social skills, and even psychological well-being. How different is PISA going to be in 2021?
In 2018, we added the concept of what we call global competency, which tests your capacity to appreciate different ways of thinking different ways of working different cultures. In 2021, we will add the aspect of creative thinking, which has become a very important dimension of success today.
Looking back, why do you think India pulled out of PISA? What is OECD doing to address those concerns in the 2021 test cycle?
In 2009, India joined the assessment at a very rapid speed; that also meant the results were not productive and meaningful. Probably sufficient attention was not paid to adapting the instrument to the Indian context at the time. A lot has happened in India and in PISA since then. PISA is generally paying greater attention to make sure the instruments reflect the cultural and linguistic context of countries closely. India, too, is preparing carefully and we are trying to ensure the (test) instruments are translated into an Indian context.
How does PISA try to adapt its instruments to social cultural contexts of countries where it is conducted?
Most tasks in PISA are ones where students have to face a real situation. For example, when we test mathematics we don’t start with a formula. We ask students to translate a real-world problem into a mathematical problem, solve the mathematics and then to use the mathematical results to interpret the real-world problem. That means that real problem needs to be authentic to students. So if you are in a tropical country, you don’t ask questions related to the heating system, probably.
Many have attributed the success of China in PISA to the OECD practice of permitting countries to choose a representative sample, as opposed to one geographical entity. What this means in practice is the ability of some education administrators choosing their top-performing students from smaller samples in cities or city-states. How fair is to rank countries in this context?
You should never associate the results of Shanghai or Beijing with China. We never do that. China is a very diverse country with very poor regions, and they surely will not deliver the same results as Shanghai. PISA is a very demanding assessment. It’s very difficult to implement because the assessment is very innovative and requires a lot of monitoring.
We are technically not capable to do that in the whole country. So we started the first with one region in China – Shanghai – (and) then expanded to four. We would never say that those results represent China. So that’s why the tables make that distinction in the international ranking.
What are your views on the Indian schooling system? What do you think are the positives and deficiencies?
I have limited experience (on this). It’s almost difficult to speak of an Indian school system because of the diversity of approaches and outcomes. There are some world-class schools, but many are also struggling with the foundation. I think one of the biggest assets in India is the value that society, especially the poor, places on education. You wouldn’t find this in the Middle East, South America or even in Europe.
The challenge in India is around social backgrounds. You can see significant social gaps in Indian school systems that are bigger, for example, than what we see in countries like China or Vietnam. I think PISA is a good instrument to illuminate those challenges in a comparative context.
India recently scrapped its no-detention policy under the Right to Education Act. Is there any direct relation between detaining a child and improvement in academic performance?
Many people think that retention is a way to help students to catch up, but PISA shows very clearly that this is not a good approach. We have seen that students, even in the second year, often have similar results. So retention doesn’t lead to improvement in learning outcomes. What those students need is individualised support to help them where they fail.
PISA also shows that retention is very expensive, because such students enter the labour market one year later, and the government loses taxes. Retention also leads to stigmatisation of those students and makes it too convenient for teachers, who can pass a student on to someone else. That creates a mindset that I am doing the right lesson but I have got the wrong students. Whereas in countries where there is no retention, teachers have a much greater sense of responsibility to take care of students and to redouble their effort when those students fall behind. So you can see clearly in PISA that countries that do not use retention policies – such as Canada and Nordic counties in Europe – generally have superior learning outcomes and also more equitable learning outcomes.
Japan is a country that never had retention because it has a philosophy that teachers should do whatever it takes to have students succeed.
Of course, retention does not mean automatic promotion. The idea is that instead of retention, you actually provide additional support for students and redouble your effort to make sure that every student succeeds. That is essential. If no-retention policy is just automatic promotion, I agree that’s not a very good approach.
Have you followed the changes adopted by the Delhi government in schools education? What do you think of it?
I can only follow it through what I read; I have not really seen it myself. But I think these are very important changes. If infrastructure is not there, learning outcomes will not be present – and so often infrastructure is not only physically necessary, but (is) also a symbol. What message do we give children if the shopping centre is the most beautiful building and the neighbourhood school is falling apart? What I also like in Delhi is the increasing emphasis on what we call social and emotional skills. And I think that’s a very important paradigm shift in the education of tomorrow.
The academic community has expressed concerns over the impact of rankings such as PISA. Academics say this has contributed to an obsession with standardised testing and shifting focus from long-term and enduring solutions to stop-gap measures.
I have a different view on this. I think standardised tests have an important place in the world of education. Without standardised tests, we often see that disadvantaged students just fall off the radar – standardised tests put everybody on a level playing field. I agree it shouldn’t be everything, and assessment should always have multiple perspectives. There should be a formative assessment in school. Teachers should have a strong relationship with the student.
But standardised testing is basically the guarantor of fairness and equity. And it’s no surprise that actually countries which have a strong history of standardized testing — think about China, Vietnam, Singapore — are also some of the most equitable and fairest education systems. The difficulty of standardised testing is that some of the things that are very important are very hard to measure. So we need to find other ways to measure things like social emotional skills.
But how can one create an exam with standard, but meaningful measures for nearly 80 countries? How can we one decide what should be measured?
I think that’s really one of the biggest challenges that PISA faces. To have an assessment framework that adequately represents their world. But this is a hypothesis that you can empirically test. We have, over 20 years, worked very carefully to understand the cultural and linguistic and instructural context of each country. The PISA test is constructed with material from almost all of the countries. The big difference btween PISA and other standardized tests is that actually the test is developed together with the (participating) countries. Yes, PISA may emphasize aspects of instruction that are not given emphasis in one country. In the Russian Federation, for example, content knowledge is given greater emphasis and so it does less well than on other international tests. Even then PISA is considered a relevant test by the Russian authorities.
The question of what is done with the results is a different one. In my own country in Germany when the first results of PISA out came out in 2008, people were very unhappy with it. The overall performance was not great, but more importantly the social disparities were very large. So Germany looked carefully at how do countries deal with social disparities that are doing better. They sent people out to different countries to study education systems there. They came back, they redesigned education policy and practice and 10 years later actually the social disparities have diminished. I think there’s a good example how the results can be used to inspire policy development. Brazil was the country with the lowest results on PISA in the first assessment because it was the only kind of emerging economy in the PISA test then. And when the results came out, public expenditure in education in Brazil was almost doubled.So PISA would never tell you what you need to do, but it tells you what everybody else is doing. It’s basically giving you new eyes and ears to see what’s happening in the world. So I am not aware of any countries that has implemented short term, stopgaps as you call them. I am not aware of it because PISA is not a high stakes test. It doesn’t sort of have consequences for individuals or schools. It’s a sample test. In fact we are very conscious to administer PISA never in the whole of a country even in small countries. We always take a sample because we don’t want that the test gets used to sort of rank schools. So it’s a sample assessment, it’s low stakes and no concern that people would over utilize those results.