6 months, over a dozen experts: Making of a CBSE question paper

This is the second year in a row that the maths paper has been criticised — raising questions over the process by which CBSE designs question papers. According to Board officials, the system, governed by the CBSE’s examination bylaws, contains elaborate and multiple levels of monitoring before, during and after the students take the exam.

[related-post]

Before the exam

The CBSE’s guidelines for curriculum lay down the number of periods to be allotted to each topic in class, and weightage in the question paper. For Class XII maths, calculus has been allotted 80 periods and 44 marks — the most. Algebra has 50 periods but only 13 marks; vectors and 3D geometry, 30 periods and 17 marks.

In 2014, the guidelines redefined the “typology” and “design” of the question paper. In maths, application-based questions, which “use abstract information in concrete situation, to apply knowledge to new situations”, should get 29% marks, it said. The High Order Thinking Skills (HOTS) questions, which test “analysis and synthesis — classify, compare, contrast, or differentiate between different pieces of information, organise and/or integrate unique pieces of information from a variety of sources”, were allotted 15%. “Remembering” or recall-based questions got 20%, and “understanding” or comprehension of concepts, 22%.

Paper-setting, Stage I

Setting the question paper for each subject takes six months or more. It is prepared by the efforts of 9-17 people with specific qualifications for the job. They come from all over the country, and none of them knows the identity of the others.

The process begins in August-September, when Class XII students are still finishing their first term exams. For each subject, the Board appoints 4-7 paper “setters”, each of whom has a postgraduate degree in the concerned subject or an allied subject, at least 10 years’ experience teaching the concerned subject at secondary or senior secondary level, or is employed at a state- or national-level education agency set up by the government. According to the rules, the paper setters should not have “written or revised a guide-book, help-book, key or similar other matter, with whatsoever name, relating to the subject”. They should also not have given private tuition that year, and no member of their immediate family should be appearing in that year’s exams.

The setters are given question papers of previous years, and the curriculum design and structure of the papers — their “typology” and “design”. Following these guidelines, a setter prepares a question paper in about two months on average, and sends it to CBSE.

Paper-setting, Stage II

A second set of experts enter the picture now. These “moderators” check if laid-down standards have been followed on the syllabus, difficulty level, and length of the paper. About 5-10 moderators are appointed for each subject every year. They have the same qualifications as setters, but are a different set of individuals.

“Since students from different socio-economic backgrounds and intellectual levels take the paper, this is an effort to ensure there is some parity. This is our second level of monitoring,” said a senior official.

According to the bylaws, the moderators should ensure that “each question paper has been set according to the syllabus of the subject, blue print, design and text books/recommended books”, and complies with the unit-wise weightage given in a subject’s curriculum. The bylaws state that variations of marks, if any, under different sub-units of the subject should be kept “at the minimum”.

The moderators also prepare a marking scheme, which includes expected answers, distribution of marks, and the marks to be allotted for every step of the solution. They have to mention against each question the “approximate time” needed to solve it by an “average student who has carefully studied the course and has prepared for the examination methodically”. The moderators must ensure no question is erroneously or ambiguously worded, which could lead to “an interpretation different from the question intends to convey”.

After the exam

Subject experts from across the country meet after the examination that same day to consider feedback received during and after the exam. For this year’s controversial maths paper, 17 senior teachers from all over India met. For English (Core), 11 subject experts were identified, 15 for Physics, 17 each for Biology and Chemistry. Based on examination-day feedback, the experts revise the marking scheme. “They define the marks to be allotted for every step of every question afresh. This is given to our evaluators afresh, with modifications, if any, to the marking scheme defined by paper setters and moderators,” the official explained.

Evaluators can’t see the correct roll numbers of examinees. For each of the 10 regions under CBSE, a Professor or Principal of a college, or a Reader or Senior Lecturer is appointed Chief Secrecy Officer, who then notifies a team — made of individuals who are at least college lecturers — to put fictitious roll numbers on every answer sheet. Again, no immediate relative of a secrecy officer should be appearing for the exam that year.

A Principal, Vice-Principal, or Post Graduate Teacher of an affiliated school is appointed Head Examiner for every subject. The Head Examiner has to monitor the evaluation to ensure uniform evaluation principles are being followed by all evaluators.