The announcement of computerized online entrance examinations for Jawaharlal Nehru University (JNU), in the form of multiple choice questions (MCQ), appears, on the face of it, to be merely yet another bureaucratic, ill-thought-out, and hasty change in higher education policy. We need to be alert to what is going on because there are two kinds of more substantial drastic transformations in higher education underway. One is the devastation of the public university system and its replacement by private players who would be given free rein to run universities as profit-making organizations. This process, however, predates the Narendra Modi regime.
The second change is related to the Hindutva ideology and the violation of every rule and norm, to not only fill universities with faculty members vetted and passed by the Rashtriya Swayamsevak Sangh (RSS) but also to control the ideological colour of the student body in universities, especially JNU. That is the context in which, in JNU, it is being argued that multiple choice questions are more “objective” than essay-type questions, that online examinations facilitate access, and are more secure and less susceptible to bias and tampering. However, these assertions are far from substantiated by facts on the ground.
Are MCQs More ‘Objective’ Than Essay Type Questions?
The university administration’s statements on the shift to online entrance exams refer to the elimination of “bias” in the selection of students. There are two kinds of misrepresentations in such allegations. First, like elsewhere, the JNU faculty’s political opinions range across the entire political spectrum. In any case, for all conscientious teachers (in JNU and everywhere else), the goal of teaching is to equip students to think critically, even about what we teach.
For example, consider two typical JNU entrance examination questions:
“Is the multiplicity of languages in India a problem or its asset?”
“Is affirmative action for disadvantaged groups contrary to the principle of equality? Give reasons for your answer. Compare the policy of reservations in India with an affirmative-action policy in the USA.”
Whether the answer considers linguistic diversity to be an asset or a problem, whether it supports affirmative action or does not, is irrelevant. Hundreds of teachers evaluate these answer scripts over weeks, then moderated by another smaller set for consistency of marking. What can be a more objective process?
47. Given below are two premises (a) and (b). From those two premises four conclusions (i), (ii), (iii) & (iv) are drawn. Select the code that states the conclusions validly drawn from the premises (taking singly or jointly).
Premises: (a) Untouchability is a curse (b) All hot pans are untouchable
Conclusions: (i) All hot pans are curse (ii) Some untouchable things are hot pans (iii) All curses are untouchability (iv) Some curses are untouchability
(1) (i) and (ii)
(2) (ii) and (iii)
(3) (iii) and (iv)
(4) (ii) and (iv)
Consider what the question assumes: that untouchability is a “curse” (a term that implies divine or supernatural intervention), not a phenomenon produced by and deliberately maintained through socio-economic-cultural structures of oppression and exclusion; that “untouchability” across different registers of the word can be equated – fingers would be burnt by touching hot pans; some humans are polluted by other humans’ touch. The question thus is politically loaded.
The second example is from the UGC NET examination conducted in July 2018:
- Given below are two statements, one labelled as Assertion A and other labelled as Reason R. Select the correct answer from the code given below.
Assertion A: The Cold War and decolonization process had discouraged more active involvement by United Nations within the state.
Reason R: States have national interest as its prime interest.
- Both A and R are true and R is the correct explanation of A
- Both A and R are true but R is not the correct explanation of A
- A is true but R is false
- R is true but A is false.
The “correct” option according to UGC here is- 1. Now, both A and R are highly debatable, and to state that either is “correct” is ridiculous. They are not facts; both are statements that come out of ideological and theoretical assumptions that are deeply contested within the concerned discipline. Why did the role of the United Nations decline? Do states act as rational individuals do? Do states have clear internally homogeneous “national interests”? The last two questions would be answered in the affirmative by the realist school of international relations, but there are substantial critiques of this school of thought as well. If A and R were posed as questions, they would have elicited the range of debates around these issues. Instead, a student has no option but to agree that both A and R are “correct” and that A follows from R. What if a thoughtful student has a critique of realism in international relations theory and chooses option 3? Even someone who accepts realism could choose 2, which seems to be an equally viable option. The point is that all these are controversial opinions, not correct and wrong answers.
Online MCQ and Human Error
Further, there is a possibility that MCQ exams may, through human error, have the wrong answer key. In 2015, not a single candidate passed the UGC NET examination for Persian, because out of 125 questions in the paper, 94 had wrong answers.
Now, here are four more disturbing features of the new entrance examinations. They are going to be entirely online. That has implications for all entrance examinations in India, not only for JNU. The remaining three features are more JNU-specific.
Two, the JNU administration requires that the answer key is provided in advance before conducting the exams. Three, the questions would be selected from a question bank prepared not only by JNU faculty members but by “experts from across the country in addition to JNU faculty”. Four, it is not clear how JNU’s deprivation-point system would be applied if at all, and what this would mean for social justice. Let us consider this one by one.
We begin with two relatively smaller (though deeply problematic) issues, then move on to a much more serious problem.
Technically Unsound and Discriminatory
In JNU, the tender for the service provider has been floated. Bids were opened on 19 September 2018, and the selected company will be entrusted with the task of conducting the JNU entrance exam in December 2018. This is hasty and is bound to result in massive glitches. Each exam site all over India has to be tested for connectivity, several test runs have to be done, and a contingency plan in the case of failure has to be planned and tested as well. How is this to be accomplished in such a short period?
In the last year alone, several online tests in different parts of India have faced serious problems. Servers have been down, and the university website has had “technical problems”; the examination paper did not open in the Common Law Admission Test held in Lucknow at 15 different exam centres, thereby substantially reducing the time candidates had to answer the questions. Earlier in 2012, there had been over a hundred hacking attempts in the medical postgraduate entrance examination, an experience that had been replicated in the National Eligibility cum Entrance Exam (NEET) (for medical and dental undergraduate programmes) too. It is noteworthy that the NEET exam is now being reversed from being a bi-annual online exam to, once again, being conducted once a year, and as a written test.
Another concern driving the change is that the computer-based exam may put aspirants from rural areas at a disadvantage.
The IIT JEE is being cited as a successful example of entrance exams with MCQ conducted online, but this is misleading. Looking at the examples above, wherein tens of thousands of students’ futures will be at risk, it is apparent that this system is unstable and full of technical glitches that have not been ironed out. The reversal of the NEET experiment is an essential example that JNU must study seriously.
Online Entrance Exams – What Are the Logistics and Who Profits?
Any evidence does not back the claim that online exams are cheaper than pen-and-paper exams. The contract for conducting the IIM CAT examination is $40 million. About 2 lakh candidates take that exam. In other words, if we assume a dollar to be worth Rs 70, the cost will be Rs 14000 per student. JNU costs (for 1 lakh candidates) may be less, but still a considerable amount for a public university.
As for logistics, most people have no clue about what is involved. We need to understand that the company that wins a tender would have to outsource the nationwide conduct of any entrance examination to the identified smaller firms all over the country that would conduct the actual examinations. These entities would be unsupervisable entities. Given the numbers of candidates involved, it is clear that computer terminals cannot be provided for every single candidate at once. The exams will, therefore, be conducted in batches at every centre. That requires, for every annual examination, multiple sets of questions, because in MCQ format, questions cannot be repeated. For example, for every 100 candidates, there would have to be at least five sets of questions, assuming every batch is about 20 candidates. There would have to be a trade-off here between the number of examination centres one can have and the minimum size of the facility that one is willing to accept.Moreover, none of these questions can be repeated in the following years – all of this because there is only one correct answer to every question. Essay-type questions, on the other hand, can be re-circulated over time because there is no one correct answer, and the reasoning and arguments are what matter. So, large banks of questions have to be built up. What would be the quality of questions generated in such numbers? As a matter of interest, the JNU authorities have asked for only two sets of questions from the JNU faculty. Who would set the remaining questions?
There is widespread cost-cutting in universities nationwide, but at the same time, there is massive investment in biometric machines and online entrance exams. So where are universities cutting costs to pay for online examinations? In JNU, cost-cutting is affecting the library, for one. Faculty members of JNU have learnt from meetings in library committees that starting next year, and we are looking at massive budget cuts for the library that will leave us without access to online databases such as JStor and Muse. So, it is not as if all online solutions are being prioritized.
The real question is – what are the forces behind the massive thrust to transform all entrance exams into online ones? Who exactly are the “stakeholders” interested in making a strong pitch for online examinations (and other high-expense technological fixes like biometric attendance)? What kinds of monetary incentives are circulating in our systems to push institutions into this disastrous, technologically driven mode of conducting entrance exams and perhaps eventually, all examinations?
The experience of the United States is instructive. British publishing giant Pearson has a virtual monopoly over all aspects of education from testing to online classes and student data systems. An investigation by Politico found that:
“Pearson stands to make tens of millions in taxpayer dollars and cuts in student tuition from deals arranged without competitive bids in states from Florida to Texas. The review also found Pearson’s contracts set forth specific performance targets – but don’t penalize the company when it fails to meet those standards. Also, in the higher education realm, the contracts give Pearson extensive access to personal student data, with few constraints on how it is used.”
However, eventually, as Jonathan Zimmerman, an education historian at New York University, said: “The people we’ve elected have created a landscape that’s allowed Pearson to prosper.” It needs to be asked who are the big players in the education software business in India, and what kinds of incentives are being provided to decision makers?
Now for the JNU specific issues.
Problems with Giving the Answer Key in Advance
The administration did not initially explain why the answer key is needed before the examination is conducted. When this point came up in a press note circulated by the students’ union, the administration’s public response to the media was that candidates should know their score at the moment they finish the exam (JNU 2018). This is a fresh point, not made by the administration earlier, and it is not clear what purpose is served by students knowing the number of questions they have got right, if, in any case, they would know their position on the merit list and whether they have been selected. The norm is for the answer key to be revealed after the examination in the presence of the paper setter. However, the JNU administration had initially insisted that the answer key must be supplied to it on a pen drive before the examination is conducted. Further, it also gave itself two weeks to proofread the question papers. Both these requirements made the JNU faculty suspicious.
Faced with sustained questioning by the faculty, the JNU administration has made some amendments to the answer-key policy and allowed it to be retained with the dean of the school, but the policy requiring submission of the answer key in advance has not been revised. Furthermore, this provision is only relevant for questions framed by JNU faculty, but not for those set by the outside experts. A new development on this is a point in the latest guidelines that states that after the examinations are completed, the question papers and answer keys will be uploaded on the JNU website, giving applicants the opportunity to review and question the designated answers within a week or so and raise concerns with the Director of Admissions on “the appropriateness and precision” of an answer. The final merit list will be prepared after this process. Our concern is that this goes beyond allowing the examinee to check if the computer has been correct in its processing. The terms “appropriateness and precision” allow the possibility of the student questioning the rationale of choice given as correct in the multiple-choice question. How will differences be adjudicated if there is a variance between the understanding of the examiner(s) and the examinee? We fear that the final merit list will become open to manipulation at this stage.
Outside Experts to Set Questions
Trained academics have distinct competencies in their fields, and nobody can be an expert across a discipline. Different centres in JNU have their academic focus and areas they specialize in, and entrance examinations should reflect this. Most of JNU’s centres are interdisciplinary, and the syllabi for each programme have many subject-area components in them. As a consequence, questions are also framed to elicit acquaintance with more than one discipline simultaneously.
An entrance examination is fundamentally different in its goals from standardized tests like the UGC NET exam or even the Graduate Records Examination (GRE), which evaluate a candidate’s state of knowledge. An entrance examination, however, is conducted to evaluate the eligibility of a candidate for a particular programme. How then can faculty, who are not teaching in that programme, claim to bring in any expertise? Finally, and most importantly, who are the experts from outside JNU who will set questions, how will they be selected, and by whom?
From the Perspective of Admission Policy and Social Justice
Since the 2017-18 admissions, JNU’s progressive admission policy has been laid to waste by the present vice-chancellor. The 2017-18 admissions saw an 83 per cent cut in seats for the research degrees of the university, and final admissions revealed that only 159 students were admitted into the university for these programmes and 131 seats were left vacant. One of the reasons why JNU could not fill its seats in 2017-18 and why the social-justice component in JNU research degree admissions has weakened so drastically, is because of the policy that sets the minimum marks in the written exam to qualify for the viva-voce part of the examination, at 50 per cent. No other institution sets this high a qualifying percentage for research examinations across disciplines. Most do not set this high a mark for any programme of study. JNU faculty were first orally informed by officials that the online test uniformly across programmes at every level – undergraduate, postgraduate and research – will adopt this very percentage of 50 per cent as the qualifying mark, but the two sets of guidelines that have been issued do not make any mention of what the qualifying marks for any of the tests are going to be. This will affect all students, those who may be encountering the format for the first time or have had no time or the means to develop familiarity with it; but it will hit first-generation learners the hardest. The JNU entrance examination until now has enabled teachers to select students with the best potential, because it employed a fair benchmark of minimum attainment, which was not affected by factors beyond the student’s control.
The apprehension is that this unique innovation of JNU, made in service of the requirements of the JNU Act, is now being abolished across the board. Similarly, no clarification is forthcoming on the mandatory relaxations in pass marks on written examinations to be given to reserved categories.
All in all, the JNU faculty’s concern about the changes being wrought on the JNU admission policy is the intent to alter the very character of JNU and the quality of its programmes, and that this is being done in a way that fosters profiteering and discrimination. That this could breed corruption is a distinct possibility, particularly because the process has now entirely been taken out of the JNU faculty’s hands.
For the first time in JNU’s history, the entire admission policy has been taken out of the control of the Academic Committee. The committee formally set up by the JNU administration to oversee admissions for all programmes comprises a few members of the JNU faculty selected by the vice chancellor (with no representation across schools), and some faculty from IIT Delhi and University of Delhi.
Finally, as stated earlier, this question goes far beyond JNU or any particular university. Every Indian with an interest in affordable and high-quality higher education should ask questions about the massive push towards online examinations. Do students benefit from this? And who profits?
(This is an abridged version of an article originally published with the title ‘Does the MCQ Format work For Social Sciences and Humanities Entrance Examinations?’ in Economic and Political Weekly Vol 53, Issue No 42, 20 October 2018. It has been republished here with the authors’ permission. The full article may be accessed at the following link: https://www.epw.in/journal/2018/42)
Forward Press also publishes books on Bahujan issues. Forward Press Books sheds light on the widespread problems as well as the finer aspects of Bahujan (Dalit, OBC, Adivasi, Nomadic, Pasmanda) society, culture, literature and politics. Contact us for a list of FP Books’ titles and to order. Mobile: +917827427311, Email: firstname.lastname@example.org)
The titles from Forward Press Books are also available on Kindle and these e-books cost less than their print versions. Browse and buy: