CAT 2009: What lies beneath...

Well, well, well. That the CAT (the Common Admission Test, the entrance test to the Indian Institutes of Management) is a tough exam is hardly a state-secret. What is perhaps not so openly debated is its fairness in conduct, content, reporting style and responsibility towards the student community.

The idea of this blog is to share whatever little yours truly knows about cracking the CAT (as you will soon see, the little is pretty little), but that can wait for a while. The blog will, for now, attempt to examine the underbelly (pun unintended) of the CAT to find how just has it been over the years. The blog will also seek to understand whether, as a student, you have been getting worth out of the not-so-small amount you pay (Rs.1400 for 2009) towards the test. (The amount Rs.1400 for two-and-half-hours translates to Rs.560 per hour, near about what a good physician would charge for a consultation, but that’s quite another story.)

The blog will break down the argument in four parts: Conduct, Content, Reporting Style and Responsibility towards the student community. I shall start with discussing the first only.

Conduct: The CAT is going “on-line” (Computer-based). Big deal. What could have been the reason for doing so? Recessed amidst the frenzied furor of the news-media that the mighty CAT is going hi-tech – basically Indian exams coming of age – lies a question. Why? Since we have no way of knowing the exact compelling and convincing reasons, let us speculate.

Perhaps the candidate size was growing too large for the CAT committee to handle (reportedly around three hundred thousand)? Tch, tch, tch. If you teach management and you can’t manage numbers, aren’t you accepting the adage that consultants are those impotents who can preach but not practice? The recruitment exams of the SBI and the passports-to-babudom CSE (Civil Services Examinations) apparently manage larger numbers, with almost no complaints from either side.

There is a perfectly plausible reason no one dare admit: face-saving. How? Here it is.

Perhaps I may be wrong, but not too much. Of the last six CATs that come to my mind, perhaps no more than one was error-free. Yes, at least five of the previous six CATs had some error or the other. Not something to write home about if you claim to be (not without reason) the best management institutes of the country. Not fair if you can’t set 75 to 150 correct questions, especially when you are judging a student size that runs in hundred-thousands. Much less fair if you charge in four digits for a twenty-page booklet. The student community had, rightfully, in 2005 or 2006, approached the Supreme court questioning the validity of the error-prone CAT. (A recent bump like that was noticed in the GCET 2009, Gujarat State’s entrance test, but the results of both the court proceedings went in favor of the testing bodies.)

A Computer-Based Test (CBT) can sweep all this under the carpet. When the students have hard copy, a physical paper, they also leave the examination hall with the evidence of errors. But when the exam is a CBT, the student leaves with no evidence. File closed, reputation intact, and one is any wiser.

The second issue regarding the conduct is the fact that a CBT, purely out of the scarcity of sufficient number of computers, would be spread over a ‘window’. In other words the CAT is not a one-day affair as was till 2008, but will be spread over about a period of ten days. Candidates write their exams on different days, and hence they write different paper. How the CAT will decide that the three hundred thousand students were given different questions with a comparable level of difficulty beats me. If there is a software or an individual out there that/who can exactly judge the level of difficulty of each question, I have perhaps not heard of it/him/her. It is a personal belief of mine that the larger you are, the more answerable you are. The CAT must not only ensure the level of difficulty remains the same, but must also inform and satisfy the students that there was a level playing field.

An issue interlinked with this would be pertinent to raise: if the CAT has been unable to set a one single paper without errors in the near past, how and why does it want us to believe that all the multiple papers it will set will be error-free? If Yuvi has not performed in the last six matches, is it not a tad optimistic to believe that he will play consistently well in the whole of the world-cup matches and make no mistakes?

The final question that remains here is: will luck play a role here? I strongly think so. What has the CAT been doing with questions that were later found to be incorrect? It used to remove the question from the scoring. Since all the three hundred thousand students wrote the same paper, the effect of removing one question on all the three hundred thousand was uniform and hence the element of luck was reduced to near-zero. (Why near-zero? Well, the CAT used to give the same paper but with the questions in different order. That meant a safe-guard against copying. But the CAT has also had the dubious distinction of having made a different number of errors in different sets of papers. That meant that one candidate had an error-free paper while the other had two errors in his paper.) Doesn’t canceling the question really correct the mistake? Not necessarily so. If you were a student struggling for every single question, every single minute and every single mark, you wouldn’t be particularly pleased when, after struggling for five minutes and leaving the question you later were told the question was incorrect anyway.

Now that people will write different papers altogether, the probability that they will receive papers with different number of mistakes will also increase. That sounds a lot like 100m-sprinters running on tracks with different base and different lengths. Bolt, where are you?