Tuesday, November 11, 2008

Tinkering with a flawed exam vs. an evidence-based admission policy


The IITs are tinkering with the cut-off marks. Again.

During the last three three years, the IITs first selected the top 80 percent of the exam takers in each subject. The survivors after this step were then ranked according to their aggregate scores.

In JEE-2009, the first cut-off is not at the 20th percentile, but at the average score in each subject. If the students' scores in each subject fall on a true bell curve (do they?), then the average would correspond to the 50th percentile. I think this is the only difference in the ranking procedure between JEE-2008 and JEE-2009.

All of which leads to questions like these: Why the 50th percentile? Why not, say, a cut-off at the 80th percentile? Or, what was wrong with the earlier procedure -- with the cut-off at the 20th percentile? I don't think there will be any public articulation of answers to these questions.

* * *

Meanwhile, Boalt Hall, UC-Berkeley's law school, has issued a report from a 10-year study to identify elements of an ideal standardized exam that would predict success as a lawyer (while LSAT predicts success in the first year of law school). The thrust of the report is clear: one has to look beyond LSAT, and Boalt Hall thinks it now has a good model that can be deployed. It will be interesting to see how long it takes to implement its own ideas.

The more important take-away (at least for me) is the serious effort expended by UC-Berkeley's law school in doing such a serious, academic study that could be used later for developing appropriate admission policies and procedures.

Both these news stories appeared today. The contrast is striking, indeed.

* * *

Thanks to Pratik Ray for the e-mail alert about the Telegraph story.

2 Comments:

  1. Yogesh K. Upadhyaya said...

    Abi:

    Noticed the new look of your blog.

    About IIT-JEE rank, the matter is made unnecessarily complicated by introducing logic (for cut-off) when there is none exists. Every year the cut-off formula is changed just based on public criticism, and not based on scientific rational.

    Whe I took IIT-JEE in 1972, it was the 5th year of Joint Entrance Exam. There were 3 subjects of physics, chemistry and maths, of 100 marks each. The ranks were awarded based on aggregate marks out of 300 total. There were no cut-offs for individual subject. The topper got 88% marks and I got 979 AIR rank with 52% marks, that simple.

    Yogesh Upadhyaya
    New Jersey

  2. Giri@iisc said...

    There was no intense media scrutiny in 1972. Consider the case where 90/300 gets a rank of 2000 in JEE. For example, you can score 90 in three ways,
    30,30,30; 0, 45, 45; 0,0,90. Will you give the same JEE ranks to everyone? If you do not qualify the third candidate, you will be accused of ignoring a Math Olympiad champion. If you qualify the third candidate, the country (media) will be shocked to know that a person who scored 0 in Physics and Chemistry qualified. So, the committee agrees on sectional cutoffs and whether it is 20, 50, 80 percentile is anyone's guess.

    So, it is not that simple. Because of the confusion in JEE, GATE is unwilling to implement sections (read my blog on this).