Normalization of Board Marks for Admission to Engineering Courses

Findings and recommendations of the Joshi Committee

The Issue

• Students leaving high school look for various career opportunities, including in Engineering.

• Many courses, many examinations.• A lot of load on students.• High stakes.• Students turn to specialized coaching classes.• General education neglected.• Coaching distorts outcome of merit assessment examinations.• Institutes of Technology/ Engineering Colleges suffer from this

distortion.

MHRD’s Response

• Eliminate multiple entrance examinations by clubbing them together– JEE (Main) for all engineering institutions/colleges,– Another specialized examination for IITs.

• Take into consideration Board performance for engineering entrance decisions– As an eligibility condition for admission to IITs,– As a component in composite score for merit lists

of NITs and other institutions/colleges.

Committees Formed

• Committees arising from deliberation in Council of IIT’s– Damodar Acharya Committee (8 March, 2010):

Inadequacy of present admission system brought into focus.

– T. Ramsami Committee (11 November, 2010): Feasibility of utilizing Board marks (in the form of percentiles) recognized.

• Committee arising from deliberation in Council of NIT’s– S.K. Joshi Committee (13 August, 2012):

Findings presented here.

Objectives of Joshi Committee

• Specified Terms of Reference– Validating the normalization formula using actual results of

various Board and refining it based on its validation.

• Detailed objectives– To evaluate implementation methodologies and

effectiveness of various possible schemes,– To validate the chosen scheme for its consistency and

application for JEE (Main) 2013,– To identify and enlist relevant issues, which are not

covered under the scope of current study, for proper implementation of the scheme.

Inputs Received by Joshi Committee

• Inputs obtained by formed by the Chairman of CBSE (implementing agency) from– A ‘Core Committee’ formed by Chariman-CBSE, comprising

experts from ISI, IIT’s and other prominent institutions,– Glenn Rowley of Australian Council for Educational Research

(ACER),– Jim Tognolini and Jon Twing of Indian Centre for Assessment,

Evaluation and Research (CAER).

• Further analysis/validation by members of the Joshi Committee, including those participating from the ‘Core Committee’.

Decisions of Joshi Committee

Objective 1: Evaluation of methodologies/schemes

Issue: How to Equate Board Marks• Framework provided by NIT Council :

60% weight on JEE-Main, 40% weight on Board scores.• Discarded option : Use of linear transformation

– Adjustment only for mean and standard deviation ,– 2012 Boards data showed board-to-board difference in score

distribution – even after such transformation.• Accepted (with modification) option : Use of Board percentiles

– Percentiles of different boards treated at par,– Recommended by Ramasami Committee.

• Modification : Use of transformed Board percentiles– Modification of all percentiles to bring them to JEE (Main) scale– No change in relative ranking,– Makes Board scores ready for combination with JEE (Main) score.

Issue: How to Address Difference of Difficulty Levels of Subjects

• ‘Rasch model’ explicitly take into account difficulty level of a subject .

• Specialized computational methods based on such models are still at developmental stage.

• Such model can ‘compensate’ for differences of difficulty levels of subjects– within a board, but– not across boards.

• Use of this model also amounts to moving away from percentiles, recommended by Ramasami Committee.

• Joshi committee does not recommend the use of such a model.

Issue: How to Address Differences in Marking Pattern of Different Subjects• Possible solution: Equate Board subject marks by using

percentiles BEFORE aggregating.• Possible difficulty

– Much heavier computational burden,– Greater chances of unforeseen hitches,– Greater need for communication with several boards to resolve confusion,– Time frame for computation is short,– Data gathering/validation mechanism is not yet in place.

• Joshi Committee Decision: Equate Board marks AFTER aggregation.

• Saving grace: Usual aggregates are widely used (and accepted) for determining ranks within Boards.

Issue: Which Subjects to Aggregate

• Number of subjects should be as large as possible (emphasis on school education as a whole).

• Most boards have at least five subjects.• Mathematics and Physics are REQUIRED for JEE-Main.• Joshi Committee decision is to use five subjects:

– Physics,– Mathematics,– Any one of Chemistry, Biology, Biotechnology, Computer Science,– One language,– Any subject other than above four.

Issue: Basis Group for Normalization

• Discarded Option : All students– All students do not have appropriate subject combinations.

• Discarded Option : Passed students with appropriate subject combinations– Pass percentage varies from board to board,– Truncating at pass-mark would create board to board

disparity.

• Chosen Option : All students with appropriate subject combinations.

Issue: Nature of Calibration with JEE (Main) Marks

• Normalize Board aggregate marks to make their distribution match – JEE (Main) aggregate marks of all appearing

candidates (Option 1)– JEE (Main) aggregate marks of candidates from

that board only (Option 2)• Choice between the two options were made

on the basis of additional data analysis for validation.

Decisions of Joshi Committee

Objective 2: Validation and fine tuning of chosen scheme

Assumptions Behind the Options

• Assumption behind Option 1– All boards have same merit distribution.

• Assumption behind Option 2– Different boards have different merit distributions.– This difference can be measured (and adjusted for)

by the performance levels of students of different boards in JEE (Main)/AIEEE.

Risks of Adjustment through JEE (Main) Performance

• Students of some boards perform poorly in JEE (Main) / AIEEE.• This disparity may be due to

– Poor merit/ability,– Non-alignment of board examination pattern with JEE (Main) / AIEEE

(rank correlations indicate this),– Lack of instruction in English and Hindi

(only available languages for JEE-Main / AIEEE),– Less access to coaching,– Load of an extra subject in board (for some boards).

• All these effects are confounded.• If performance disparity is attributed only to merit disparity,

confounding factors are ignored.• Solution may be worse than the problem.

Findings from Analysis of 2012 Data

• Option 2 requires different treatment of Board percentiles; Option 1 does not.

• The differential treatment of Board percentiles under Option 2 can be quite extreme:– 80th percentile of Maharashtra Board equated with 50th percentile of

CBSE;– Topper of Maharashtra Board has normalized Board score 331; Topper

of Jharkhand Board has 274;– A CBSE student with AIEEE marks 130 and Board percentile 93.1 has a

rank of about 18,000; a Maharashtra Board student with that profile has a rank of about 34,000.This amounts to penalizing the Maharashtra Board student for poor AIEEE performance of peers from that Board.

Key Findings

• Vastly different treatment of percentiles of different Boards (Option 2) would not be fair in the presence of confounding factors.

• Representation of various boards in different sections of the merit list– Option 2 changes the present (2012) pattern substantially,– Option 1 has less drastic impact.

• Option 1 would produce more equitable performance across boards.

• Option 1 would be the right choice.

Decisions of Joshi Committee

Objective 3: Issues relevant for implementation

Issues and Actions

• How to implement the selected method– Algorithm and Flowchart provided.

• Operational issues– Timelines for Processing and Analysis provided.

• Other issues (no action within purview of Joshi Committee)– Collection of data,– Formatting of data,– Validation / Authentication of data,– Adherence of time frames for data delivery.

Further Recommendation

• A Core group may be formed by CBSE for implementation of the normalization scheme focussing on – Data Collection,– Nature of Data,– Validation of Data,– Timeline for Data Collection,– Data Processing.