making sense of rankings

Making Sense of Rankings

Are rankings fact, fiction or somewhere in between? Learn more about university rankings, what they mean and how you

can use them.

Top 5 Colleges & Universities

1. MIT2. Harvard3. Cambridge4. UCL5. Imperial

6. Cal Tech7. Oxford8. Harvard9. Stanford10.MIT

11.Harvard12.Stanford13.Berkeley14.MIT15.Cambridge

What Criteria Did You Use?

• Number of citations by university researchers?• Number of alumni winning Nobel Prizes and

Fields Medals?• Academic-athletic balance?• Proportion of international faculty?• 6 year graduation rate?• Salaries of alumni? Salaries of faculty?• Acceptance rate?

AN HISTORICAL PERSPECTIVE

Outcomes-Based Rankings

• 1900 UK – Where We Get Our Best Men– Published a list of universities ranked in order by

the absolute number of eminent men who attended them

• 1910 US – American Men of Science– “Students should certainly use every effort to

attend institutions having large proportions of men of distinction.”

Reputational Rankings

• 1924 US – Raymond Hughes’ study on graduate school quality

• 1966 US – Assessment of Quality in Graduate Education, The Cartter Report

– Surveyed leading profs from different graduate disciplines

Pre-US News World of Rankings

• Rankings were the realm of professors and higher education administrators, not the general public

• They were published as studies in little-circulated academic books and journals

• Even if located, they were often too obscure and esoteric

US NEWS AND WORLD REPORT

The Beginning

• First university ranking system to be published in a popular publication (1983)

• The first three versions were published every other year and were entirely based on reputational surveys of college presidents

• In 1988, US News began its annual publication of rankings and incorporated additional criteria (input and output)

Impact of US News

• Impact on high school student (and parent) decisions– Students with high academic achievement are

highly likely to find rankings “very important”

• Impact of decisions made by universities on:– Admissions rates and practices– Fundraising and spending– Hiring

Gaming the System

• Universities have falsified data

• Reporting academic information only for certain demographics

• Universities use unfair admissions practices– Huge numbers on waitlists– Part I of an application

PRESENT RANKINGS

USA

• US News• Forbes– Based on academic outcomes and student reviews

of professors– Combines universities and colleges

• Princeton Review– Aspects of college life

• Kiplinger’s Best Values

UK

Comprehensive and by Subject– Involves the National Student Survey (data from 2 to 3 years ago)– Graduate prospects – How is this measured?

• The Guardian– Focuses on teaching; ignores research

• The Times Good University Guide– Research quality is from 2008!

• The Complete University Guide

Canada

• Maclean’s– Library = 15%– National Awards by faculty = 20%

* DIY Rankings! Cool * Includes information from the National Survey of Student Engagement

Where’s the competition?

WorldInclude Subject-based Rankings

• QS World University Rankings– 40% Global Reputation Survey– Proportion of international students/faculty

• Times Higher Education World University Rankings

• Academic Ranking of World Universities– China– Simple formula based on academic distinctions and research

Where are US colleges?

HOW DO THEY COMPARE?

Rankings• Quality cannot be quantified• They can tell you, in general terms, about the academic

credentials of the students they attract

Unscientific Methods

Universities A B C

Stanford 2 6 9

MIT 7 6 1

Notre Dame 8 17 57

Duke 13 8 12

Boston College 14 31 72

Tufts 15 28 51

Dartmouth 17 10 34

Cornell 26 15 9

NYU 41 32 18

METHODOLOGIES

US NewsCriteria

Academic ReputationSelectivityFaculty ResourcesGraduation/Retention RatesFinancial ResourcesAlumni GivingGraduation Rate Per-formance

QS World University RankingsCriteria

Academic Reputation

Employer Reputation

Faculty-Student Ratio

Citations Per Faculty

International Students

International Faculty

CRITICISMS

Quality?

• Quality cannot be quantified

• Rankings measure how an institution grades out for specific criteria

• When publications change their criteria, the rankings noticeably change – sometimes drastically

• They can tell you, in general terms, about the academic credentials of the students they attract

One Size Fits All?

• Institutions are so different with different missions. Aren’t we unfairly comparing institutions? Worldwide?

• Are rankings really personal for you? Aren’t they generic?

WHAT DOES THIS MEAN FOR YOU?

How Can You Use Rankings?

• Read the methodology. Know what you’re looking at.

• Group universities (and colleges together)

• Use a variety of publications

• Use them as one small factor in your search

RECOMMENDATIONS

Alternatives to Rankings

Would you date someone based purely on someone else’s opinion?

Set Your Criteria

• Location

• Learning Style – size of institution

• Specific programs

• Opportunities

Read Websites

• Know the school– Columbia vs. Brown– MIT vs. U Chicago

• Who are the professors?

• Special opportunities– NYU Abu Dhabi

Guides

• UCAS– Why study this course?– Special programs

• US websites– College Prowler– Unigo

• Books

People

• College Reps!

• Counselors

• ISM Alumni

Final Points

• Know yourself.

• Spend time. Do your research.

• Be thorough and open-minded.

• Use rankings responsibly.