International Rankings and the Assessment of Teaching: A Regional

PerspectiveRichard Holmes

University Ranking Watch

Contents

• International university rankings• Assessment of teaching in rankings• Arab universities and international

rankings• Arab universities and teaching

indicators• The way forward

Teaching indicators in global rankings

• Current global rankings emphasise research

• URAP, NTU, Leiden, US News 100%

• ARWU at least 90% (10% award winning alumni)

continued

• QS 60% - 80% (Staff Student Ratio includes research only staff)

• THE -- effectively 84.75% (teaching survey 90% + correlation with research

• RUR – 76%

continued

• Webometrics – 83% internet data

• IC4U – entirely web data

• U-Multirank includes several teaching related dimensions -- student staff ratio, graduating on time, graduation rate, overall learning experience, etc

Assessment of teaching in national university rankings

• National rankings put more emphasis on teaching

US News• Input – quality of students (high school

rank, tests) staff student ratio, class size

• Output – completion, value added,

continued

Guardian•Input – Staff Student Ratio, spending•Output -- student satisfaction, graduate success•Times Higher Education•Simulation of Teaching Excellence Framework includes student satisfaction, completion and employment

continued

Shanghai Best Chinese Universities

•National college entrance exam scores

•Graduate employment rate

continued

Perspektywy

•Employer preferences

•Assessment by academic staff

Teaching indicators in international rankings -- input

• Staff student ratio

• (THE, QS, RUR, U-Multirank)

• Income (THE)

• Academic staff with doctorates (U- Multirank, QS Arab region)

Teaching indicators in international rankings -- outcomes

• Nobel and Fields awards (Shanghai)• CEOs and prizes (CWUR)• Employer review (QS)• Survey of postgraduate teaching by

researchers (THE, RUR)• Survey of teaching by students (U

Multirank)• THE and QS employability surveys

Arab universities in international rankings

Webometrics lists 26000 universities•989 in the Arab world•270 have an “excellence” rank above 5778Challenge for Arab universities is• to assess universities ca 300 with any significant research capacity•and those ca 700 with no research profile

Arab universities in global rankings

Shanghai rankings – Saudi Arabia 4, Egypt 1 in top 500 (King Abdulaziz University and King Saud University in top 200)

continued

Times Higher Education world rankings-- 26 from 10 countries(8 from Egypt) -- highest is King Abdulaziz University is 201-250

QS -- 30 (7 from Saudi Arabia) -- King Fahd University of Petroleum and Minerals 187th

continued

Round University Rankings – 12 (4 from Egypt) – Qatar University 360th

U-Multirank – 23 (5 from Egypt)

continued

• Current global rankings include only a fraction of Arab universities

• Best is QS with 30

• Assessment of teaching is very limited

Performance of Arab universities on teaching indicators

• THE teaching group – 26 Arab universities listed

Top 5• KFUPM, Saudi Arabia• American University Beirut, Lebanon• King Abdulaziz University, Saudi Arabia• Khalifah University, UAE• American University Cairo, Egypt

RUR teaching indicators

• 12 Arab universitiesTop five• American University Beirut, Lebanon• American University Cairo, Egypt• Qatar University• KFUPM, Saudi Arabia• Kufa University, Iraq

QS world rankings – faculty student ratio

• 18 Arab universities with scores

Top five• KFUPM, Saudi Arabia• King Saud University, Saudi Arabia• KAUST, Saudi Arabia• Khalifa University, UAE• Universite Sainte Joseph, Lebanon

QS world rankings – employer survey

• 12 Arab universitiesTop Five• American University in Beirut, Lebanon• American University in Cairo, Egypt• Cairo University, Egypt• American University in Sharjah, UAE• American University in Dubai, UAE

ARWU alumni

1 Arab university -- Cairo

CWUR –quality of education (prizes)

• 3 Arab universities (377 total)

• American University Beirut • Alexandria University• Cairo University

CWUR – alumni employment

• 6 Arab universities

• KFUPM, Saudi Arabia• King Saud University• American University Beirut• Cairo University• United Arab Emirates University• Alexandria University

QS arab region – faculty student ratio

Top seven with score of 100•Jubail Industrial College, Saudi Arabia•Gulf University, Bahrain•Dar Al Hekma College for Women, Saudi Arabia•Yanbu Industrial College, Saudi Arabia•Al Nahrain University, Iraq•Hawler Medical University, Iraq•Université Antonine, Beirut

QS Arab region -- employer reputation

50 universitiesTop 5 [same as world rankings]•American University Beirut, Lebanon•American University Cairo, Egypt•Cairo University, Egypt•American University Sharjah, UAE•American University Dubai, UAE

US News Arab region rankings employer survey

124 universitiesEmployer reputation top seven•KFUPM•American University Sharjah•American University Cairo•King Abdulaziz University (5)•King Abdullah University of Science and Technology (5)•Ain Shams University (5)

To summarise

• At present none of the international rankings is satisfactory

• Only a fraction of Arab universities are represented – best international ranking is QS WUR with 30 and US News Arab region with 124 universities

• Some indicator scores suggest problems with collecting data and validity

continued

• Looking at existing rankings, we should forget about overall rankings and focus on specific indicators

• These should not be bundled with other indicators (THE and RUR)

• Preferably not submitted by institutions but derived from third party sources not created for ranking purposes

• Assess reasonable number of universities

continued

• It is now possible to assess research capability although there are obvious problems with indicators weighting

• Assessing teaching is more of a problem – accurate data is not available, only a few places are covered, there are data collection issues

continued

• At the moment the best available data would appear to be the US News employer survey

• 124 universities, third party data, reasonable results

Alternative indicators ?

• LinkedIn “interested in attending” (60)• Top 5• Helwan university• American University in Cairo• Arab Academy of Science Technology and

Maritime Transport• Cairo University• Ain Shams University

continued

• Looking at the UK and USA, some these things can predict student outcomes such as completion, satisfaction, employment, “value added”

• Test/exam scores• Research quality of staff (REF)• Staff student resources• Spending• Vice-chancellor salaries

continued

Suggestions•Arab universities and governments work on harmonization and standardization of secondary school exams and admissions tests •Establish a regional body to collect comparable data on income, staff, faculty, salaries, qualifications•Encourage international assessments to assess university learning outcomes

continued

ABOVE ALL•Ignore overall ranking scores and focus on valid and relevant indicators