times higher education world university rankings - · pdf filetimes higher education world...
TRANSCRIPT
Times Higher Education
World University Rankings:
what quality measures?
Cover:
• Background to the World University Rankings
• History
• Impact
• Some student perspectives
• WUR
• Methodology
• Indicators and measures
Background to the
World University Rankings
Times Higher Education
1971 First issue of
supplement
World University
Rankings launched
THE becomes
standalone
publication
independent of The
Times
THE relaunched in
magazine format
World University
Rankings
production moved
to Thomson
Reuters
World University
Rankings brought
in-house
2004
2005
2008
2010
2015
A brief history of the World University Rankings
2004 1st edition of World University
Rankings commissioned by Times Higher Education
2010
Improved methodology
Move to Thompson Reuters
“Universities deserve a rigorous, robust and
transparent set of rankings”
2015 Times Higher Education move
production in-house
Switch to Elsevier for bibliometrics
Impact of Rankings
“Rankings prompt change in areas that directly improve student learning”
US Institute for Higher Education Policy (IHEP). May 2009
Rankings "help by encouraging… more informed policy making... they can stimulate national debate and focused analysis... which in turn may lead to positive policy changes at system level.”
European Universities Association, April 2013
“The data collected for the THEWUR provide a useful set of indicators which enable us to analyse the dynamics of higher education development and to comparatively relate excellence to policies”
Dirk Van Damme, Head of the Innovation and Measuring Progress Division (IMEP) at the OECD
“The THE rankings are the principal yardstick we should look to”
Shashi Tharoor, former Minister of State for Human Resource Development, India.
“Times Higher Education rankings – now increasingly seen as the gold standard.”
Ferdinand Von Prondzynski, Vice Chancellor, Robert Gordon University
Impact of Rankings
“More universities [compared to earlier study] monitor their peers worldwide (77% vs nearly 50%) and that an "overwhelming majority" use rankings to inform strategic decisions” – Hazelkorn
Rankings and the Reshaping of Higher Education – Hazelkorn 2011:
“… the evidence is compelling that universities do use rankings as a strategic tool and a management instrument”
“Rankings have become an ‘explicit part of target agreements’ or contracts ‘between presidency and departments’ and featured in individual faculty performance contracts.”
“Rankings have taken the function of data collection and analysis out of the back-office, and placed it at the centre of strategic decision making and performance measurement.”
Rankings and
Key Performance Indicators “We have not targeted an actual ranking position in our strategic plan, but we have 20 key performance indicators and there is a connection with rankings -- several match the THE rankings indicators
Anton Muscatelli, principal, University of Glasgow – UK
“A majority of institutions stated that they had responded to league tables in the areas of KPIs and, in many cases, used the league tables variables to review these indicators.”
Case study pre-1992 university “some of the key performance indicators used by Council [ the Universities governing body] now are modelled to dovetail with elements of the rankings.”
Counting what is measured or measuring what counts? League tables and their impact on Higher Education in England - HEFCE
Some Student Perspectives
Student perspectives – decision factors
“The ever-increasing usage of rankings [by students] is fast becoming a recurring theme.” Hazelkorn
How important are these factors when deciding which university to apply to? Answers 6 and 7 on a 7 point scale; 7= Strongly Agree
Source: Hobsons ‘Beyond the data: Influencing international student decision making’ May 2014
0 20 40 60 80 100
Teaching quality
Subject available
Tuition fees
Academic reputation rankings
Living expenses
Safety of campus / local area
Scholarships
Entry requirements
Responsiveness of university in…
Links with employers (internships)
Graduate employment rates
Student perspectives – teaching quality factors
When thinking about teaching quality, how important are the following factors? Answers 6 and 7 on a 7 point scale; 7= Strongly Agree
Source: Hobsons ‘Beyond the data: Influencing international student decision making’ May 2014
0 20 40 60 80 100
Academic reputation / ranking
Subject / course ranking
Student satisfaction with university
Tuition fees
Use of technology in teaching
Academic profile of lecturers
Student satisfaction ranking
Links with employers
Quality of facilities
% student in employment / furtherstudy after graduation
Teaching hours per week
Student perspectives – best teaching quality?
Conjoint factors of importance Shown 8-10 sets of institutions with random combination of levels for each attribute and asked to choose which one represented best teaching quality
Source: Hobsons ‘Beyond the data: Influencing international student decision making’ May 2014
0 5 10 15 20 25 30
Tuition Fee
Subject ranking
University ranking
% students in employment
Teaching hours per week
Student satisfaction
Student perspectives
“The quality and impact of our research is officially acknowledged. In the 2014 Research Excellence Framework, 62 per cent of our research submissions were classified as ‘world-leading’ or ‘internationally excellent’”
From a UK university international student application webpage
World University Rankings:
Methodology and Data
Sources
THE WUR methodology
Source 13 metrics 5 areas/pillars Ranking Width of line corresponds to weighting of metric
Data sources: portal
• 1667 institutions
invited
• 1126
participated
• Details at
institutional and
subject level
Data sources: reputation survey
• Survey of
academics
selected by
Elsevier
• Balanced by
geography
• OECD data
• 10,000
responses
Data sources: Scopus
• Move from
Thompson
Reuters’ Web of
Science to
Elsevier’s Scopus
• Three measures
• Field Weighted
Citations to avoid
subject bias
THE WUR Universe
• More Universities than ever
before
• 88 countries involved
• 70 countries with ranked
institutions
• 801 Universities will be ranked
with banding
Institutions approached
Institutions that submitted data
Institutions with sufficient publications
Institutions with sufficient data
1,667
1,126
861
Institutions that have undergraduates 851
845
Ranked 801
The Indicators:
quality measures?
3 Ps model of teaching and learning J B Biggs, 1993 Higher Education Research and Development
Presage
Exists before a student starts
learning or being taught
Process What is going on in
teaching and learning
Product Outcomes of the
educational process
Dimensions of quality Graham Gibbs, HEA 2010
Presage
Funding
Student Staff ratio
Quality of teaching staff
Quality of students
Process
Class size
Contact hours
Quality of teaching
Research Environment
Intellectual challenge
Assessment & Feedback
Other - reputation
Product
Student Retention
Performance and degree classification
Employability
THE WUR methodology
Teaching
Presage
Teaching
Presage
Process
Research
Process
WUR - less able to capture
• Quality of teaching staff, quality of students [Presage]
• Class sizes and contact hours, assessment and feedback, quality of teaching (qualifications / students’ judgment), level of intellectual challenge (curriculum, depth, engagement) [Process]
• Student retention, performance/degree classification, employability [Product]
Challenges of choosing indicators and sourcing
data
Relevant and valid - globally
Clear sense of what is better performance
Transparent and easily understood
Available - globally
Independent source or straightforward for institutions to report
Conclusion
• WUR aims to capture broad performance of universities and not just
teaching quality
• Rankings are one part of the approaches to the assessment of higher
education and bring into focus the indicators included
• Used by students, institutions, funders, government bodies
• THE is open to discussion and suggestions about how the element of
teaching quality in World University Rankings could be enhanced
Thank You [email protected]