universities evaluations and rankings philippe vincke
DESCRIPTION
UNIVERSITIES EVALUATIONS AND RANKINGS Philippe VINCKE Rector of the Université Libre de Bruxelle s. Evolution of higher Education. New Actors of higher education and research Increasing mobility of students and researchers Accountability of the universities, transparency - PowerPoint PPT PresentationTRANSCRIPT
UNIVERSITIES EVALUATIONS AND RANKINGS
Philippe VINCKERector of the Université Libre de Bruxelles
Evolution of higher Education
New Actors of higher education and research
Increasing mobility of students and researchers
Accountability of the universities, transparency
Evaluations, comparisons, rankings
Criticisms of the existing rankings (1)
Competencies of the authors of the rankings
Impossibility, for the reader, to reconstruct and verify the results (rankings are not« scientific »)
No information about the goals, the intended uses, the aimed public
Precise definition of « university » : are they all comparable ?
Choice of the criteria and of their relative importance
Research Education Costs Services Social aspects National context, legislation Financial ressources
Choice of the indicators Data validation
Criticisms of the existing rankings (2)
Criticisms of the existing rankings (3)
Bibliometry Quality of the data Discrimination among the scientific fields Different traditions (journals, books, proceedings,
number of authors, time span of valid research) Supremacy of the publications in English Which indicators ? (IF, citation index, h index, …)
Experts Do they exist ? How to choose them ? Which questions ? How to treat the answers ?
Numerical « manipulations » (1)
How is it possible to imagine that complex objects such as universities can be characterized by one number ?
The weighted mean can exclude good candidates
Example : A 41 97B 100 38C 68 68
Curious effects of normalization
(1) (2)
A 2000 500
B 1120 175
C 400 370
D 1600 45
E 880 240
F 160 435
G 1360 110
H 640 305
(1) (2) Global score
A 100 100 100
B 56 35 45,5
C 20 74 47
D 80 9 44,5
E 44 48 46
F 8 87 47,5
G 68 22 45
H 32 61 46,5RANKING : A, F, C, H, E,B,G,D
(1) (2)
A 1600 500
B 1120 175
C 400 370
D 1600 45
E 880 240
F 160 435
G 1360 110
H 640 305
(1) (2) Global score
A 100 100 100
B 70 35 52,5
C 25 74 49,5
D 100 9 54,5
E 55 48 51,5
F 10 87 48,5
G 85 22 53,5
H 40 61 50,5
RANKING : A,D,G,B,E,H,C,F
Before : A, F, C, H, E, B, G, D
One modification of the score of A on one criterion.No change in the scores of the other universities
After : A, D, G, B, E, H, C, F
Inverse ranking !!
Other comments
Rankings are contested but used
Rankings have an influence on reality
Excesses are possible (financial bonus, or incitements,…)
Standardization effect
Conclusions
The rankings relayed by the media are not scientifically valid at this stage
Evaluation of research and higher education is a necessity
But it must be realized by competent people in the context of a clear policy and with explicit goals
There does not exist a unique method applicable in all institutions
Main questions (1)
Wich « objects » ?
Universities (definition?) Education programmes Diploma’s Research centers Research programmes …
Main questions (2)
What does one want to do ?
To compare To select the « best(s) » To rank To define « homogene » categories To detect strong and weak points To assign ressources …
Main questions (3)
For whom ? External autorities, government, … Potential partners (universities, research centers,
companies,…) External teachers or researchers Potential students Funding agencies Sponsors Public opinion, media Alumni Internal authorities Internal teachers or researchers Internal students …
Main questions (4)
For each « situation » (characterized by the answers to the 3 previous questions):
Which indicators ?
Quality of the data ?
Numerical treatment of the data !
Different approaches for different concrete questions
Choose the « best » education programme for this student ? (« best » for him)
Allocate financial resources to research centers Select the universities which could be « good »
partners for this company Identify the strong points of these universities for
students interested in studies in that field
Necessity of an interactive decision-aid toolbox for each possible user and question.