international panorama of scientometrics...scientometrics wolfgangglänzel structure introduction...

96
International Panorama of Scientometrics Wolfgang Glänzel Structure Introduction Perspective shift International panorama The situation Extending bibliometrics New databases Subject delineation Bibliometrics and funding Meso/micro-level bibliometrics Ranking Conclusions . . . international panorama of scientometrics wolfgang glänzel 1,2 1 katholieke universiteit leuven, steunpunt o&o indicatoren, dept. msi, leuven (belgium) 2 hungarian academy of sciences, ispr, budapest (hungary) rio de janeiro 2008 1/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Upload: others

Post on 06-Jul-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

.

.

. ..

.

.

international panorama of scientometrics

wolfgang glänzel1,2

1katholieke universiteit leuven, steunpunt o&o indicatoren, dept. msi, leuven (belgium)2hungarian academy of sciences, ispr, budapest (hungary)

rio de janeiro 2008

1/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 2: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

.. .1 Structure of presentation

.. .2 Introduction

.. .3 Perspective shift

.. .4 International panorama

The situationExtending bibliometricsNew databasesSubject delineationBibliometrics and fundingMeso/micro-level bibliometricsRanking

.. .5 Conclusions

2/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 3: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

.The roots..

.

. ..

.

.

Scientometrics, as an inter-disciplinary eld, has its roots in several eldsof the sciences and socials sciences.

.The ‘Perspective Shift’..

.

. ..

.

.

During the last four decades, scientometrics evolved from sub-disciplineof information science to an evaluation tool for science policy andresearch management.

.Challenges and present tasks..

.

. ..

.

.

New challenges force the development of new techniques andmethodologies, the exploration and creation of new data sources and thefoundation of regional and national research and service centres.

3/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 4: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

.The roots..

.

. ..

.

.

Scientometrics, as an inter-disciplinary eld, has its roots in several eldsof the sciences and socials sciences.

.The ‘Perspective Shift’..

.

. ..

.

.

During the last four decades, scientometrics evolved from sub-disciplineof information science to an evaluation tool for science policy andresearch management.

.Challenges and present tasks..

.

. ..

.

.

New challenges force the development of new techniques andmethodologies, the exploration and creation of new data sources and thefoundation of regional and national research and service centres.

3/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 5: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

.The roots..

.

. ..

.

.

Scientometrics, as an inter-disciplinary eld, has its roots in several eldsof the sciences and socials sciences.

.The ‘Perspective Shift’..

.

. ..

.

.

During the last four decades, scientometrics evolved from sub-disciplineof information science to an evaluation tool for science policy andresearch management.

.Challenges and present tasks..

.

. ..

.

.

New challenges force the development of new techniques andmethodologies, the exploration and creation of new data sources and thefoundation of regional and national research and service centres.

3/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 6: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

In order to understand the present situation of scientometrics we have togo back to the roots of the eld:

History of science (D. de Solla Price)

Philosophy (V.V. Nalimov)

Information science (E. Gra eld)

Sociology of science (R.K. Merton)

Mathematics (S.D. Haitun, A.I. Yablonsky)

4/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 7: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

Bibliometrics represented a statistical approach to master thegrowing ood of scienti c information and to analyse and tounderstand the cognitive characteristics of ”big science” bymeasuring quantitative aspects of communication in science and byproviding the results to scientists and users outside the scienti ccommunity.

Monitoring, describing and modelling of the production,dissemination and use of knowledge was originally in theforeground.

5/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 8: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

Bibliometrics represented a statistical approach to master thegrowing ood of scienti c information and to analyse and tounderstand the cognitive characteristics of ”big science” bymeasuring quantitative aspects of communication in science and byproviding the results to scientists and users outside the scienti ccommunity.

Monitoring, describing and modelling of the production,dissemination and use of knowledge was originally in theforeground.

5/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 9: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

First scientometric applications were developed to improve use ofbibliographic databases and to extend information services.

Citations were considered documented use of information, and haveconsequently applied rst in the context of librarianship, scienti cinformation and information retrieval.

The journal Impact Factor (Gar eld & Sher, 1963) was used to helpselect journals for the Science Citation Index (SCI).

The co-citation based Atlas of Science developed and issued by theInstitute for Scienti c Information (ISI) was considered a new kind of“review literature” which is also suited to help students in choice ofcareer in science (Gar eld, 1975).

6/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 10: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

First scientometric applications were developed to improve use ofbibliographic databases and to extend information services.

Citations were considered documented use of information, and haveconsequently applied rst in the context of librarianship, scienti cinformation and information retrieval.

The journal Impact Factor (Gar eld & Sher, 1963) was used to helpselect journals for the Science Citation Index (SCI).

The co-citation based Atlas of Science developed and issued by theInstitute for Scienti c Information (ISI) was considered a new kind of“review literature” which is also suited to help students in choice ofcareer in science (Gar eld, 1975).

6/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 11: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

First scientometric applications were developed to improve use ofbibliographic databases and to extend information services.

Citations were considered documented use of information, and haveconsequently applied rst in the context of librarianship, scienti cinformation and information retrieval.

The journal Impact Factor (Gar eld & Sher, 1963) was used to helpselect journals for the Science Citation Index (SCI).

The co-citation based Atlas of Science developed and issued by theInstitute for Scienti c Information (ISI) was considered a new kind of“review literature” which is also suited to help students in choice ofcareer in science (Gar eld, 1975).

6/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 12: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Historical background

First scientometric applications were developed to improve use ofbibliographic databases and to extend information services.

Citations were considered documented use of information, and haveconsequently applied rst in the context of librarianship, scienti cinformation and information retrieval.

The journal Impact Factor (Gar eld & Sher, 1963) was used to helpselect journals for the Science Citation Index (SCI).

The co-citation based Atlas of Science developed and issued by theInstitute for Scienti c Information (ISI) was considered a new kind of“review literature” which is also suited to help students in choice ofcareer in science (Gar eld, 1975).

6/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 13: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

A consequence of the growth of knowledge and the evolution fromlittle science to big science: Need for supplementing researchevaluation with quantitative methods and of linking funding toperformance indicators.

The application to science policy has brought a new perspective, andresulted in re-interpretation of bibliometric conceptions.

The ’science indicators’ movement in the US with the discussionabout the possible use of bibliometrics in science policy in the 1970smarked the begin of a new era in bibliometrics.

7/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 14: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

A consequence of the growth of knowledge and the evolution fromlittle science to big science: Need for supplementing researchevaluation with quantitative methods and of linking funding toperformance indicators.

The application to science policy has brought a new perspective, andresulted in re-interpretation of bibliometric conceptions.

The ’science indicators’ movement in the US with the discussionabout the possible use of bibliometrics in science policy in the 1970smarked the begin of a new era in bibliometrics.

7/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 15: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

A consequence of the growth of knowledge and the evolution fromlittle science to big science: Need for supplementing researchevaluation with quantitative methods and of linking funding toperformance indicators.

The application to science policy has brought a new perspective, andresulted in re-interpretation of bibliometric conceptions.

The ’science indicators’ movement in the US with the discussionabout the possible use of bibliometrics in science policy in the 1970smarked the begin of a new era in bibliometrics.

7/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 16: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

Bibliometrics evolved from a sub-discipline of library andinformation science to an instrument for evaluation andbenchmarking. I called this “perspective shift”.

As a consequence of this perspective shift, new elds of applicationsand challenges opened to bibliometrics; but many tools were stilldesigned for use in scienti c information, information retrieval andlibraries. Those became used in a context for which these were notdesigned.

Data sources have to be improved and extended, new methods haveto be developed to make bibliometrics t for present-day and futuretasks.

8/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 17: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

Bibliometrics evolved from a sub-discipline of library andinformation science to an instrument for evaluation andbenchmarking. I called this “perspective shift”.

As a consequence of this perspective shift, new elds of applicationsand challenges opened to bibliometrics; but many tools were stilldesigned for use in scienti c information, information retrieval andlibraries. Those became used in a context for which these were notdesigned.

Data sources have to be improved and extended, new methods haveto be developed to make bibliometrics t for present-day and futuretasks.

8/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 18: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

Bibliometrics evolved from a sub-discipline of library andinformation science to an instrument for evaluation andbenchmarking. I called this “perspective shift”.

As a consequence of this perspective shift, new elds of applicationsand challenges opened to bibliometrics; but many tools were stilldesigned for use in scienti c information, information retrieval andlibraries. Those became used in a context for which these were notdesigned.

Data sources have to be improved and extended, new methods haveto be developed to make bibliometrics t for present-day and futuretasks.

8/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 19: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

Gar eld himself recognised the power of the IF for journal evaluationand considered it later also a journal performance indicator.

Sociology of science laid the theoretical groundwork for theparadigmatic perspective shift.

However, advancement of bibliometric methodology could notalways keep pace with the demands and the breathtakingdevelopment in data processing.

9/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 20: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

Gar eld himself recognised the power of the IF for journal evaluationand considered it later also a journal performance indicator.

Sociology of science laid the theoretical groundwork for theparadigmatic perspective shift.

However, advancement of bibliometric methodology could notalways keep pace with the demands and the breathtakingdevelopment in data processing.

9/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 21: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

Gar eld himself recognised the power of the IF for journal evaluationand considered it later also a journal performance indicator.

Sociology of science laid the theoretical groundwork for theparadigmatic perspective shift.

However, advancement of bibliometric methodology could notalways keep pace with the demands and the breathtakingdevelopment in data processing.

9/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 22: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

Further important changes in bibliometric application could be observed.

The level of aggregation decreased from the national level over theinstitutional level down to the level of the evaluation of researchgroups or even individual scientists.

The role of bibliometrics changed from a rather “passive” monitoringand reporting to a more “active” one in terms of using bibliometricindicators for research funding and promotion.

A necessary requirements are high quality of data and watertightmethodology (cf., Glänzel, 1996).

Consequences are possible repercussions on the scientists’publication and citation behaviour (Glänzel & Debackere, 2003).

10/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 23: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

Further important changes in bibliometric application could be observed.

The level of aggregation decreased from the national level over theinstitutional level down to the level of the evaluation of researchgroups or even individual scientists.

The role of bibliometrics changed from a rather “passive” monitoringand reporting to a more “active” one in terms of using bibliometricindicators for research funding and promotion.

A necessary requirements are high quality of data and watertightmethodology (cf., Glänzel, 1996).

Consequences are possible repercussions on the scientists’publication and citation behaviour (Glänzel & Debackere, 2003).

10/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 24: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The evolution

Further important changes in bibliometric application could be observed.

The level of aggregation decreased from the national level over theinstitutional level down to the level of the evaluation of researchgroups or even individual scientists.

The role of bibliometrics changed from a rather “passive” monitoringand reporting to a more “active” one in terms of using bibliometricindicators for research funding and promotion.

A necessary requirements are high quality of data and watertightmethodology (cf., Glänzel, 1996).

Consequences are possible repercussions on the scientists’publication and citation behaviour (Glänzel & Debackere, 2003).

10/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 25: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The situation

The new situation implies several new tasks and challenges in bibliometricresearch and technology requiring new methods and instruments.

Extending bibliometrics to the technical sciences, social sciences andhumanities

Exploring and building new (“bibliometrics-ready”) databases

Development of advanced subject-delineation techniques(“bibliometrics-aided retrieval”) and improvement of subjectclassi cation

Inclusion of bibliometrics in funding formulas

Using bibliometrics in the evaluation of institutes, departments andresearch groups

Ranking issues and “multi-dimensional bibliometrics”

11/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 26: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The situation

The new situation implies several new tasks and challenges in bibliometricresearch and technology requiring new methods and instruments.

Extending bibliometrics to the technical sciences, social sciences andhumanities

Exploring and building new (“bibliometrics-ready”) databases

Development of advanced subject-delineation techniques(“bibliometrics-aided retrieval”) and improvement of subjectclassi cation

Inclusion of bibliometrics in funding formulas

Using bibliometrics in the evaluation of institutes, departments andresearch groups

Ranking issues and “multi-dimensional bibliometrics”

11/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 27: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The situation

The new situation implies several new tasks and challenges in bibliometricresearch and technology requiring new methods and instruments.

Extending bibliometrics to the technical sciences, social sciences andhumanities

Exploring and building new (“bibliometrics-ready”) databases

Development of advanced subject-delineation techniques(“bibliometrics-aided retrieval”) and improvement of subjectclassi cation

Inclusion of bibliometrics in funding formulas

Using bibliometrics in the evaluation of institutes, departments andresearch groups

Ranking issues and “multi-dimensional bibliometrics”

11/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 28: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The situation

The new situation implies several new tasks and challenges in bibliometricresearch and technology requiring new methods and instruments.

Extending bibliometrics to the technical sciences, social sciences andhumanities

Exploring and building new (“bibliometrics-ready”) databases

Development of advanced subject-delineation techniques(“bibliometrics-aided retrieval”) and improvement of subjectclassi cation

Inclusion of bibliometrics in funding formulas

Using bibliometrics in the evaluation of institutes, departments andresearch groups

Ranking issues and “multi-dimensional bibliometrics”

11/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 29: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The situation

The new situation implies several new tasks and challenges in bibliometricresearch and technology requiring new methods and instruments.

Extending bibliometrics to the technical sciences, social sciences andhumanities

Exploring and building new (“bibliometrics-ready”) databases

Development of advanced subject-delineation techniques(“bibliometrics-aided retrieval”) and improvement of subjectclassi cation

Inclusion of bibliometrics in funding formulas

Using bibliometrics in the evaluation of institutes, departments andresearch groups

Ranking issues and “multi-dimensional bibliometrics”

11/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 30: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The situation

The new situation implies several new tasks and challenges in bibliometricresearch and technology requiring new methods and instruments.

Extending bibliometrics to the technical sciences, social sciences andhumanities

Exploring and building new (“bibliometrics-ready”) databases

Development of advanced subject-delineation techniques(“bibliometrics-aided retrieval”) and improvement of subjectclassi cation

Inclusion of bibliometrics in funding formulas

Using bibliometrics in the evaluation of institutes, departments andresearch groups

Ranking issues and “multi-dimensional bibliometrics”

11/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 31: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The situation

The new situation implies several new tasks and challenges in bibliometricresearch and technology requiring new methods and instruments.

Extending bibliometrics to the technical sciences, social sciences andhumanities

Exploring and building new (“bibliometrics-ready”) databases

Development of advanced subject-delineation techniques(“bibliometrics-aided retrieval”) and improvement of subjectclassi cation

Inclusion of bibliometrics in funding formulas

Using bibliometrics in the evaluation of institutes, departments andresearch groups

Ranking issues and “multi-dimensional bibliometrics”

11/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 32: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

The situation

In order to take on these challenges, also new centres for science,technology and innovation research have been founded world wide.Nowadays public and commercial institutes are searching for newsolutions in bibliometrics and related elds.

Among these institutes, we nd research centres with long tradition likeISI (USA), ISSRU (Hungary), VINITI (Russia), CWTS (Netherlands) or NISTADS(India) but also newer ones like OST (France), REPP (Australia), NIFU STEP(Norway), SOOI (Belgium), KAWAX (Chile), WISELAB (China) and IFQ(Germany), the companies Science-Metrix (Canada), Evaluametrics (UK),Evindence (UK) and others.

12/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 33: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Extending bibliometrics

Scientometrics was designed to measure scholarly communication inbasic research. But what happens when information sources and/ortargets are outside the community (Glänzel & Schoep in, 1999)?

Extending bibliometrics to the technical sciences, social sciences andhumanities is faced with the following problems.

Coverage (document type)

Journal coverage

National representation

Language representation

Role and interpretation of citations

Quality criteria and comparability

13/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 34: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Extending bibliometrics

Scientometrics was designed to measure scholarly communication inbasic research. But what happens when information sources and/ortargets are outside the community (Glänzel & Schoep in, 1999)?

Extending bibliometrics to the technical sciences, social sciences andhumanities is faced with the following problems.

Coverage (document type)

Journal coverage

National representation

Language representation

Role and interpretation of citations

Quality criteria and comparability

13/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 35: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Extending bibliometrics

Scientometrics was designed to measure scholarly communication inbasic research. But what happens when information sources and/ortargets are outside the community (Glänzel & Schoep in, 1999)?

Extending bibliometrics to the technical sciences, social sciences andhumanities is faced with the following problems.

Coverage (document type)

Journal coverage

National representation

Language representation

Role and interpretation of citations

Quality criteria and comparability

13/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 36: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Extending bibliometrics

Scientometrics was designed to measure scholarly communication inbasic research. But what happens when information sources and/ortargets are outside the community (Glänzel & Schoep in, 1999)?

Extending bibliometrics to the technical sciences, social sciences andhumanities is faced with the following problems.

Coverage (document type)

Journal coverage

National representation

Language representation

Role and interpretation of citations

Quality criteria and comparability

13/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 37: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Extending bibliometrics

Scientometrics was designed to measure scholarly communication inbasic research. But what happens when information sources and/ortargets are outside the community (Glänzel & Schoep in, 1999)?

Extending bibliometrics to the technical sciences, social sciences andhumanities is faced with the following problems.

Coverage (document type)

Journal coverage

National representation

Language representation

Role and interpretation of citations

Quality criteria and comparability

13/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 38: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Extending bibliometrics

Scientometrics was designed to measure scholarly communication inbasic research. But what happens when information sources and/ortargets are outside the community (Glänzel & Schoep in, 1999)?

Extending bibliometrics to the technical sciences, social sciences andhumanities is faced with the following problems.

Coverage (document type)

Journal coverage

National representation

Language representation

Role and interpretation of citations

Quality criteria and comparability

13/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 39: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Extending bibliometrics

Scientometrics was designed to measure scholarly communication inbasic research. But what happens when information sources and/ortargets are outside the community (Glänzel & Schoep in, 1999)?

Extending bibliometrics to the technical sciences, social sciences andhumanities is faced with the following problems.

Coverage (document type)

Journal coverage

National representation

Language representation

Role and interpretation of citations

Quality criteria and comparability

13/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 40: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Extending bibliometrics

Academic output by field of research and publication type

0%

20%

40%

60%

80%

100%

Mat

hem

atic

s

Che

mis

try

Phy

sics

Bio

logy

Med

ical

Agr

icul

ture

Ear

th

Soc

ial S

ci

App

lied

Sci

Eng

inee

ring

Hum

aniti

es

Info

rm, C

ompu

t

Journal Articles (refereed) Journal Articles (non-refereed) Books

Book Chapters Conference Creative Works

Other

Academic output by field of research and publication type

0%

20%

40%

60%

80%

100%

Mat

hem

atic

s

Che

mis

try

Phy

sics

Bio

logy

Med

ical

Agr

icul

ture

Ear

th

Soc

ial S

ci

App

lied

Sci

Eng

inee

ring

Hum

aniti

es

Info

rm, C

ompu

t

Journal Articles (refereed) Journal Articles (non-refereed) Books

Book Chapters Conference Creative Works

Other

Source: ISI (2000) and Linda Butler (2003)

14/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 41: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

New databases

While Scopus (Elsevier) evolves to a competitive product to the WoS withsimilar coverage and similar features, Google Scholar is rather a productfor the semi-professional user.

The advantage of Google Scholar is its availability and open access, thedisadvantage is the lacking documentation and the unclear standards.

At present, none of these databases is able to meet the requirements ofpresent-day research evaluation.

15/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 42: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

New databases

In order to meet the emerging requirements of research evaluation, thepossibility of extending existing databases (CWTS, SOOI) or building new,“bibliometric-ready”, databases (e.g., REPP, Australia; NIFU STEP, Norway;VABB-SHW, Flanders) is considered. Some of these databases are alreadyin use.

Different quality criteria, the weight of document types and lackinginternational harmonisation remain problematic.

Building advanced bibliometric indicators for data extracted from thesedatabases remains an unresolved problem as well. Measures of receptionand impact are practically not available.

16/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 43: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Subject delineation

The lack a appropriate subject-classi cation schemes for elds, sub eldsand topics resulted in the following tasks.

Improving subject delineation and classi cation based on hybridmethods, i.e., combinations of text mining, and bibliometrictechniques (Zitt & Bassecoulrad, 2006, Glänzel et al., 2006, Glenissonet al., 2005, Janssens et al., 2008, 2009, Glänzel et al., 2009)

Identi cation of emerging and converging elds, visualisation ofstructure and eld dynamics, domain studies (University of Granada;Information Visualization Lab, Indiana University; Science-Metrix;CWTS, Leiden University; SOOI, KU Leuven; ASCoR, UniversityAmsterdam)

17/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 44: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Subject delineation

The lack a appropriate subject-classi cation schemes for elds, sub eldsand topics resulted in the following tasks.

Improving subject delineation and classi cation based on hybridmethods, i.e., combinations of text mining, and bibliometrictechniques (Zitt & Bassecoulrad, 2006, Glänzel et al., 2006, Glenissonet al., 2005, Janssens et al., 2008, 2009, Glänzel et al., 2009)

Identi cation of emerging and converging elds, visualisation ofstructure and eld dynamics, domain studies (University of Granada;Information Visualization Lab, Indiana University; Science-Metrix;CWTS, Leiden University; SOOI, KU Leuven; ASCoR, UniversityAmsterdam)

17/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 45: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Subject delineation

The lack a appropriate subject-classi cation schemes for elds, sub eldsand topics resulted in the following tasks.

Improving subject delineation and classi cation based on hybridmethods, i.e., combinations of text mining, and bibliometrictechniques (Zitt & Bassecoulrad, 2006, Glänzel et al., 2006, Glenissonet al., 2005, Janssens et al., 2008, 2009, Glänzel et al., 2009)

Identi cation of emerging and converging elds, visualisation ofstructure and eld dynamics, domain studies (University of Granada;Information Visualization Lab, Indiana University; Science-Metrix;CWTS, Leiden University; SOOI, KU Leuven; ASCoR, UniversityAmsterdam)

17/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 46: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Subject delineation

In several studies, “intellectual” subject-classi cation schemes have beenanalysed for “goodness” of classi cation and compared with the results ofhybrid clustering.

Example: The 22- eld subject classi cation scheme of the EssentialScience Indicators (ESI) is evaluated. In a second step, thehybrid clustering method is applied to classify WoSjournals. The choice of 22 clusters also allows a directeld-to-cluster comparison, and we substantiate that the

science areas resulting from cluster analysis form a morecoherent structure than the “intellectual” referencescheme, the ESI subject scheme. Observedmigration ofjournals among science elds can then be adopted toimprove classi cation.

18/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 47: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Subject delineation

In several studies, “intellectual” subject-classi cation schemes have beenanalysed for “goodness” of classi cation and compared with the results ofhybrid clustering.

Example: The 22- eld subject classi cation scheme of the EssentialScience Indicators (ESI) is evaluated. In a second step, thehybrid clustering method is applied to classify WoSjournals. The choice of 22 clusters also allows a directeld-to-cluster comparison, and we substantiate that the

science areas resulting from cluster analysis form a morecoherent structure than the “intellectual” referencescheme, the ESI subject scheme. Observedmigration ofjournals among science elds can then be adopted toimprove classi cation.

18/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 48: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Subject delineation

Three-dimensional MDS map visualising distances between the centres (centroids) ofthe 22 ESI elds and the 22 journal clusters

19/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 49: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and funding

In several countries bibliometric indicators are used as components offunding formulas.

Example: One of the major funding mechanisms for basic sciencein Flanders is the BOF-mechanism. The BijzonderOnderzoeksfonds is a research fund of the Flemishgovernment that is distributed to the six Flemishuniversities on the basis of a speci c allocation model.This allocation key comprises both input and outputmeasures on the research performance of theuniversities involved. Part of the key is based onpublication and citation data as derived from WoS andISI Proceedings.

20/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 50: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and funding

The use of bibliometrics in fundingmechanimsmight also have undesiredconsequences, as has reported by Linda Butler.

Example: The publications component of the Composite Indexused for allocating university block funding in Australiahas stimulated an increased publication activity in thelower impact journals (Butler, 2004).

21/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 51: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and funding

0.0

0.5

1.0

1.5

2.0

2.5

1981-8

5

1983-8

7

1985-8

9

1987-9

1

1989-9

3

1991-9

5

1993-9

7

1995-9

9

Austr

alia

n U

niv

ers

itie

s' S

hare

of S

CI

Quartile 1

Quartile 2

Quartile 3

Quartile 4

Introduction of

publications collection

Source: ISI (2000) and Linda Butler (2004)

Australian universities’ share of publication in the SCI, by journal impact quartile:ve-year windows, 1981–1985 to 1996–2000

22/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 52: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and funding

Scientific community

Publication & citation behaviour

Science policy &

Research management

Funding & Promotion

Bibliometrics

Scientific community

Publication & citation behaviour

Science policy &

Research management

Funding & Promotion

Bibliometrics

Schematic visualisation of the feedback of policy use of bibliometricson the scienti c community

23/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 53: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Meso- andmicro-level bibliometrics

Speci c problems

Institute/group-speci c specialisation (different pro les mightcause biases)

Diversi cation (might result in deviations from the assumed eldstandard)

Avoiding “comparing apples with pears” (e.g. medical school withbusiness school)

24/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 54: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Meso- andmicro-level bibliometrics

Speci c solutions

...1 Clustering institutions by publication pro les

...2 Standardisation and normalisation for intra- and inter-classcomparison whenever possible

...3 Development of indicators for identifying the “high-impact” ofresearch and for comparison of performance across different levelsof aggregation

25/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 55: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

1. Pro le clusters

0

100

200

300

400

500

600

700

800

900

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Number of Clusters

Duda/ Hart Je(2)/Je(1) x100

Calinski / Harabasz Pesudo-F

Optimum

(1) (2)

0

100

200

300

400

500

600

700

800

900

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Number of Clusters

Duda/ Hart Je(2)/Je(1) x100

Calinski / Harabasz Pesudo-F

Optimum

(1) (2)

Optimum number of classes according to Duda & Hart

26/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 56: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

1. Pro le clusters

The eight clusters resulting from the second optimum solution

Cluster Code

Cluster 1 (Biology) BIOCluster 2 (Agriculture) AGRCluster 3 (Multidisciplinary) MDSCluster 4 (Geo & Space Science) GSSCluster 5 (Technical & Natural Sciences) TNSCluster 6 (Chemistry) CHECluster 7 (General & Research Medicine) GRMCluster 8 (Specialised Medicine) SPM

27/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 57: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

1. Pro le analysis

0%

5%

10%

15%

20%

25%

30%

35%

40%

BIO AGR MDS GSS TNS CHE GRM SPM

Belgium Finland Spain

Examples for different national cluster pro les

28/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 58: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

2. The effect of subject normalisation

0.2

0.4

0.6

0.8

1.0

1.2

1.4

0.2 0.4 0.6 0.8 1.0 1.2 1.4MECR/FECR

NMCR

RCR = 1.0

low observedimpact

low observedimpact

low journalimpact

low journalimpact

high journalimpact

high journalimpact

high observedimpact

high observedimpact observation

higher than

journal impact

observation

lower than

journal impact

Prototype of a subject-normalised relational chart plottingNMCR vs. MECR/FECR

29/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 59: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

2. The effect of subject normalisation

HT

HM

HE

SUSL

SK

PT

PP

PC

NA

NU

NL

RT

RD

RN

IM

IB IS

FT

FO

FH

EV

EB

EC

KC

KT

KA

DB

DH

DM

BB

BG

BL

AG

AI

AV

2

3

4

5

6

7

8

9

2 3 4 5 6 7 8 9MECR

MO

CR

RCR = 1.0

AV

AI

AG

BL

BG

BB

DMDH

DB

KA

KT

KC

EC

EB

EV

FH

FOFTIS

IBIMRN

RD

RT

NL

NU

NA

PC

PP

PT

SK

SLSU

HEHM HT

0.50

0.75

1.00

1.25

1.50

1.75

2.00

0.50 0.75 1.00 1.25 1.50 1.75 2.00

MECR/FECR

NM

CR

=MO

CR

/FEC

R

RCR = 1.0

Real vs. expected performance (not normalised)

Real vs. expected performance (field-normalised)

30/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 60: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

2. The effect of subject normalisation

A

B

C

D

E

F

G

H

I

J

K

L

M

NO

P

QR

S

T

UV

X Y

0.6

0.8

1.0

1.2

1.4

1.6

1.8

0.6 0.8 1.0 1.2 1.4 1.6 1.8

MECR/FECR

NMCR

Sub eld-based relational chart of expected and observed citationimpact for 24 selected universities

31/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 61: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

3. Characteristic Scores and Scales

Characteristic scores and scales (Glänzel & Schubert, 1988) providesubject-sensitive, self-adjusting thresholds for citation distributions.

This method can be summarised as iteratively truncating samplesat their mean value and recalculating the mean of the truncatedsample until the procedure is stopped or the sample is empty.

⇒ bk = E(X|X> bk−1) with b0 := 0.

32/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 62: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

3. Characteristic Scores and Scales

Characteristic scores and scales (Glänzel & Schubert, 1988) providesubject-sensitive, self-adjusting thresholds for citation distributions.

This method can be summarised as iteratively truncating samplesat their mean value and recalculating the mean of the truncatedsample until the procedure is stopped or the sample is empty.

⇒ bk = E(X|X> bk−1) with b0 := 0.

32/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 63: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

3. Characteristic Scores and Scales

Characteristic scores and scales (Glänzel & Schubert, 1988) providesubject-sensitive, self-adjusting thresholds for citation distributions.

This method can be summarised as iteratively truncating samplesat their mean value and recalculating the mean of the truncatedsample until the procedure is stopped or the sample is empty.

⇒ bk = E(X|X> bk−1) with b0 := 0.

32/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 64: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

3. Characteristic Scores and Scales

b1

b2

b3

Class 2

Class 1

Class 3

Class 4

Visualisation of characteristic scores and scales for four classes

33/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 65: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

3. Characteristic Scores and Scales

As was shown in earlier studies (e.g., Glänzel, 2007), the scores bk aretime-dependent and strongly sensitive to the subject matter but theshare of papers in the individual zones is relatively stable over thecitation window, and does not noticeably vary among differentsub elds either.

Characteristic Scores and Scales allow gauging citation impact of anysample (publication output of individual scientists, research groups,departments, institutes, etc.) against a suitable reference standard.

34/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 66: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

3. Characteristic Scores and Scales

As was shown in earlier studies (e.g., Glänzel, 2007), the scores bk aretime-dependent and strongly sensitive to the subject matter but theshare of papers in the individual zones is relatively stable over thecitation window, and does not noticeably vary among differentsub elds either.

Characteristic Scores and Scales allow gauging citation impact of anysample (publication output of individual scientists, research groups,departments, institutes, etc.) against a suitable reference standard.

34/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 67: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

Ranking of college and universities, has become one of the most favouriteissues in the assessment of higher education institutions and thus one ofthe new challenges in policy-relevant evaluation.

Since the rst edition of the Shanghai Jiao Tong world university rankingand the successor lists, such as the Times Higher Education Supplement -QS World University Rankings, the comparative evaluation of the qualityof Education Institutions has been brought into the focus of public andpolicy interest.

Methodology and discussion is, however, controversal. Even the recentlyelaborated “Berlin principles” (2006) could not solve the problems. And,as expected, bibliometrics is part of this ranking as well.

35/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 68: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

In order to clarify the notion of ranking, the following comprehensible butnevertheless precise verbal de nition is given.

.De nition..

.

. ..

.

.

Ranking is positioning comparable objects on an ordinal scale based on a(non-strict) weak order relation among (statistical) functions of, or a combinationof functions of measures or scores associated with those objects.

.Remark..

.

. ..

.

.

These (mainly statistical) functions, which are usually called indicators. Differentindicators Xi representing different aspects of quality, form the components of acomposite indicator Y, being the basis of the ranking, particularly,

Y=∑λiXi ,

where i (i= 1,2, . . . ,n) are n pre-de ned weights. Without loss of generality, wecan assume that

∑λi = 1.

36/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 69: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

The most problematic issues in applying composite indicators are asfollows.

The “cleanness” and hence the reliability of underlying data

The time-variant nature of the underlying data sources

Possible interdependence of components

Altering weights can result in different ranking

Methodology might result in obscure and irreproducible ranking

Random errors of statistical functions are usually ignored

The multi-dimensional space is crashed into linearity (loss ofinformation)

37/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 70: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

The most problematic issues in applying composite indicators are asfollows.

The “cleanness” and hence the reliability of underlying data

The time-variant nature of the underlying data sources

Possible interdependence of components

Altering weights can result in different ranking

Methodology might result in obscure and irreproducible ranking

Random errors of statistical functions are usually ignored

The multi-dimensional space is crashed into linearity (loss ofinformation)

37/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 71: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

The most problematic issues in applying composite indicators are asfollows.

The “cleanness” and hence the reliability of underlying data

The time-variant nature of the underlying data sources

Possible interdependence of components

Altering weights can result in different ranking

Methodology might result in obscure and irreproducible ranking

Random errors of statistical functions are usually ignored

The multi-dimensional space is crashed into linearity (loss ofinformation)

37/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 72: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

The most problematic issues in applying composite indicators are asfollows.

The “cleanness” and hence the reliability of underlying data

The time-variant nature of the underlying data sources

Possible interdependence of components

Altering weights can result in different ranking

Methodology might result in obscure and irreproducible ranking

Random errors of statistical functions are usually ignored

The multi-dimensional space is crashed into linearity (loss ofinformation)

37/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 73: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

The most problematic issues in applying composite indicators are asfollows.

The “cleanness” and hence the reliability of underlying data

The time-variant nature of the underlying data sources

Possible interdependence of components

Altering weights can result in different ranking

Methodology might result in obscure and irreproducible ranking

Random errors of statistical functions are usually ignored

The multi-dimensional space is crashed into linearity (loss ofinformation)

37/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 74: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

The most problematic issues in applying composite indicators are asfollows.

The “cleanness” and hence the reliability of underlying data

The time-variant nature of the underlying data sources

Possible interdependence of components

Altering weights can result in different ranking

Methodology might result in obscure and irreproducible ranking

Random errors of statistical functions are usually ignored

The multi-dimensional space is crashed into linearity (loss ofinformation)

37/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 75: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

The most problematic issues in applying composite indicators are asfollows.

The “cleanness” and hence the reliability of underlying data

The time-variant nature of the underlying data sources

Possible interdependence of components

Altering weights can result in different ranking

Methodology might result in obscure and irreproducible ranking

Random errors of statistical functions are usually ignored

The multi-dimensional space is crashed into linearity (loss ofinformation)

37/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 76: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

The most problematic issues in applying composite indicators are asfollows.

The “cleanness” and hence the reliability of underlying data

The time-variant nature of the underlying data sources

Possible interdependence of components

Altering weights can result in different ranking

Methodology might result in obscure and irreproducible ranking

Random errors of statistical functions are usually ignored

The multi-dimensional space is crashed into linearity (loss ofinformation)

37/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 77: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

In order to account for the complexity of university activities, two basicapproaches are possible:

The selective ranking focuses on measuring and ranking according toone selected activity whereas

the integrated or ’holistic’ ranking procedure attempts to capture thecomplex set of all or at least of the most important activities.

The advantages of the rst method are obvious as information loss andincommensurability can be reduced and reliability can be increased. Inthe following we give a concise description of examples for selective andintegrated college and university ranking.

38/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 78: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

In order to account for the complexity of university activities, two basicapproaches are possible:

The selective ranking focuses on measuring and ranking according toone selected activity whereas

the integrated or ’holistic’ ranking procedure attempts to capture thecomplex set of all or at least of the most important activities.

The advantages of the rst method are obvious as information loss andincommensurability can be reduced and reliability can be increased. Inthe following we give a concise description of examples for selective andintegrated college and university ranking.

38/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 79: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

In order to account for the complexity of university activities, two basicapproaches are possible:

The selective ranking focuses on measuring and ranking according toone selected activity whereas

the integrated or ’holistic’ ranking procedure attempts to capture thecomplex set of all or at least of the most important activities.

The advantages of the rst method are obvious as information loss andincommensurability can be reduced and reliability can be increased. Inthe following we give a concise description of examples for selective andintegrated college and university ranking.

38/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 80: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

In order to account for the complexity of university activities, two basicapproaches are possible:

Evaluation of education In 1993 a national education-relateduniversity ranking was published in Germany (Spiegel, 1993). Theranking was survey-based.Because of differences and peculiarities of national educational andaccreditation systems, such endeavours are practically restricted tothe national level.

Research performanceWith the ’Shanghai Ranking’, rst published in2003, the focus was shifted to research assessment. This ranking islargely based on theWeb of Science and its derivatives.

39/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 81: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

In order to account for the complexity of university activities, two basicapproaches are possible:

Evaluation of education In 1993 a national education-relateduniversity ranking was published in Germany (Spiegel, 1993). Theranking was survey-based.Because of differences and peculiarities of national educational andaccreditation systems, such endeavours are practically restricted tothe national level.

Research performanceWith the ’Shanghai Ranking’, rst published in2003, the focus was shifted to research assessment. This ranking islargely based on theWeb of Science and its derivatives.

39/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 82: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

In order to account for the complexity of university activities, two basicapproaches are possible:

Evaluation of education In 1993 a national education-relateduniversity ranking was published in Germany (Spiegel, 1993). Theranking was survey-based.Because of differences and peculiarities of national educational andaccreditation systems, such endeavours are practically restricted tothe national level.

Research performanceWith the ’Shanghai Ranking’, rst published in2003, the focus was shifted to research assessment. This ranking islargely based on theWeb of Science and its derivatives.

39/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 83: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

‘Holistic approach’ The broader approach by THES-QS actually marksa new direction in university ranking, particularly, the trend towardsintegrated evaluation. It largely relies on peer review score. Theholistic approach and a world-wide ranking based on all HEIactivities, including education, research and third mission, however,remains at least for the present utopian.

The question arises of whether there is really any need for anintegrated ranking.

40/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 84: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

• Education

• Research

• Third mission

BibliometricsBibliometricsBibliometrics

University activities

Research outputResearch outputResearch output

Scholarly communicationScholarly communicationScholarly communication

Function of bibliometrics in quantifying and measuring activities of HEIs

41/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 85: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

Robustness is easily lost by building composite indicators withpartially interdependent or incompatible components and arbitraryweights.

Reality is more complex than can be described this way. Instead ofany linear ranking a complex analysis is necessary to capture allaspects of performance among the manifold of a university’sactivities.

Bibliometrics can contribute to evaluate at least one of these aspects.Standardisation and normalisation help eliminate biases andfacilitate longitudinal analysis as well.

Even normalisation of indicators cannot disguise that comparingHEIs with completely different pro les remains an exercise of’comparing apples with oranges’.

42/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 86: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

Robustness is easily lost by building composite indicators withpartially interdependent or incompatible components and arbitraryweights.

Reality is more complex than can be described this way. Instead ofany linear ranking a complex analysis is necessary to capture allaspects of performance among the manifold of a university’sactivities.

Bibliometrics can contribute to evaluate at least one of these aspects.Standardisation and normalisation help eliminate biases andfacilitate longitudinal analysis as well.

Even normalisation of indicators cannot disguise that comparingHEIs with completely different pro les remains an exercise of’comparing apples with oranges’.

42/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 87: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

Robustness is easily lost by building composite indicators withpartially interdependent or incompatible components and arbitraryweights.

Reality is more complex than can be described this way. Instead ofany linear ranking a complex analysis is necessary to capture allaspects of performance among the manifold of a university’sactivities.

Bibliometrics can contribute to evaluate at least one of these aspects.Standardisation and normalisation help eliminate biases andfacilitate longitudinal analysis as well.

Even normalisation of indicators cannot disguise that comparingHEIs with completely different pro les remains an exercise of’comparing apples with oranges’.

42/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 88: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Bibliometrics and ranking

Robustness is easily lost by building composite indicators withpartially interdependent or incompatible components and arbitraryweights.

Reality is more complex than can be described this way. Instead ofany linear ranking a complex analysis is necessary to capture allaspects of performance among the manifold of a university’sactivities.

Bibliometrics can contribute to evaluate at least one of these aspects.Standardisation and normalisation help eliminate biases andfacilitate longitudinal analysis as well.

Even normalisation of indicators cannot disguise that comparingHEIs with completely different pro les remains an exercise of’comparing apples with oranges’.

42/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 89: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Conclusions and discussion

Beyond any doubt, growing policy application turned ´littlebibliometrics’ into ‘big bibliometrics’. However, this evolution alsoresulted in a clear ‘perspective shift’.

The changing shape of scientometrics can be observed world wise.New research and service centres are founded, new journals andconference series are launched. The “geographical” centre of gravityis further shifted towards East.

This shift, in turn, has severe consequences for the eld itself and thewhole scienti c community as well.

Bibliometrics is faced with new challenges which have the potentialto further transform our eld.

43/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 90: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Conclusions and discussion

Beyond any doubt, growing policy application turned ´littlebibliometrics’ into ‘big bibliometrics’. However, this evolution alsoresulted in a clear ‘perspective shift’.

The changing shape of scientometrics can be observed world wise.New research and service centres are founded, new journals andconference series are launched. The “geographical” centre of gravityis further shifted towards East.

This shift, in turn, has severe consequences for the eld itself and thewhole scienti c community as well.

Bibliometrics is faced with new challenges which have the potentialto further transform our eld.

43/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 91: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Conclusions and discussion

Beyond any doubt, growing policy application turned ´littlebibliometrics’ into ‘big bibliometrics’. However, this evolution alsoresulted in a clear ‘perspective shift’.

The changing shape of scientometrics can be observed world wise.New research and service centres are founded, new journals andconference series are launched. The “geographical” centre of gravityis further shifted towards East.

This shift, in turn, has severe consequences for the eld itself and thewhole scienti c community as well.

Bibliometrics is faced with new challenges which have the potentialto further transform our eld.

43/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 92: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Conclusions and discussion

Beyond any doubt, growing policy application turned ´littlebibliometrics’ into ‘big bibliometrics’. However, this evolution alsoresulted in a clear ‘perspective shift’.

The changing shape of scientometrics can be observed world wise.New research and service centres are founded, new journals andconference series are launched. The “geographical” centre of gravityis further shifted towards East.

This shift, in turn, has severe consequences for the eld itself and thewhole scienti c community as well.

Bibliometrics is faced with new challenges which have the potentialto further transform our eld.

43/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 93: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Conclusions and discussion

Bibliometrics is determing the design of a new generation ofbibliographic databases, and has become established in researchfunding.

On the other hand, related elds like Webometrics have in uencedbibliometric methodology as well.

Both open-minded interaction with users and “hard-minded”scienti c accuracy are precondition for being up to the new tasks.

44/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 94: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Conclusions and discussion

Bibliometrics is determing the design of a new generation ofbibliographic databases, and has become established in researchfunding.

On the other hand, related elds like Webometrics have in uencedbibliometric methodology as well.

Both open-minded interaction with users and “hard-minded”scienti c accuracy are precondition for being up to the new tasks.

44/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 95: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

Conclusions and discussion

Bibliometrics is determing the design of a new generation ofbibliographic databases, and has become established in researchfunding.

On the other hand, related elds like Webometrics have in uencedbibliometric methodology as well.

Both open-minded interaction with users and “hard-minded”scienti c accuracy are precondition for being up to the new tasks.

44/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven

Page 96: international panorama of scientometrics...Scientometrics WolfgangGlänzel Structure Introduction Perspectiveshift International panorama Thesituation Extending bibliometrics Newdatabases

InternationalPanorama ofScientometrics

Wolfgang Glänzel

Structure

Introduction

Perspective shift

Internationalpanorama

The situation

Extendingbibliometrics

New databases

Subject delineation

Bibliometrics andfunding

Meso/micro-levelbibliometrics

Ranking

Conclusions

.

.

. ..

.

.

Thank you very much

45/45 Wolfgang Glänzel Steunpunt O&O Indicatoren – K.U. Leuven