Bibliometrics and PublishingPeter Sjögårde, Bibliometric analyst

KTH Royal Institute of Technology, ECE School of

Education and Communication in Engineering Sciences

(ECE), Unit for Publication Infrastructure

v2 2015-06-05

Outline

• KTH context – Aims at KTH• Introduction to bibliometrics• Bibliometric indicator for funding allocation at KTH• Annual Bibliometric Monitoring

KTH context – Aims at KTH

• Strengthen its position as one of Europe’s leading technical universities

• Advance in relevant rankings• Increase citation scores

What is bibliometrics?

Bibliometrics can be defined as the quantitative study of publication collections

Statistics on:• Publications, like journal articles (research output)• Citations (research impact)• Co-publishing (research collaboration)

Example of publication output

2004 2005 2006 2007 2008 2009 2010 20110

20

40

60

80

100

120

Publication frequencies

ReportPatentLicentiate ThesisDoctoral ThesisConference Proceedings (editor)Conference PaperCollection/Anthology (ed-itor)Chapter in bookBook ReviewBookArticle, review/surveyArticle in journal (other)Article in journal (peer reviewed)

Nu

mb

er

of

pu

blic

ati

on

s

Example

Examples of citation impact

2004 2005 2006 2007 2008 2009 20100.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

Time series of field normalized citation rate

Year

Fie

ld n

orm

aliz

ed

cit

ati

on

rate

(3

year

mov-

ing

avera

ge),

cf3

Example

Examples of a co-publishing network

Example

Research Publication A

ReaderWriter Publication B

with Reference to the publication A

Citation

Impact?

• On society• On industry• On research community

As measured by the use of bibliometrics (citations)

Differences between research fields affecting bibliometric indicators

• Publication pace (time for publishing and peer review process)• Length of publications• Number of references• Coverage of publications in citation database• Document type (e.g. reviews get more citations, proceedings

papers are often excluded)• Size of research field

Field normalized citation rate

For a publication:

Same research field same yearsame document type

Average = 1

Citation indicators > Research Quality

• Correlation between peer review assessments and bibliometric indicators.

• In general, high quality (as judged by peers) is needed to receive high citation scores at aggregated levels.• But not enough

• There are research groups that get good judgments by reviewers but has low citation scores.

1. Research with impact outside the research community (e.g. applied research)

2. Research for which the bibliometric methods are improper.

Lutz Bornmann and Hans-Dieter Daniel, “Selection of Research Fellowship Recipients by Committee Peer Review. Reliability, Fairness and Predictive Validity of Board of Trustees’ Decisions,” Scientometrics 63, no. 2 (April 1, 2005): 297–320, doi:10.1007/s11192-005-0214-2; A. F. J. van Raan, “Advanced Bibliometric Methods as Quantitative Core of Peer Review Based Evaluation and Foresight Exercises,” Scientometrics 36, no. 3 (July 1, 1996): 397–420, doi:10.1007/BF02129602; Charles Oppenheim, “The Correlation between Citation Counts and the 1992 Research Assessment Exercise Ratings for British Research in Genetics, Anatomy and Archaeology,” Journal of Documentation 53, no. 5 (December 1, 1997): 477–87, doi:10.1108/EUM0000000007207; Fler studier är refererade i Bornmann and Daniel, “What Do Citation Counts Measure?”; och i Blaise Cronin, The Hand of Science: Academic Writing and Its Rewards (Lanham, Md: Scarecrow Press, 2005), 125–129

Application and use

Peer review

Sybille Hinze. 2014. Nordic workshop on bibliometrics and research policy. Reykjavik, Iceland.

Bibliometrics

Macro

Micro

Meso

Bibliometrics at KTH

• Research Assessment Exercise (research groups, UoA)• Funding allocation• Annual Bibliometric Monitoring• Reports and analyzes on demand from management and

schools• Consultancy work for other universities

Bibliometric indicator for funding allocation at KTH

Mean field normalized citation rate for journals and serials indexed by Web of Science• Average of the citation rate of the journals in which KTH:s

researchers have published the last three years. • Calculated for each department. Multiplied with the

number of faculty at each department.• Aggregated to Schools and used for allocation to the

Schools.

Bibliometric indicator for funding allocation at KTHExample. Let A be a department. Assume that A has three publications, P1, P2, and P3, published in three different journals. Further assumptions:

Then the mean field normalized journal impact for A, mjcf(A), is equal to

 

Let the number of faculty for A be 3. Then the value of the bibliometric indicator for funding allocation at KTH is

  A author fraction (of publication)

Jcf (of journal for A publication)

[A Author fraction] x Jcf

P1 1 (1/1) 0.8 0.8P2 0.25 (1/4) 1 0.25P3 0.5 (5/10) 2 1Sum 1.75   2.05

Purpose of the bibliometric funding allocation indicator

• Incentives to: • publish in highly cited journals indexed by Web of

Science. • publish few publications of high quality rather than a

high quantity of publications.

Annual Bibliometric Monitoring

https://intra.kth.se/bibliometri/public/start

Comments/questions to:

Per Ahlgren, Bibliometric Analyst

perahl@kth.se

Peter Sjögårde, Bibliometric Analyst

sjogarde@kth.se

Unit for Publication Infrastructure

pi@ece.kth.se

KTH Royal Institute of Technology

School of Education and Communication in Engineering Sciences (ECE)

Unit for Publication Infrastructure