From econometrics to bibliometrics

Ludo WaltmanCentre for Science and Technology Studies, Leiden University

Econometrics and BeyondRotterdamMay 27, 2016

2

About myself

• 2003: BSc in Informatics & Economics• 2005: MSc in Economics & Informatics• 2005–2009: PhD student at Econometric Institute

• 2009–2016: Researcher at CWTS

3

About CWTS

• Centre for Science and Technology Studies (CWTS)

• Research center at Leiden University focusing on science and technology studies

• Strong emphasis on bibliometric and scientometric research

• About 30 staff members• 25th anniversary in 2015

4

Current activities

• Leading the Advanced Bibliometric Methods research program at CWTS– Indicators– Network analysis– Text mining– Visualization

• Bibliometric consultancy for CWTS BV• Coordinating the CWTS Leiden Ranking• Deputy director CWTS• Editor-in-Chief Journal of Informetrics

5

CWTS Leiden Ranking

6

From econo-metrics to bibliometrics

7

From econometrics to bibliometrics: Two examples• Visualization of network data• Indicator design

8

Visualization of network data

9

2006: 50 Years of Econometrics

10

2006: Multidimensional scaling vs. visualization of similarities (VOS)

11

2007: Concept map viewer

12

2009: VOSviewer 0.9.0

13

2016: VOSviewer 1.6.4

14

Co-authorship network based on EUR economics and business publications

15

Publications using VOSviewer

Visualizations of bibliometric networks nowadays are produced mostly using the VOSviewer software and the VOS technique; MDS is hardly used anymore

16

Indicator design

17

Bibliometric indicators

• Bibliometric indicators:– Scientific activity– Scientific impact– Collaboration– Interdisciplinarity– Etc.

• Bibliometric indicators tend to be justified based on ad hoc arguments rather than rigorous systematic reasoning

18

CWTS ‘crown indicator’

• For each publication i of a research group, we know the actual number of citations ci and the ‘expected’ number of citations ei

• On average, how often have the publications of a research group been cited relative to expectation?

crown indicator

19

Consistency

• Suppose group A and B have the same number of publications

• According to some indicator, group A is performing better than B

• Suppose group A and B jointly produce a new publication

• Then, if our indicator is consistent, group A should still be performing better than B

20

Inconsistency of crown indicator

• Crown indicator group A: 25 / 10 = 2.50• Crown indicator group B: 10 / 5 = 2.00• Group A and B co-author a new publication• Crown indicator group A: 65 / 20 = 3.25• Crown indicator group B: 50 / 15 = 3.33• Ranking of group A and B has reversed

Pub. 1 Group A 25 10Pub. 2 Group B 10 5Pub. 3 Group A and B 40 10

Pub. 1 Group A 25 10Pub. 2 Group B 10 5

21

Solving crown indicator inconsistency by changing order of operations• Inconsistent crown indicator:

• Consistent crown indicator:

𝑐1+⋯+𝑐𝑛𝑒1+⋯+𝑒𝑛

1𝑛 (𝑐1𝑒1+⋯+

𝑐𝑛𝑒𝑛 )

22

Adoption in major analytical tools

23

h-index

A scientist has index h if h of his papers have at least h citations each and the other papers have

at most h citations each

24

Consistency requirement

If two scientists achieve the same performance improvement, their ranking relative to each other

should remain unchanged

25

Inconsistency of the h-index

26

From econo-metrics to bibliometrics

27

From econometrics to bibliometrics: Conclusions• Bibliometrics is a quantitative field of science, but

level of sophistication of quantitative analyses is much lower than in econometrics

• Background from Econometric Institute has been very helpful to increase level of sophistication of bibliometric research

• Illustrates the need to move toward an integrated data science field, combining methods from econometrics, statistics, operations research, computer science, network science, etc.

28

Congratulations to theEconometric Institute!

Thank you for your attention