bibliometric measures to demonstrate impact

17
Prestigious Fellows Seminar The University of Queensland Dr Berenika M Webster 13 April 2010 Using bibliometric measures to demonstrate the impact of research

Upload: berenika-webster

Post on 12-Jan-2017

139 views

Category:

Data & Analytics


0 download

TRANSCRIPT

Page 1: Bibliometric measures to demonstrate impact

Prestigious Fellows SeminarThe University of Queensland

Dr Berenika M Webster13 April 2010

Using bibliometric measures to demonstrate the impact of

research

Page 2: Bibliometric measures to demonstrate impact

James McKeen Cattell and American Men of Science

1

Page 3: Bibliometric measures to demonstrate impact

Hulme’s statistical bibliography

2

Page 4: Bibliometric measures to demonstrate impact

Samuel C Bradford’s law of scattering

3

Page 5: Bibliometric measures to demonstrate impact

Derek de Solla Price’s Little Science, Big Science

4

Page 6: Bibliometric measures to demonstrate impact

Sociology of science at Columbia University

5

Page 7: Bibliometric measures to demonstrate impact

Eugene Garfield’s “association of ideas index”

6

Page 8: Bibliometric measures to demonstrate impact

(Bibliometric* OR Citation*) AND Evaluation

Publication year

Number of publications

Publication year

Number of publications

1963 1 1993 71967 1 1994 101972 1 1995 221976 3 1996 241979 2 1997 181980 4 1998 211981 5 1999 231982 4 2000 301983 4 2001 241984 2 2002 361985 2 2003 271986 5 2004 291987 3 2005 341988 5 2006 331989 3 2007 681990 9 2008 681991 10 2009 1091992 22 2010 25

RAE in UK

RQF in Australia

ERA and REF

Source: Web of Science, 20107

Page 9: Bibliometric measures to demonstrate impact

Things to look out for…

8

Can the available data answer the question?Are you comparing like with like?Relative and multiple measures.Do the results make sense?

Page 10: Bibliometric measures to demonstrate impact

Can the available data answer the question?

Disciplinary differencesPatterns of scholarly communicationInterdisciplinarity

Place vs. people

Source of publication informationExternal data sourceUniversity records Individual’s CV

Data set – size matters

9

Page 11: Bibliometric measures to demonstrate impact

Compare like with like – two case studies

Cancer research charities

Two schools of psychology

10

Page 12: Bibliometric measures to demonstrate impact

Absolute measure – volume of outputs

11

Page 13: Bibliometric measures to demonstrate impact

Relative measure: % of world output

12

Page 14: Bibliometric measures to demonstrate impact

Relative measures: subject profile of a researcher

Subject area Paps CitsAuthor

CPP Aus CPP Wld CPP Aus RCI WldRCIREPRODUCTIVE BIOLOGY 19 379 19.95 13.27 12.47 1.50 1.60DEVELOPMENTAL BIOLOGY 3 106 35.33 17.82 26.68 1.98 1.32OBSTETRICS & GYNECOLOGY 4 104 26.00 5.15 4.75 5.05 5.47CELL BIOLOGY 2 74 37.00 19.26 20.93 1.92 1.77AGRICULTURE, DAIRY & ANIMAL SCIENCE 1 83 83.00 4.85 6.19 17.11 13.41ENDOCRINOLOGY & METABOLISM 6 67 11.17 11.76 9.44 0.95 1.18VETERINARY SCIENCES 3 23 7.67 1.96 1.58 3.91 4.85ZOOLOGY 1 12 12.00 6.86 6.41 1.75 1.87IMMUNOLOGY 1 2 2.00 14.7 13.58 0.14 0.15PATHOLOGY 1 0 0.00 6.1 5.15 0.00 0.00Total 41 850 20.73 10.173 10.718 2.04 1.93

Source: Custom Citation Report, 201013

Page 15: Bibliometric measures to demonstrate impact

Multiple measures: how does my organisation compare against others?

Source: InCites (UQ data set), 2009

Publications Citations CPPAgricultural Sciences 1 2 7Biology & Biochemistry 1 1 3Chemistry 3 6 6Clinical Medicine 3 4 5Computer Science 3 1 2Economics & Business 5 4 5Engineering 5 5 5Environment/Ecology 1 1 3Geosciences 6 6 5Immunology 4 3 5Materials Science 3 1 2Mathematics 5 5 5Microbiology 1 2 5Molecular Biology & Genetics 2 1 1Neuroscience & Behavior 5 4 2Pharmacology & Toxicology 3 4 5Physics 5 4 2Plant & Animal Science 1 1 7Psychiatry/Psychology 1 3 5Social Sciences, general 2 3 3Space Science 7 6 2

Subject Area Rank

Page 16: Bibliometric measures to demonstrate impact

Source: InCites (UQ/Glasgow Univ data sets), 200915

Page 17: Bibliometric measures to demonstrate impact

Do the results make sense?

Use healthy scepticism

Is the data used relevant to the question asked?Are the methods used robust?Are the conclusions not reaching beyond the

limits of the data collected and methods applied?

Are all caveats stated clearly?

Publication and citation data should be a supplement to peer review

16