bibliometric measures to demonstrate impact
TRANSCRIPT
Prestigious Fellows SeminarThe University of Queensland
Dr Berenika M Webster13 April 2010
Using bibliometric measures to demonstrate the impact of
research
James McKeen Cattell and American Men of Science
1
Hulme’s statistical bibliography
2
Samuel C Bradford’s law of scattering
3
Derek de Solla Price’s Little Science, Big Science
4
Sociology of science at Columbia University
5
Eugene Garfield’s “association of ideas index”
6
(Bibliometric* OR Citation*) AND Evaluation
Publication year
Number of publications
Publication year
Number of publications
1963 1 1993 71967 1 1994 101972 1 1995 221976 3 1996 241979 2 1997 181980 4 1998 211981 5 1999 231982 4 2000 301983 4 2001 241984 2 2002 361985 2 2003 271986 5 2004 291987 3 2005 341988 5 2006 331989 3 2007 681990 9 2008 681991 10 2009 1091992 22 2010 25
RAE in UK
RQF in Australia
ERA and REF
Source: Web of Science, 20107
Things to look out for…
8
Can the available data answer the question?Are you comparing like with like?Relative and multiple measures.Do the results make sense?
Can the available data answer the question?
Disciplinary differencesPatterns of scholarly communicationInterdisciplinarity
Place vs. people
Source of publication informationExternal data sourceUniversity records Individual’s CV
Data set – size matters
9
Compare like with like – two case studies
Cancer research charities
Two schools of psychology
10
Absolute measure – volume of outputs
11
Relative measure: % of world output
12
Relative measures: subject profile of a researcher
Subject area Paps CitsAuthor
CPP Aus CPP Wld CPP Aus RCI WldRCIREPRODUCTIVE BIOLOGY 19 379 19.95 13.27 12.47 1.50 1.60DEVELOPMENTAL BIOLOGY 3 106 35.33 17.82 26.68 1.98 1.32OBSTETRICS & GYNECOLOGY 4 104 26.00 5.15 4.75 5.05 5.47CELL BIOLOGY 2 74 37.00 19.26 20.93 1.92 1.77AGRICULTURE, DAIRY & ANIMAL SCIENCE 1 83 83.00 4.85 6.19 17.11 13.41ENDOCRINOLOGY & METABOLISM 6 67 11.17 11.76 9.44 0.95 1.18VETERINARY SCIENCES 3 23 7.67 1.96 1.58 3.91 4.85ZOOLOGY 1 12 12.00 6.86 6.41 1.75 1.87IMMUNOLOGY 1 2 2.00 14.7 13.58 0.14 0.15PATHOLOGY 1 0 0.00 6.1 5.15 0.00 0.00Total 41 850 20.73 10.173 10.718 2.04 1.93
Source: Custom Citation Report, 201013
Multiple measures: how does my organisation compare against others?
Source: InCites (UQ data set), 2009
Publications Citations CPPAgricultural Sciences 1 2 7Biology & Biochemistry 1 1 3Chemistry 3 6 6Clinical Medicine 3 4 5Computer Science 3 1 2Economics & Business 5 4 5Engineering 5 5 5Environment/Ecology 1 1 3Geosciences 6 6 5Immunology 4 3 5Materials Science 3 1 2Mathematics 5 5 5Microbiology 1 2 5Molecular Biology & Genetics 2 1 1Neuroscience & Behavior 5 4 2Pharmacology & Toxicology 3 4 5Physics 5 4 2Plant & Animal Science 1 1 7Psychiatry/Psychology 1 3 5Social Sciences, general 2 3 3Space Science 7 6 2
Subject Area Rank
Source: InCites (UQ/Glasgow Univ data sets), 200915
Do the results make sense?
Use healthy scepticism
Is the data used relevant to the question asked?Are the methods used robust?Are the conclusions not reaching beyond the
limits of the data collected and methods applied?
Are all caveats stated clearly?
Publication and citation data should be a supplement to peer review
16