impact factor and citation metrics: what do they really mean? session 1206 room 203a january 30,...

Impact Factor and Citation Impact Factor and Citation Metrics: What do they Really Metrics: What do they Really Mean?Mean?

Session 1206 Room 203AJanuary 30, 2009

Debbie Chaves, Wilfrid Laurier UniversityBrian Cameron, Ryerson University

OutlineOutline

  • Measurement of the Business of Science

• Background on various methods

• Where things can go wrong

• What that means for librarians

Why Measure?Why Measure?

• Security of knowledge

• Predictability

• Comparison analysis

• Subject and domain dependent

• Quantifiability fallacy

Historical Background on Historical Background on Impact FactorImpact Factor

Eugene Garfield, "Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas," Science 122, 3159 (July 15, 1955): 108

What is an Impact FactorWhat is an Impact Factor

ratio of articles published to articles cited during a rolling two-year window

A/BA = citations in 2008 : articles cited in

2006-7B = citable items published in 2006-7

year 2008 citations to 2006 + 2007 articlesarticles published in 2006 + 2007

Your (real) Impact FactorYour (real) Impact Factor

Use of Impact FactorsUse of Impact FactorsCollection developmentChoosing journals for publicationJournal assessment/marketing by

publishersEvaluation of scholarly research

& individual performance, for purposes of tenure and promotion, and funding

Evaluation of departments, institutions, and nations

Problems with IFProblems with IFTwo-year window# of journals published in

disciplineISI coverageLanguagePublication typeNot representativeJournal size

Other IssuesOther Issues

Citation errorsCitation clubsCitation bartering

ManipulationManipulation

Review articlesCase studiesEditorial interference

hh-index-index

Hirsch, J. E. (2005). "An index to quantify an individual's scientific research output". PNAS 102 (46): 16569–16572.

hh-index-indexh-index, developed by Jorge

Hirsch

A scientist has index h if h of [their] Np papers have at least h citations each, and the other (Np - h) papers have at most h citations each.

Criticisms of the Criticisms of the hh-index-indexComparisonsAgeInsensitiveContextLimitations of citation databases# of authors

From From hh to to ggGiven a set of articles ranked in

decreasing order of the number of citations that they received, the g-index is the (unique) largest number such that the top g articles received on average at least g citations.

What do the numbers really mean?What do the numbers really mean?

Author inaccuracies

ISI calculation inaccuracies

What does an H-index of 1 really mean?

Citation Game-Playing – How to Citation Game-Playing – How to make it Work for YOU!make it Work for YOU!• Become famous so everyone gives you

authorship on their paper• HEP in ArXve submission timing• Ride the bias – multiple authors: choose the

middle• Folia Phoniatrica et Logopaedica: 1 articles

cited all the papers published in the last 2 years for IF=1.439 compared to 0.655

• Find and ride the trend – Martin Fleischmann was once the most highly cited chemist in Britain

Nature 451 2008 pg 766-767

Search Web of Science for Wilfrid LaurierSearch Web of Science for Wilfrid Laurier

Wilfrid Laurier as Author in 2007?Wilfrid Laurier as Author in 2007?

Wilfrid Laurier as Author in 2007?Wilfrid Laurier as Author in 2007?

Ann Arbor as Author (MUCH worse in Ann Arbor as Author (MUCH worse in Google Scholar)Google Scholar)

Ann Arbor as Author in 2007?Ann Arbor as Author in 2007?

William H. Gates = Bill GatesWilliam H. Gates = Bill Gates

Researcher ID = A-1930-2009Researcher ID = A-1930-2009

Researcher ID Publication ListResearcher ID Publication List

Anyone can write a letterAnyone can write a letter

NATURE Vol 53  June 2008 pg. 718

Chemistry Department Scopus H-IndexChemistry Department Scopus H-IndexCreated Feb 9, 2008 with the faculty at that time. Dependent on information included in SCOPUS.

09-Feb-08

Position h-index patents total number of papers

first-year publication

last-year publication

Publications/year

president 5 0 13 1984 2008 0.54

dean science 15 0 51 1988 2008 2.55

assistant 5 0 14 1999 2007 1.75

assistant 5 0 6 1998 2007 0.67

assistant 15 2 48 1998 2007 5.33

assistant 6 0 13 1998 2007 1.44

assistant 7 0 12 1998 2007 1.33

assistant 9 0 14 2000 2005 2.80

CAS 3 0 4 1999 2002 1.33

CAS 10 33 25 1979 2007 0.89

professor 8 0 36 1967 2007 0.90

associate 7 0 17 1992 2005 1.31

CAS 0 0 2 2007 2008 2.00

CAS 17 0 24 1984 2001 1.41

Average 8 2.50 19.93 1992.21 2006.14 1.73

ISI Accuracy?ISI Accuracy?

Sciencewatch.com Sciencewatch.com

Institutional Metrics Institutional Metrics http://webometrics.infohttp://webometrics.info

What does this mean for What does this mean for Librarians?Librarians?Must be familiar and knowledgeable

about the current calculations of Science Worthiness

International students are aware

Administrations and granting agencies will ask for these statistics but may not be aware of their limitations

Avoid the citation culture

Take Home MessageTake Home Message

Avoid Disregard Syndrome Practice Citation Vigilance