The Use & The Use & Abuse of Abuse of

Citation DataCitation Data

Sophie BurySophie BuryPeter F. Bronfman Peter F. Bronfman Business LibraryBusiness Library

&&

Ilo MaimetsIlo MaimetsSteacie Science & Steacie Science &

Engineering LibraryEngineering Library

What is Citation Data?What is Citation Data?

Generated by counting numbers of Generated by counting numbers of times publications have been citedtimes publications have been cited

Used to generate widely used impact Used to generate widely used impact factors e.g. journal impact factorfactors e.g. journal impact factor

Another name for this field: Another name for this field: BibliometricsBibliometrics First proposed by Eugene Garfield in First proposed by Eugene Garfield in

19551955 Garfield founded Institute for Scientific Garfield founded Institute for Scientific

Information (ISI)Information (ISI)

3 Kinds of Citation Data3 Kinds of Citation Data

ArticlesArticles Citation ImpactCitation Impact

AuthorsAuthors E.g. hE.g. h-index-index

JournalsJournals Journal Impact FactorJournal Impact Factor

Kinds of Citation DataKinds of Citation Data

Citation ImpactCitation ImpactTimes Cited Times Cited

(Web of Science (Web of Science Example)Example)

Kinds of Citation DataKinds of Citation Data

Author ImpactAuthor ImpactH-indexH-index

““An index to quantify an An index to quantify an individual’s scientific individual’s scientific research output”research output”

hh-index: -index: a scholar with an index of h a scholar with an index of h has published h papers with has published h papers with at least h citations eachat least h citations each

Proposed by Jorge E. Proposed by Jorge E. Hirsch in 2005, Physics Hirsch in 2005, Physics Professor, University of Professor, University of California at San DiegoCalifornia at San Diego

Kinds of Citation DataKinds of Citation Data

Journal Impact FactorJournal Impact Factor

Measuring Journal Measuring Journal QualityQuality

Definition: Definition: The average The average number of times number of times published papers are published papers are cited in the two calendar cited in the two calendar years following years following publication publication

The Uses of Citation DataThe Uses of Citation Data

By Researchers:By Researchers: To look for research which To look for research which

up-dates earlier findingsup-dates earlier findings Identify related articles on Identify related articles on

same topicsame topic To look for replications of To look for replications of

empirical researchempirical research To look for who is citing ones To look for who is citing ones

own researchown research To follow a scholarly debate To follow a scholarly debate

through time through time Trace the development of an Trace the development of an

accepted theory over timeaccepted theory over time To identify “significant” To identify “significant”

publications or authors in a publications or authors in a disciplinediscipline

The Uses of Citation DataThe Uses of Citation Data

Administrative Purposes:Administrative Purposes:

Hiring, tenure & promotionHiring, tenure & promotion

Funding / grant proposalsFunding / grant proposals

Augment Institutional Augment Institutional prestige and ranking by prestige and ranking by emphasizing research in emphasizing research in high impact fieldshigh impact fields

The Uses of Citation DataThe Uses of Citation Data

Government funding Government funding agenciesagencies

Publishers / EditorsPublishers / Editors

Libraries use impact Libraries use impact factors to make factors to make collection decisionscollection decisions

Potential Pitfalls Potential Pitfalls Problems with the finite window (2 years)

Areas where ideas turn over quickly are favoured because they publish more

More articles published per issue and more issues published per year will also increase the JIF

Journals that specialize in publishing reviews are cited more frequently

Authors that publish reviews are cited more frequently

Manipulation of RankingsManipulation of Rankings

Self-citation Journals publish more

review articles to increase IF.

Some journals encourage citation from same journal

Resulting EffectsResulting Effects

““The rich get richer and poor get The rich get richer and poor get poorer”poorer”

A centrifugal effectA centrifugal effect High IF rankings in journals are High IF rankings in journals are

self-perpetuatingself-perpetuating Hard for new journals to break Hard for new journals to break

into high rankingsinto high rankings Anglo-American biasAnglo-American bias Non-English language citations Non-English language citations

undercountedundercounted

What Does It Really Measure?What Does It Really Measure? Citation does not Citation does not

necessarily equal value or necessarily equal value or even usage of an article even usage of an article e.g. Professional application: e.g. Professional application:

clinical article being used to clinical article being used to “improve diagnoses and save “improve diagnoses and save lives”lives”

Recent study proves that Recent study proves that many references are not many references are not actually read by authors – actually read by authors – reference lists are simply reference lists are simply copied.copied.

Sources of Citation DataSources of Citation Data

ISI ProductsISI ProductsWeb of Science (ISI)Web of Science (ISI)

ScopusScopus

Google ScholarGoogle Scholar

Harzing’s Publish Harzing’s Publish or Perishor Perish

Databases Compared:Databases Compared:Scope/ComprehensivenessScope/Comprehensiveness

Subjects CoveredSubjects Covered

Number of Sources CoveredNumber of Sources Covered

Years CoveredYears Covered

Types of Sources CoveredTypes of Sources Covered

Geographic CoverageGeographic Coverage

Language CoverageLanguage Coverage

Web of Science - IntroductionWeb of Science - Introduction

From Thomson ScientificFrom Thomson Scientific Pioneer in fieldPioneer in field Includes: Includes:

Science Citation Index (1900-present)Science Citation Index (1900-present) Social Sciences Citation Index (1956-present)Social Sciences Citation Index (1956-present) Arts & Humanities Citation Index (1975-Arts & Humanities Citation Index (1975-

present)present) Can search across all or limit to one or Can search across all or limit to one or

moremore

Web of Science - ScopeWeb of Science - Scope Best source where historic Best source where historic

coverage is importantcoverage is important Citation data available from Citation data available from

1945 onwards for Science, 1945 onwards for Science, 1956 for Social Sciences and 1956 for Social Sciences and 1975 for Arts and 1975 for Arts and HumanitiesHumanities

Limited to citations from Limited to citations from some 9,300 scholarly some 9,300 scholarly journalsjournals

No citations from books and No citations from books and few conference proceedingsfew conference proceedings

Covers predominantly North Covers predominantly North American & English American & English language titleslanguage titles

Offers coverage in all of the Offers coverage in all of the academic disciplines academic disciplines although is strongest in although is strongest in ScienceScience

Source: Jasco, P. 2005Source: Jasco, P. 2005

Scopus - IntroductionScopus - Introduction

Launched Nov. 2004Launched Nov. 2004 Elsevier productElsevier product Includes its own web search engine ScirusIncludes its own web search engine Scirus Draws on Medline, Embase, Compendex, Draws on Medline, Embase, Compendex,

World Textile Index, Fluidex, Geobase, World Textile Index, Fluidex, Geobase, Biobase Biobase Uses all of their controlled vocabularies Uses all of their controlled vocabularies

making searching more powerfulmaking searching more powerful

Scopus - ScopeScopus - Scope Best source where Best source where

international coverage is international coverage is important important

Citation data available from Citation data available from 1996 onwards (bibliographic 1996 onwards (bibliographic data back to 1966)data back to 1966)

Covers over 15,000 titles Covers over 15,000 titles total total

Broader range of types of Broader range of types of publications than WoSpublications than WoS

Covers more serialsCovers more serials Very heavy weighting toward Very heavy weighting toward

coverage of Biomedical coverage of Biomedical literature, though some literature, though some social sciences and physical social sciences and physical sciences coveragesciences coverage

Source: SCOPUS Help File 2008Source: SCOPUS Help File 2008

Scopus - scopeScopus - scope

Source: SCOPUS Help File 2008Source: SCOPUS Help File 2008

Covers mainly Covers mainly Western European, Western European, North American, & North American, & English language English language titlestitles

Google Scholar - ScopeGoogle Scholar - Scope Unknown number of records, unknown Unknown number of records, unknown

number of sources, unknown year coveragenumber of sources, unknown year coverage

Various studies conducted; some general Various studies conducted; some general conclusions are possible:conclusions are possible: Over 30 different document types – especially Over 30 different document types – especially

noted for much better coverage of conference noted for much better coverage of conference proceedings and grey literatureproceedings and grey literature

Coverage of social sciences, arts and humanities Coverage of social sciences, arts and humanities generally better than WoS and Scopusgenerally better than WoS and Scopus

Coverage of international, non-English journals Coverage of international, non-English journals better than WoS and Scopus better than WoS and Scopus

Sources: Meho and Yang, 2007Sources: Meho and Yang, 2007

Brief DemonstrationBrief DemonstrationSources of Citation DataSources of Citation Data

Web of Science Web of Science

&&

SCOPUSSCOPUS

Google ScholarGoogle Scholar Citations are not of the same quality as in Citations are not of the same quality as in

Wos and SCOPUS because there is no Wos and SCOPUS because there is no selection or indexing by subject specialistsselection or indexing by subject specialists

Search functionality is very basic and Search functionality is very basic and limited:limited: Not possible to limit to scholarly high impact Not possible to limit to scholarly high impact

journals journals No reliable way of searching or limiting searchesNo reliable way of searching or limiting searches Can sort by « recent articles » but this does not Can sort by « recent articles » but this does not

result in most recent publicationsresult in most recent publications Significant improvement in results display Significant improvement in results display

neededneeded

Introducing Harzing’s Introducing Harzing’s Publish or PerishPublish or Perish

Like a metasearch gatewayLike a metasearch gateway

Draws on citation data in Google ScholarDraws on citation data in Google Scholar

Freely available for download on the web:Freely available for download on the web:http://www.harzing.com/pop.htmhttp://www.harzing.com/pop.htm

Attractive interface featuresAttractive interface features

So which citation source So which citation source should I use and when?should I use and when?

No one or best source (Meho and Yang, No one or best source (Meho and Yang, 2007, Bakkalbasi et al., 2006, Gavel and 2007, Bakkalbasi et al., 2006, Gavel and Iselid, 2008)Iselid, 2008) Overlaps vary with discipline, date range, type Overlaps vary with discipline, date range, type

of publication, place of publication, etc. of publication, place of publication, etc. Depends on the kind of analysis and Depends on the kind of analysis and

searching you need to do searching you need to do

Source: Gavel & Iselid, 2008Source: Gavel & Iselid, 2008

Based on a study done in 2008 in Sweden