Gaining Insights Through Bibliometric Analysis

Elaine M. Lasda BergmanPresentation for NCAR/UCARDecember 21, 2015

Bibliometrics: Context and Background Term coined by Pritchard (1969)

Why Bibliometric Analysis? Factual evidence that scholarly output was useful/utilized by

others (with caveats)

Objective, not subjective (vs. surveys, reputation)

Popular methodology for LIS research

“Ingredients” in a Bibliometric Study• Research question with a possible bibliometric solution

• underrepresented/emerging/interdisciplinary fields• institutional metrics (professional associations, countries, universities, etc.)

• Body of research from which to draw citations• groups of scholars, specific journal titles, countries, universities, etc.• date range• cited or citing references?

• Analysis• “core collections” /Bradford’s Law• subject scatter• database indexing/overlap

• Insights gained

Article 1: Social GerontologyResearch questions:

(1)To what extent do social gerontology researchers rely on social science literature vs. research from other disciplines? (i.e., WHAT SHOULD I BE BUYING FOR THESE SCHOLARS?!)

(2)What are the best databases to search for social gerontology research?

(3)To what degree do these findings demonstrate whether Social Gerontology is “integrative” in nature, and to what extent does social gerontology have its own “territory” [Winter, 1991]

(4) Is Social Gerontology a discipline in its own right?

Social Gerontology• Corpus:

• randomized sample of cited refs in 3 key social gerontology journals 2005-2009

• Database: • Scopus (found mistakes - cited references in Age and Ageing!)

• Metadata added:• OCLC subject headings, Ulrich’s database indexing information

• Analysis:• subject scatter, database indexing, core titles in field

Social Gerontology(1)Social gerontology researchers rely roughly upon half social science, half medical

literature in their research.

(2)The best databases for social gerontologists to search are broad multidisciplinary (Scopus, Web of Science, PubMed) and not subject specific databases like Ageline or Abstracts in Social Gerontology.

(3)Social gerontology is clearly integrative but there is clearly a small body of periodicals that is unique in territorial scope to social gerontology.

(4)Is social gerontology its own distinct discipline? (maybe)

Article 2: Social WelfareResearch Questions:

(1)Which is the “best” resource for finding citing references to Social Work literature? Scopus, Web of Science, or Google Scholar?

(2)How do the three databases differ in terms of general patterns of coverage; for example: database overlap, languages covered, document types covered, etc.

(3)When is each resource most (or least) appropriate?

Social Welfare• Corpus:

• Citing references to all 2005 articles appearing in the top 5 social work journals from a “reputation approach” article [Sellers, et al. , 2004]

• Databases:• Web of Science, Scopus, and Google Scholar

• Metadata added: • document type, format, language

• Analysis:• overlap (and lack thereof) between WoS, GS and Scopus, patterns of citing

reference coverage as a whole and by journal, variation in document types, language distribution

Social Welfare(1)What constitutes “the best” depends on what is counted as a citing reference, and

what question is being asked of the data.

(2)Google Scholar did not return many more unique citations from peer reviewed scholarly sources than were returned by proprietary databases.

(3)Other scholarly formats could be considered meaningful indications of research productivity: conf proceedings, dissertations and theses, etc.

(4)For the most accurate measurement of impact, all three databases should be utilized.

OA vs Paywall: an Impact Anecdote

Gerontology article:

cited 1 time in Scopus

not indexed in WoS

cited 1 time in GS______________________174 postprint downloads

from UAlbany’s IR

Social Welfare article:

cited 5 times in Scopus

cited 5 times in WoS

cited 17 times in GS_______________________

4 postprint downloads from UAlbany’s IR

So What? Who Cares? • Bibliometrics continue to provide insights to researchers in LIS and other

disciplines [Harzing, 2015]

• Bibliometric methodologies are easier than ever to execute

• Dirty data is getting harder to find

• Expanded variety of impact metrics may lead to additional questions/investigation instead of quick answers

• Understand. Educate. Interpret. Proceed with Caution.

ReferencesHarzing, A.-W., & Alakangas, S. (2015). Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary

comparison.Scientometrics, 1–18. http://doi.org/10.1007/s11192-015-1798-9

Lasda Bergman, E. M. (2011). Social Gerontology—Integrative and Territorial Aspects: A Citation Analysis of Subject Scatter and Database

Coverage. Behavioral & Social Sciences Librarian, 30(3), 154–175. http://doi.org/10.1080/01639269.2011.592803

Lasda Bergman, E. M. (2012). Finding Citations to Social Work Literature: The Relative Benefits of Using Web of Science, Scopus, or Google

Scholar. The Journal of Academic Librarianship,38(6), 370–379. http://doi.org/10.1016/j.acalib.2012.08.002

Winter, M. F. (1991). Specialization and interdisciplinary growth in the social sciences. Behavioral and Social Sciences Librarian, 10(2), 1–7.

Pritchard, A. (1969). Statistical bibliography or bibliometrics. Journal of Documentation, 25, 348-9.