normalization of mendeley reader impact on the reader- and paper-side
TRANSCRIPT
Normalization of Mendeleyreader impact on the reader- andpaper-sideRobin Haunschild Lutz Bornmann
09/16/2016
Introduction
AltmetricsAlternative metrics, closely related to article level metricsFacebook: posts, likes, ...Twitter: tweets, retweets, ...Mendeley, CiteULike, Zotero, ...: readers (reader counts,bookmarks, saves, ...)News outlets: stories, mentionsBlogs: stories, mentions...
Two major fields of altmetrics research1. Meaning of altmetrics counts2. Normalization of altmetrics counts
Mendeley reader impact Robin Haunschild et al. 2
Introduction
MendeleyOnline reference managerDesktop and mobile applicationsSocial, academic networking componentAPI for user statistics
Global web traffic rank from Alexa
Mendeley reader impact Robin Haunschild et al. 3
Introduction
Research Questions1. Is normalization important for Mendeley reader counts?2. Which normalization procedures are possible?3. Can analogous versions to citing-side and cited-side
normalizations be done using Mendeley reader counts?4. How do the methods differ?
Mendeley reader impact Robin Haunschild et al. 4
Data set
Data setDOIs from WoS papers from 2012 (Na = 1,133,224 articlesand Nr = 64,960 reviews)Search via Mendeley API for DOI in December 2014Overall 94.8% of the articles and 96.6% of the reviews werefound on Mendeley9,352,424 Mendeley reader counts for the articles and1,335,764 for the reviews.0.05% of the article readers and 0.04% of the reviewreaders did not share their (sub-)disciplinary information
Mendeley reader impact Robin Haunschild et al. 5
Example data from Mendeley
Data available from MendeleyInput DOI, PubMedID, ... (e.g., 10.1063/1.4769790, Insensitivityof the error of the minimally empirical hybrid functionalrevTPSSh to its parameters):
Total reader count (here: 9)Reader count per academic status (here: 2 Researchers, 1Other, 3 Professors, 2 PhD Students, and 1 AssociateProfessor)Reader count per Mendeley discipline (here: 1 inMaterials Science, 1 in Physics, 5 in Chemistry, 1 in SocialSciences, and 1 in Economics)Reader count per country (here: 1 in USA, 1 in the Vatican,and 1 in South Korea)
Mendeley reader impact Robin Haunschild et al. 6
Mendeley readers per document type
Mendeley readers per article in the Mendeley disciplines
Mendeley reader impact Robin Haunschild et al. 7
Mendeley readers per document type (cont’d)
Mendeley readers per review in the Mendeley disciplines
Mendeley reader impact Robin Haunschild et al. 8
Mendeley readers per document type (cont’d)
Mendeley readers per article in the top WoS subjectcategories
Mendeley reader impact Robin Haunschild et al. 9
Mendeley readers per document type (cont’d)
Mendeley readers per review in the top WoS subjectcategories
Mendeley reader impact Robin Haunschild et al. 10
Cited-side and citing-side methods
Cited-side1. Counting all citations a paper has received2. Normalization of citation counts with respect to the
scientific field of the cited paper
Analogous version for Mendeley reader counts: paper-side
Citing-side1. Counting all citations a paper has received separately for
each scientific field of the citing paper2. Normalization of citation counts with respect to the scientific
field of the citing paper
Analogous version for Mendeley reader counts: reader-side
Mendeley reader impact Robin Haunschild et al. 11
Method
Paper-side NormalizationAverage number of readers per paper (ρ) in a scientific field(here: WoS subject category), document type, and publicationyear:
ρc =1
Nc
Nc∑i=1
Ri (1)
Nc : Number of papers in a WoS subject category, document type,and publication yearRi : Number of reader counts of paper i
NRSi =Ri
ρc(2)
NRSi : Normalized Reader Score for paper iMultiplicative (or fractional or full counting) for papers with WoSsubject categories
Mendeley reader impact Robin Haunschild et al. 12
Method
Paper-side Normalization (cont’d)Average over a set of papers of a specific unit:
MNRS =1N
N∑i=1
NRSi (3)
N: Number of papers in a specific research unit
InterpretationAnalogous to MNCS:
(M)NRS ≈ 1: average reader impact of paper(M)NRS < 1: below average reader impact of paper(M)NRS > 1: above average reader impact of paper
Mendeley reader impact Robin Haunschild et al. 13
Method
Reader-side NormalizationAverage number of readers per paper (ρ) in a scientific field(here: Mendeley discipline), document type, and publicationyear:
ρd =1
Nd
Nd∑i=1
Rid (4)
Nd : Number of papers in a Mendeley discipline, document type, andpublication yearRid : Number of reader counts of paper i in Mendeley discipline d
βid =Rid
ρd(5)
βid : Normalized Reader Score for paper i in Mendeley discipline d
Mendeley reader impact Robin Haunschild et al. 14
Method
Reader-side Normalization (cont’d)Sum over all Mendeley disciplines yields a paper-based readerimpact value:
DNRSi =D∑
d=1
βid (6)
D: Number of Mendeley disciplines where paper i has readers.DNRSi : Discipline Normalized Reader Score of paper iAverage over a set of papers of a specific unit:
MDNRS =1N
N∑i=1
DNRSi (7)
N: Number of papers in a specific research unit
Mendeley reader impact Robin Haunschild et al. 15
Spearman correlation coefficients for journals
MDNRS vs. MNRS for different OECD categories
OECD category rs No. of journalsNatural sciences 0.75 3337Engineering and technology 0.81 1556Medical and health sciences 0.82 2855Agricultural sciences 0.89 385Social sciences 0.83 1920Humanities 0.42 563
Mendeley reader impact Robin Haunschild et al. 16
Additional Information
More detailsDOI: 10.1016/j.joi.2016.04.015
Submission history22 February, 2016: Upload of Manuscript (version 1) toFigshare and submission of revised version to JoI.07 March, 2016: Submission of abstract for this contribution14 March/29 March, 2016: Submission deadline for STIcontributions22 April, 2016: Manuscript accepted by JoI and upload offinal manuscript version to Figshare.30 May, 2016: Formal acceptance of this contribution to thisconference.DOI: 10.1016/j.joi.2016.04.015, published in the August2016 issue of JoI.
Mendeley reader impact Robin Haunschild et al. 17
Summary and Conclusions
Answering the Research QuestionsYes, normalization is important for Mendeley reader counts.Raw reader counts should not be used for impactassessment.Paper-side and reader-side normalizations are possible.The paper-side normalization is the analogue of thecited-side normalization, and the reader-side normalizationis the analogue of the citing-side normalization.The reader-side and paper-side normalization methodsprovide slightly different rankings, e.g. for journals.
Mendeley reader impact Robin Haunschild et al. 18
Summary and Conclusions
Normalization of Mendeley reader countsTwo different methods for normalization of Mendeley readercounts were presented.Both methods correlate larger than expected for mostjournals.
OutlookNormalization with respect to Mendeley disciplines hasbeen done.Is it useful to normalize with respect to:
academic status groups orcountry affiliations
of Mendeley readers?
Mendeley reader impact Robin Haunschild et al. 19
53 23 1653 23 16
Mendeley reader impact Robin Haunschild et al. 20