what is the impact of the publications read by the different mendeley users? could they help to...
TRANSCRIPT
What is the impact of the publications read by the different Mendeley users? Could they help to
identify alternative types of impact?
Zohreh Zahedi, Rodrigo Costas & Paul Wouters
Centre for Science and Technology Studies (CWTS)
ALM Workshop, San Francisco, CA, USA
October 10-12, 2013
Outline
• Introduction
• Objectives
• Research Questions
• Methodology
• Findings
• Conclusions, Discussions & Limitations
2
Introduction
Altmetrics Tools: Mendeley , Impact Story , altmetric.com, PLOS ALM, F1000, Plum Analytics, ….
3
Previous research
Altmetrics & Citation Correlation:
Henning (2010);Priem, Piwowar & Hemminger (2012); Bar-Ilan et. al. (2012a & 2012b); Li, Thelwall & Giustini (2012); Li & Thelwall (2012); Zahedi, Costas & Wouters (2013); Mohammadi & Thelwall (2013); Schlögl et. al. (2013); Haustein et.al. (2013a & 2013b)
Altmetrics & Citation as predictors:
Wardle, 2010
Eysenbach, 2011
Waltman & Costas, 2013
4
Objectives & Research Questions:To distinguish patterns in terms of impact depending on the types of Mendeley users
Q1. What do the different Mendeley users read in terms of document types and Subject fields?
Q2. To what extent do the readerships of the different users in Mendeley correlate with citation indicators?
Q3. What is the impact of publications read by different users in Mendeley?
5
Methodology (1)
Random Samples:
1. 20,000 WOS publications from all disciplines between 2005-2011
2. 200,000 WOS publications from all disciplines between 2011-2012
Metrics: Mendeley & Impact Story APIs
6
Methodology (2)
Collecting altmetrics on the basis of DOIs of the publications
Using Mendeley & Impact Story APIs
Linking and matching with WOS
Adding bibliometric indicators
Analyzing the data
7
Types of Mendeley users
• Professors (Associate, Assistant)
• Lecturers (Senior)
• Postdocs
• Researchers (Academics/non-Academics)
• Students (Bachelor, Master, Postgraduate)
• PhD/Doctoral
• Librarian
• Other professionals
• Unknown
8
Distribution of readerships in the samples by types of users
9
Chart TitleSample 1 Sample 2
34%
25%
17%
10%
7%
4% 3% 1% 0%
33%
28%
13%
10%
7%
5%3%1% 0% PhD
Unknown
Students
PostDocs
Researchers
Professors
Other Pro-fessionals
Lecturer
librarians
Modeling impact by Mendeley users:
Scientific: Professors, PhD, Postdocs, Academic Researchers
Educational: Lecturers, Bachelor, Master & Postgraduate Students
Professional: Librarians, Other Professionals, non Academic Researchers
Unknown: unidentified users
10
Sample 2
Sample 1
53%
14%5%
28%
SCIENTIFIC READERS
EDUCATIONAL READERS
PROFESSIONAL READERS
Unknown
52%
18%
5%
25%
What document type are more read by the different users? (sample 1)
11
Articles Reviews Non Citables Letter0%
5%
10%
15%
20%
25%
30%
35%
40%
PhD
Unknown Professions
Students
PostDoc
Researchers
Professors
Other Professionals
Lecturers
Librarians
Which fields are more read by types of users?(sample 1)
12
MED
ICAL
AND LIF
E SC
IENCES
NATURAL
SCIE
NCES
SOCIA
L AND
BEH
AVIORAL
SCIE
NCES
ENGIN
EERIN
G SCIE
NCES
MULT
IDIS
CIPLINARY
JOURNALS
LANG
UAGE,
INFO
RMATI
ON A
ND C
OMMUNIC
ATION
LAW
, ARTS
AND H
UMAN
ITIE
S
0%5%
10%15%20%25%30%35%40%45%
PHD
Unknown
Students
PostDocs
Researchers
Professors
Other Pro-fessionals
Lecturers
librarians
Which fields are more cited/read per publication?(sample 1)
13
MULT
IDIS
CIPLINARY
JOURNALS
MED
ICAL
AND LIF
E SC
IENCES
NATURAL
SCIE
NCES
ENGIN
EERIN
G SCIE
NCES
SOCIA
L AND
BEH
AVIORAL
SCIE
NCES
LANG
UAGE,
INFO
RMATI
ON A
ND C
OMMUNIC
ATION
LAW
, ARTS
AND H
UMAN
ITIE
S
05
1015202530354045
Readers per Paper (RPP)Citations per Paper (CPP)
Which fields are more cited/read per publication? (sample 2)
14
MULT
IDIS
CIPLINARY
JOURNALS
SOCIA
L AND
BEH
AVIORAL
SCIE
NCES
NATURAL
SCIE
NCES
MED
ICAL
AND LIF
E SC
IENCES
LANG
UAGE,
INFO
RMATI
ON A
ND C
OMMUNIC
ATION
ENGIN
EERIN
G SCIE
NCES
LAW
, ARTS
AND H
UMAN
ITIE
S
0
1
2
3
4
5
6
7
8
Readers per Paper (RPP)Citations per Paper (CPP)
To what extent do the different types of users in Mendeley correlate with citation indicators?
Correlation Readers Unknown PhDs PostDocs Students
Researchers
ProfessorsOther
Professional Librarian Lecturers
Sample 1
Citations
0,52 0,51 0,46 0,43 0,34 0,15 0,09 0,02 -0,01 -0,01
Sample 2 0,35 0,33 0,29 0,24 0,22 0,1 0,03 0,04 -0,01 -0,01
15
CorrelationReaders Unknown Scientific Educational Professional
Sample 1
Citations
0.52 0,51 0,48 0.34 0,05
Sample 20,35 0,33 0,30 0.21 0.07
What are the impact of publications read by different types of readers?
16
Unkno
wn
post
docs
PhD
Stud
ents
Resea
rche
rs
Profe
ssor
s
Other
Profe
ssiona
ls
Lect
urer
s
libra
rians
0%
5%
10%
15%
20%
25%
Sample 1sample2
PP Top 10%
Limitations
• Access only to the top 3 categories of readers in Mendeley
• Data collection (time consuming)
• Speed of use the APIs (API limit)
• Scalability (limitations for the medium-large scale analysis)
• Not perfect data matching with WOS (DOIs, ….)
17
Conclusions & Discussions• Potential advantage of Mendeley over citations:
– for publications from social sciences and humanities
– for recent publications [!]
• Scientific users are more correlated with citations than educational and professional users
• The other users could help to identify other types of impact: educational, professional [?]
• Some users tend to read more highly cited papers than others: Postdoc, PhD Students vs Professors
• Identifying the unknown users can shed some light in detecting these other types of impact
• Further analysis needs to be done to dig into the content of reading by different types of users 18
