alt metric s - wur...2016/03/14  · let’s vote altmetrics is meant 1. to filter information 2. to...

30
ALT METRIC S March 2016 Ellen Fest, Hugo Besemer

Upload: others

Post on 28-Jun-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

ALT METRIC

S March 2016

Ellen Fest, Hugo Besemer

Page 2: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Let’s vote

Altmetrics is meant

1. to filter information

2. to assess impact

Altmetrics measures the impact of publications that are not covered

by Web of Science or Scopus

1. yes

2. no

Altmetrics is not about citations

1. yes

2. no

Page 3: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

How it started

2010

“Altmetrics manifesto”

www.altmetrics.org

What they wanted to address

1. Peer review is slow (and

everything gets published

somewhere eventually)

2. Citation counts (and related

measures like H-index) are

slower)

3. Journal Impact Factor is

incorrectly used to assess the

impact of individual articles

Page 4: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Six years later (1)

Page 5: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Six years later (2)

Page 6: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Six years later (3)

Page 7: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Six years later (4)

Page 8: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

But there are limitations

Page 9: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

And more limitations

Page 10: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Some publishers provide their own metrics

PLOS

● Article Level Metrics

Elsevier

● researcher dashboard

BMC

Frontiers in

Page 12: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Plum analytics: institutional metrics

dashboard

http://plumanalytics.com/learn/about-metrics/

Page 13: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Personal profiles: Impactstory (paid)

Page 14: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Personal profiles: Kudos (1)

Page 15: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Personal profiles: Kudos (2)

Page 16: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Some publishers provide authors with

metrics (1)

Page 17: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Some publishers provide authors with

metrics (2)

Page 18: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Some publishers provide authors with

metrics (3)

Page 19: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

What meaning can be read into these

numbers?

They may seen as a proxy for

● Scientific impact

● “Societal impact”

● Buzz

Haustein, S. (2014). Social media in scholarly communication, 42. Retrieved from

http://www.cirst.uqam.ca/Portals/0/docs/Conference/PPT/StefanieHaustein_PPT.pdf

Page 20: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Intermezzo: citation metrics depends on

subject area

Scientist Zacharias Math has a publication from 2003 with 17 citations

Scientist Molecula Biology has a publication from 2009 with 24 citations

Page 21: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Baselines for Mathematics

Page 22: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Baselines for Molecular Biology

0

100

200

300

400

0 2 4 6 8 10 12

Years after publication

Cu

mu

lati

ve

no

. c

ita

tio

ns Baseline

top 10%

top 1%

Page 23: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

How do Altmetrics baselines compare to

traditional baselines - traditional

Page 24: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

How do Altmetics baselines compare to

traditional baselines – “Alt”

Page 25: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Tweets vs traditional metrics

“It is concluded that the scientific citation process acts relatively independently of the social dynamics on

Twitter.” [1]

“A moderate negative correlation (ρ=-0.390*) is found between the number of publications and tweets

per day, while retweet and citation rates do not correlate.” [2]

“automated Twitter accounts create a considerable amount of tweets to scientific papers and that they

behave differently than common social bots, which has critical implications for the use of raw tweet

counts in research evaluation and assessment” [3]

“The results showed that approximately 76% of the sampled accounts were

maintained by individuals (rather than organizations), 67% of these accounts were maintained by a

single man, and 34.4% of the individuals were identified as possessing a Ph.D, suggesting that the

population of Twitter users who tweet links to academic articles does not reflect the demographics of the

general public” [8]

Page 26: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Mendeley vs citation metrics and reads

“The overall correlation between Mendeley readership counts and citations for the social sciences was higher than for the humanities.” [4]

“…..Mendeley readership can reflect usage similar to traditional citation impact, if the data is restricted to readers who are also authors” [5]

“….it is reasonable to use Mendeley bookmarking counts as an indication of readership because most (55%) users with a Mendeley library had read or intended to read at least half of their bookmarked publications"

Page 27: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

Blogs vs citation metrics

“….articles receiving blog citations close to their publication time receive more journal citation later…..” “….7 out of 12 journals (58%) in 2009 and 13 out of 19 journals (68%) in 2010.” [7]

Page 29: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

References (1)

[1] J. C. F. de Winter, “The relationship between tweets, citations, and article views for PLOS ONE articles,” Scientometrics, vol.

102, no. 2, pp. 1773–1779, 2014. http://dx.doi.org/10.1007/s11192-014-1445-x

[2] S. Haustein, T. D. Bowman, K. Holmberg, I. Peters, V. Lariviere, and V. Larivière, “Astrophysicists on Twitter: An in-depth

analysis of tweeting and scientific publication behavior,” Aslib J. Inf. Manag., vol. 66, no. 3, pp. 279–296, 2014.

http://www.emeraldinsight.com/10.1108/AJIM-09-2013-0081\nhttp://www.emeraldinsight.com.globalproxy.cvt.dk/journals.htm?issn=2050-

3806&volume=66&issue=3&articleid=17112729&show=html

[3] S. Haustein, T. D. Bowman, K. Holmberg, A. Tsou, C. R. Sugimoto, and V. Larivière, “Tweets as impact indicators: Examining

the implications of automated ‘bot’ accounts on Twitter,” J. Assoc. Inf. Sci. Technol., p. n/a–n/a, 2015.

http://doi.wiley.com/10.1002/asi.23456

[4] E. Mohammadi and M. Thelwall, “Mendeley readership altmetrics for the social sciences and humanities: Research evaluation

and knowledge flows,” J. Assoc. Inf. Sci. Technol., vol. 65, no. 8, pp. 1627–1638, 2014. http://dx.di.org/10.1002/asi.23071

Page 30: ALT METRIC S - WUR...2016/03/14  · Let’s vote Altmetrics is meant 1. to filter information 2. to assess impact Altmetrics measures the impact of publications that are not covered

References (2)

[5] E. Mohammadi, M. Thelwall, S. Haustein, and V. Larivière, “Who reads research articles? An altmetrics analysis of Mendeley

user categories,” J. Assoc. Inf. Sci. Technol., vol. 66, no. 9, pp. 1832–1846, 2015. http://dx.doi.org/10.1002/asi.23286

[6] E. Mohammadi, M. Thelwall, K. Kousha “Can Mendeley Bookmarks Reflect Readership ? A Survey of User Literature review

Changes in scholarly reading habits in the digital era,” 2014.

http://www.scit.wlv.ac.uk/~cm1993/papers/CanMendeleyBookmarksReflectReadershipSurvey_preprint.pdf

[7] H. Shema, J. Bar-Ilan, and M. Thelwall, “Do blog citations correlate with a higher number of future citations? Research blogs

as a potential source for alternative metrics,” J. Assoc. Inf. Sci. Technol., vol. 65, no. 5, pp. 1018–1027, 2014.

http://dx.doi.org/10.1002/asi.23037

[8] A. Tsou, T. Bowman, A. Ghazinejad, and C. Sugimoto, “Who Tweets about Science ?,” Issi2015, no. 1, pp. 95–100, 2015.

http://www.issi2015.org/files/downloads/all-papers/0095.pdf