Horizon 2020 Policy Support

Facility

Different types of altmetrics (Part 1/2) 07 / 04 / 2017

Kim Holmberg

kim.j.holmberg@utu.fi

http://kimholmberg.fi/

Page 2 Horizon 2020 Policy Support Facility

Content

Part 1 / 2

Key points from the kick-off meeting and the

questionnaire

Objectives of the working meeting

Background information (what are altmetrics)

Part 2 / 2

What is impact?

Benefits and challenges of different types of altmetrics

Altmetrics data aggregators

Page 3 Horizon 2020 Policy Support Facility

Key points from the kick-off meeting

Great diversity in the approaches to and the degree of

adoption of open science exist between the member

states.

It is also clear that although there are some ongoing

discussions and a keen desire to learn more about

altmetrics, none of the member states are using

altmetrics for research evaluation and only few

examples of other types of use were mentioned.

Page 4 Horizon 2020 Policy Support Facility

Key points from the questionnaire (1/5)

Question 1: Are you aware of ways in which altmetrics

are being used to assess research in your country? At

governmental/institutional/individual level?

Altmetrics are not used for research evaluation at any

level in the member states. However, member states are

aware of altmetrics and some examples of use of them

do exist.

Page 5 Horizon 2020 Policy Support Facility

Key points from the questionnaire (2/5)

Question 2: Are you aware of any other ways in which

altmetrics are being used in your country?

Besides from some national journals and national or

institutional repositories altmetrics are not widely used

in the member states for other purposes either.

In some of the member states researchers themselves

add altmetrics to their CVs and homepages.

Page 6 Horizon 2020 Policy Support Facility

Key points from the questionnaire (3/5)

Question 3: What kind of change do you wish to see in

your country in regard to open science and altmetrics?

Learn about what altmetrics could do to present the

impact/attention (both scientific and societal) of locally

relevant research published in a national language

Clear national policies and operational action plans to

realize open science principles in research and education

Development of a broader and more appropriate set of

indicators and article level metrics

A cultural change towards open science

Page 7 Horizon 2020 Policy Support Facility

Key points from the questionnaire (4/5)

Question 4: What are your specific expectations to get

out of this MLE in regards to altmetrics?

Learn about what altmetrics can be used for and what

they should not be used for and with that

Learn about open science policies and regulations and

about the experiences from different countries

Learn about approaches to ensure open research data,

and exchange and re-use of it

Learn about good practice examples of open science

training and education programs

Learn about successful reward and evaluation systems

Page 8 Horizon 2020 Policy Support Facility

Key points from the questionnaire (5/5)

Question 5: Any failures or success stories regarding

open science initiatives and/or (alt)metrics that you

wish to share?

The success stories mentioned revolve around the

promotion of open science via institutional Research

Information Systems (CRIS)

In some cases some altmetrics have been displayed at

article level (in for instance repositories or researchers’

profiles)

No failures were mentioned (with the exception of a

conservative attitude towards more traditional metrics)

Page 9 Horizon 2020 Policy Support Facility

Objectives of the working meeting

The objectives for the 1st Working Meeting include:

1. Discussion about what impact means for different

stakeholders and how different types of impact could

and should be responsibly measured.

2. Learn about altmetrics in general and more

specifically about the possibilities and challenges with

using them for research evaluation

Page 10

Background information A (very) short introduction to altmetrics (”alternative metrics”

Altmetrics the study and use of scholarly

impact measures based on

activity in online tools and

environments

Priem, J. (2014). Altmetrics. In Cronin, B. & Sugimoto,

C.R. (Eds.). Beyond Bibliometrics: Harnessing

Multidimensional Indicators of Scholarly Impact. MIT

Press, 2014.

“…altmetrics presents an

alternative to the current

practice of relying only on

citation counts and

journal impact factors for

the quantitative analysis

of impact by introducing

new complementary

approaches and

sources of data.”

Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level

discussion and metrics. Learned Publishing, 26(1).

“No one can read everything. We rely on

filters to make sense of the scholarly

literature, but the narrow, traditional filters

are being swamped. However, the growth of

new, online scholarly tools allows us to

make new filters; these altmetrics reflect the

broad, rapid impact of scholarship in this

burgeoning ecosystem. We call for more

tools and research based on altmetrics.”

Altmetrics: a manifesto - http://altmetrics.org/manifesto/

Search

Altmetrics data providers

Page 17 Horizon 2020 Policy Support Facility

Altmetrics data providers

Including but not limited to:

Policy documents published online

Mainstream media (news sites)

Social reference managers (e.g., Mendeley, CiteULike)

Peer-review platforms (e.g., Publons, PubPeer, F1000)

Wikipedia

Blogs (curated list of scientific blogs)

Social media (e.g., Twitter, Facebook, Google+, LinkedIn)

Other sources such as YouTube, Reddit and Pinterest

Page 18 Horizon 2020 Policy Support Facility

Associations between altmetrics and citations

“Statistically significant associations were found

between higher metric scores and higher citations…

(Twitter, Facebook wall posts, research highlights, blogs,

mainstream media and forums)...”

“…the coverage of all the altmetrics except for Twitter

seems to be low and so it is not clear if they are

prevalent enough to be useful in practice.”

Thelwall M, Haustein S, Larivière V, Sugimoto CR (2013) Do Altmetrics

Work? Twitter and Ten Other Social Web Services. PLoS ONE 8(5): e64841.

doi:10.1371/journal.pone.0064841

Altmetrics are not alternatives to citations but rather complements

Measure of

attention,

engagement

or influence

Altmetric.com 2014 Top 10 scientific articles (Source: http://www.altmetric.com/top100/2014/)

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

0

50

100

150

200

250

300

total_posts_blog

total_posts_policy

total_posts_fbwall

total_posts_wikipedia

total_posts_gplus

total_posts_linkedin

total_posts_tweet

Engineering, Altmetric (2016): Altmetric Top 50 2016. figshare. https://doi.org/10.6084/m9.figshare.c.3590951.v2. Retrieved: Apr 04, 2017

Stories behind the numbers?

Measure of societal impact

on different audiences (?)

Usage and demographics of social media platforms. Pew Research, http://www.pewinternet.org/2016/11/11/social-

media-update-2016/

Mapping interactions,

contexts and networks

“… instead of emulating the indicators used for scientific impact --based on outputs or citation approaches--, societal impact assessment should develop a different methodological approach --based on interaction approaches.”

Robinson-Garcia, van Leeuwen, & Ràfols (2017). Using altmetrics for contextualised mapping of societal impact: from hits to networks. SSRN. Available at:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2932944

Holmberg, K., Bowman, T.D., Haustein, S., & Peters, I. (2014). Astrophysicists’ Conversational Connections on Twitter. PLoS ONE,

vol. 9, no. 8: e106086. DOI:10.1371/journal.pone.0106086. Available

at: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0106086

Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I. (2014). Astrophysicists on Twitter: An in-depth analysis of

tweeting and scientific publication behavior. Aslib Journal of Information Management, vol. 66, no. 3, pp. 279-296. DOI:10.1108/AJIM-

09-2013-0081. Available at: http://www.emeraldinsight.com/doi/abs/10.1108/AJIM-09-2013-0081.

Astrophysicists

on Twitter –

results from two

studies

Communities of attention and

conversational networks

Communities of attention and

conversational networks

Publications and tweets per day: -0,339*

Citation rate and tweet per day: -0,457**

Astrophysicists on Twitter

Altmetrics can be… … a measure of attention,

engagement or influence

… a measure of societal impact

on different audiences

… used to map interactions,

contexts and networks

… used to tell the narratives

behind the numbers

But…

More research is needed!