Altmetrics –evidence of societal impact of

science or just buzz?

Kim HolmbergRUSE - Research Unit for the Sociology of Education

University of Turku(e) kim.j.holmberg@utu.fi

(w3) http://kimholmberg.fi(w3) https://impact.utu.fi

@kholmber

Altmetricsthe study and use of scholarly

impact measures based on activity in online tools and environments

Priem, J. (2014). Altmetrics. In Cronin, B. & Sugimoto, C.R. (Eds.). Beyond Bibliometrics: Harnessing MultidimensionalIndicators of Scholarly Impact. MIT Press, 2014.

“No one can read everything. We rely on filters to make sense of the scholarly literature, but the narrow, traditional filters are being swamped. However, the growth of new, online scholarly tools allows us to make new filters; these altmetrics reflect the broad, rapid impact of scholarship in this burgeoning ecosystem. We call for more tools and research based on altmetrics.”

Altmetrics: a manifesto - http://altmetrics.org/manifesto/

Search

“…altmetrics presents an alternative to the current practice of relying only on citation counts and journal impact factors for the quantitative analysis of impact by introducing new complementary approaches and sources of data.”

Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level discussion and metrics. Learned Publishing, 26(1).

Altmetrics data providers

Altmetrics data providers

Including but not limited to:• Policy documents published online• Mainstream media (news sites)• Social reference managers (e.g., Mendeley, CiteULike)• Peer-review platforms (e.g., Publons, PubPeer, F1000)• Wikipedia• Blogs (curated list of scientific blogs)• Social media (e.g., Twitter, Facebook, Google+, LinkedIn)• Other sources such as YouTube, Reddit and Pinterest

Advantages with altmetrics

0 2 4 6 8 10 12 14 16 18 20

Time after publication (days or weeks perhaps)

Altm

etric

sAltmetrics areaccumulated

quickly after the publication

0 2 4 6 8 10 12 14 16 18 20

Time after publication (years)

Cita

tions

Time of research assessment

Option 1: Potential future impactOption 2: Past success and impact

Altmetrics are notlimited to scientificarticles and books

Altmetrics can reflect how researchers are engaging with the public

Altmetrics can reflect different types of impact and the context of the impact

Science

Education

Nature

Economy

Culture

Health

Society

Not just scientific impact…

Altmetrics canreflect different

levels of impact orengagement

RESEARCH OBJECT

Haustein, S., Bowman, T.D., Costas, R. (2016). Interpreting “altmetrics”: Viewing acts on social media through the lens of citation and social theories. In Cassidy R. Sugimoto (Ed.). Theories of Informetrics and Scholarly Communication.

Preprint available at: https://arxiv.org/abs/1502.05701

“Altmetrics expand our view of what impact looks like, but also of what’s making the impact.”

Altmetrics: a manifesto - http://altmetrics.org/manifesto/

Challengeswith altmetrics

The meaning and applicability of altmetrics generated on different platforms is still unclear

Lack of data standards leads to issues with transparency, replicability and accuracy

Question: Can we expect some common standards and/or tools for altmetrics to be provided at EU level?

doiDigital Object Identifier

Commonly known tracking issues; Most altmetrics identified from:

Altmetrics are time-dependent and thus comparing altmetrics of documents from different time periods should be avoided

Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014). Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature. Journal of the Association for Information

Sciences and Technology. doi: 10.1002/asi.23101

(Some) Altmetrics only reflect the actions of those that actively use social media

Altmetrics may be skewed towards humorous and curious titles

Bots?Haustein, S., Bowman, T.D., Holmberg, K., Tsou, A., Sugimoto, C.R. & Lariviére, V. (2015). Tweets as impact indicators: Examining the implications of

automated “bot” accounts on Twitter. JASIST. arXiv version available at: http://arxiv.org/ftp/arxiv/papers/1410/1410.4139.pdf.

Altmetrics are easily manipulated

Holmberg, K. & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics, vol. 101, no. 2, pp. 1027-1042. DOI:10.1007/s11192-014-1229-3.

Disciplinary differences in Twitter scholarly communication

Need for normalization

1) Mohammadi, E., Thelwall, M. & Kousha, K. (2016). Can Mendeley bookmarks reflect readership? A survey of user motivations. Journal of the Association for Information Science and Technology, 67(5), 1198-1209. doi:10.1002/asi.23477

2) All altmetric research on Twitter

Mainly academics

Wider audience

1)

2)

- Mendeley readership and tweets reflect different things- They cannot be compared with each other without normalization

Goodhart’s Law: When a measure

becomes the target, it ceases to be a

good measure

Goodhart, 1975

Altmetrics in research

evaluation

Measure of attention,

engagementor influence

https://fimpact.utu.fi(under development, but not for long anymore)

Research project financed by The Finnish Ministry of Education and Culture’s Open Science and Research Initiative.

Stories behind the numbers

Mapping interactions, contexts and networks

“… instead of emulating the indicators used for scientific impact --based on outputs or citation approaches--, societal impact assessment should develop a different methodological approach --based on interaction approaches.”

Robinson-Garcia, van Leeuwen, & Ràfols (2017). Using altmetrics for contextualised mapping of societal impact: from hits to networks. SSRN. Available at:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2932944

Holmberg, K., Bowman, T.D., Haustein, S., & Peters, I. (2014). Astrophysicists’ Conversational Connections on Twitter. PLoS ONE, vol. 9, no. 8: e106086. DOI:10.1371/journal.pone.0106086. Available at: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0106086

Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I. (2014). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Journal of Information Management, vol. 66, no. 3, pp. 279-296. DOI:10.1108/AJIM-09-2013-0081. Availableat: http://www.emeraldinsight.com/doi/abs/10.1108/AJIM-09-2013-0081.

Astrophysicists on Twitter – resultsfrom two studies

Communities of attention and conversational networks

Communities of attention and conversational networks

Publications and tweets per day: -0,339*Citation rate and tweet per day: -0,457**

Astrophysicists on Twitter

Measure of societal impacton different audiences?

Usage and demographics of social media platforms. Pew Research, http://www.pewinternet.org/2016/11/11/social-media-update-2016/

Altmetrics –evidence of societal

impact of science or just buzz?

Tack för eruppmärksamhet!

Kim HolmbergRUSE - Research Unit for the Sociology of Education

University of Turku(e) kim.j.holmberg@utu.fi

(w3) http://kimholmberg.fi(w3) https://impact.utu.fi

@kholmber