altmetrics - measuring the impact of scientific activities
TRANSCRIPT
Background image: http://commons.wikimedia.org/wiki/File:Water_drop_impact_on_a_water-surface_-_(5).jpg
Altmetrics Measuring the impact of scientific activities
Kim HolmbergResearch Unit for the Sociology of Education
University of Turku
(w3) http://kimholmberg.fi
@kholmber
Oulu, 8.5.2015
Bibliometricsthe study of quantitative aspects of the production, dissemination, and use of recorded information
Tague-Sufcliffe, 1992
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
201420132012201120102009200820072006200520042003200220012000199919981997199619951994
The increase of publications about bibliometrics as indexed by Web of Science between 1994 and 2014 (data retrieved from Web of Science on 9 February, 2015).
Goodhart’s Law: When a measure becomes the target, it ceases to be a good measure
Goodhart, 1975
Outreach and Impact Leader, - Ref:1455401
“ We are seeking to recruit … to support, develop and promote outreach and impact activities …Our aim is to maximize the widest possible impact of the research programme in whichever way is most appropriate … The salary range will be £41,430 - £48,873 per annum...”
Taking impact seriously
Problems with citation-based indicators
Problem 1: Time
0 2 4 6 8 10 12 14 16 18 20
Impact factor window
Immediacy index window
Cited half-life
50% ofcitations
50% ofcitations
Time after publication (Years)
Cit
atio
ns
0 2 4 6 8 10 12 14 16 18 20
Time after publication (Years)
Cit
atio
ns
Time of research assessment
Option 1: Journal impact factor, Julkaisufoorumi, …
Potentialfutureimpact
0 2 4 6 8 10 12 14 16 18 20
Time after publication (Years)
Cit
atio
ns
Time of research assessment
Option 2: Citations (after n years)
Pastsuccess and
impact
0 2 4 6 8 10 12 14 16 18 20
Time after publication (Years)
Cit
atio
ns
How can we measureimpact in realtime?
Problem 2: Journals
1
vs.
Fenner, M. & Lin, J. (2013). Altmetrics in Evolution: Defining & Redefining the Ontology of Article-Level Metrics. http://dx.doi.org/10.3789/isqv25no2.2013.04
2
Problem 3: Impact
Science
IMPACT ON:
Science
Society
Environment
Culture
Economy
Education
IMPACT ON:
SOLUTION?
Altmetricsthe study and use of scholarly impact measures based on activity in online tools and environments
Priem, J. (2014). Altmetrics. In Cronin, B. & Sugimoto, C.R. (Eds.). Beyond Bibliometrics: Harnessing MultidimensionalIndicators of Scholarly Impact. MIT Press, 2014.
Created byresearchers
Created bythe public
Altmetrics
Created byresearchers
Indicatingfuture
scientificimpact?
Altmetrics
Indicatingother types of impact, such
as societalimpact?
Created bythe public
Altmetrics
Researchers
The public
Rank Title Altmetric score News Blogs Tweets
1 Experimental evidence of massive-scale emotional
contagion through social networks
5044 301 130 3801
2 Variation in Melanism and Female Preference in Proximate
but Ecologically Distinct Environments
4823 4 11 6669
3 Artificial sweeteners induce glucose intolerance by
altering the gut microbiota
3500 127 34 3303
4 Stimulus-triggered fate conversion of somatic cells into
pluripotency
2986 159 92 3295
5 Dogs are sensitive to small variations of the Earth's
magnetic field
2956 54 33 2668
6 Christmas 2013: Research: The survival time of chocolates
on hospital wards: covert observational study
2734 3 4 3084
7 Epidemiological modeling of online social network
dynamics
2392 156 19 1649
8 Searching the Internet for evidence of time travelers 2246 34 22 2262
9 Conscious Brain-to-Brain Communication in Humans Using
Non-Invasive Technologies
2160 82 32 1660
10 Were James Bond's drinks shaken because of alcohol
induced tremor?
2146 127 14 1559
Top 10 articles according to the Altmetric Score (Source: http://www.altmetric.com/top100/2014/)
0
50
100
150
200
250
300
350
0
1000
2000
3000
4000
5000
6000
7000
8000
1 2 3 4 5 6 7 8 9 10
Altmetric score
Tweets
News
Blogs
The volume of tweets, blog posts, and news stories for the top ten research articles from 2014 as measured with altmetrics, and their Altmetric Score (number of news stories and blog posts are depicted on the secondary axis to the right in the figure)
Researchers
The public
Low
imp
act
Hig
him
pac
t
Altmetrics
Provide a more nuancedimage of the total impactthat research has made?
Citations Altmetrics
Altmetrics are not alternatives to citations
Citations Tweets
Older publications Newer publications
The effects of time on the volume of citations and tweets (Haustein, Peters, Sugimoto, Thelwall, & Larivière, 2014)
The effects of time
Heterogeneity of altmetrics
doiDigital Object Identifier
Most altmetrics identified from:
Bollen J, Van de Sompel H, Hagberg A, Bettencourt L, Chute R, Rodriguez MA, et al. (2009) ClickstreamData Yields High-Resolution Maps of Science. PLoS ONE 4(3): e4803. doi:10.1371/journal.pone.0004803
Disciplinary differences
Bots?Haustein, S., Bowman, T.D., Holmberg, K., Tsou, A., Sugimoto, C.R. & Lariviére, V. (in press). Tweets as impact indicators: Examining the implications of
automated “bot” accounts on Twitter. To appear in JASIST. arXiv version available at: http://arxiv.org/ftp/arxiv/papers/1410/1410.4139.pdf.
Gaming altmetrics?
Stories behind the altmetrics?
Altmetrics and open science?
Measuring the Societal Impact of Open ScienceAvoimen Tieteen Yhteiskunnallisen Vaikuttavuuden Mittarit
Project is financed by the Ministry of Education and Culture in Finland (2015-2016)
Goals of the research project:1) Investigate altmetrics (at country-level, i.e.
Finland) and create a prototype to demonstrate them
2) Develop methods to map societal impact of research (and to analyze the stories behindthe altmetrics)
Shameless selfpromotion:
Bookforthcominglater in 2015
Kim [email protected]://kimholmberg.fi@kholmber
Kiitos!
Slides available athttp://www.slideshare.net/kholmber