altmetrics for measuring research output
DESCRIPTION
Webinar slides for the Southern Ontario Health Libraries Information Network (SOHLIN) presented on October 30th, 2014.TRANSCRIPT
Robin Featherstone, MLISResearch Librarian, Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta@rmfeatherstonehttp://www.slideshare.net/featherr
Altmetrics for Measuring Research Output
http://www.ualberta.ca/ARCHE/@arche4evidence
SOHLIN, Fall 2014
Conflict of interest disclosure• I have had free access to the
Altmetric.com Explorer since September 2013
Image by Nik Papageorgiou: http://theupturnedmicroscope.com
Altmetrics (AKA Social Media Metrics, Alternative Assessment Metrics) …[…] capture ways in which articles are disseminated throughout the expanding scholarly ecosystem, and reach beyond the scope of traditional trackers and filters.1
[…] measure research impact by including references outside of traditional scholarly publishing.2
1. Public Library of Science (PLOS). Altmetrics. [29 April 2014]. Available from http://article-level-metrics.plos.org/alt-metrics/
2. Baynes G. Scientometrics, bibliometrics, altmetrics: Some introductory advice for the lost and bemused. Insights. 2012;25(3):311-5. doi: 10.1629/2048-7754.25.3.311.
Altmetric tools measure data collected from:• Tweets• Blog mentions• Facebook posts• Presentations• News articles• Shared citations (e.g., Mendeley,
CiteULike)• Data uploads• Etc.
BIG DATA
Relevant for Research Evaluation?• Datasets• Conference posters• Theses & Dissertations• Lectures, online classes• Software• “Nanopublications”
What’s your impact1?
1. Emerald Group Publishing. Impact of Research [11 April 2014]. Available from: http://www.emeraldgrouppublishing.com/authors/impact/index.htmhttp://blogs.lse.ac.uk/impactofsocialsciences/2011/07/14/publishers-measuring-impact/
Agenda• Researcher-level vs. article-level
altmetrics• Product overview• Measurable mentions & unique IDs• Gaming the system• Altmetric research• Altmetrics for biomedical and health
sciences• Future directions: standards, research,
assessment
Researcher case study – Heather Piwowar
• h-index: 8 – 23 articles cited 258 times
http://www.scopus.com/authid/detail.url?authorId=25122628200
Researcher Case Study – Heather Piwowar• Slideshare followers: 85
http://www.slideshare.net/hpiwowar
• ImpactStory research products: 117http://impactstory.org/HeatherPiwowar
• Figshare views: 1384http://figshare.com/authors/Heather_Piwowar/96399
• GitHub contributions: 1410https://github.com/hpiwowar
• Google Scholar h-index: 13 (51 publications cited 678 times)http://scholar.google.ca/citations?user=1YLq0XMAAAAJ&hl=en
• ResearchGate score: 14.33• Research Gate Impact Points: 50.29
https://www.researchgate.net/profile/Heather_Piwowar2
Article Case Study – Marshall 2013
• Web of Science times cited: 1• Google Scholar times cited: 21• Scopus times cited: 6
Article Case Study – Marshall 2013• Altmetric score: 76
Altmetrics Players
Q: Which twitter post will be counted by altmetrics measurement instruments?
A. Great review on pet ownership for children with autism by Gretchen Carlisle
B. Great review by @gretchgretch15 on pet ownership - #autism
C. Great review - http://dx.doi.org/10.1016/j.pedn.2013.09.005
What is gaming? • Alice has a new paper out. She asks those
grad students of hers who blog to write about it.
• Alice has a new paper out. She believes that it contains important information for diabetes patients and so signs up to a ’100 retweets for $$$’ service.
5. Case examples from: Adie, E. Gaming Altmetrics. [29 April 2014]. Available from http://www.altmetric.com/blog/gaming-altmetrics/
Auto-spamming
Spectrum of social media self-promotion
6. Adie, E. Gaming Altmetrics. [29 April 2014]. Available from http://www.altmetric.com/blog/gaming-altmetrics/
Altmetrics research• PLOS Altmetrics Collection: http://
www.ploscollections.org/altmetrics
• Many studies compared altmetrics with citation counts and Impact Factors
• Positive but weak correlations found
Altmetrics in the health sciences• ‘Biomedical and health sciences’
showed highest share of publications with Altmetric.com scores7
• Increasing numbers of PubMed citations were being tweeted8
2010 2.4%2011 10.9%2012 20.4%
7. Costas R, Zahedi Z, Wouters P. Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. arXiv preprint arXiv:14014321. 2014.8. Haustein S, Peters I, Sugimoto CR, Thelwall M, Larivière V. Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Science and Technology. 2013.
Rank Score Article Title Year Journal
1 1350 Early Television Exposure and Subsequent Attentional Problems in Children
2004 Pediatrics
2 558 Do Television and Electronic Games Predict Children’s Psychosocial Adjustment? Longitudinal Research Using the UK Millennium Cohort Study
2013 Archives of Disease in Childhood
3 521 Microbial Contamination of Human Milk Purchased via the Internet
2013 Pediatrics
4 457 Effective Messages in Vaccine Promotion: A Randomized Trial
2014 Pediatrics
5 383 Gun Violence Trends in Movies 2013 Pediatrics
Using Altmetrics.com rankings– Pediatric Journals
Using Altmetrics.com rankings- ARCHE publicationsRank Score Article Title Journal Notes1 152 How confidence
intervals become confusion intervals
BMC Medical Research Methodology
One of the highest ever scores in this journal (ranked #1 of 364) – article was mentioned by 219 tweeters
2 148 Lifestyle Interventions for Patients With and at Risk for Type 2 Diabetes: A Systematic Review and Meta-analysis
Annals of Internal Medicine
Among the highest ever scored in this journal (ranked #37 of 2470) – article was mentioned by 128 tweeters
3 122 Social media use among patients and caregivers: a scoping review
BMJ Open Among the highest ever scored in this journal (ranked #22 of 1501) – article was mentioned by 150 tweeters
Future Directions – Publisher-driven altmetrics
Future Directions –Product development• Increase transparency of self-promotion
activites• Implement anti-spamming/anti-gaming
software solutions
Future Directions – Standards development
• NISO Alternative Assessment Metrics (Altmetrics) Initiative
Future Directions – Research• Is there a correlation between altmetrics
rankings and…– full-text downloads?– qualitative measures of interest? – …
Future Directions - Assessment• Will be used by: – funding agencies– research centres– libraries– Knowledge Translation (KT) researchers
Future Directions
The Alberta Research Centre for Health EvidenceEdmonton, Alberta, Canada
Robin Featherstone, MLIS@[email protected]
Slides available: http://www.slideshare.net/featherr