alternative metrics to evaluate scientific impact
DESCRIPTION
Presented at the “Science evaluation in the 21st century and its impact on scientific formation” session of the Biased Science and Alternatives for the Publication System Symposium - Rio de Janeiro, September 15 2014.TRANSCRIPT
![Page 1: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/1.jpg)
Alternative metrics to evaluate scientific impact
Prof. Dra. Nanci OddonePrograma de Pós-Graduação em Biblioteconomia/UNIRIO
With Iara Vidal & Andréa do Nascimento
Simpósio “Biased Science and Alternatives for the Publication System”15 de setembro de 2014 ● UFRJ, Rio de Janeiro, Brasil
![Page 2: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/2.jpg)
ALTMETRICS
“The creation and study of new metrics based on the Social Web for analyzing and informing scholarship”
![Page 3: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/3.jpg)
ORIGINS
![Page 4: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/4.jpg)
ORIGINS• Altmetrics: a manifesto / Jason
Priem, Dario Taraborelli, Paul Groth, Cameron Neylon (2010)
• Scholarship’s main filters for importance – peer review, citation counts, JIF – are failing. Altmetrics offers possible solutions
![Page 5: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/5.jpg)
ALTERNATIVE WAYS• Alternative Metrics: shares, tags, notes,
comments, bookmarks, social mentions...
• Alternative Products: blogs, raw data, code, grey literature...
• Alternative Audiences: organizations, doctors, patients, educators, students, citizen scientists...
![Page 6: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/6.jpg)
MULTIPLE DIMENSIONS
IMPACT
USAGEDownloadsViews
PEER REVIEW
Expert opinion
CITATIONSALTMETRICS
StorageLinks
BookmarksConversations
Adapted from altmetrics.org
![Page 7: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/7.jpg)
ALTMETRIC INDICATORS
Extraído de: KONKIEL, S. Altmetrics: A 21st-Century Solution to Determining Research Quality. Online searcher, v. 37, n. 4, p. 10-15, 2013.
Dado Descrição Exemplos
Shares / Mentions A public post to share news about an article or research output
Twitter, Topsy, Facebook, reddit, news articles, blog posts, Google+, YouTube, Figshare, Mendeley
Saves / CapturesSocial bookmarks, favorites, online reference managers
Mendeley, CiteULike, Delicious, Github, Twitter, and Slideshare
Reviews / Social Media Discusion, including additional commentary
Faculty of 1000 (F1000), blog posts, comentários de artigos, comentários no Facebook
AdaptationsCreation of derivative work using an article or other research output Github
Views / UsageDownloads or views in web services and social media sites
Figshare, Slideshare, Dryad, Facebook, YouTube, LinkedIn
Cites Citations in academic literature Pubmed, Scopus, Web of Science
Adaptado de: KONKIEL, S. Altmetrics: A 21st-Century Solution to Determining Research Quality. Online searcher, v. 37, n. 4, p. 10-15, 2013; e TANAMBAUM, G. Article-Level metrics: a SPARC primer. 2013.
![Page 8: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/8.jpg)
ADVANTAGES• Diversity: diverse metrics, for diverse
products, capturing multiple dimensions of scientific impact
• Complementing traditional metrics (San Francisco Declaration on Research Assessment – DORA, 2012)
• Speed: almost real-time data
• Openness: open data and tools
![Page 9: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/9.jpg)
DISADVANTAGES• Gaming
• Lack of credibility: a measure of impact or popularity?
• Altmetric sources and indicators are heterogeneous and ephemerous
• CONTEXT IS KEY!
![Page 10: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/10.jpg)
ARTICLE-LEVEL METRICS (ALMS)
• The combination of traditional and alternative data to capture the impact of a particular article
• A clearer picture of how articles are being discussed, shared, and used
![Page 11: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/11.jpg)
ALMS VS. ALTMETRICS
ALMs = attempts to
measure impact at the article-
level
Altmetrics = refers to data sources (social media tools)
![Page 12: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/12.jpg)
IMPACT MEASURES
From TANAMBAUM, G. Article-Level metrics: a SPARC primer. 2013.
![Page 13: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/13.jpg)
PUBLISHERS & ALTMETRICS•Elsevier•EBSCO•PLOS•BioMed Central•Nature•Springer
•Wiley•Emerald•High Wire Press•OJS/PKP•Rede SciELO•(…)
![Page 14: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/14.jpg)
TOOLSFour Main Providers:•Altmetric•ImpactStory•Plum Analytics•PLOS ALM
![Page 15: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/15.jpg)
![Page 16: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/16.jpg)
![Page 17: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/17.jpg)
![Page 18: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/18.jpg)
![Page 19: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/19.jpg)
CHALLENGES• Need for standards/best practices to
guide data collection and treatment• Altmetrics Initiative, National
Information Standards Organization (NISO)
• Need for ALM/altmetrics studies in different contexts – including peripheric countries.
![Page 20: Alternative metrics to evaluate scientific impact](https://reader033.vdocument.in/reader033/viewer/2022061203/547e74ecb47959a7508b4bc2/html5/thumbnails/20.jpg)
THANK YOU
Iara Vidal@iaravps / [email protected]
Andréa do Nascimento@deiafix / [email protected]
Prof. Dra. Nanci [email protected]