altmetrics for team science
Embed Size (px)
TRANSCRIPT
IUPUI OFFICE OF
ACADEMIC AFFAIRSA
ltm
etri
cs
PRESENTERS:
Heather Coates, Assistant Librarian, University LibraryWillie Miller, Assistant Librarian, University Library
Citation Metrics
Willie Miller
Assistant Librarian, Liaison to the
Schools of Informatics, Journalism,
University Librarian
Altmetrics
Heather Coates
Digital Scholarship & Data
Management Librarian, Liaison to
the Richard M. Fairbanks School
of Public Health, University Library
Traditional Metrics
Team Science & Collaborative Translational Research
March 5, 2015
Willie Miller, MLS Informatics and Journalism Librarian
Traditional Metrics: Levels
• Article-level: citation counts
• Journal-level: impact factor
• Author-level: h-index
Sources of Data
• Web of Science
– Journal Citation Reports (JCR)
• Scopus
• Google Scholar Citation
WEB OF SCIENCE
For author-level, article-level, and journal level metrics (JCR)
SCOPUS
For author-level, article-level, and institution-level metrics
GOOGLE SCHOLAR CITATIONS
For author-level and article-level metrics
Citation Metrics: Tracking
March 5, 2015
Citation Metrics: Tracking
• Team outputs• # of team publications
• Journals for team publications
• High profile citation sources – seminal article, leading researcher in the field, cited in a new policy, etc.
• Personal outputs• Contribution toward key publications or other outputs
• Clearly identify ouputs created as part of a team versus individual or smaller group work
March 5, 2015
Citation Metrics: Presentation
March 5, 2015
5 Top-ranked Journals
JournalNo. of
PublicationsJournal Impact Factor
(2013)5-yr Impact Factor
The Lancet Neurology 3 21.823 24.075
American Journal of Psychiatry 3 13.559 15.062
Brain 3 10.226 10.846
Neurology 11 8.303 8.375
Biological Psychiatry 5 9.472 10.347
Presentation: Examples
March 5, 2015
Team H-index Values
Author H-Index Team Rank
Ira Shoulson 54 1
Elizabeth H. Aylward 53 2
*Jane S. Paulsen 45 3
Julie C. Stout 42 4
Douglas R. Langbehn 33 5
Kevin M. Duff 25 6
AltmetricsTeam Science & Collaborative
Translational Research
March 5, 2015
Heather Coates, MLS, MS Digital Scholarship & Data Management Librarian
Evaluating ImpactTeam Science & Collaborative Translational Research
March 5, 2015
March 5, 2015
Impact Indicators
Research Output& Activities
Biological materials, collaborations, data, databases, repositories, techniques & procedures, grey literature, invention disclosures, mobile apps, patents, trainees, etc.
Advancement of Knowledge
Books/chapters (inclusion in bibliographies, library ownership, textbook use), change in understanding (paradigm shift, lead to new approach), citations (first & second generation citations, countries and institutions represented), conference themes, new centers/institutes
Clinical Implementation(or TRIP)
Biological materials, study cited in clinical decision aid, clinical/practice guidelines, diagnostic application, instruments, quality measure guidelines (gov’t or NPO), reporting requirements
Legislation & Policy
Committee participation, study cited in guidelines, study cited in policy, study cited in enactment of standards
Economic Benefit
Findings cited in reduced costs for delivery of healthcare services, findings result in enhancement of existing resources and expertise, license agreements for use of IP, spinoff or startup company
Community Benefit
Public awareness of risk factors, patient decision materials, cited in public/private insurance benefit plan
AltmetricsTeam Science & Collaborative Translational Research
March 5, 2015
Traditional measures of impact
• Proxy for expert evaluation
• Typically citation-based
• Levels of evidence• Journal-level
• Article-level
• Scholar-level
• How can you use these in your dossier?• What is the value of these metrics?
March 5, 2015
Altmetrics typically measure impact of individual products
Citation-based Metrics
March 5, 2015
Journal Level Metrics
Output/Article Level Metrics
Author Level Metrics
Altmetrics
March 5, 2015
Web MetricsSocial Media
Metrics
View UseShare Recommend Discuss
Altmetric Manifesto
NO ONE CAN READ EVERYTHING. We rely on filters to make sense of the scholarly literature, but the narrow, traditional filters are being swamped. However, the growth of new, online scholarly tools allows us to make new filters; these altmetrics reflect the broad, rapid impact of scholarship in this burgeoning ecosystem. We call for more tools and research based on altmetrics…
March 5, 2015
Altmetrics
• Supplement traditional metrics
• Scope extends beyond the formal published scholarly record (journals & books)
• Timeframe: immediate to short-term impact
• Sources: focus is on social media/engagement
• DOES capture qualitative data (ex: blog snippets, tweet content)
March 5, 2015
Types of scholarly outputs
• Articles
• Books & chapters
• Conference presentations, panels, etc.
• Blogs & blog posts
• Grey literature, conference materials, white papers, unpublished reports
• Learning objects, instructional content, assessment tools, online courses, innovative use of technology, syllabi, etc.
• Impact on diverse populations or communities; impact on the community through media, changes in policy, law, or programs
March 5, 2015
Altmetric Sources
• Publishers (e.g., PLoS, PubMed, etc.)• Aggregation services
– Impact Story: http://impactstory.org/– Altmetric.com: http://www.altmetric.com/– Plum Analytics: https://plu.mx/g/samples
• Subject repositories– PubMedCentral (PMC)– arXiv– SSRN
• Institutional repositories– IUPUI ScholarWorks: http://scholarworks.iupui.edu– IUPUI DataWorks: http://dataworks.iupui.edu
March 5, 2015
Aggregator Sources
• News media
• Social media platforms
• Blogs
• Reference managers (e.g., Mendeley)
• other online conversations captured include YouTube, Wikipedia, Reddit, F1000, Q&A, Policy Docs
• NOT server-side views & download data
March 5, 2015
March 5, 2015
March 5, 2015
March 5, 2015
March 5, 2015
March 5, 2015
March 5, 2015
ImpactStory: Example
March 5, 2015
Plum Analytics Example
March 5, 2015
March 5, 2015
The problem is not a lack of data, but how to evaluate
and make sense of it.
Using Altmetrics
What is your case? What statements do you need to support?
• Identify priorities
• Choose tools/platforms
• Gather
• Select
• Store
• Visualize
March 5, 2015
Gathering Data
• Gather manually– Free
– Time-consuming
– Messy & redundant
– Tailored to your scholarship & argument
– Include unusual or field-specific sources
• Use an aggregator – Minimal cost (individual)
– Less time-consuming
– Generic, broad presentation
– Heavy emphasis on major social media channels
March 5, 2015
Gathering Data: Altmetric.it
• Install Altmetric.it Bookmarklet– http://www.altmetric.com/bookmarklet.php
– Drag the Altmetric.it icon to your browser bookmarks toolbar
• Get altmetrics for your outputs– Go to URL for your article/blog post/column/code/etc
– Click on the Altmetric.it shortcut
– Gather data from the donut in a spreadsheet
• Note: reliant on unique identifiers like DOI, handles, arXiv ID, SSRN ID, PMC ID, etc. but NOT limited to articles
March 5, 2015
Gathering Data: Scopus
• Register for an ORCID (unique author ID)
• Sign up for a Scopus account
• Link your ORCID to your Scopus account
• Maintain an accurate list of articles associated with your profile
• Gather data for each article (track in a spreadsheet)
March 5, 2015
Manual Tracking: Example
March 5, 2015
NOTE: The usage, views, and mentions data are fabricated for this example.
Presentation: Examples
March 5, 2015
Top 5 Articles
Title Year Citations Usage Views Mentions
Apathy is not depression 1998 351 1947 4208 0
*A new model for prediction of the age of onset and penetrance for Huntington's disease based on CAG length 2004 303 2691 5712 2
Generalized cognitive deficits in schizophrenia: A study of first- episode patients 1999 303 1658 3309 0
*Venezuelan kindreds reveal that genetic and environmental factors modulate Huntington's disease age of onset 2004 288 2147 4348 0
*Detection of Huntington's disease decades before diagnosis: The Predict-HD study 2008 234 5819 12381 9
*Product of the Huntington Study Group
NOTE: The usage, views, and mentions data are fabricated for this example.
Presentation: Examples
March 5, 2015
Top 5 Non-Article Outputs
Title Impact Evidence
Huntington Study Group website CommunityReceives more than 2,000 unique visitors per month.
HSG participant database Community 4300 registered potential participants
Research sites Community 63 research sites in 11 countries
Physician's Guide to the Management of Huntington Disease
Clinical Implementation/TRIP Downloaded more than 2,400 times
UHDRSClinical Implementation/TRIP Downloaded more than 13,000 times
NOTE: The evidence details are fabricated for this example.
Sources for Altmetrics by product
• Articles• Publisher website
• PubMedCentral
• SSRN/arXiv/PMC
• IUPUI ScholarWorks
• Presentations• Slideshare
• IUPUI ScholarWorks
• Blogs & blog posts• Wordpress
• Google Analytics
• Data• Figshare
• DataDryad
• Data Journals
• IUPUI DataWorks
March 5, 2015
Using Altmetrics
• Complementary - Portray just part of the picture– More immediate measure of impact
– Measure of impact outside your discipline and academia (public policy, practice, community engagement, public awareness, etc.)
– Reflect engagement with and impact upon a broader audience, beyond academia and your own area of expertise
• Flavors of impact (Priem et al, 2012)– “popular hit” – highly tweeted and shared on non-academic social
media sites
– “expert pick” – good F1000 ratings and subsequent citations, few shares or social media mentions
March 5, 2015
Resources
• ImpactStory Guide to Altmetrics: http://impactstory.demo.libguides.com/c.php?g=211074&p=1392895
• ImpactStory blog (http://blog.impactstory.org/)
• Piwowar, H. & Priem, J. (2013). The power of altmetrics on a CV. Bulletin of the Association for Information Science & Technology. https://www.asis.org/Bulletin/Apr-13/AprMay13_Piwowar_Priem.html
• Roemer, R. C. & Borchardt, R. (2012). From bibliometrics to altmetrics: A changing scholarly landscape. College and Research Libraries News, 73(10), 596-600. http://crln.acrl.org/content/73/10/596.full
• Dinsmore, A., Allen, L., & Dolby, K. (2014). Alternative perspectives on impact: The potential of ALMs and altmetrics to inform funders about research impact. PLoS Biology, 12(11), e1002003. doi: 10.1371/journal.pbio.1002003
• Steven Roberts website & P&T dossier, both incorporating altmetrics: http://faculty.washington.edu/sr320/?p=2806
• Altmetric webinar: Applied Altmetrics: http://godigitalscience.com/view/mail?iID=Y9PXAVDJMH5JVAUPJU79
March 5, 2015
References
• Beacham, B., Kalucy, L., McIntyre, E. (2005). Focus on Understanding and Measuring Research Impact. Retrieved from http://www.phcris.org.au/phplib/filedownload.php?file=/elib/lib/downloaded_files/publications/pdfs/phcris_pub_3236.pdf
• Garfield, E. (2000). The use of JCR and JPI in measuring short and long term journal impact. Presented at the Council of Scientific Editors Annual Meeting: San Diego, CA.
• Leyesdorff, L. (2009). How are new citation-based journal indicators adding to the bibliometric toolbox? Journal of the American Society for Information Science and Technology, 60(7), 1327-1336.
• Priem et al. (2010). Altmetrics Manifesto: http://altmetrics.org/manifesto/
• Priem, J., Piwowar, H., Hemminger, B. M. (2012). Altmetrics in the wild: Using social media to explore scholarly impact. http://arxiv.org/abs/1203.4745
March 5, 2015
QUESTIONS
Concluding Remarks
For additional P&T resources visit
http://academicaffairs.iupui.edu/
P&T Guidelines
Dossier Samples
Online Programs