june 18 niso virtual conference: keynote speaker: altmetrics at the portfolio level
DESCRIPTION
June 18 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends Keynote Speaker: Altmetrics at the Portfolio Level - Paul Groth, Ph.D., Assistant Professor at the VU University AmsterdamTRANSCRIPT
Paul Groth @pgroth
Web & Media GroupDepartment of Computer ScienceThe Network InstituteVU University Amsterdamhttp://www.few.vu.nl/~pgroth
Altmetrics at the portfolio level
NIH Data Book – (http://report.nih.gov/ndb/index.aspx) Data provided by the Division of Information Services, Reporting Branch
Research Project Grants Applications, awards, and success rates
Metrics: Do metrics matter?Nature 2010http://doi.org/10.1038/465860a
"Outside letters basically trump everything," says Robert Simoni, chairman of the biology department at Stanford University in California.
“Imagine how the academic appointment process might change if search and review committees had access—within an appropriately tagged or linked online CV, for example, or via the ORCID system—to information about the specific contributions made by a candidate to each of his/her works, including contributions that might not otherwise have qualified for ‘authorship’ status?”
Point of view: Faculty appointments and the record of scholarshipAmy Brandhttp://dx.doi.org/10.7554/eLife.00452
Point of view: Faculty appointments and the record of scholarshipAmy Brandhttp://dx.doi.org/10.7554/eLife.00452
Slide 3: http://article-level-metrics.plos.org/files/2013/10/Brand.pptx
Opportunities• Individuals and institutions need better
tools for curating and networking their own record of scholarship
• Institutions need more information about scholarly contribution
• ALMs that reliably differentiate sources of input (general; academic; expert; etc.) would be more useful”
ENTER ALTMETRICS
Altmetrics is the study and use of scholarly impact measures based on activity in online tools and environments.
http://doi.org/10.1371/journal.pone.0048753
http://blog.peerj.com/post/65345738206/changing-the-currency-of-science-to-solve-our-greatest
Thanks Ian Mulvany
ALTMETRICS AS MEASURES OF IMPACT?
“It took approximately a generation (20 years) for bibliographic citation analysis to achieve acceptability as a measure of academic impact."
(Vaughan and Shaw, 2003)
The Research is Happening
htt
p:/
/jaso
npri
em
.org
/self-a
rchiv
ed/5
uni-
post
er.
png
(Birkholtz et al. 2013)
(Fausto et al 2012)
http://ploscollections.org/altmetricshttp://asis.org/Bulletin/Apr-13/AprMay13_Piwowar.html
http://www.cwts.nl/pdf/CWTS-WP-2013-003.pdf
Altmetrics 2014
Zahedi, Zohreh; Fenner, Martin; Costas, Rodrigo (2014): How consistent are altmetrics providers? Study of 1000 PLOS ONE publications using the PLOS ALM, Mendeley and Altmetric.com APIs. figshare. http://dx.doi.org/10.6084/m9.figshare.1041821
http://www.slideshare.net/paulwouters1/issi2013-wg-pw
Bottom Line: use altmetrics as evidence in a larger story
WHAT’S THE LARGER STORY?
DIFFERENT AGGREGATIONS
Aggregations
• Institutional Level– See the next talk
• Faculty• Department• Group
Aggregations
• Institutional Level– See the next talk
• Faculty• Department• Group• Most interesting: adhoc Groups
Projects
• Typical lifetime 2-5 years
• Networks of sub-groups
• Networks of projects• Different measures
of success
29
Published AND discussed AND cited
30
31
32
Events
• Interval lifetime• Changing networks• Virtual teams• Affiliation is transient• Aggregates• Central to scholarly communication
Summary: a broader view
• Almetrics gives a broader view• Altmetrics is must be situated in
context• We should begin looking at the
portfolio level– Multiple aggregations– What cannot be measured
Challenge: Architecture
Thanks
Collaborators: Peter van den Besselaar, Julie Birkholz, Frank van Harmelen, Shenghui Wang, Rinke Hoekstra, Thomas Gurney, Mike Taylor, Anita de Waard, Jason Priem, Dario Taraborelli, Cameron Neylon, Ian Mulvany