lisa vii: the scientific and technical foundation for altmetrics in the united states
DESCRIPTION
TRANSCRIPT
Scientific and technical foundation for altmetrics in the US
William Gunn, Ph.D. Head of Academic Outreach
Mendeley
@mrgunn
https://orcid.org/0000-0002-3555-2054
Why altmetrics?
http://www.stm-assoc.org/2009_10_13_MWC_STM_Report.pdf
• 978 data repositories • 19 funder policies • 16 data journals
New forms of scholarship need
new metrics.
Problems with Impact Factor
Country Documents Citable
documents Citations
Self-
Citations
Citations
per
Document H index
United States 7,063,329 6,672,307 129,540,193 62,480,425 20.45 1,380
China 2,680,395 2,655,272 11,253,119 6,127,507 6.17 385
United Kingdom 1,918,650 1,763,766 31,393,290 7,513,112 18.29 851
Germany 1,782,920 1,704,566 25,848,738 6,852,785 16.16 740
Japan 1,776,473 1,734,289 20,347,377 6,073,934 12.11 635
France 1,283,370 1,229,376 17,870,597 4,151,730 15.6 681
Canada 993,461 946,493 15,696,168 3,050,504 18.5 658
Italy 959,688 909,701 12,719,572 2,976,533 15.26 588
Spain 759,811 715,452 8,688,942 2,212,008 13.89 476
India 750,777 716,232 4,528,302 1,585,248 7.99 301
Problems with Impact Factor
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0030291
Problems with Impact Factor
During discussions with Thomson Scientific over which article types in PLoS Medicine the company deems as “citable,” it became clear that the process of determining a journal's impact factor is unscientific and arbitrary.”
The higher the impact factor, the more likely the research is to be retracted, partly due to intense competition. http://bjoern.brembs.net/news766.html.11
What matters is who is reading your work!
There is no correlation between the number a citations an article receives and the impact factor of the journal. http://www.bmj.com/content/314/7079/497.1.full
Adams, Jonathan. "Collaborations: the fourth age of research."
Nature 497.7451 (2013): 557-560.
King, Christopher (2012) Thomson Reuters Annual Report
http://ar.thomsonreuters.com/_files/pdf/MultiauthorPapers_ChrisKing.pdf
What are altmetrics?
Research impacts more than authors
htt
p://d
x.d
oi.org
/10.3
789/isqv25no2.2
01
3.0
4
Citations are slow
Research is fast
...and aggregates research data in the cloud
Mendeley extracts research data…
Install Mendeley Desktop
Collecting rich signals from domain experts.
Home Work Mobile
Cloud Library
Shared Folder
Mendeley Research Catalog
Mendeley Research Catalog
Read papers + keep track of notes
470M documents
Taking some misery out of writing
Information Extraction
We are publishing this data to the LOD cloud
http://code-research.eu/
Defining readership
• Each document addition is a “read”
• stamped with metadata describing the context of the read event
• a read is like a citation, but faster and captures more
Professors on Mendeley tend to be in applied math, stats, and physics.
Graduate students on Mendeley tend to be in engineering disciplines.
http://dx.doi.org/10.6084/m9.figshare.1041819
Cell Biology and Neuroscience are highly active disciplines, relative to their output.
Social sciences are highly active relative to their citations/paper
http://dx.doi.org/10.6084/m9.figshare.1041819
Readership vs. citations
• it comes with a payload of metadata
• it accrues faster
• it illuminates previously hidden impact
http
://arx
iv.org
/htm
l/1203.4
745v1
altmetrics show broader impact
http://www.asis.org/Bulletin/Apr-13/AprMay13_Lin_Fenner.html
altmetrics show broader impact
Issues To Be Addressed
• Identity
• Privacy
• Attribution
• Gaming
• Filtration
standards/ best practice
Consistency is key
http
://ww
w.n
iso.o
rg/p
ublic
atio
ns/is
q/2
013/v
25no2/c
ham
berla
in/
NISO Altmetrics Standards Alfred P. Sloan Foundation American Library Association California Institute of Technology Center for Research Libraries EBSCO Elsevier Harvard Internet Archive Wiley Library and Information Technology Association Library of Congress Los Alamos National Laboratory National Institutes of Health National Library of Medicine OCLC Princeton Columbia Smithsonian Stanford …
NISO Altmetrics Standards
• Types of sources: Mendeley, Twitter, Views, Downloads, Github
• Quality of sources: collection, reporting, aggregation methods; provenance; availability
• Use cases: discovery and assessment (of people and objects)
Types of sources
Most altmetrics providers use the following:
• Page views or downloads
• Mendeley readers (articles only for now)
• Tweets
• Comments:Blog posts, Pubmed Commons
• Github
Quality of sources
Collection methods vary & counts are inconsistent. Further study is needed.
For reporting, transparency is key. Show raw data, not just a derived number.
Aggregation of raw data is generally done by the recipient (institution, funder, publisher, author, etc) according to their need, instead of using one central source.
Quality of sources
Understanding and open reporting of provenance is important for community buy-in and long term stability.
Raw data should be available under open license, via API, with identifiers.
Identifiers include DOI:object, ORCID:person, ISNI/Ringgold: institution
Ex. This person(ORCID), at this institution (ISNI), released this object (DOI).
Use cases
Two main use cases exist: Discovery and Evaluation
Because the data sources remain variable, discovery can be done now. Accuracy of numbers matters less in recommendation than in assessment. Precision important for both.
Next Steps
• NISO white paper will be in public comment period soon.
• Working Groups will be established to develop best practices and standards.
• Pending approval, NISO will issue recommended practice or published standard.
• NISO to develop training to implement and adopt any recommended standards.
Amgen: 47 of 53 “landmark” oncology publications could not be reproduced
Bayer: 43 of 67 oncology & cardiovascular projects were based on contradictory results
Dr. John Ioannidis: 432 publications purporting sex differences in hypertension, multiple sclerosis, or lung cancer. Only one data set was reproducible
http://reproducibilityinitiative.org
There is no gold standard
