icsti tacc 2014: how mendeley illuminates a broader definition of impact

How Mendeley illuminates a broader definition of impactWilliam Gunn, Ph.D.Head of Academic OutreachMendeley

@mrgunnhttps://orcid.org/0000-0002-3555-2054

...and aggregates research

data in the cloud

Mendeley extracts research data…

Install Mendeley Desktop

Collecting rich signals from domain experts.

Home Work Mobile

Cloud Library

Shared Folder

Read papers + keep track of notes

470M documents

Taking some misery out of writing

Mendeley Research Catalog

Mendeley Research Catalog

New forms of discovery

• Mendeley Suggest– personalized recommendations based

on reading history• related articles

– relatedness based on document similarity

• recommender frameworks– implement recommendations as a

service• third-party recommender services

– serve niche audiences

Information Extraction

We are publishing this data to the LOD cloud

http://code-research.eu/

Connecting researchers

Search

ReadAnnotateOrganize

Write

Import

Data we collect

• Metadata about the paper–title, author, tags, annotations, reads, shares, groups

• Metadata about the researcher–discipline, educational status, country, papers they’ve authored, presentations, expertise

2.7M researchers

Defining readership

• Each document addition is a “read”

• stamped with metadata describing the context of the read event

• a read is like a citation, but faster and captures more

research impacts more than authors

Problems with Impact Factor

Highly Tweeted articles are 11x more likely to be highly cited. (Eysenbach 2011) http://www.jmir.org/2011/4/e123/

The higher the impact factor, the more likely the research is to be retracted, partly due to intense competition.http://bjoern.brembs.net/news766.html.11

What matters is who is reading your work!

http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0030291

Problems with Impact Factor

During discussions with Thomson Scientific over which article types in PLoS Medicine the company deems as “citable,” it became clear that the process of determining a journal's impact factor is unscientific and arbitrary.”

Citations are slow

Mendeley is fast

Readership vs. citations

• it comes with a payload of metadata

• it accrues faster• it illuminates previously hidden impact

http://arxiv.org/html/1203.4745v1

altmetrics show broader impact

http://dx.do

i.org/10.37

89/isqv

25no

2.20

13.04

http://www.asis.org/Bulletin/Apr-13/AprMay13_Lin_Fenner.html

altmetrics show broader impact

Issues To Be Addressed

• Identity• Privacy• Attribution• Gaming• Filtration

standards/best practice

Amgen: 47 of 53 “landmark” oncology publications could not be reproduced

Bayer: 43 of 67 oncology & cardiovascular projects were based on contradictory results

Dr. John Ioannidis: 432 publications purporting sex differences in hypertension, multiple sclerosis, or lung cancer. Only one data set was reproducible

There is no gold standard

Cultural skew is important

South America is weak on N.A. social media, strong on Mendeley

What would people build if they could get the data?

• Impact Story – get credit for all your work

• PLOS ALM – article-level metrics for papers

• Plum Analytics – bespoke analytics for libraries (EBSCO)

• Altmetric.com – altmetrics for publishers. (Digital Science)

www.mendeley.com

william.gunn@mendeley.com@mrgunn