altmetrics and impact - committee on publication ethicspublicationethics.org/files/euan adie...

39
Altmetrics and impact Altmetric.com Euan Adie COPE, 17 th April 2015

Upload: others

Post on 27-Feb-2021

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Altmetrics and impact

Altmetric.com

Euan Adie COPE, 17th April 2015

Page 2: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Altmetrics and impact

Altmetric.com

Euan Adie

Getting credit where credit is due

COPE, 17th April 2015

Page 3: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

This talk

• Why are altmetrics of interest to authors & institutions?

• How are they used

• Things we’ve learned

• How are they abused

Page 4: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Several different tools available

Page 5: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

You say tomato…

Page 6: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 7: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

But there is another driver

Page 8: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Bad news for researchers?

• You’re under pressure to justify

– Yourself

– Your research

• Both internally and externally

Page 9: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Good news for researchers?

Funders and institutions are increasingly

looking for or considering other types of:

• Impact

• Research output

• Contribution

Page 10: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

The Evaluation Gap

http://citationculture.wordpress.com/

Page 11: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Altmetrics

Take a broader view of

impact to help give credit

where credit is due

Page 12: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Example: social & mainstream

mediaBlogs, reviews,

commentsIncluding Faculty of 1000, PubPeer,

MathOverflow and the world’s

largest curated index of academic

blogs.

Newspapers &

magazinesInternational titles, both mainstream

and niche.

Social media

Page 13: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Example: policy documents

World Health Organization (WHO)

“WHO policy on collaborative TB/HIV activities:

guidelines for national programmes and other

stakeholders”

National Institute for Health and

Care Excellence (NICE)

“Delivering Accident Prevention at local level in the

new public health system: Road safety policy and

links to wider objectives”

Intergovernmental Panel on

Climate Change (IPCC)

“Managing the Risks of Extreme Events and

Disasters to Advance Climate Change Adaptation”

Page 14: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Example: popular non-fiction

Gulp

“’America’s funniest science writer’ (Washington Post)

takes us down the hatch on an unforgettable tour.”

The Black Swan

“Since being published in 2007, as of February 2011

has sold close to 3 million copies. It spent 36 weeks

on the New York Times Bestseller list list; 17 as

hardcover and 19 weeks as paperback. It was

published in 32 languages.”

Thinking Fast and Slow

“The basis for his Nobel prize, Kahneman developed

prospect theory to account for experimental errors he

noticed in Daniel Bernoulli's traditional utility theory.”

Page 15: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

How people use altmetrics data

Page 16: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 17: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

• To gauge the overall popularly of the article• 87% of respondents strongly agreed or agreed

• To discover and network with researchers who are interested in

the same area of their work • 77% strongly agreed or agreed

• To understands a paper’s influence on the scientific community • 66% strongly agreed or agreed

• To determine what journal to submit their next paper to • 60% of respondents strongly agreed or agreed

• To determine areas of research to explore • Only 37% of respondents strongly agreed or agreed

Page 18: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Browsing by author

Browsing by

department

Page 19: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Some things we’ve learned

Page 20: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 21: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

People are very keen to relate it to citations!

Scholarly altmetrics correlate with citations.

Public engagement / policy & practice

altmetrics don’t.

Page 22: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 23: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 24: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

How people (ab)use altmetrics data

Page 25: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

"The more any

quantitative social

indicator (or even

some qualitative

indicator) is used for

social decision-

making […] the more

apt it will be to distort

and corrupt the social

processes it is

intended to monitor.”

Donald Campbell,

1976

Page 26: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Altmetric

score

Quantifying attention

Page 27: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Why score at all? To allow ranking

Page 28: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 29: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 30: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 31: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 32: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Gaming the system

Page 33: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Gaming?

• Alice asks her friends to retweet her.

Page 34: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Gaming?

• Bob likes Alice’s paper. He shares it with all his friends and asks them to retweet him.

Page 35: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Gaming?

• Alice pays $5 for 100 retweets

Page 36: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Four types of suspicious attention

Page 37: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,
Page 38: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

What can be done?

• Make underlying data available, visible

• Only track sources that can be audited

– Some interesting sources fail this test e.g. downloads and private Facebook activity

• Automatically flag up suspicious activity, then manually curate

• Have a standard process in place to deal with gamed articles, notify the journal

Page 39: Altmetrics and impact - Committee on Publication Ethicspublicationethics.org/files/Euan Adie COPE_FINAL.pdf · 2015. 4. 28. · Altmetrics and impact Altmetric.com Euan Adie COPE,

Thanks for listening!

@altmetric

[email protected]