uncovering the impact story of open research

Heather  Piwowar  @researchremix  Postdoc  with  NESCent  and  Dryad,  at  Duke  and  UBC

UBC  2012  

Uncovering the Impact Story

of Open Research

some photos NC, SA

Research evaluation matters

who gets hired

who gets grants

which journals succeed

which journal innovations succeed

whether we publish in journals at all

whether we actually write articles

the whole structure of scientific communication

right now, impact factor

problems!

run-away hits

decision of 4 anonymous people

only one type of impact

favours the established

let me tell you something that is going to start changing the game

A few weeks ago, on Oct 4, NSF announced it is going to change

its Biosketch Template to call for “Products” instead of

“Publications”

Big deal. So it is a new title.

Whole lot more than that.

1.

A major science funder believes status quo isn't the only way to disseminate research, and traditional peer-reviewed articles in high impact journals are not all that matters.

Other research products can be considered first-class research products.

2. It signals this to all the researchers who apply to the NSF.

This is a big deal, because it moves the discussion away from just OA advocates and open data tweeps to all scientists. All scientists now get to think,

"hrmm, did I release a dataset that is as good as my papers?"

3.

It necessitates a move away from journal impact factors.

The quality and impact of data and software can't be evaluated with the journal impact factor.

Item level metrics and altmetrics are going to be totally necessary for grant reviewers to know whether these alternative products have made a difference.

4.

Once we are in the grant review door with item-level metrics, watch out!

Alternative metrics are empowering for innovations in publishing. All the new journal experiments that are springing up... eLife, Peerj, F1000 Research, PLoS Currents.... are invited to compete, invited to help authors make the case that the articles published in those containers were impactful.

Very empowering for innovative journals, most of which are OA. And empowering for dismantling the "article" yet further into its component parts.

5.

It demonstrates a willingness on the part of funders to refine what they consider evaluation-worthy criteria.

Transformation is going to happen as major funders encourage their applicants to brag about their openness, and the impact that their openness has enabled.

what can the world look like

Beyond the Impact Factor

Beyond the Article

Quality

Impact

A Single NumberMultiple Dimensions

CC-BY-NC by maniacyak on flickrhttp://www.flickr.com/photos/maniacyak/3432589472

Multiple Flavours

Open

Context

Agileexperimentation

Tell the full story of research impact.

altmetrics.org/tools

Altmetric.comImpactStoryPLOS article-level metricsPlum AnalyticsReader MeterScience Card

http://www.flickr.com/photos/pixscapes/4331070047

born as total-impactHackathon!

incorporating as nonprofitBoard: Cameron Neylon, John Wilbanks

http://orcid.org

what can the world look like

Beyond the Impact Factor

Beyond the Article

datasets!software!

slides!preprints!

blogs!

context

type of engagementtype of audience

comparisons, reference sets

not just one number

not just one flavour

open!

rOpenSciPlum

Ubiquity PressPeerEv

dissertationsIR experiments

...

barrier-based $$$

http://www.flickr.com/photos/quinnanya/2055471833

http://flickr.com/photos/nieve44/2152712789/

Changes are happening in scholarly communication.

http://www.flickr.com/photos/myklroventine/892446624/

http://www.flickr.com/photos/jo-h/2688026447/

Build great things.

Reward impactful work.

Tell the full story of our research impact.

thank you!Jason Priem: cofounder of ImpactStory

Also: Todd Vision, Mike Whitlock, the open science community, and those who release their articles, datasets and photos openly.

blog.ImpactStory.orgteam@ImpactStory.org

@ImpactStory

ImpactStory.org