prescore presentation by adam etkin for highwire fall 2013 meeting

preSCOREpeer review evaluation SCORE –

A new set of metrics for today’s information ecosystem

Traditional peer review seems to be under constant criticism

Cases of faulty research being published seem to support the critics

The recent Bohannon “sting” highlights the problem

But this problem isn’t new & isn’t just an OA problem

It’s about Peer Review.

What is preSCORE?

These are extreme examples which make news, however surveys indicate that most researchers value peer review. Different approaches are not easy to identify. preSCORE is a new tool that generates a metric indicating the level of peer review conducted.

Both article and journal level metrics are included in preSCORE. 

Creating a Different Metric

Calculated metric includes other “best practices” Not a lagging indicator, a leading indicator Newer initiatives such as altmetrics offer more immediate,

useful info, but this is also post-publication. We focus on pre-publication peer-review, which sets it apart from other metrics

Supports filtering and differentiation, both critical to readers and researchers

Who Benefits?

Readers Authors Publishers/Journals Librarians

How Does It Work?

Export tagged metadata from peer review system

Pull reviewer h-index from Thomson-Reuters Metadata and h-index analyzed by preSCORE,

and the score is calculated preSCORE passed as needed via API and

made available on journal page displays, within article metrics, and so on

preSCORE and preVAL

preSCORE is the “heart” of the system Not focused on good/bad numbers preVAL answers the first, most basic

question…“Was this peer reviewed?” This allows users to drill in for more details if

they want Allows journals to share additional info

Examples

Drill Down for Details

Project Status

Currently completing development with ScholarOne and Aries

Beta testing underway Expect to launch in 2014 Anyone who would like more info OR

publishers/journals who wish to participate in beta testing, please contact me directly

Thank You!

Adam EtkinpreSCORE Founder

aetkin@prescore.org