2014 crossref annual meeting peer review panel: pre: securing trust & transparency in peer...
DESCRIPTION
Adam Etkin presents the PRE system of evaluating peer review systems at the 2014 CrossRef annual meeting on a panel on innovations in peer reviewTRANSCRIPT
![Page 1: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/1.jpg)
Securing Trust & Transparency in Peer Review
http://www.pre-val.org
@PeerReviewEval
Adam Etkin, Founder and Managing Director
November 2014
![Page 2: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/2.jpg)
A service that works with the publisher and journal to provide independent validation of the review process
A badge that publishers can display in various places – search results, article pages, article-level metrics – to signal to readers that quality peer review has been conducted
A window into a given journal’s peer review process accessible by end users
PRE-val answers the most basic, and important, question about scholarly works: “Has this article really been peer reviewed?”
PRE-val supports quality peer review.
What is PRE-val?
2
![Page 3: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/3.jpg)
Why do we need PRE-val?
Traditional peer review & scholarly publishersseem to be under constant criticism.
Emergence of “predatory” publishers
High-profile cases of faulty research being published
Difficulty distinguishing peer-reviewed research from non-reviewed content in journals – publishing in a peer-reviewed journal does not equate with peer review for every article
“Publish then filter” and “Publishing is a button” attitude
3
![Page 4: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/4.jpg)
Why do we need PRE-val?
4
![Page 5: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/5.jpg)
Surveys show peer review is valued by researchers & authors.
Sources: Sense About Science; Taylor & Francis; CIBER Research; NPG/Palgrave Macmillan Author Insights survey
“The qualitative data also point to the fact that peer review is the central pillar of trust.”
University of Tennessee and CIBER Research Ltd, December 2013
Why do we need PRE-val?
5
Most (69%) researchers are satisfied with the current system of peer review but only a third think
that the current system is the best we can do
Most (69%) researchers are satisfied with the current system of peer review but only a third think
that the current system is the best we can do
Most (84%) believe that without peer review there would be no
control in scientific communication
Most (84%) believe that without peer review there would be no
control in scientific communication
78% of OA authors prefer traditional,
rigorous peer review
78% of OA authors prefer traditional,
rigorous peer review
Only 20% want basic check followed by post-
publication review
Almost all researchers (91%) believe that their last paper was
improved as a result of peer review
Almost all researchers (91%) believe that their last paper was
improved as a result of peer review
While many want a faster process with fewer rounds, the overwhelming majority (~70%) prefer to wait for
thorough review
While many want a faster process with fewer rounds, the overwhelming majority (~70%) prefer to wait for
thorough review
93% of science authors consider quality of
PEER REVIEW when deciding where to
publish
![Page 6: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/6.jpg)
Less time to review more information
6
*B.-C. Björk, R. Annikki, and M. Lauri. Global annual volume of peer reviewed scholarly articles and the share available via different Open Access options*The National Science Board estimates the average annual growth of the indexes within the Web of Science to be 2.5% (See: Science and Engineering Indicators 2010, chapter 5, page 29)*The stm report: An overview of scientific and scholarly journal publishing (2009)*http://dan.corlan.net/cgi-bin/medline-trend?Q=
Total Peer Reviewed Journals
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
2006 2007 2008 2009 2010 2011 2012 2013
Year
Jou
rnal
s
28,000+ journals, and growing Total Peer Reviewed Articles Published
0
500,000
1,000,000
1,500,000
2,000,000
2,500,000
2006 2007 2008 2009 2010 2011 2012 2013
Year
Art
icle
s
2 million articles in 2013
More information than ever, less time to sort through it all.
![Page 7: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/7.jpg)
Not just one “right” approach
The number of varied approaches to peer review is increasing, yet readers often assume it is a uniform practice
Journals with high standards have no clear way of routinely showing these standards at the article level
Trends toward “lite” peer review & critics are dominating the discussion, while quality has a harder time finding a voice
Peer review is not one thing, and it cannot be assumed.
This leads to confusion and questions about trust . . .
7
![Page 8: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/8.jpg)
Measurements Abound
![Page 9: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/9.jpg)
Metrics/Filters = Knowledge = Power
A Lesson From The Car Industry
Looking for a new car?• MPG• Size• Speed• Features• Safety• Cost
Evaluating a journal?• Impact Factor• Audience• Speed• Altmetrics• Peer Review (PRE) • Cost
![Page 10: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/10.jpg)
10
Community
Different approaches, different services….one goal: Working together to educate and support
![Page 11: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/11.jpg)
We believe in peer review
Recognizes journals with an editor-in-chief or other overseeing editor
PRE-val encourages journals to use quality reviewers
Provides transparency into iteration through review to improve the science and clarity of a paper
Helps promote use of best practices, which are markers of commitment to better peer review approaches
PRE-val creates incentives to use best practices in peer review.
11
![Page 12: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/12.jpg)
Technical flow – Low/No work by publisher
12
First, we sit down together to learn about your process. PRE-val configuration takes into account your unique processes and preferences related to what you want to make public/transparent.
Publisher Manuscript Submission/Peer Review Tracking System
<metadata>PEER REVIEW</metadata>
PRE API
#1
Low/no development work on part of publisher or
manuscript submission/peer review system
Publisher places badge on - Journal article page - Search results - Aggregator sites - Article metrics - anywhere else a signal of peer review is important
![Page 13: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/13.jpg)
![Page 14: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/14.jpg)
Info about peer review process:•Type of Review•Rounds of review•Roles participating
Other measures of screening quality:•COPE member?•Plagiarism screening?•Retraction policy?
Optional info:•Reviewer comments•Reviewer names
14
![Page 15: 2014 CrossRef Annual Meeting Peer Review Panel: PRE: Securing Trust & Transparency in Peer Review](https://reader035.vdocument.in/reader035/viewer/2022062419/5592bcb81a28ab00418b459e/html5/thumbnails/15.jpg)
Securing Trust & Transparency in Peer Review
http://www.pre-val.org
@PeerReviewEval
Adam Etkin, Founder and Managing Director
November 2014