We know that the increasing pressure to publish has led to more cases of fraud and scientific misconduct, and that there is increasing demand in the broader academic ecosystem for rewarding or recognising reviewers, as well as considering alternative forms of peer review. Overview: • Trust report • Current challenges • What’s new? • Taylor & Francis initiatives, support and services

1

December 2013 saw the publication of a scholarly research project looking at Trust and Authority in Scholarly Communications in the Light of the Digital Transition. This builds on other work done by the research group CIBER about researcher behaviour in relation to scholarly journals. The report ranges widely, looking at citation behaviour, usage, metrics, etc, but we are just looking at their findings on peer review:

• what researchers liked, • didn’t like, and • were unsure about in the peer review process.

2

3

4

5

There are a number of other challenges in peer review of which we are already aware.

6

There are a number of new forms of peer review that are emerging, which we’ll discuss in more detail

7

Methodological review, according to publication criteria – but no “filter”. No responsibility for reviewers to contribute to improvement of content or readability. Few rounds of revision, so quick.

8

The trend towards ‘openness’ is fuelling the greater demand for transparent peer review. “Open” peer review is often allied with making data openly available as well, but is usually used to refer to the practice of making all of the reviewing and decision making process open, including sometimes making the reviewer’s names public. Just to quickly mention here that T&F, together with a society partner, are working on an exciting new open peer review journal to be launched in 2016. Look out for further announcements on this towards the end of the year.

9

We’ve always had post publication peer review, in the form of Letters to the Editor, going as far back as the Philosophical Magazine in 1799 Published comments and rejoinders. Many journals now adding ability to comment on articles, but very few showing multiple comments, and of those many are quite trivial notes, far from the depth and quality of a full peer review.

10

In a nutshell, these are various initiatives employing different new forms of post-publication peer review: variations of commenting on sites with some form of ranking or rating articles. You might think that all such post-publication commentary would be completely “open”, and that these “reviewers” would sign their names to their comments. However this is not always the case and their have been some instances of authors being mobbed by anonymous hostile reviews. T&F have asked some questions about author attitudes to peer review as part of one of our Open access surveys and found that there’s a gender bias in how authors view post publication peer review – female researchers are more nervous about it than men. This also reported in an online survey conducted by BioMed Central on Views on the peer review system of biomedical journals where female researchers held a higher perception of unfairness in peer review: http://www.biomedcentral.com/content/pdf/1471-2288-13-74.pdf

11

The idea behind pre-registration is simple: Before you conduct your experiment, you publicly list exactly what kind of experiment you are going to conduct, how many participants you will test, and what the predicted outcome is. Once you have done this, you have very few degrees of freedom to tweak your results afterwards. For example, if your results are not statistically significant, you cannot keep running participants until you obtain the desired result (i.e. optional stopping), because you have specified the number of participants in advance. Similarly, if you obtain results that don’t match your hypothesis, you cannot confabulate a post-hoc hypothesis that matches the outcome of your study.

12

New peer review consortium of the above publishers that builds on the existing journal peer review process. Group of journals across publishers, all in same subject area: Goal: By reducing the number of times a manuscript is reviewed, the consortium aims to speed publication of research results. Authors who submit to a participating journal in the consortium, and are not accepted by that journal, will be able to redirect their paper, with the referee’s reports, to any other journal in the consortium. Referees will be given the opportunity to opt out of having their reports forwarded, or to forward them anonymously (in all cases, the referee’s identity will be anonymous to the author – referees will choose whether they wish to remain anonymous to the editors of the secondary journal.

13

Rubriq’s model focuses on pre-publication – indeed pre-submission (to a journal) review. Not a publisher, to provide independent reviews. ‘We provide rigorous reviews by the same qualified peers who review for journals, but with a standardized scorecard that can be used in any publishing model. Enable faster, more consistent reviews, and will help match papers with the right journals.’

14

We are constantly exploring new ways/services to improve Peer Review processes, and I’ll just mention two interesting new services that we are in the very early stages of investigating: Publons’ aim is to help provide recognition and support for reviewers, which David has referred to in his presentation. Step included to verify peer review activity, so that there is an ‘official reviewer record’ that can be used in promotion and funding applications. Pre-Val is concerned with verifying and qualifying peer review. Partnering with PRE-val would provide independent, third party verification of the peer review process at the journal and article level, reinforcing our commitment to peer review integrity.

15

At T&F we’re flexible – we’ll work with you to trial new systems and ideas and we know that one size doesn’t fit all. Traditional peer review still holds sway but it is encouraging and exciting to see other models developing and being supported by the research community. Also encouraging is the impact that some of these new modes may have on the wider dissemination and transparency of information and discovery of unethical behaviour. We are working with our Author Services team in hosting Peer Review focus

groups in South Africa in April which will feed into a White Paper, alongside

data from other focus groups around the world.

16

We also have our Editorial Resources site – specific sections on peer review, providing information and support and also industry examples.

17

18