PRISMA-P: Preferred Reporting

Items for Systematic Reviews and Meta-

Analyses Protocols

Larissa Shamseer, MSc

Reporting Guidelines Research Coordinator

Knowledge Synthesis Group

Overview

• What is a systematic review protocol?

• Why are protocols important?

• Where can protocols be accessed?

• Protocols & Selective reporting

• How can reporting guidelines help?

• PRISMA-P

• Development

• Next steps

What is a systematic review protocol?

• States intentions for planned research

• Description of methodological approach

• Defines selection criteria

• Provides operational definitions

• Document written prior to starting a systematic review

(SR) stating rationale, intended purpose and content (i.e.

methods)

Why are reporting protocols important?

• Potential to reduce bias & enhance rigour

• Enhance transparency and confidence in ensuing review

• Reduce duplication of effort and foster collaboration

• Assist in critical appraisal

• Protect/uphold integrity in the research process

Where can protocols be accessed?

• Few options to register protocols; repeated calls for

protocol registration...

• http://www.crd.york.ac.uk/prospero/

• Launched Feb 2011

• April 16, 2012: 448 registered protocols from 27

countries

• 61 from Canada

Where can protocols be accessed?

• Paucity of protocol production & publication

• <50% (of non-Cochrane SRs) on MEDLINE report

working from protocols (Moher 2007)

• Systematic Reviews, open-access journal launched in

Feb 2012, publishes protocols (and other SR products)

• Theme series on importance of protocol registration

Selective reporting in primary studies

• Gaps in evidence of selective reporting in observational

studies

• What we know from the randomized trial literature

• Major discrepancies in primary outcome reporting

between protocols and trial reports:

• changed in 33-67% of trial reports

• omitted in 13-42% of trial reports

• introduced in 11-50% of trial reports (Dwan 2011)

• When trials are included in systematic reviews, the

problem is magnified

• Do systematic reviews suffer from similar selective

reporting?

• Of 60 Cochrane reviews (Issue 3, 2000) 47 had

published protocols (Silagy 2002)

• 91.5% contained a major change from protocol

relating to methods and selected outcomes

• 22% (n=64) of Cochrane reviews published over a 9-

month period had discrepant outcomes from their

published protocols (Kirkham 2010).

• 75% changed primary outcome(s)

• Discrepant outcomes were more likely to be

significant than those not changed.

• Represents a fraction of the problem since majority of

SRs are published in peer-reviewed journals

Selective Reporting in SRs

Barrier to publishing protocols

• Authors lack knowledge re: what to include in protocol

• Existing guidance addresses conduct

• Existing guidance overwhelming

• Solution: reporting guidelines!

Reporting Guidelines

• Over the last 25 years, guidance for authors preparing reports for publication have emerged

• www.EQUATOR-Network.org

• CONSORT

• Guidance for reporting parallel-group randomized trials

• SPIRIT

• Upcoming guidance for reporting protocols of randomized trials;

• mirrors many CONSORT items

• PRISMA

• Reporting guidance for systematic reviews and meta-analyses

• Newly implemented across all EPC programs

• PRISMA for Protocols (PRISMA-P)

• Aims to mirror PRISMA items; reduce workload

Scope of PRISMA-P

• Primarily aimed at reporting protocols of SRs for

evaluating clinical intervention efficacy [RCTs]

• All-encompassing/extrapolation to other SR types

Development of PRISMA-P • Follows recommended EQUATOR process (Moher

J Clin Epi 2010)

1 Paucity of protocol registration and publication outside

of select groups

• “Extend” PRISMA guidance

2 Review the literature

• PROSPERO group

3 Funding:

• AHRQ

• Canadian Institutes of Health Research

Initial Steps

Pre-meeting activities

4 27 international experts (journal editors, systematic

review methodologists, reporting guideline developers,

systematic review funders)

5 PROSPERO Delphi (Booth 2011)

6 Source of items: PROSPERO Delphi, PROSPERO

register, PRISMA, IOM CER standards, SPIRIT

7 PRISMA-P meeting: June 2011, Washington, D.C.

8 38 items 22 items

• Agreement on reporting key methodological items

• Disagreement/discussion/consensus on many non-

scientific issues (e.g. conflict of interest)

• PRISMA-P Statement & Explanation and Elaboration

document planned for 2012

• Statement currently being reviewed by steering

committee

• Early support (i.e. planned implementation) by a

number of groups

Face-to-face meeting

9 Checklist refinement currently underway

1st major revision – Oct 2011

• 22 items 18 items (22 including sub-items)

• 3 sections:

• Administrative information

• Introduction

• Methods

2nd major revision – March 2012

• 18 items in progress

• Phrasing and wording nuances

Post-meeting activities

PRISMA-P CHECKLIST

Administrative Information

Section/topic # Checklist item Reported

on page #

ADMINISTRATIVE INFORMATION

Title 1a Identify the report as a protocol of a systematic review.

1b If an update of a previous review, identify as such

Protocol

registration 2

If registered, name of the registry (e.g. PROSPERO, Joanna Briggs

Institute) and registration number

Review authors 3 Name, institutional affiliation, e-mail and physical mailing address of

all protocol authors;

Protocol version 4 Whether the report represents the original protocol or an amendment

to a previously published protocol

Support 5a

Names and contact information of organizations providing monetary

or material support for the review (e.g. funding agency, foundation,

company, institution), and for review sponsor, if different from funder

5b

Role(s) and responsibilities of systematic review funders, sponsors

and/or institution(s), if any, in development of review methods and

the decision to submit the protocol for publication, including who will

have ultimate authority over each of these activities.

Contributorship 6 Contributions of protocol authors; state guarantor of protocol

Section/topic # Checklist item Reported on page #

INTRODUCTION

Rationale 7 Describe the rationale for the review in the context of what is already known

Objectives 8

Provide an explicit statement of the question(s) the review will

address with reference to participants, interventions, comparisons,

outcomes (PICO), as well as study design, setting, and time frame, if applicable

Introduction

Section/topic # Checklist item Reported on page #

PLANNED METHODS Eligibility criteria 9 Specify study characteristics (e.g., PICOS, setting, time frame) to be

used as criteria for eligibility for review, giving rationale Information sources

10 Describe all intended information sources (e.g., names of electronic

databases, unpublished literature sources, included study authors) in

the search and planned date of search

Search strategy 11 Present full search strategy to be used for at least one electronic

database, including any planned limits (e.g., years considered, language, publication type), such that it could be repeated

Study selection process

12 State the process that will be used for selecting studies (e.g. abstract and full text consideration, independently, in duplicate).

Data management & collection

13a Describe planned method of extracting data from reports (e.g., piloting

forms, independently, in duplicate), any processes for obtaining and confirming data from investigators and how data will be managed.

Data items 14 List and define all variables for which it is intended data will be sought

(e.g., PICO, funding sources), any anticipated data assumptions and

simplifications, and any anticipated prioritization of review outcomes

(e.g., primary, secondary).

Planned Methods

Risk of bias in individual studies

15 Describe anticipated methods for assessing risk of bias of individual

studies (including whether this will be done at the study or outcome level), and how this information is to be used in any data synthesis.

Data Synthesis 16a Conditions under which data will be quantitatively synthesized, including any planned exploration of completeness and/or consistency (e.g. I2)

16b If data are appropriate for synthesis, describe planned summary

measures, methods of synthesis, and additional analyses (e.g. sensitivity or subgroup analyses, meta-regression)

16c If quantitative synthesis is not appropriate, type of summary planned

Reporting biases 17 Specify any planned assessment of risk of bias (e.g., publication bias

across studies, selective reporting within studies) and their potential impact on overall findings if present

Strength of evidence

18 Describe how confidence in cumulative evidence across studies will be assessed (e.g., using GRADE tool), if planned.

Section/topic # Checklist item Reported on page #

PLANNED METHODS

Planned methods (cont’d)

Next steps

• Will approach current PRISMA endorsers

• Standard endorsement text

• Develop & pilot eTool to facilitate use by authors,

peer reviewers, editors

• Evaluate effect of endorsement

eTools

• Why?

• DM is into colour!

• Poor implementation to date

Steering committee

• David Moher, Director, Ottawa EPC

• Mike Clarke, Professor, Queen’s University Belfast, Ireland

• Davina Ghersi, Expert Knowledge Development Officer, National

Health and Medical Research Council, Australia

• Lesley Stewart, Director, Centre for Reviews and Dissemination,

UK

• Alessandro Liberati, Professor, University of Modena, Italy

• Mark Petticrew, Professor, London School of Hygiene and Tropical

Medicine, UK

• Paul Shekelle, Director, Southern California Evidence-based

Practice Centre

• Project Coordinator: Larissa Shamseer, Ottawa EPC

EPC PRISMA-P participants

• Stephanie Chang, Agency for Healthcare Research and Quality (Rockville, USA)

• Mark Helfand, Oregon Evidence-based Practice Centre (Portland, USA)

• Joseph Lau, Tufts Medical Centre Evidence-based Practice Centre (Boston, USA)

• Kathleen Lohr, Research Triangle Institute-University of North Carolina Evidence-based Practice Centre (Research Triangle Park, USA)

• Jennifer Tetzlaff, Ottawa Evidence-based Practice Centre (Ottawa, Canada)

• Thomas Trikalinos, Tufts Medical Centre Evidence-based Practice Centre (Boston, USA)

• Evelyn Whitlock, Oregon Evidence-based Practice Centre and United States Preventive Services Task Force (Portland, USA)

Other PRISMA-P Participants

• Douglas G Altman, DSc, Centre for Statistics in Medicine (Oxford, UK)

• Alison Booth, Centre for Reviews and Dissemination, University of York (York, UK)

• An-Wen Chan, University of Toronto (Toronto, Canada)

• Tammy Clifford, Canadian Agency for Drugs and Technologies in Health (CADTH)

• Peter Gøtzsche, Nordic Cochrane Centre (Copenhagen, Denmark)

• Jeremy Grimshaw, Ottawa Hospital Research Institute (Ottawa, Canada); KT Canada

• Trish Groves, British Medical Journal (London, UK)

• Toby Lasserson, Cochrane Editorial Unit (London, UK)

• Jessie McGowan, University of Ottawa (Ottawa, Canada)

• Melissa Norton, PLoS Medicine (London, UK)

• Iveta Simera, EQUATOR Network (Oxford, UK)

• Bill Summerskill, the Lancet (London, UK)

Questions?