Meta-analysis

Systematic ReviewsDr Sharon MickanCentre for Evidence-based Medicine University of Oxford

1Learning Objectives - overviewReview purpose of a Systematic Review

Types of systematic reviewBest question for each study type

Process of designing a systematic review

Critical appraisal of a systematic review

What do you do?For an patient with a painful sore throat, you wonder whether corticosteroids will help with pain relief? You do a search and find several studies: some suggest that steroids reduce pain; some do not What do you do? Ask a consultant? Peer? Patient?Ask research student to find all studies & select the best?How do you know which study to believe?You find this review

How confident are you of the evidence?

Purpose of systematic reviewsProvide up to date summary of all published research literatureAllow large amounts of data to be assimilated Provide an objective collation of results of researchProvide reliable recommendations

Clarify the differencesSystematic ReviewNarrative ReviewMeta-analysis

Any other similar terms?

Systematic Review or meta-analysis?A Systematic Review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review.

Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies.9Narrative vs systematic reviewNarrativeMany questionsNo search methodsNo inclusion criteriaNo combining studiesProne to random and systematic errorProvide conflicting summariesSystematicOne questionExplicit searchReproducibleExplicit inclusion criteriaCombine study results(meta-analysis)

WHY do we need Systematic Reviews?10Benefits of systematic reviews Up to date resource for cliniciansStarting point for clinical guidelinesPolicy guidanceBasis for new primary researchImportant for grant funding bodiesManagement guidance

Research training tool???Useful ResourcesThe Cochrane Collaboration www.thecochranelibrary.com/Cochrane Handbook for Systematic Reviews of Interventions (version 5 updated March 2011)CRD www.crd.york.ac.uk/The Centre for Reviews and Dissemination is a department of the University of York and is part of the National Institute for Health ResearchEPPI-Centre www.eppi.ioe.ac.uk/The Evidence for Policy and Practice Information and Co-ordinating Centre, Social Science Research Unit, Institute of Education, University of London. Steps of a systematic review Clear answerable questionReproducible search strategyAssessment of literature qualitySummary of the evidenceStatistical, sensitivity analysesInterpretation Conclusions, recommendationsPublished protocol and reviewTypes of systematic reviewDifferent research questions require different study designs generate different types of reviewVariations occur inResearch questions askedPrimary study designs includedMethods for synthesisApproaches to being systematicTypes of evidence included

Best evidence for different questionsTreatmentPrognosisParticular perspectiveSystematic Review of Systematic Review of Systematic Review of Randomised trialsInception CohortsQualitative studies15Types of Systematic ReviewsCross-sectional analysis Nov 2004300 Systematic ReviewsTherapeutic = 213 (71%)Cochrane = 125 (59%)Non-Cochrane = 88 (41%)Diagnosis/Prognosis = 23 (7%)Epidemiology = 38 (13%)

Getting startedKEY = systematic, rigorous, transparent, reproducibleDefine the research questionClear background, scope, settingResearch question determines method of review (PICO)Specify inclusion and exclusion criteria

Find the published researchClear, comprehensive, reproducible search strategySearch termsDatabasesOther strategies for grey literature

Manage the research evidenceOrganise database, hand searchingUse of forward citation searching, reference listsManage referencesReference Management software eg Endnote Screen studies to check fit2 reviewers, process of agreementRecord decisions about whether studies meet criteria

Assess quality of the literatureDual, independent assessment of design aspects likely to cause bias depends on study designsResource http://www.equator-network.org/home/

The Cochrane risk of bias toolRisk of biasInterpretation

Within a study

Across studiesLow risk of biasPlausible bias unlikely to seriously alter the results.Low risk of bias for all key domains.

Most information is from studies at lowrisk of bias.Unclear risk of biasPlausible bias that raises some doubt about the resultsUnclear risk of bias for one or more keydomains.

Most information is from studies at low or unclear risk of bias.High risk of biasPlausible bias that seriously weakensconfidence in the results.

High risk of bias for one or more keyDomains.The proportion of information from studies at high risk of bias is sufficient to affect the interpretation of the results.A visual representation - RCTs

Describe included studiesDesign data extraction formsGeneral descriptive informationResearch methodsKey results 2 reviewers, process of agreement

Decide on process of synthesisFactors to considerConsistency of outcome measuresSub groups HeterogeneityCommon sense test

Details of data synthesisLook for consistent measurement of data, with 95% confidence intervals

Primary outcome/s Basis for meta-analysis

Sub group analysisIdentify in protocol with justificationTo enhance usefulness of research answers

HeterogeneityCommon sense test of study design, outcome measurements, forest plot Are syntheses meaningful (apples vs oranges)Influences statistics within meta-analysis

Sensitivity analysesdetermine whether the assumptions or decisions made have a major effect on the results of the review.

Protocol developmentDefine and justify the research questionFind and manage the research evidenceDescribe included studiesSynthesise the evidenceInterpret and disseminate

Registration of Systematic ReviewsPROSPEROInternational prospective register of systematic reviewshttp://www.crd.york.ac.uk/PROSPERO/BenefitsProvides a public record of planned methodsRaises awareness of the reviewTracks use and impact of published reviewsPermanent record whether final report published or not

Cochrane review process1. Register title with Review Group2. Write the protocolProtocol reviewed & revisedPublished on CDSR3. Write the reviewReview reviewed and revisedPublished on CDSR4. Update (every 2-3 years)Is the review any good FAITH? FINDINGDid they find most studies?APPRAISALDid they use appropriate inclusion criteria?INCLUDEDid they include valid studies for question asked?TOTAL UpDid they synthesise similar outcomes?HETEROGENEITY

A quick reviewWhy look for a SR?

What types of SR exist?

What are the key steps in a SR?

Why is a protocol important?

How do you appraise a SR?