-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
1/79
SYSTEMATIC REVIEWAND META-ANALYSIS
Dr. Detty S Nurdiati, MPH, PhD, SpOG(K)1,2
1Clinical Epidemiology and Biostatistics Unit2Dept of Obstetrics & Gynecology
Faculty of Medicine, Universitas Gadjah Mada Yogyakarta, Indonesia
The 6th Board Meeting and Scientific Meeting of ICE-EBM NetworkFaculty of Medicine, Universitas Maranatha, Bandung, December 13-15, 2013
EVIDENCE-BASED MEDICINE WORKSHOP
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
2/79
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
3/79
Levels of Evidence,
in Context
Anderson, P.F. (2006).
Chain of Trust / Level of Evidence – Vertical.
Available at: http://www-personal.umich.edu/~pfa/
pro/courses/ChainOfTrustLoEVert2.pdf.
Top pyramid is from:
Medical Research Library of Brooklyn.
Guide to Research Methods, The Evidence Pyramid. Available
at: http://library.downstate.edu/ebm/2100.htm
Anderson PF. 2011. Systematic Reviews: Context & Methodology
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
4/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
You find this review
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
5/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
6/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
How confident are you of the evidence?
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
7/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Why systematic reviews?
• efficient way to access the body of research
• saves time required for searching
•critical appraisal
• interpretation of results
• explore differences between studies
•
reliable basis for decision making• unbiased selection of relevant information
• useful for health care, policy, future research
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
8/79
SYSTEMATIC REVIEW?
A review of a clearly formulated
question that uses systematic and
explicit methods to identify,
select and critically appraiserelevant research, and to collect
and analyse data from the studies
that are included in the review.
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
9/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Narrative vs Systematic Review
Narrative
– Many questions
– No search methods
–No inclusion criteria
– No combining studies
– Prone to random and
systematic error
– Provide conflictingsummaries
Systematic
– One question
– Explicit search, reproducible
–Explicit inclusion criteria
– Combine study results
(with/without meta-analysis)
WHY do we need Systematic Reviews?
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
10/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Purpose of systematic reviews• Provide up to date summary of all published
research literature
• Allow large amounts of data to be assimilated
• Provide an objective collation of results of
research
• Provide reliable recommendations
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
11/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Useful Resources
• The Cochrane Collaboration www.thecochranelibrary.com/ – Cochrane Handbook for Systematic Reviews of
Interventions (version 5 updated March 2011)
• CRD www.crd.york.ac.uk/ – The Centre for Reviews and Dissemination is a
department of the University of York and is part ofthe National Institute for Health Research
• EPPI-Centre www.eppi.ioe.ac.uk/ – The Evidence for Policy and Practice Information and
Co-ordinating Centre, Social Science Research Unit,Institute of Education, University of London.
Mickan S. 2013. Systematic Reviews
http://www.thecochranelibrary.com/http://www.crd.york.ac.uk/http://www.eppi.ioe.ac.uk/http://www.eppi.ioe.ac.uk/http://www.crd.york.ac.uk/http://www.thecochranelibrary.com/
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
12/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Key features of a systematic review• clearly stated objectives
• pre-defined eligibility criteria
• explicit, reproducible methodology
• systematic search
• assessment of validity of included studies
• systematic synthesis and presentation offindings
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
13/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
The review team• review must be undertaken by more than one person
• allows double-checking
–
eligibility of included studies – data collection and entry
– risk of bias assessment
• different areas of expertise
–
clinical (multidisciplinary) – systematic review methods (including statistics)
– user perspective (consumer, professional, settings)
• consider establishing an advisory group
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
14/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Steps of a systematic review1. Clear answerable question
2. Reproducible search strategy
3. Assessment of literature quality
4. Summary of the evidence
5. Statistical, sensitivity analyses
6. Interpretation
7. Conclusions, recommendations
8. Published protocol and review
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
15/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Types of systematic review
• Different research questions require differentstudy designsgenerate different types ofreview
• Variations occur in – Research questions asked
– Primary study designs included
–Methods for synthesis
– Approaches to being systematic
– Types of evidence included
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
16/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Best evidence for different questions
Treatment Prognosis Particularperspective
Systematic
Review of …
Systematic
Review of … Systematic
Review of …
Randomisedtrials InceptionCohorts Qualitativestudies
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
17/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Best evidence for different questions
Treatment Prognosis Particularperspective
Systematic
Review of …
Systematic
Review of … Systematic
Review of …
Randomisedtrials InceptionCohorts Qualitativestudies
The most type of
SR published
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
18/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Risk of bias
• use risk of biasassessment
• for studies that provideddata for sleep disruption:
– what is the overall risk ofbias of these studies
–does the risk of biasreduce our confidence inthe effect?
F lt f M di i U i it G dj h M d Y k t I d i
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
19/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Review level
↓
Effect measure
Study A Effect measureOutcome data
Effect measureOutcome dataStudy B
Effect measureOutcome dataStudy C
Effect measureOutcome dataStudy D
Study level
↓
Source: Jo McKenzie & Miranda Cumpston
F lt f M di i U i it G dj h M d Y k t I d i
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
20/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
What is a meta-analysis?
• combines the results from two or more studies
• estimates an ‘average’ or ‘common’ effect
•
optional part of a systematic review
Source: Julian Higgins
Systematicreviews Meta-analyses
F lt f M di i U i it G dj h M d Y k t I d i
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
21/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Why perform a meta-analysis?
• quantify treatment effects and theiruncertainty
• increase power
• increase precision
• explore differences between studies
• settle controversies from conflicting studies
• generate new hypotheses
Source: Julian Higgins
F lt f M di i U i it G dj h M d Y k t I d i
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
22/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
When not to do a meta-analysis
• mixing apples with oranges
• each included study must address same question
• consider comparison and outcomes
• requires your subjective judgement
• combining a broad mix of studies answers broad
questions
• answer may be meaningless and genuine effectsmay be obscured if studies are too diverse
Source: Julian Higgins
Faculty of Medicine Universitas Gadjah Mada Yogyakarta Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
23/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
When not to do a meta-analysis
• garbage in – garbage out
• a meta-analysis is only as good as the studies in it
• if included studies are biased:
• meta-analysis result will also be incorrect
• will give more credibility and narrower confidenceinterval
• if serious reporting biases present:
• unrepresentative set of studies may give misleadingresult
Source: Julian Higgins
Faculty of Medicine Universitas Gadjah Mada Yogyakarta Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
24/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
When can you do a meta-analysis?
• more than one study has measured an effect
• the studies are sufficiently similar to produce
a meaningful and useful result• the outcome has been measured in similar
ways
•data are available in a format we can use
Faculty of Medicine Universitas Gadjah Mada Yogyakarta Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
25/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Steps in a meta-analysis
• identify comparisons to be made
• identify outcomes to be reported and
statistics to be used• collect data from each relevant study
• combine the results to obtain the summary
of effect• explore differences between the studies
• interpret the results
Faculty of Medicine Universitas Gadjah Mada Yogyakarta Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
26/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Selecting comparisons
• break your topic down into pair-wise comparisons
• each review may have one or many• use your judgement to decide what to group
together, and what should be a separatecomparison
Hypothetical review: Caffeine for daytime drowsiness
vscaffeinated coffee decaffeinated coffee
Faculty of Medicine Universitas Gadjah Mada Yogyakarta Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
27/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Selecting outcomes & effect measures
• for each comparison, select outcomes
• for each outcome, select an effect measure• may depend on the available data from included
studies
• asleep at end of trial (RR)
• irritability (MD/SMD)
• headaches (RR)
Hypothetical review: Caffeine for daytime drowsiness
vscaffeinated coffee decaffeinated coffee
Faculty of Medicine Universitas Gadjah Mada Yogyakarta Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
28/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Calculating the summary result
• collect a summary statistic from eachcontributing study
• how do we bring them together?
• treat as one big study – add intervention &control data?
• breaks randomisation, will give the wrong answer
•
simple average?• weights all studies equally – some studies closer to
the truth
• weighted average
Faculty of Medicine Universitas Gadjah Mada Yogyakarta Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
29/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Weighting studies
• more weight to the studies which give moreinformation
• more participants, more events, narrower
confidence interval
• calculated using the effect estimate and its variance
• inverse-variance method:
2
SE
1
estimateof variance
1weight
weightsof sum
)weightestimate(of sumestimatepooled
Faculty of Medicine Universitas Gadjah Mada Yogyakarta Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
30/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Headache Caffeine Decaf Weight
Amore-Coffea 2000 2/31 10/34
Deliciozza 2004 10/40 9/40
Mama-Kaffa 1999 12/53 9/61
Morrocona 1998 3/15 1/17
Norscafe 1998 19/68 9/64
Oohlahlazza 1998 4/35 2/37
Piazza-Allerta 2003 8/35 6/37
For example
Faculty of Medicine Universitas Gadjah Mada Yogyakarta Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
31/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
Headache Caffeine Decaf Weight
Amore-Coffea 2000 2/31 10/34 6.6%
Deliciozza 2004 10/40 9/40 21.9%
Mama-Kaffa 1999 12/53 9/61 22.2%
Morrocona 1998 3/15 1/17 2.9%
Norscafe 1998 19/68 9/64 26.4%
Oohlahlazza 1998 4/35 2/37 5.1%
Piazza-Allerta 2003 8/35 6/37 14.9%
For example
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
32/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
• for dichotomous or continuous data
• inverse-variance
• straightforward, general method
• for dichotomous data only
• Mantel-Haenszel (default)
• good with few events – common in Cochrane reviews
•
weighting system depends on effect measure• Peto
• for odds ratios only
• good with few events and small effect sizes (OR close to
1)
Meta-analysis options
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
33/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
A forest of lines
Trees Joyce Kilmer Forest by charlescleonard http://www.flickr.com/photos/charlescleonard/3754931947/
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
http://i.creativecommons.org/l/by-nc/3.0/88x31.png
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
34/79
y , j , gy ,
Forest plots
Headache at 24 hours
• headings explain the comparison
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
35/79
y , j , gy ,
Forest plots
Headache at 24 hours
• list of included studies
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
36/79
y , j , gy ,
Headache at 24 hours
• raw data for each study
Forest plots
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
37/79
y j gy
Forest plots
Headache at 24 hours
• total data for all studies
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
38/79
Headache at 24 hours
• weight given to each study
Forest plots
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
39/79
Headache at 24 hours
• effect estimate for each study, with CI
Forest plots
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
40/79
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
41/79
Headache at 24 hours
• scale and direction of benefit
Forest plots
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
42/79
Headache at 24 hours
• pooled effect estimate for all studies, with CI
Forest plots
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
43/79
Interpreting confidence intervals
• always present estimate with a confidence interval
• precision
• point estimate is the best guess of the effect
• CI expresses uncertainty – range of values we can be
reasonably sure includes the true effect
• significance
• if the CI includes the null value
• rarely means evidence of no effect
• effect cannot be confirmed or refuted by the available evidence
• consider what level of change is clinically important
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
44/79
Heterogeneity• consider heterogeneity in the meta- analysis
– overlap in confidence intervals
– I2 statistic
• is there unexplained statistical heterogeneity?
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
45/79
Identifying heterogeneity
• visual inspection of the forest plots
• chi-squared (c2) test (Q test)
•I2 statistic to quantify heterogeneity
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
46/79
Visual inspection
Forest plot A Forest plot B
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
47/79
The chi-squared (c2) test
• tests the null hypothesis of homogeneity
• low power with few studies
•
may detect clinically unimportant differences withmany studies
• narrow question (yes/no) not useful if
heterogeneity is inevitable
• calculated automatically by RevMan
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
48/79
The I2 statistic
• I2 statistic describes the percentage of
variability due to heterogeneity rather than
chance (0% to 100%)
• low values indicate no, or little, heterogeneity
• high values indicate a lot of heterogeneity
• calculated automatically by RevMan
• be cautious in interpreting
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
49/79
What to do about heterogeneity
• check that the data are correct
• if heterogeneity is very high – interpret fixed-effect results with caution
•
consider sensitivity analysis – would random-effects havemade an important difference?
– may choose not to meta-analyse• average result may be meaningless in practice
• consider clinical & methodological comparability of studies
– avoid• changing your effect measure or analysis model
• excluding outlying studies
• explore heterogeneity
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
50/79
Reporting biases•
consider whether the outcome is at risk – no large studies
– industry sponsorship
–
few of your studies report this outcome – consider funnel plot results or statistical tests for
small study effects, if appropriate
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
51/79
External validity
• how well, and how
completely, do the studies
address your review
question?
• compare your inclusion
criteria to the ‘Characteristics
of included studies’ table – PICO
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
52/79
The Results section of your review
• a systematic, narrative summary of results
• forest plots
– key forest plots linked as figures
•
usually primary outcomes – all forest plots will be published as supplementary data
– avoid forest plots with only one study
• may also add other data tables
–results of single studies
• summary data for each group, effect estimates, confidence intervals
– non-standard data
• not helpful to report trivial outcomes or results at high risk of bias
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
53/79
RAPID CRITICAL APPRAISAL OF SYSTEMATIC REVIEW
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
54/79
Time Spent
P
r o j e c t s W o r k e d O
n
Step 1What
question
did thestudy ask?
Step 4
How do theresults
apply to the
care of mypatients
Rapid Critical Appraisal of SR
Step 3
What do theresultsmean? Step 2
How well
was the
study
done?
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
55/79
Step 1
What question did the study ask?
P :...................
I :...................
C :...................
O :...................
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
56/79
Step 2
How well was the study done?
... Internal Validity...
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
57/79
QUESTION
Does the SR address a focusedquestion?
... And use it to direct the searchand select articles for inclusion?
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
58/79
Getting started
KEY = systematic, rigorous, transparent, reproducible• Define the research question
– Clear background, scope, setting
– Research question determines method of review (PICO)
– Specify inclusion and exclusion criteria
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
59/79
FIND
Did the search find all therelevant evidence?
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
60/79
Find the published researchClear, comprehensive, reproducible search strategy
• Search terms
• Databases
• Other strategies for grey literature
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
61/79
Manage the research evidence
• Organise database, hand searching
– Use of forward citation searching, reference lists
•
Manage references – Reference Management software eg Endnote
• Screen studies to check fit
–2 reviewers, process of agreement
– Record decisions about whether studies meet
criteria
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
62/79
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
63/79
APPRAISEHave the studies been critically
appraised?
... And was the overall qualityadequate?
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
64/79
Assess quality of the literature•
Dual, independent assessment of design aspectslikely to cause bias – depends on study designs
• Resource: online www.cochrane.org/training/cochrane-handbook or www.equator-network.org/home/
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
http://www.cochrane.org/training/cochrane-handbookhttp://www.cochrane.org/training/cochrane-handbookhttp://www.equator-network.org/home/http://www.equator-network.org/home/http://www.equator-network.org/home/http://www.equator-network.org/home/http://www.cochrane.org/training/cochrane-handbookhttp://www.cochrane.org/training/cochrane-handbookhttp://www.cochrane.org/training/cochrane-handbook
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
65/79
The Cochrane risk of bias tool
Risk of bias Interpretation Within a study Across studies
Low risk of bias Plausible bias unlikely toseriously alter the
results.
Low risk of bias for all
key domains.
Most information is from
studies at low
risk of bias.
Unclear risk of bias Plausible bias that raisessome doubt about theresults
Unclear risk of bias forone or more key
domains.
Most information is fromstudies at low or unclear
risk of bias.
High risk of bias Plausible bias thatseriously weakens
confidence in the results.
High risk of bias for one
or more key
Domains.
The proportion of
information from studies
at high risk of bias issufficient to affect the
interpretation of the
results.
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
66/79
A visual representation - RCTs
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
67/79
Describe included studies• Design data extraction forms
– General descriptive information
– Research methods
– Key results
–2 reviewers, process of agreement
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
68/79
SYNTHESISE
Have the results been synthesisedwith appropriate tables and plots?
... And were the results similarbetween the studies?
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
69/79
Decide on process of synthesisFactors to consider
• Consistency of outcome measures
• Sub groups
• Heterogeneity
• Common sense test
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
70/79
Details of data synthesis
• Look for consistent measurement of data,with 95% confidence intervals
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
71/79
Primary outcome/s
•
Basis for meta-analysis
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
72/79
Sub group analysis•
Identify in protocol with justification• To enhance usefulness of research answers
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
73/79
Heterogeneity• Common sense test of study design, outcome
measurements, forest plot
• Are syntheses meaningful (apples vs oranges)
• Influences statistics within meta-analysis
Mickan S. 2013. Systematic Reviews
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
74/79
Sensitivity analyses• determine whether the assumptions or decisions
made have a major effect on the results of thereview.
Mickan S. 2013. Systematic Reviews
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
75/79
Step 3
What do the results mean?
What measure was used, howlarge was the effect (could ithave been due to chance)?
Other comments?
Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
76/79
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
77/79
Step 4
How do the results apply to the care
of my patients? Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
78/79
Is the review any good – FAITH?
FINDING
Did they find most studies?
APPRAISAL
Did they use appropriate inclusion criteria?
INCLUDE
Did they include valid studies – for question asked?
TOTAL UP
Did they synthesise similar outcomes?
HETEROGENEITY
-
8/18/2019 20141110-Detty-IKK-EBM-Systematic Review and Meta-analysis.pdf
79/79
The Cochrane Collaboration
International non-profit organisation that
prepares, maintains, and disseminates
systematic up-to-date reviews of
health care interventions