meta-analysis the ebm workshop a.a.haghdoost, md; phd of epidemiology [email protected]
TRANSCRIPT
![Page 1: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/1.jpg)
Meta-analysis
The EBM workshopThe EBM workshop
A.A.Haghdoost, MD; PhD of EpidemiologyA.A.Haghdoost, MD; PhD of [email protected]@kmu.ac.ir
![Page 2: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/2.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Definition
Meta-analysis: a type of systemic review that uses statistical techniques to quantitatively combine and summarize results of previous research
A review of literature is a meta-analytic review only if it includes quantitative estimation of the magnitude of the effect and its uncertainty (confidence limits).
![Page 3: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/3.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Function of Meta-Analysis(1)
1-Identify heterogeneity in effects among multiple studies and, where appropriate, provide summary measure
2-Increase statistical power and precision to detect an effect
3-Develop ,refine, and test hypothesis
continuedcontinued
![Page 4: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/4.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Function of Meta-Analysis(2)
continuationcontinuation
4-Reduce the subjectivity of study comparisons by using systematic and explicit comparison procedure
5-Identify data gap in the knowledge base and suggest direction for future research
6-Calculate sample size for future studies
![Page 5: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/5.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Historical background
Ideas behind meta-analysis predate Glass’ work by several decades– R. A. Fisher (1944)
• “When a number of quite independent tests of significance have been made, it sometimes happens that although few or none can be claimed individually as significant, yet the aggregate gives an impression that the probabilities are on the whole lower than would often have been obtained by chance” (p. 99).
• Source of the idea of cumulating probability values
– W. G. Cochran (1953)• Discusses a method of averaging means across independent
studies• Laid-out much of the statistical foundation that modern meta-
analysis is built upon (e.g., inverse variance weighting and homogeneity testing)
![Page 6: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/6.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Basic concepts
The main outcome is the overall magnitude of the effect.
It's not a simple average of the magnitude in all the studies.
Meta-analysis gives more weight to studies with more precise estimates. – The weighting factor is 1/(standard error)2.
![Page 7: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/7.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Main magnitude of effects
DescriptiveDescriptiveMeanPrevalence
AnalyticalAnalyticalAdditiveAdditive
Mean differenceStandardized mean differenceRisk, rate or hazard differenceCorrelation coefficient
MultiplicativeMultiplicativeOdds ratio, Risk, Rate or Hazard Ratio
![Page 8: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/8.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Statistical concepts(1)
You can combine effects from different studies only when they are expressed in the same units.
Meta-analysis uses the magnitude of the effect and its precision from each study to produce a weighted mean.
![Page 9: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/9.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Statistical concepts(2)
The impact of fish oil consumption on Cardio-vascular diseases
![Page 10: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/10.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Forest plot
the graphical display of results from individual studies on a common scale is a “Forest plot”.
In the forest plot each study is represented by a black square and a horizontal line (CI:95%).The area of the black square reflects the weight of the study in the meta-analysis.
A logarithmic scale should be used for plotting the Relative Risk.
![Page 11: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/11.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Forest plot
![Page 12: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/12.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Statistical concepts(3)
There are two basic approach to Quantitative meta –analysis:
Weighted-sum
Fixed effect model
Random effect model
Meta-regression model
![Page 13: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/13.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Fixed effect model
General Fixed effect model- the inverse variance – weighted method
Specific methods for combining odds ratio
Mantel- Haenszel method
Peto’s method
Maximum-Likelihood techniques
Exact methods of interval estimation
![Page 14: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/14.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Fixed effect model
In this model, all of the observed difference between the studies is due to chance
Observed study effect=Fixed effect+ error
Xi= θ + ei ei is N (0,δ2 )
Xi = Observed study effect
θ = Fixed effect common to all studies
![Page 15: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/15.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
General Fixed effect model
Ť=∑ wiTi/ ∑ wi
The weights that minimize the variance of Ť are inversely proportional to the conditional variance in each study
Wi=1/vi
Var(Ť)=1/ ∑ wi
![Page 16: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/16.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Mantel- Haenszel method
Each study is considered a strata.
Ť=∑ai di / ni / ∑bi ci /ni
![Page 17: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/17.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Random effect model
The “random effect” model, assumes a different underlying effect for each study.
This model leads to relatively more weight being given to smaller studies and to wider confidence intervals than the fixed effects models.
The use of this model has been advocated if there is heterogeneity between study results.
![Page 18: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/18.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Source of heterogeneity
Results of studies of similar interventions usually differ to some degree.
Differences may be due to:- inadequate sample size- different study design- different treatment protocols- different patient follow-up- different statistical analysis- different reporting- different patient response
![Page 19: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/19.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
An important controversy has arisen over whether the primary objective a meta-analysis should be the estimation of an overall summary or average effect across studies (a synthetic goal)
or the identification and estimation of differences among study-specific effects (analytic goal)
![Page 20: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/20.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Test of Homogeneity
This is a test that observed scatter of study outcomes is consistent with all of them estimating the same underlying effect.
Q= X2homo=∑i=1
nwi (mi -M)2
df=n-1
wi =weight
M=meta analytic estimate of effect
mi =effect measure of each study
![Page 21: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/21.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Dealing with statistical heterogeneity
The studies must be examined closely to see if the reason for their wide variation in effect. If it’s found the analysis can be stratified by that factor.
Subgroup analysisExclusion of studyChoose another scale
Random effect modelMeta-regression
![Page 22: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/22.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Random effect model
Assume there are two component of variability:
1)Due to inherent differences of the effect being sought in the studies (e.g. different design, different populations, different treatments, different adjustments ,etc.) (Between study)
2)Due to sampling error (Within study)
![Page 23: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/23.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Random effect model
There are two separable effects that can be measured
The effect that each study is estimating
The common effect that all studies are estimating
Observed study effect=study specific (random )effect + error
![Page 24: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/24.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Random effect model
This model assumes that the study specific effect sizes come from a random distribution of effect sizes with a fixed mean and variance.
There are five approach for this model:Weighted least squares
Un-weighted least squares
Maximum likelihood
Restricted Maximum likelihood
Exact approach to random effects of binary data.
![Page 25: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/25.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Random effect
Xi= θi + ei ei is N (0,δ2 )
Xi = Observed study effect
θi = Random effect specific to each study θi =U+di
U=Grand mean (common effect)
di is N (0, 2ד ) – Random term
![Page 26: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/26.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Weighted least squares for Random Effect
Ŵ=∑wi/k
S2w=1/k-1(∑wi
2-k Ŵ2)
U=(k-1)(Ŵ-S2w/kŴ)
0=2ד if Q<k-1
2ד =(Q-(k-1))/U if Q>k-1
wi* = 1/var.+ 2ד var.=within study variances
![Page 27: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/27.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Weighted least squares for Random Effect (WLS)
Ť.RND=∑ wi* Ti/ ∑ wi*
Var(Ť.RND)=1/ ∑ wi*
Where Ti is an estimate of effect size and θi
is the true effect size in the ith study
Ti = θi +ei ei is the error with which Ti estimates θi
var(Ti)= דθ2 +vi
![Page 28: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/28.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
random versus fixed effect models
Neither fixed nor random effect analysis can be considered ideal.
Random effect models has been criticized on grounds that unrealistic distributional assumption have to be made.
Random effect models are consistent with the specific aims of generalization.
![Page 29: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/29.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Peto’s advocates
He suggested a critical value .01 instead of usual .05 to decide whether a treatment effect is statistically significant for a fixed effect model.
This more conservative approach has the effect of reducing the differences between fixed and random effect models.
![Page 30: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/30.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Meta-regression
If more than two groups of studies have been formed and the characteristic used for grouping is ordered, greater power to identify sources of heterogeneity may be obtained by regressing study results on the characteristic .
With meta-regression, it is not necessary or even desirable to groups the studies.
The individual study results can be entered directly in the analysis.
![Page 31: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/31.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Meta-Regresion
1- meta-Regression model( extension of fixed effect model)
2- Mixed model( extension of random effect model)
![Page 32: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/32.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Fixed-effects regression
Θi=B0+B1xi1+...+Bpxip
It’s the covariate predictor variables that are responsible for the variation not a random effect; the variation is predictable, not random.
![Page 33: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/33.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Mixed model
Θi=B0+B1xi1+...+Bpxip+ui
This model assumes that part of the variability in true effects is unexplainable by the model.
![Page 34: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/34.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Between studies variation
You can and should allow for real differences between studies–heterogeneity–in the magnitude of the effect.
– The τ2 statistic quantifies % of variation due to real differences.
![Page 35: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/35.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Fixed effects model and heterogeneity
In fixed-effects meta-analysis, you do so by testing for heterogeneity using the Q statistic.
If p<0.10, you exclude "outlier" studies and re-test, until p>0.10.
When p>0.10, you declare the effect homogeneous.
But the approach is unrealistic, limited, and suffers from all the problems of statistical significance.
![Page 36: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/36.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
In random-effect meta-analysis, you assume there are real differences between all studies in the magnitude of the effect.
The "random effect" is the standard deviation representing the variation in the true magnitude from study to study.
You need more studies than for traditional meta-analysis.
The analysis is not available in a spreadsheet.
Random effects model and heterogeneity
![Page 37: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/37.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Fixed effects models: within-study variability
– "Did the treatment produce benefit on average
in the studies at hand?"
Random effects models: between-study and
within-study variability
– "Will the treatment produce benefit ‘on
average’?"
Concept of analysis in random versus fixed effect models
![Page 38: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/38.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Limitations
It's focused on mean effects and differences between studies. But what really matters is effects on individuals.
(Aggression bias)
A meta-analysis reflects only what's published or searchable.
![Page 39: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/39.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Aggregation bias
Relation between group rates or and means may not resemble the relation between individual values of exposure and outcome.
This phenomenon is known as aggregation bias or ecologic bias.
![Page 40: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/40.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Ecological fallacy
BP
Education
![Page 41: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/41.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Meta-analysis of neoadjuvant chemotherapy for cervical cancer
Word of Mouth14%
Trial Registers 14%
Hand Searching14%
Medline/Cancerlit
58%
![Page 42: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/42.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Type of reporting
Published in full47%
Published as abstract24%
Unpublished24%
Ongoing5%
![Page 43: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/43.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Selection bias in Meta analysis
English language bias Database biasPublication biasBias in reporting of data Citation biasMultiple publication biasSample size
![Page 44: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/44.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Publication bias
The results of a meta-analysis may be biased if the included studies are a biased sample of studies in general.
The classic form of this problem is publication bias, a tendency of journals to accept preferentially papers reporting an association over papers reporting no association
![Page 45: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/45.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Publication bias
If such a bias is operating, a meta-analysis based on only published reports will yield results biased away from the null.
Because small studies tend to display more publication bias, some authors attempt to avoid or minimize the problem by excluding studies below a certain size.
![Page 46: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/46.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Some meta-analysts present the effect magnitude of all the studies as a funnel plot, to address the issue of publication bias.
A plot of 1/(standard error) vs effect magnitude has an inverted funnel shape.
Asymmetry in the plot can indicate non-significant studies that weren’t published.
![Page 47: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/47.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Funnel plot
effectmagnitude
0
1/SE
“funnel” ofunbiasedstudies
![Page 48: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/48.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Funnel plot
![Page 49: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/49.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
1- Linear Regression Approach (Egger’s method)
SND=a + b. precisionSND=OR/SE
The intercept “a” provides a measure of asymmetry- the larger its deviation from zero the more pronounced the asymmetry.
Measures of Funnel Plot Asymmetry
![Page 50: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/50.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
2- A rank correlation test
This method is based on association between the size of effect estimates and their variance. If publication bias is present, a positive correlation between effect size and variance emerges because the variance of the estimates from smaller studies will also be large.
Measures of Funnel Plot Asymmetry
![Page 51: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/51.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Funnel plot
![Page 52: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/52.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Key Messages
Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from the Cochrane Database of Systematic Reviews.
Critical examination of systematic reviews for publication and related biases should be considered a routine procedure.
![Page 53: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/53.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Sources of Funnel Plot asymmetry
Selection BiasTrue HeterogeneitySize of effect differs according to study size:
– Intensity of interventions– Difference on underlying risk– Data irregularities– Poor methodological design of small studies– Inadequate analyses– Fraud– Artefactual– Choice of effect measure– Chance
![Page 54: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/54.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Sample size as source of bias
Consider a hypothetical literature summary stating, “of 17 studies to date, 5 have found a positive association,11 have found no association, and 1 has found a negative association; thus, the preponderance of evidence favors no association”.
Mere lack of power might cause most or all of the study results to be reported as null.
![Page 55: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/55.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Quality score
Some meta-analysts score the quality of a study.– Examples (scored yes=1, no=0):
• Published in a peer-reviewed journal?• Experienced researchers?• Research funded by impartial agency? • Study performed by impartial researchers?• Subjects selected randomly from a population?• Subjects assigned randomly to treatments?• High proportion of subjects entered and/or finished the
study?• Subjects blind to treatment?• Data gatherers blind to treatment?• Analysis performed blind?
![Page 56: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/56.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Quality score
Use the score to exclude some studies, and/or…
Include as a covariate in the meta-analysis, but…
Some statisticians advise caution when using quality.
![Page 57: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/57.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Quality scoring
A very common practice is to weight studies on a quality score usually based on some subjective assignment .
For example, 10 quality points for a cohort design, 8 points for a nested case control design, and 4 points for a population based case control design.
![Page 58: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/58.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Quality scoring
Quality scoring submerges important information by combining disparate study features into a single score.
It also introduces an unnecessary and somewhat arbitrary subjective element in to the analysis.
![Page 59: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/59.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Quality scores as weighing factors
study weight=1/var.
Quality adjusted weight= quality score /var.
![Page 60: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/60.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
The judgment that the studies should or should not be combined should be stated and justified explicitly.
There is some of a tendency to make this judgment on the basis of the quantitative results, but it’s critical to make a qualitative judgment.
Quality scores
![Page 61: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/61.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
What is an IPD Meta-analysis?
Involves the central collection, checking and analysis of updated individual patient data
Include all properly randomised trials, published and unpublished
Include all patients in an intention-to-treat analysis
![Page 62: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/62.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
IPD Meta-analysis
Individual patient data used
Analysis stratified by trial
IPD does not mean that all patients are combined into a single mega trial
![Page 63: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/63.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
IPD Analyses
Collect raw data from related studies, whether or not the studies collaborated at the design stage, exposures measures and other covariates that can be applied uniformly across the studies combined.
The major advantage of a IPD over an MA is the use of individual-based rather than group-based data.
![Page 64: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/64.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
sensitivity analysis
In sensitivity analysis, the sensitivity of inference to variations in or violations of certain assumptions is investigated.
For example, the sensitivity of inference to the assumption about the bias produced by failure to control for smoking can be checked by repeating the meta-analysis using other plausible values of the bias.
![Page 65: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/65.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
sensitivity analysis
If such reanalysis produces little change in an inference, one can be more confident that the inference is insensitive to assumptions about confounding by smoking.
In influence analysis, the extent to which inferences depend on a particular study or group of studies is examined; this can be accomplished by varying the weight of that study or group.
![Page 66: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/66.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
sensitivity analysis
Thus , in looking at the influence of a study, one could repeat the meta-analysis without the study, or perhaps with half its usual weight .
If change in weight of a study produces little change in an inference, inclusion of the study can not produce a serious problem, even if unquantified biases exist in the study
![Page 67: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/67.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Sensitivity and influence analysis
On the other hand, if an inference hinges on a single study or group of studies, one should refrain from making that inference
![Page 68: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/68.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
conclusion
Most meta-analysis will require from each study both a point estimate of effect and an estimate of its standard error .
A point estimate accompanied only by a P value will generally not provide for accurate computation of a standard error estimate, and should not be considered sufficient for reporting purposes.
![Page 69: Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir](https://reader035.vdocument.in/reader035/viewer/2022070308/551be130550346be588b5dfb/html5/thumbnails/69.jpg)
EBM workshopEBM workshop Meta analysisMeta analysis A.A.HaghdoostA.A.Haghdoost
Like large epidemiologic studies, meta-analysis run the risk of appearing to give results more precise and conclusive that warranted.
The lager number of subjects contributing to a meta-analysis will often lead to very narrow confidence intervals for the effect estimate.
Over conclusion