meta-analysis of correlated data. common forms of dependence multiple effects per study –or per...
TRANSCRIPT
![Page 1: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/1.jpg)
Meta-Analysis of Correlated Data
![Page 2: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/2.jpg)
Meta-Analysis of Correlated Data
![Page 3: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/3.jpg)
Common Forms of Dependence
• Multiple effects per study– Or per research group!
• Multiple effect sizes using same control
• Phylogenetic non-independence
• Measurements of multiple responses to a common treatment
• Unknown correlations…
![Page 4: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/4.jpg)
Multiple Sample Points per Study!
Study Experiment in Study Hedges D V Hedges D
Ramos & Pinto 2010 1 4.32 7.23
Ramos & Pinto 2010 2 2.34 6.24
Ramos & Pinto 2010 3 3.89 5.54
Ellner & Vadas 2003 1 -0.54 2.66
Ellner & Vadas 2003 2 -4.54 8.34
Moria & Melian 2008 1 3.44 9.23
![Page 5: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/5.jpg)
Hierarchical Models
• Study-level random effect
• Study-level variation in coefficients
• Covariates at experiment and study level
![Page 6: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/6.jpg)
Hierarchical Models• Random variation within study (j)
and between studies (i)
Tijij,ij2)
ijj,j2
j,2
![Page 7: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/7.jpg)
Study Level Clustering
![Page 8: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/8.jpg)
Hierarchical Partitioning of One Study
Grand Mean
Study Mean
Variation due to
Variation due to
![Page 9: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/9.jpg)
Example: Data Set 1
Group Effect Variance
1 A 0.2 0.10
2 A 0.6 0.15
3 A 0.5 0.05
4 A 0.1 0.06
5 B 0.8 0.08
6 B 0.4 0.05
7 B 0.9 0.04
8 C 0.2 0.09
...
![Page 10: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/10.jpg)
A Two-Step SolutionTijij,ij
2)
ijj,j2
j,2library(plyr)
data1_study <- ddply(data1, .(Group), function(adf){
mod <- rma(Effect, Variance, data=adf)
cbind(theta_j = coef(mod),
se_theta_j = coef(summary(mod))[1,2],
omega2 = mod$tau2)
})
![Page 11: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/11.jpg)
A Two-Step Solution
Tijij,ij2)
ijj,j2
j,2
> data1_study
Group theta_j se_theta_j omega2
1 A 0.3312500 0.1369306 0.00000000
2 B 0.7005364 0.1654476 0.02854676
3 C 0.6788453 0.1987595 0.17151248
4 D 0.7836646 0.2677693 0.26470540
5 E 0.8552760 0.1556476 0.14561528j j
![Page 12: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/12.jpg)
A Two-Step Solution
Tijij,ij2)
ijj,j2
j,2
> rma(theta_j, I(se_theta_j^2), data=data1_study)
Random-Effects Model (k = 5; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.0272 (SE = 0.0414)
...
estimate se zval pval ci.lb ci.ub
0.6472 0.1087 5.9545 <.0001 0.4342 0.8603 ***
2
![Page 13: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/13.jpg)
Multiple Effects per Research Group
![Page 14: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/14.jpg)
Solutions to Multiple Hierarchies
• Multiple-Step Meta-analyses
• Multi-level hierarchical model fits– Better estimator– Accommodates more complex data
structures–May need to go Bayesian (don't be scared!)
• Model correlation…
![Page 15: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/15.jpg)
Common Forms of Dependence
• Multiple effects per study– Or per research group!
• Multiple effect sizes using same control
• Phylogenetic non-independence
• Measurements of multiple responses to a common treatment
• Unknown correlations…
![Page 16: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/16.jpg)
Multiple Effect Sizes with Common Control
Effect of each treatment calculated using same control!
![Page 17: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/17.jpg)
The Control Keeps Showing Up!
• nc and sdc are going to be the same for all treatments
• Effect sizes will covary
![Page 18: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/18.jpg)
Calculating Covariance
Formulae available or derivable for all effect sizes
![Page 19: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/19.jpg)
A Mixed Effect Group Model
• Group means, random study effect, and then everything else is error
Tiim,i2)
where
imm,2
![Page 20: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/20.jpg)
A Mixed Effect Group Model
• Group means, random study effect, and then everything else is error
TiMVNi,i)
where
iMVN Xi, 2
![Page 21: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/21.jpg)
What are i and i?
i =
i=
TiMVNi,i)
![Page 22: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/22.jpg)
What about the treatment effects?
Xi =
i= iMVN Xi, 2
![Page 23: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/23.jpg)
What if treatments are correlated?
i =
TiMVNi,i)
![Page 24: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/24.jpg)
Why does covariance matter?
x-y =
x + y + 2
x,y
• In asking if two treatments differ, cov helps tighten confidence intervals
• High cov more weight for a study as treatments share information
![Page 25: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/25.jpg)
Multiple Treatments
study trt m1i m2i sdpi n1i n2i
1 1 1 7.87 -1.36 4.2593 25 25
2 1 2 4.35 -1.36 4.2593 22 25
3 2 1 9.32 0.98 2.8831 38 40
4 3 1 8.08 1.17 3.1764 50 50
5 4 1 7.44 0.45 2.9344 30 30
6 4 2 5.34 0.45 2.9344 30 30CommonControl!
http://www.metafor-project.org/doku.php/analyses:gleser2009
![Page 26: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/26.jpg)
Calculating the Variance/Covariance Matrix
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.113 0.060 0.000 0.000 0.000 0.000
[2,] 0.060 0.098 0.000 0.000 0.000 0.000
[3,] 0.000 0.000 0.105 0.000 0.000 0.000
[4,] 0.000 0.000 0.000 0.064 0.000 0.000
[5,] 0.000 0.000 0.000 0.000 0.098 0.055
[6,] 0.000 0.000 0.000 0.000 0.055 0.082
http://www.metafor-project.org/doku.php/analyses:gleser2009
![Page 27: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/27.jpg)
Fitting a Model with a VCOV Matrix
> rma.mv(yi ~ factor(trt)-1,
V,
random =~ 1|study,
data=dat)
![Page 28: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/28.jpg)
Comparison to No Correlation Model
With correlation estimate se zval pval ci.lb ci.ub
factor(trt)1 2.3796 0.1641 14.4984 <.0001 2.0579 2.7013
factor(trt)2 1.5784 0.2007 7.8662 <.0001 1.1851 1.9716
Without correlation estimate se zval pval ci.lb ci.ub
factor(trt)1 2.3759 0.1511 15.7196 <.0001 2.0797 2.6722
factor(trt)2 1.5177 0.2125 7.1405 <.0001 1.1011 1.9343
![Page 29: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/29.jpg)
Common Forms of Dependence
• Multiple effects per study– Or per research group!
• Multiple effect sizes using same control
• Phylogenetic non-independence
• Measurements of multiple responses to a common treatment
• Unknown correlations…
![Page 30: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/30.jpg)
Effect Size on Related Organisms Not Independent
Warming onLitterfall
Pine TreesRedwoodsFir Trees
Oak Trees{
![Page 31: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/31.jpg)
Phylogenetic Distances Determines Covariances for
Weights
![Page 32: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/32.jpg)
![Page 33: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/33.jpg)
What about Multiple Studies of Some Species?
![Page 34: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/34.jpg)
Common Forms of Dependence
• Multiple effects per study– Or per research group!
• Multiple effect sizes using same control
• Phylogenetic non-independence
• Measurements of multiple responses to a common treatment
• Unknown correlations…
![Page 35: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/35.jpg)
Common Treatments
Treatment
Response 1 Response 2 Response 3
![Page 36: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/36.jpg)
Common Treatments
CO2
CO2 Assimilation
GS
Stomatal Conductance
PN
![Page 37: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/37.jpg)
Correlation Between Responses
![Page 38: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/38.jpg)
What does Correlation between effects mean?
Xi =
i= iMVN Xi, 2
![Page 39: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/39.jpg)
What Do We Do?1. Create a 'composite' measure
– Average
– Weighted Average
2. Estimate different coefficients directly
3. Robust Variance Estimation (RVE)
![Page 40: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/40.jpg)
The CO2 Effect Data
experiment Paper Measurement Hedges Var
1 1 121 GS -0.4862 0.3432
2 1 121 PN 0.9817 0.3735
3 2 121 GS 0.1535 0.3343
4 2 121 PN 2.0668 0.5113
5 3 121 GS 0.0965 0.3337
6 3 121 PN 2.6101 0.6172
7 4 121 GS 0.0000 0.2857
8 4 121 PN 3.6586 0.7638
9 5 168 GS -1.5271 0.4305
10 5 168 PN 1.8355 0.4737
![Page 41: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/41.jpg)
Direct Estimation
rma.mv(Hedges ~ Measurement,
Var,
random =~ Measurement|Paper,
data=co2data,
struct="HCS")
![Page 42: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/42.jpg)
and Different Correlation Structures
• Different structures for different data
• We do not always know which one is correct!
![Page 43: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/43.jpg)
Estimates of Variance, Covariance
Multivariate Meta-Analysis Model (k = 68; method: REML)
Variance Components:
outer factor: Paper (nlvls = 18)
inner factor: Measurement (nlvls = 2)
estim sqrt k.lvl fixed level
tau^2.1 4.5098 2.1236 34 no GS
tau^2.2 3.5799 1.8921 34 no PN
rho 0.4751 no
![Page 44: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/44.jpg)
Disadvantages to Multivariate Meta-Analysis
1. Difficult to estimate with few studies
2. Additional assumptions of covariance structure
3. Often little improvement over univariate meta-analysis
4. Publication bias exacerbated if data not missing at random
Jackson et al. 2011 Satist. Med.
![Page 45: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/45.jpg)
Robust Variance Estimation
• Essentially, bound weights within a group j to 1/mean varj and assume a value of
– Test sensitivity to choice of
– Correct DF for small sample sizes
• Methods developed by Hedges, Tipton, and others
• robumeta package in R
![Page 46: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/46.jpg)
robumeta & RVE
library(robumeta)
robu(Hedges ~ Measurement, data=co2data,
studynum=Paper,
var.eff.size=Var)
![Page 47: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/47.jpg)
RVE: Correlated Effects Model with Small-Sample Corrections
Model: Hedges ~ Measurement
Number of studies = 18
Number of outcomes = 68 (min = 2 , mean = 3.78 , median = 4 , max = 10 )
Rho = 0.8
I2 = 85.59992
Tau.Sq = 2.561661
Struct="CS" only so far
![Page 48: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/48.jpg)
Often, Choice of Matters Little
> sensitivity(co2modRVE)
Type Variable rho=0 rho=0.2 rho=0.4 rho=0.6 rho=0.8 rho=1
1 Estimate intercept 0.00454 0.00457 0.00459 0.00462 0.00464 0.00467
2 - MeasurementPN 1.03149 1.03139 1.03128 1.03118 1.03107 1.03097
3 Std. Err. intercept 0.51173 0.51179 0.51185 0.51192 0.51198 0.51204
4 - MeasurementPN 0.61984 0.61990 0.61996 0.62003 0.62009 0.62015
5 Tau.Sq - 2.55334 2.55542 2.55750 2.55958 2.56166 2.56374
![Page 49: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/49.jpg)
Results May Differ…
Multivariate Meta-AnalysisModel Results:
estimate se zval pval ci.lb ci.ub
intrcpt -0.0503 0.5221 -0.0963 0.9233 -1.0735 0.9730
MeasurementPN 1.0579 0.5359 1.9742 0.0484 0.0076 2.1082 *
Robust Variance EstimationModel Results:
Estimate StdErr t-value df P(|t|>) 95% CI.L 95% CI.U Sig
1 intercept 0.00464 0.512 0.00907 16.7 0.993 -1.077 1.09
2 MeasurementPN 1.03107 0.620 1.66278 16.7 0.115 -0.279 2.34
![Page 50: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/50.jpg)
Other Sources of Unknown Correlations
• Shared system types
• Shared environmental events
• Labs or investigators
• Re-sampling experiments
• Experiments repeated in a region
• More…
![Page 51: Meta-Analysis of Correlated Data. Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same](https://reader030.vdocument.in/reader030/viewer/2022032802/56649e155503460f94aff04e/html5/thumbnails/51.jpg)
Why Model Correlation instead of Hierarchy?
• Depends on question
• Analytical difficulty
• Leveraging correlation to aid with missing data