![Page 1: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/1.jpg)
1
Meta-analysis with missing data: metamiss
Ian White and Julian HigginsMRC Biostatistics Unit, Cambridge, UK
Stata users’ group, London
10 September 2007
![Page 2: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/2.jpg)
2
Motivation
• Missing outcome data compromise trials
• So they also compromise meta-analyses
• We may want to – correct for bias due to missing data– down-weight trials with more missing data
• NB missing data within trials, not missing trials
![Page 3: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/3.jpg)
3
Plan
Meta-analysis of binary data
• Haloperidol example
• Standard approaches to missing data
• Imputation methods
• IMORs
• Methods that allow for uncertainty
• Demonstration
![Page 4: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/4.jpg)
4
Haloperidol meta-analysis Haloperidol Placebo
% missing r1 f1 m1 n1 r2 f2 m2 n2
Arvanitis 25 25 2 52 18 33 0 51 2%
Beasley 29 18 22 69 20 14 34 68 41%
Bechelli 12 17 1 30 2 28 1 31 3%
Borison 3 9 0 12 0 12 0 12 0%
Chouinard 10 11 0 21 3 19 0 22 0%
Durost 11 8 0 19 1 14 0 15 0%
Garry 7 18 1 26 4 21 1 26 4%
Howard 8 9 0 17 3 10 0 13 0%
Marder 19 45 2 66 14 50 2 66 3%
Nishikawa 82 1 9 0 10 0 10 0 10 0%
Nishikawa 84 11 23 3 37 0 13 0 13 6%
Reschke 20 9 0 29 2 9 0 11 0%
Selman 17 1 11 29 7 4 18 29 50%
Serafetinides 4 10 0 14 0 13 1 14 4%
Simpson 2 14 0 16 0 7 1 8 4%
Spencer 11 1 0 12 1 11 0 12 0%
Vichaiya 9 20 1 30 0 29 1 30 3%
r=successes
f=failures
m=missing
n=total
![Page 5: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/5.jpg)
5
Standard approaches to missing data
• Available cases (complete cases): ignore the missing data– assumes MAR: missingness is independent of
outcome given arm
• Assume missing=failure– implausible, but not too bad for health-related
behaviours
• Neither assumption is likely to be correct
![Page 6: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/6.jpg)
6
Other ideas• Sensitivity analyses, e.g. do both
missing=failure and available cases– but these could agree by chance
• Explore best / worst cases• Use reasons for missingness• Explicit assumptions about informative
missingness (IM)– IM: missingness is dependent on outcome
![Page 7: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/7.jpg)
7
metamiss.ado• Processes data on successes, failures and missing by arm
& feeds results to metan• Available cases analysis (ACA)• Imputed case analyses (ICA):
– impute as failure: ICA-0– impute as success: ICA-1– best-case: ICA-b (missing=success in E, failure in C)– worst-case: ICA-w– impute with same probability as in control arm: ICA-pC– impute with same probability as in experimental arm: ICA-pE– impute with same probability as in own arm: ICA-p (agrees with
ACA)– impute using IMORs: ICA-IMOR (see next slide)
![Page 8: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/8.jpg)
8
More general imputation: IMORs• Measure Informative Missingness using the
Informative Missing Odds Ratio (IMOR):– Odds ratio between outcome and missingness
• Can’t estimate IMOR from the data, but given any value of IMOR, we can analyse the data
• Generalises other ideas: e.g. – ICA-0 uses IMORs 0, 0– ICA-1 uses IMORs , – ICA-b uses IMORs , 0– ICA-p uses IMORs 1, 1– ICA-pC uses IMORs OR, 1 where OR is odds ratio
between arm and outcome in available cases
![Page 9: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/9.jpg)
9
Getting standard errors (weighting) right
• Weight 1: treat imputed data as real• Weight 2: use standard errors from ACA• Weight 3: scale imputed data to same sample size
as available cases• Weight 4: algebraic standard errors
– same as weight 1 for ICA-0, ICA-1, ICA-b, ICA-w– same as weight 2 for ICA-p– uses Taylor expansion for ICA-IMOR– for ICA-pC & ICA-pE, we condition on the IMOR (I
can explain…)
![Page 10: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/10.jpg)
10
![Page 11: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/11.jpg)
11
![Page 12: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/12.jpg)
12
Overall (I-squared = 41.4%, p = 0.038)
Arvantis
Vichaiya
Selman
Chouinard
Nishikawa_84
Howard
Serafetinides
Durost
Simpson
Mander
Reschke
Study ID
Garry
Borison_92
Spencer
Beasley
Bechelli
Nishikawa_82
1.57 (1.28, 1.92)
1.42 (0.89, 2.25)
19.00 (1.16, 311.96)
1.48 (0.94, 2.35)
3.49 (1.11, 10.95)
9.20 (0.58, 145.76)
2.04 (0.67, 6.21)
8.40 (0.50, 142.27)
8.68 (1.26, 59.95)
2.35 (0.13, 43.53)
1.36 (0.75, 2.47)
3.79 (1.06, 13.60)
ES (95% CI)
1.75 (0.58, 5.24)
7.00 (0.40, 122.44)
11.00 (1.67, 72.40)
1.05 (0.73, 1.50)
6.21 (1.52, 25.35)
3.00 (0.14, 65.90)
100.00
18.86
0.52
19.11
3.10
0.53
3.27
0.51
1.09
0.48
11.37
2.48
Weight
3.37
%
0.49
1.14
31.22
2.05
0.42
1.57 (1.28, 1.92)
1.42 (0.89, 2.25)
19.00 (1.16, 311.96)
1.48 (0.94, 2.35)
3.49 (1.11, 10.95)
9.20 (0.58, 145.76)
2.04 (0.67, 6.21)
8.40 (0.50, 142.27)
8.68 (1.26, 59.95)
2.35 (0.13, 43.53)
1.36 (0.75, 2.47)
3.79 (1.06, 13.60)
ES (95% CI)
1.75 (0.58, 5.24)
7.00 (0.40, 122.44)
11.00 (1.67, 72.40)
1.05 (0.73, 1.50)
6.21 (1.52, 25.35)
3.00 (0.14, 65.90)
100.00
18.86
0.52
19.11
3.10
0.53
3.27
0.51
1.09
0.48
11.37
2.48
Weight
3.37
%
0.49
1.14
31.22
2.05
0.42
1.1 1 10 100
ACA
![Page 13: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/13.jpg)
13
Overall (I-squared = 25.8%, p = 0.158)
Study ID
Borison_92
Chouinard
Durost
Vichaiya
Selman
Beasley
Nishikawa_82
Howard
Simpson
Nishikawa_84
Mander
Spencer
Garry
Arvantis
Reschke
Bechelli
Serafetinides
1.90 (1.51, 2.39)
ES (95% CI)
7.00 (0.40, 122.44)
3.49 (1.11, 10.95)
8.68 (1.26, 59.95)
19.00 (1.16, 312.42)
2.43 (1.19, 4.96)
1.43 (0.90, 2.27)
3.00 (0.14, 65.90)
2.04 (0.67, 6.21)
2.65 (0.14, 49.42)
8.47 (0.53, 134.46)
1.36 (0.74, 2.47)
11.00 (1.67, 72.40)
1.75 (0.58, 5.27)
1.36 (0.85, 2.17)
3.79 (1.06, 13.60)
6.20 (1.51, 25.40)
9.00 (0.53, 152.93)
100.00
Weight
0.65
4.06
1.42
0.68
10.42
%
25.01
0.56
4.29
0.62
0.70
14.75
1.50
4.38
24.38
3.26
2.67
0.66
1.90 (1.51, 2.39)
ES (95% CI)
7.00 (0.40, 122.44)
3.49 (1.11, 10.95)
8.68 (1.26, 59.95)
19.00 (1.16, 312.42)
2.43 (1.19, 4.96)
1.43 (0.90, 2.27)
3.00 (0.14, 65.90)
2.04 (0.67, 6.21)
2.65 (0.14, 49.42)
8.47 (0.53, 134.46)
1.36 (0.74, 2.47)
11.00 (1.67, 72.40)
1.75 (0.58, 5.27)
1.36 (0.85, 2.17)
3.79 (1.06, 13.60)
6.20 (1.51, 25.40)
9.00 (0.53, 152.93)
100.00
Weight
0.65
4.06
1.42
0.68
10.42
%
25.01
0.56
4.29
0.62
0.70
14.75
1.50
4.38
24.38
3.26
2.67
0.66
1.1 1 10 100
ICA-0
![Page 14: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/14.jpg)
14
Overall (I-squared = 60.3%, p = 0.001)
Spencer
Study ID
Beasley
Serafetinides
Borison_92
Garry
Selman
Mander
Nishikawa_82
Durost
Howard
Chouinard
Nishikawa_84
Bechelli
Arvantis
Reschke
Vichaiya
Simpson
1.16 (1.04, 1.29)
11.00 (1.67, 72.40)
ES (95% CI)
0.93 (0.77, 1.12)
4.00 (0.51, 31.46)
7.00 (0.40, 122.44)
1.60 (0.60, 4.25)
1.12 (0.95, 1.32)
1.31 (0.75, 2.28)
3.00 (0.14, 65.90)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
3.49 (1.11, 10.95)
10.68 (0.68, 167.43)
4.48 (1.42, 14.15)
1.47 (0.93, 2.32)
3.79 (1.06, 13.60)
10.00 (1.36, 73.33)
1.00 (0.11, 9.44)
100.00
0.35
Weight
35.81
0.29
0.15
1.29
47.38
4.01
0.13
0.33
0.99
0.94
0.16
0.93
5.95
0.75
0.31
0.24
%
1.16 (1.04, 1.29)
11.00 (1.67, 72.40)
ES (95% CI)
0.93 (0.77, 1.12)
4.00 (0.51, 31.46)
7.00 (0.40, 122.44)
1.60 (0.60, 4.25)
1.12 (0.95, 1.32)
1.31 (0.75, 2.28)
3.00 (0.14, 65.90)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
3.49 (1.11, 10.95)
10.68 (0.68, 167.43)
4.48 (1.42, 14.15)
1.47 (0.93, 2.32)
3.79 (1.06, 13.60)
10.00 (1.36, 73.33)
1.00 (0.11, 9.44)
100.00
0.35
Weight
35.81
0.29
0.15
1.29
47.38
4.01
0.13
0.33
0.99
0.94
0.16
0.93
5.95
0.75
0.31
0.24
%
1.1 1 10 100
ICA-1
![Page 15: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/15.jpg)
15
Overall (I-squared = 25.9%, p = 0.156)
Study ID
Bechelli
Chouinard
Borison_92
Simpson
Nishikawa_84
Mander
Beasley
Nishikawa_82
Serafetinides
Reschke
Durost
Howard
Selman
Spencer
Arvantis
Garry
Vichaiya
2.42 (1.95, 3.00)
ES (95% CI)
6.72 (1.65, 27.28)
3.49 (1.11, 10.95)
7.00 (0.40, 122.44)
2.65 (0.14, 49.42)
10.68 (0.68, 167.43)
1.50 (0.84, 2.69)
2.51 (1.69, 3.73)
3.00 (0.14, 65.90)
9.00 (0.53, 152.93)
3.79 (1.06, 13.60)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
4.00 (2.09, 7.65)
11.00 (1.67, 72.40)
1.47 (0.93, 2.32)
2.00 (0.69, 5.83)
21.00 (1.29, 342.93)
100.00
Weight
2.37
3.57
0.57
0.54
0.62
13.68
30.05
%
0.49
0.58
2.86
1.25
3.76
11.08
1.31
22.59
4.07
0.60
2.42 (1.95, 3.00)
ES (95% CI)
6.72 (1.65, 27.28)
3.49 (1.11, 10.95)
7.00 (0.40, 122.44)
2.65 (0.14, 49.42)
10.68 (0.68, 167.43)
1.50 (0.84, 2.69)
2.51 (1.69, 3.73)
3.00 (0.14, 65.90)
9.00 (0.53, 152.93)
3.79 (1.06, 13.60)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
4.00 (2.09, 7.65)
11.00 (1.67, 72.40)
1.47 (0.93, 2.32)
2.00 (0.69, 5.83)
21.00 (1.29, 342.93)
100.00
Weight
2.37
3.57
0.57
0.54
0.62
13.68
30.05
%
0.49
0.58
2.86
1.25
3.76
11.08
1.31
22.59
4.07
0.60
1.1 1 10 100
ICA-B
![Page 16: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/16.jpg)
16
Overall (I-squared = 74.2%, p = 0.000)
Serafetinides
Arvantis
Durost
Beasley
Study ID
Simpson
Reschke
Selman
Borison_92
Mander
Nishikawa_82
Vichaiya
Chouinard
Bechelli
Nishikawa_84
Spencer
Garry
Howard
0.94 (0.79, 1.12)
4.00 (0.51, 31.46)
1.36 (0.85, 2.17)
8.68 (1.26, 59.95)
0.53 (0.39, 0.72)
ES (95% CI)
1.00 (0.11, 9.44)
3.79 (1.06, 13.60)
0.68 (0.48, 0.95)
7.00 (0.40, 122.44)
1.19 (0.67, 2.10)
3.00 (0.14, 65.90)
9.00 (1.21, 66.70)
3.49 (1.11, 10.95)
4.13 (1.29, 13.20)
8.47 (0.53, 134.46)
11.00 (1.67, 72.40)
1.40 (0.51, 3.85)
2.04 (0.67, 6.21)
100.00
0.72
13.99
0.82
33.33
Weight
0.60
%
1.87
26.57
0.37
9.35
0.32
0.76
2.33
2.26
0.40
0.86
2.98
2.46
0.94 (0.79, 1.12)
4.00 (0.51, 31.46)
1.36 (0.85, 2.17)
8.68 (1.26, 59.95)
0.53 (0.39, 0.72)
ES (95% CI)
1.00 (0.11, 9.44)
3.79 (1.06, 13.60)
0.68 (0.48, 0.95)
7.00 (0.40, 122.44)
1.19 (0.67, 2.10)
3.00 (0.14, 65.90)
9.00 (1.21, 66.70)
3.49 (1.11, 10.95)
4.13 (1.29, 13.20)
8.47 (0.53, 134.46)
11.00 (1.67, 72.40)
1.40 (0.51, 3.85)
2.04 (0.67, 6.21)
100.00
0.72
13.99
0.82
33.33
Weight
0.60
%
1.87
26.57
0.37
9.35
0.32
0.76
2.33
2.26
0.40
0.86
2.98
2.46
1.1 1 10 100
ICA-W
![Page 17: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/17.jpg)
17
Overall (I-squared = 42.0%, p = 0.035)
Spencer
Nishikawa_82
Study ID
Simpson
Serafetinides
Chouinard
Bechelli
Borison_92
Mander
Arvantis
Vichaiya
Selman
Reschke
Nishikawa_84
Garry
Beasley
Durost
Howard
1.53 (1.24, 1.88)
11.00 (1.67, 72.40)
3.00 (0.14, 65.90)
ES (95% CI)
2.35 (0.13, 43.53)
8.40 (0.50, 142.27)
3.49 (1.11, 10.95)
6.03 (1.47, 24.68)
7.00 (0.40, 122.44)
1.35 (0.74, 2.45)
1.40 (0.88, 2.23)
18.42 (1.12, 302.65)
1.30 (0.76, 2.23)
3.79 (1.06, 13.60)
8.55 (0.54, 135.71)
1.72 (0.57, 5.16)
1.03 (0.72, 1.49)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
100.00
1.21
0.45
Weight
0.51
0.54
3.29
2.17
0.53
%
12.05
19.89
0.55
14.85
2.64
0.56
3.57
32.56
1.15
3.47
1.53 (1.24, 1.88)
11.00 (1.67, 72.40)
3.00 (0.14, 65.90)
ES (95% CI)
2.35 (0.13, 43.53)
8.40 (0.50, 142.27)
3.49 (1.11, 10.95)
6.03 (1.47, 24.68)
7.00 (0.40, 122.44)
1.35 (0.74, 2.45)
1.40 (0.88, 2.23)
18.42 (1.12, 302.65)
1.30 (0.76, 2.23)
3.79 (1.06, 13.60)
8.55 (0.54, 135.71)
1.72 (0.57, 5.16)
1.03 (0.72, 1.49)
8.68 (1.26, 59.95)
2.04 (0.67, 6.21)
100.00
1.21
0.45
Weight
0.51
0.54
3.29
2.17
0.53
%
12.05
19.89
0.55
14.85
2.64
0.56
3.57
32.56
1.15
3.47
1.1 1 10 100
ICA-pc
![Page 18: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/18.jpg)
18
Overall (I-squared = 41.4%, p = 0.038)
Garry
Vichaiya
Howard
Borison_92
Selman
Durost
Nishikawa_82
Reschke
Mander
Chouinard
Simpson
Spencer
Nishikawa_84
Bechelli
Serafetinides
Study ID
Beasley
Arvantis
1.57 (1.28, 1.92)
1.75 (0.58, 5.24)
19.00 (1.16, 311.96)
2.04 (0.67, 6.21)
7.00 (0.40, 122.44)
1.48 (0.94, 2.35)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
3.79 (1.06, 13.60)
1.36 (0.75, 2.47)
3.49 (1.11, 10.95)
2.35 (0.13, 43.53)
11.00 (1.67, 72.40)
9.20 (0.58, 145.76)
6.21 (1.52, 25.35)
8.40 (0.50, 142.27)
ES (95% CI)
1.05 (0.73, 1.50)
1.42 (0.89, 2.25)
100.00
3.37
0.52
3.27
0.49
19.11
1.09
0.42
2.48
11.37
3.10
0.48
1.14
0.53
2.05
0.51
Weight
31.22
18.86
%
1.57 (1.28, 1.92)
1.75 (0.58, 5.24)
19.00 (1.16, 311.96)
2.04 (0.67, 6.21)
7.00 (0.40, 122.44)
1.48 (0.94, 2.35)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
3.79 (1.06, 13.60)
1.36 (0.75, 2.47)
3.49 (1.11, 10.95)
2.35 (0.13, 43.53)
11.00 (1.67, 72.40)
9.20 (0.58, 145.76)
6.21 (1.52, 25.35)
8.40 (0.50, 142.27)
ES (95% CI)
1.05 (0.73, 1.50)
1.42 (0.89, 2.25)
100.00
3.37
0.52
3.27
0.49
19.11
1.09
0.42
2.48
11.37
3.10
0.48
1.14
0.53
2.05
0.51
Weight
31.22
18.86
%
1.1 1 10 100
ICA-p
![Page 19: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/19.jpg)
19
Overall (I-squared = 48.2%, p = 0.014)
Vichaiya
Nishikawa_82
Garry
Bechelli
Simpson
Beasley
Howard
Selman
Borison_92
Nishikawa_84
Reschke
Mander
Durost
Chouinard
Spencer
Study ID
Serafetinides
Arvantis
1.42 (1.19, 1.69)
18.73 (1.14, 306.64)
3.00 (0.14, 65.90)
1.74 (0.59, 5.16)
6.12 (1.51, 24.87)
2.14 (0.12, 38.74)
1.00 (0.74, 1.35)
2.04 (0.67, 6.21)
1.32 (0.93, 1.86)
7.00 (0.40, 122.44)
9.57 (0.61, 151.28)
3.79 (1.06, 13.60)
1.35 (0.75, 2.45)
8.68 (1.26, 59.95)
3.49 (1.11, 10.95)
11.00 (1.67, 72.40)
ES (95% CI)
7.91 (0.47, 132.79)
1.43 (0.90, 2.28)
100.00
0.40
0.33
2.64
1.59
0.37
35.19
2.52
26.22
0.38
0.41
%
1.91
8.89
0.84
2.39
0.88
Weight
0.39
14.67
1.42 (1.19, 1.69)
18.73 (1.14, 306.64)
3.00 (0.14, 65.90)
1.74 (0.59, 5.16)
6.12 (1.51, 24.87)
2.14 (0.12, 38.74)
1.00 (0.74, 1.35)
2.04 (0.67, 6.21)
1.32 (0.93, 1.86)
7.00 (0.40, 122.44)
9.57 (0.61, 151.28)
3.79 (1.06, 13.60)
1.35 (0.75, 2.45)
8.68 (1.26, 59.95)
3.49 (1.11, 10.95)
11.00 (1.67, 72.40)
ES (95% CI)
7.91 (0.47, 132.79)
1.43 (0.90, 2.28)
100.00
0.40
0.33
2.64
1.59
0.37
35.19
2.52
26.22
0.38
0.41
%
1.91
8.89
0.84
2.39
0.88
Weight
0.39
14.67
1.1 1 10 100
ICA-IMOR 2 2
![Page 20: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/20.jpg)
20
Overall (I-squared = 35.0%, p = 0.077)
Spencer
Durost
Simpson
Study ID
Beasley
Nishikawa_82
Reschke
Bechelli
Garry
Howard
Serafetinides
Selman
Vichaiya
Mander
Borison_92
Arvantis
Chouinard
Nishikawa_84
1.70 (1.37, 2.11)
11.00 (1.67, 72.40)
8.68 (1.26, 59.95)
2.49 (0.13, 46.31)
ES (95% CI)
1.12 (0.74, 1.70)
3.00 (0.14, 65.90)
3.79 (1.06, 13.60)
6.23 (1.52, 25.49)
1.75 (0.58, 5.26)
2.04 (0.67, 6.21)
8.68 (0.51, 147.44)
1.74 (0.97, 3.12)
19.06 (1.16, 313.20)
1.36 (0.75, 2.47)
7.00 (0.40, 122.44)
1.40 (0.88, 2.23)
3.49 (1.11, 10.95)
8.91 (0.56, 141.28)
100.00
1.35
1.28
0.56
Weight
27.47
0.50
2.94
2.41
3.96
3.87
%
0.60
14.11
0.61
13.34
0.58
22.12
3.66
0.63
1.70 (1.37, 2.11)
11.00 (1.67, 72.40)
8.68 (1.26, 59.95)
2.49 (0.13, 46.31)
ES (95% CI)
1.12 (0.74, 1.70)
3.00 (0.14, 65.90)
3.79 (1.06, 13.60)
6.23 (1.52, 25.49)
1.75 (0.58, 5.26)
2.04 (0.67, 6.21)
8.68 (0.51, 147.44)
1.74 (0.97, 3.12)
19.06 (1.16, 313.20)
1.36 (0.75, 2.47)
7.00 (0.40, 122.44)
1.40 (0.88, 2.23)
3.49 (1.11, 10.95)
8.91 (0.56, 141.28)
100.00
1.35
1.28
0.56
Weight
27.47
0.50
2.94
2.41
3.96
3.87
%
0.60
14.11
0.61
13.34
0.58
22.12
3.66
0.63
1.1 1 10 100
ICA-IMOR 1/2 1/2
![Page 21: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/21.jpg)
21
Allowing for reasons (ICA-R)
• Specify number of missing individuals in each arm to be imputed by each scheme ICA-0, ICA-1, ICA-pC, ICA-pE, ICA-p, ICA-IMOR.
• Can take these data from a different outcome: metamiss scales to #missing
• If missing in a particular study, metamiss imputes using combined studies
![Page 22: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/22.jpg)
22
![Page 23: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/23.jpg)
23
Allowing for uncertainty
• So far we have pretended we really know the IMORs
• This is never really correct
• Now we allow them to be unknown but from a user-specified distribution
![Page 24: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/24.jpg)
24
Bayesian approach allowing for uncertain IMORs (Rubin, 1977)
E C
2E E E E
2E
E
E
Prior for , = log(IMOR) in experimental, control arm:
N ,
, measure your best guess about IM;
, measure your uncertainty about IM;
measu
C
C C C C
C
C
E Cres how similar you think , are:
0 is most conservative, 1 often allows little impact of IM on results.
![Page 25: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/25.jpg)
25
Bayesian analysis
• Elicit prior for E, C or use N(0,12) or N(0,22)• Get posterior distribution by integrating over the 2-
dimensional distribution of E, C. • metamiss does this fast & accurately by:
1. Standard normal approximation to posterior given E, C 2. Integrate using Gauss-Hermite quadrature.
• Alternatives: – Taylor expansion (inaccurate for large SD of log IMOR)– Full Bayesian Monte Carlo (slow, little gain in accuracy)
![Page 26: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/26.jpg)
26
Understanding priors for log IMOR: implied prior for P(success | missing)
when P(success | observed) = 1/2D
ensi
ty
0 .25 .5 .75 1P(success | missing)
N(0,0.5^2) N(0,2^2) N(-1,0.5^2) N(-1,2^2)
![Page 27: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/27.jpg)
27
![Page 28: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/28.jpg)
28
Overall (I-squared = 31.4%, p = 0.106)
Study ID
Borison_92
Simpson
Chouinard
Mander
Bechelli
Selman
Serafetinides
Spencer
Nishikawa_84
Garry
Durost
Nishikawa_82
Beasley
Reschke
Arvantis
Vichaiya
Howard
1.75 (1.38, 2.22)
ES (95% CI)
7.00 (0.40, 122.44)
2.27 (0.12, 42.39)
3.49 (1.11, 10.95)
1.35 (0.74, 2.47)
6.14 (1.50, 25.13)
1.54 (0.82, 2.88)
8.17 (0.48, 139.06)
11.00 (1.67, 72.40)
9.25 (0.58, 146.78)
1.74 (0.58, 5.23)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
1.06 (0.61, 1.85)
3.79 (1.06, 13.60)
1.42 (0.89, 2.25)
18.74 (1.14, 308.29)
2.04 (0.67, 6.21)
100.00
Weight
%
0.68
0.65
4.26
15.46
2.80
14.02
0.69
1.57
0.73
4.60
1.49
0.58
18.07
3.41
25.78
0.71
4.49
1.75 (1.38, 2.22)
ES (95% CI)
7.00 (0.40, 122.44)
2.27 (0.12, 42.39)
3.49 (1.11, 10.95)
1.35 (0.74, 2.47)
6.14 (1.50, 25.13)
1.54 (0.82, 2.88)
8.17 (0.48, 139.06)
11.00 (1.67, 72.40)
9.25 (0.58, 146.78)
1.74 (0.58, 5.23)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
1.06 (0.61, 1.85)
3.79 (1.06, 13.60)
1.42 (0.89, 2.25)
18.74 (1.14, 308.29)
2.04 (0.67, 6.21)
100.00
Weight
%
0.68
0.65
4.26
15.46
2.80
14.02
0.69
1.57
0.73
4.60
1.49
0.58
18.07
3.41
25.78
0.71
4.49
1.1 1 10 100
logimor ~ N(0,1)
![Page 29: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/29.jpg)
29
Overall (I-squared = 23.6%, p = 0.181)
Serafetinides
Vichaiya
Study ID
Nishikawa_84
Selman
Durost
Nishikawa_82
Arvantis
Bechelli
Simpson
Garry
Howard
Borison_92
Reschke
Mander
Chouinard
Beasley
Spencer
1.87 (1.44, 2.41)
7.61 (0.43, 133.33)
17.81 (1.06, 298.38)
ES (95% CI)
9.32 (0.59, 148.16)
1.60 (0.67, 3.80)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
1.42 (0.89, 2.26)
5.98 (1.45, 24.71)
2.10 (0.11, 40.70)
1.73 (0.57, 5.22)
2.04 (0.67, 6.21)
7.00 (0.40, 122.44)
3.79 (1.06, 13.60)
1.35 (0.74, 2.47)
3.49 (1.11, 10.95)
1.08 (0.51, 2.32)
11.00 (1.67, 72.40)
100.00
0.80
0.83
Weight
%
0.86
8.77
1.77
0.69
30.37
3.28
0.75
5.40
5.32
0.81
4.04
18.04
5.05
11.36
1.86
1.87 (1.44, 2.41)
7.61 (0.43, 133.33)
17.81 (1.06, 298.38)
ES (95% CI)
9.32 (0.59, 148.16)
1.60 (0.67, 3.80)
8.68 (1.26, 59.95)
3.00 (0.14, 65.90)
1.42 (0.89, 2.26)
5.98 (1.45, 24.71)
2.10 (0.11, 40.70)
1.73 (0.57, 5.22)
2.04 (0.67, 6.21)
7.00 (0.40, 122.44)
3.79 (1.06, 13.60)
1.35 (0.74, 2.47)
3.49 (1.11, 10.95)
1.08 (0.51, 2.32)
11.00 (1.67, 72.40)
100.00
0.80
0.83
Weight
%
0.86
8.77
1.77
0.69
30.37
3.28
0.75
5.40
5.32
0.81
4.04
18.04
5.05
11.36
1.86
1.1 1 10 100
logimor ~ N(0,2^2)
![Page 30: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/30.jpg)
30
Proposal: 4 sensitivity analysesIMORs Options
(e.g.)Sensitive to: Works via:
fixed equal
imor(2 2) Imbalance in missingness
Point estimates
fixed opposite
imor(2 1/2) Amount of missing data
random equal
sdlogimor(2) corr(1)
Imbalance in missingness
Weightings
random uncorrelated
sdlogimor(2) corr(0)
Amount of missing data
![Page 31: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/31.jpg)
31
Summary• Tool for sensitivity analysis• Requires thought about plausible missing data
mechanisms• Would be nice to overlay sensitivity analysis
with ACA• Further work includes combining uncertainty
with reasons• I also have a program mvmeta for
multivariate meta-analysis
![Page 32: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/32.jpg)
32
References• 1st part: Higgins JPT, White IR, Wood A. Imputation methods
for missing outcome data in meta-analysis of clinical trials. Clinical Trials, submitted.
• 2nd part: White IR, Higgins JPT, Wood AM. Allowing for uncertainty due to missing data in meta-analysis. 1. Two-stage methods. Statistics in Medicine, in press.
• Related: White IR, Welton NJ, Wood AM, Ades AE, Higgins JPT. Allowing for uncertainty due to missing data in meta-analysis. 2. Hierarchical models. Statistics in Medicine, in press.
• metamiss.ado available from http://www.mrc-bsu.cam.ac.uk/BSUsite/Software/Stata.shtml
![Page 33: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/33.jpg)
33
Extra slides
![Page 34: 1 Meta-analysis with missing data: metamiss Ian White and Julian Higgins MRC Biostatistics Unit, Cambridge, UK Stata users’ group, London 10 September](https://reader033.vdocument.in/reader033/viewer/2022051516/56649d245503460f949fad46/html5/thumbnails/34.jpg)
34
Overall (I-squared = 16.7%, p = 0.257)
Vichaiya
Howard
Nishikawa_82
Spencer
Reschke
Serafetinides
Selman
Garry
Borison_92
Mander
Bechelli
Simpson
Nishikawa_84
Study ID
Durost
Beasley
Chouinard
Arvantis
2.02 (1.51, 2.70)
19.00 (1.13, 318.79)
2.04 (0.67, 6.21)
3.00 (0.14, 65.90)
11.00 (1.67, 72.40)
3.79 (1.06, 13.60)
8.40 (0.50, 140.57)
1.48 (0.37, 5.90)
1.75 (0.52, 5.92)
7.00 (0.40, 122.44)
1.36 (0.68, 2.72)
6.21 (1.35, 28.50)
2.35 (0.12, 44.54)
9.20 (0.52, 162.90)
ES (95% CI)
8.68 (1.26, 59.95)
1.05 (0.34, 3.24)
3.49 (1.11, 10.95)
1.42 (0.86, 2.33)
100.00
1.05
6.75
0.88
2.36
5.13
1.05
4.39
5.63
1.02
17.36
3.60
0.97
1.01
Weight
2.24
6.58
6.40
33.58
%
2.02 (1.51, 2.70)
19.00 (1.13, 318.79)
2.04 (0.67, 6.21)
3.00 (0.14, 65.90)
11.00 (1.67, 72.40)
3.79 (1.06, 13.60)
8.40 (0.50, 140.57)
1.48 (0.37, 5.90)
1.75 (0.52, 5.92)
7.00 (0.40, 122.44)
1.36 (0.68, 2.72)
6.21 (1.35, 28.50)
2.35 (0.12, 44.54)
9.20 (0.52, 162.90)
ES (95% CI)
8.68 (1.26, 59.95)
1.05 (0.34, 3.24)
3.49 (1.11, 10.95)
1.42 (0.86, 2.33)
100.00
1.05
6.75
0.88
2.36
5.13
1.05
4.39
5.63
1.02
17.36
3.60
0.97
1.01
Weight
2.24
6.58
6.40
33.58
%
1.1 1 10 100
Gamble-Hollis