meta-regression with dismod-mr: how robust is the model?
Upload: institute-for-health-metrics-and-evaluation-university-of-washington
Post on 27-Jan-2015
121 views
DESCRIPTION
GHME 2013 Conference Session: Dismod MR workshop Date: June 18 2013 Presenter: Hannah Peterson Institute: Institute for Health Metrics and Evaluation (IHME), University of WashingtonTRANSCRIPT
![Page 1: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/1.jpg)
Meta-regression with DisMod-MR: how robust is the model?
June 18, 2013
Hannah M Peterson
Post-Bachelor Fellow
![Page 2: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/2.jpg)
2
Global Burden of Disease Study 2010
![Page 3: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/3.jpg)
3
YLDs• Measures morbidity
• Requires age-specific prevalenceo For 291 outcomes
o For 2 sexes
o For 187 countries
o For 3 years
![Page 4: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/4.jpg)
4
Is negative-binomial distribution the best choice?
DisMod-MR
![Page 5: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/5.jpg)
5
Alternative distributions
Distribution Probability Density Function
Normal
Lognormal
Binomial
Negative-binomial
![Page 6: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/6.jpg)
6
Alternative distributions
Distribution Probability Density Function
Normal
Lognormal
Binomial
Negative-binomial
![Page 7: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/7.jpg)
7
Alternative distributions
Distribution Probability Density Function
Normal
Lognormal
Binomial
Negative-binomial
![Page 8: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/8.jpg)
8
Alternative distributions
Distribution Probability Density Function
Normal
Lognormal
Binomial
Negative-binomial
![Page 9: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/9.jpg)
9
Potential experimental frameworks
• Data collectiono Ideal
o Impractical
• Simulationo Impossible to know true data distribution
• Out-of-sample cross validationo Do not have to choose distribution
![Page 10: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/10.jpg)
10
Out-of-sample cross validation
![Page 11: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/11.jpg)
11
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
![Page 12: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/12.jpg)
12
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
![Page 13: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/13.jpg)
13
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
• Fit the remaining 75% of data (“training data”)
![Page 14: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/14.jpg)
14
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
• Fit the remaining 75% of data (“training data”)
• Use fit to calculate statistics for test data
![Page 15: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/15.jpg)
15
Out-of-sample predictive validity
• Randomly select 25% of data to use as “test data”
• Fit the remaining 75% of data (“training data”)
• Use fit to calculate statistics for test data
• For each distribution
• For 1000 test-train splits
• For each disease data set
![Page 16: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/16.jpg)
16
Comparing distributions
How to determine the best distribution?
![Page 17: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/17.jpg)
17
Metrics of evaluation
• Biaso Measures the average difference between observation and estimate
• Median absolute error (MAE)o Measure of overall magnitude of error
• Percent coverage (PC)o Percent of time estimate uncertainty interval contains observation
![Page 18: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/18.jpg)
18
Results
Percent of wins (%)
Distribution Bias MAE PC Total
Normal 22.1 20.6 34.6 25.7
Lognormal 29.7 13.0 36.5 26.4
Binomial 26.3 48.3 1.9 25.5
Negative-binomial 21.9 18.1 27.1 22.4
![Page 19: Meta-regression with DisMod-MR: how robust is the model?](https://reader033.vdocument.in/reader033/viewer/2022061223/54c67b334a7959b6298b464d/html5/thumbnails/19.jpg)
19
Conclusions
• Choice of distribution doesn’t greatly influence results
• Best overall performance: lognormal distribution
o Contingent on method to adjust data whose value is 0
• Further investigate when each distribution performs best
o Dependent on number of covariates, priors, amount of data?