dengue forecasting · 2020. 10. 29. · centers for disease control and prevention dengue...
TRANSCRIPT
![Page 1: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/1.jpg)
Centers for Disease Control and Prevention
Dengue forecastingModel and challenges
Michael A. JohanssonEpidemic Analytics Unit, Dengue Branch, Division of Vector-Borne Diseases
San Juan, Puerto Rico
![Page 2: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/2.jpg)
Dengue – San Juan, Puerto RicoN
umbe
r of d
engu
e ca
ses
![Page 3: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/3.jpg)
Case
s
Week
![Page 4: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/4.jpg)
Case
s
Week
week8
2012/2013
![Page 5: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/5.jpg)
Case
s
Week
week16
2012/2013
![Page 6: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/6.jpg)
Case
s
Week
week24
2012/2013
![Page 7: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/7.jpg)
Case
s
Week
week32
2012/2013
![Page 8: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/8.jpg)
Case
s
Week
week40
2012/2013
![Page 9: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/9.jpg)
Case
s
Week
week48Evaluate forecasts on out-of-sample data (over multiple years for dengue).
2012/2013
![Page 10: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/10.jpg)
Mean
Johansson et al. Scientific Reports 2016
Dengue forecast error - Mexico
![Page 11: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/11.jpg)
Mean
Monthly mean
Johansson et al. Scientific Reports 2016
Dengue forecast error - Mexico
![Page 12: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/12.jpg)
Mean
Monthly meanTemperature
Johansson et al. Scientific Reports 2016
Dengue forecast error - Mexico
![Page 13: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/13.jpg)
Mean
Monthly mean
Autocorrelation
Temperature
Johansson et al. Scientific Reports 2016
Dengue forecast error - Mexico
![Page 14: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/14.jpg)
Mean
Monthly mean
Autocorrelation
Temperature
Autocorrelation + seasonality
Johansson et al. Scientific Reports 2016
Dengue forecast error - MexicoCompare to a baseline model.
![Page 15: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/15.jpg)
Forecasts - Mexico
Johansson et al. Scientific Reports 2016
Assess the uncertainty.
![Page 16: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/16.jpg)
BS Checklist for Forecasts□ Evaluate forecasts on out-of-sample data.□ Compare to a baseline model.□ Assess the uncertainty.
![Page 17: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/17.jpg)
Dengue forecasting research
“[poor prediction was] the result of the unusual behavior that occurred between 2009 and 2011”
![Page 18: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/18.jpg)
The state of dengue forecasting Many models Mostly retrospective Varying targets & evaluation metrics Little sense of appropriateness of models
for decision-making No quantitative models being routinely
used for decision-making
As of 2010: 60+ published mechanistic dengue models
Reiner & Perkins et al. J R Soc Interface 2013, Chretien et al. PLOS ONE 2014
![Page 19: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/19.jpg)
Dengue Forecasting Project Pandemic Prediction & Forecasting Science & Technology Working Group June–September, 2015
Targets: Peak week, peak incidence, and total incidence over 8 seasons in Iquitos, Peru and San Juan, Puerto Rico
16 teams; 10,000 forecasts dengueforecasting.noaa.gov, predict.cdc.gov
![Page 20: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/20.jpg)
Correlation of point forecasts is not enough
We need to assess both accuracy and confidence (i.e. certainty/uncertainty).
![Page 21: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/21.jpg)
Forecast Peak Week
ObservedPeak Week Error (weeks)
Team A 23 32 9
Team B 23 32 9
Team C 22 32 10
Error metrics are simple and straightforward
![Page 22: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/22.jpg)
Probabilistic forecasts have more informationTeam A Team B Team C
Point prediction Point predictionPoint prediction
![Page 23: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/23.jpg)
Assessing probabilistic forecastsTeam A Team B Team C
Observed peakObserved peak
Observed peak
p = 0.02 p = 0.04 p = 0
Point prediction Point predictionPoint prediction
![Page 24: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/24.jpg)
Forecast calibration
Well-calibrated Over-confident Under-confident No resolutionNo confidence
![Page 25: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/25.jpg)
Week 12 forecast for San Juan 2012/2013
![Page 26: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/26.jpg)
When are forecasts best?~12,000 forecasts 2 locations 8 seasons 19 models
![Page 27: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/27.jpg)
Nowcast/situational awarenessSEASONAL DENGUEPeak week forecasts
SHORT-TERM INFLUENZA1- to 4-week ahead forecasts
Ensemble
Historical average
![Page 28: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/28.jpg)
Promising approaches Simpler models
– No climate data (dengue)– No vector model (dengue)
Ensembles– Simple ensembles (across targets,
seasons, & diseases)– Prospectively defined– Current standard for influenza
(since 2017/18)
DENGUE INFLUENZA
Individual modelEnsemble
![Page 29: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/29.jpg)
Key questions What are the key surveillance data? How much do vectors matter? What is the contribution of weather? What is the role of immunity and enhancement? What is the role of mobility and spatial heterogeneity?
![Page 30: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/30.jpg)
Conclusions
![Page 31: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/31.jpg)
“Dengue is a disease of the tropical and subtropical regions, and within these zones it has a marked preference for the hot season - for summer.”
- Hermann Nothnagel, 1905
![Page 32: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/32.jpg)
“It is difficult to make predictions, especially about the future.”
Past Present Future
Surveillance(Hindcast)
Nowcast
Forecast
Unc
erta
inty
Time
![Page 33: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/33.jpg)
https://www.nhc.noaa.gov/verification/verify5.shtml
![Page 34: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/34.jpg)
How can infectious disease forecasting improve?(How has weather forecasting improved?) Data Analytical tools Computational power Evaluation Standardization & interoperability
![Page 35: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/35.jpg)
CDC Epidemic Prediction Initiative Connect researchers to data
– Dengue, influenza (github.com/cmu-delphi/delphi-epidata), Zika (github.com/cdcepi/zika)
Develop an analytical pipeline– predict.cdc.gov– Current: Influenza, Aedes
Build a community– Centers for Disease Control and Prevention, Researchers, Multiple US
Departments & Agencies, Council of State and Territorial Epidemiologists
![Page 36: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/36.jpg)
Conclusions Surveillance and forecasting go hand in hand. Current forecasting methods improve upon expert knowledge
and can be helpful for situational awareness. Improved analytics can improve our ability to predict and
respond effectively to arboviral disease epidemics.
![Page 37: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/37.jpg)
Key considerations Connect forecasts to decision making needs. Evaluate forecasts on out-of-sample data. Compare to a baseline model. Assess the uncertainty (including calibration). Use more than one model. Use forecasts as one input for decision making.
![Page 38: Dengue forecasting · 2020. 10. 29. · Centers for Disease Control and Prevention Dengue forecasting Model and challenges Michael A. Johansson. Epidemic Analytics Unit, Dengue Branch,](https://reader036.vdocument.in/reader036/viewer/2022081623/613f7b49f0f55d448e4cd291/html5/thumbnails/38.jpg)
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
For more information, contact CDC1-800-CDC-INFO (232-4636)TTY: 1-888-232-6348 www.cdc.gov
AcknowledgementsThe Epidemic Prediction Initiative community
CDC Epidemic Prediction Initiative
Matt BiggerstaffCraig McGowanJuan Sanchez MontalvoLuis Mier-y-Teran RomeroDania Rodriguez VargasF. Scott DahlgrenChelsea Lutz
Office of Public Health Preparedness and ResponseDivision of Vector-Borne DiseasesInfluenza DivisionCouncil of State and Territorial Epidemiologists
Nicholas Reich (University of Massachusetts Amherst)Aditi Hota (Harvard University)Mauricio Santillana (Harvard University)John Brownstein (Boston Children’s Hospital)