Some challenges to make current data-driven (‘statistical’) models even more

relevant to public health

Ottar Bjornstad Center for Infectious Disease Dynamics,

Penn State University

Focus on time series analysis of incidence analysis

0

50k

100k

150k

200k

250k

300k

350k

400k

‘44 ‘55

Quarterly measles incidence 44-55

0-1y1-3y3-5y

5-10y10-15y

15- y

Outline:

~ 1993

then –> now

current challenges

mostly anecdotal personal reflections

~ 1993

~1993 maturing* mathematical formalism incorporating: (cf yesterdays talks by Mick and Val)

– Seasonal forcing

– Age-structure & non-homogenous mixing

– Spatial diffusion & metapopulation dynamics

– Plausible scenarios of scaling of transmission with pop size

– Stochasticity

But, early days w.r.t. letting these general models loose on data …

* for directly transmitted persistent (SI), fully immunizing (SIR) or fully non-immunizing (SIS)

~1993 Early days w.r.t. letting these general models loose on data …

… because of many challenges

The obvious:

–Absence of data on key state variables (eg susceptible)

–Disparity between key state variables and observed quantities (eg incidence is not prevalence)

The less obvious:

–Weariness regarding whether a detailed quantitative match to data should be a critical characteristic of mathematical models

then –> now

Today’s expectation for match (1)

eg TSIR forecast for E&W measles (E&W)

IncidenceTSIR forecast

),(~1

1

1

ttt

tttt

tttt

INegBinI

IS

IBSS

S - Susceptibles, B - Births , I - Infected and Infective, - epidemic intensity, - correction for time discretization, β - seasonal transmission rate

Host dynamics

Transmission dynamics

Stochasticity

TSIR: Discrete time ‘piecewise constant’ B-D process (cf Chain-binomial)If l is small relative to S, then the chain can be approximated by an unconstrained B-D process; The conditional distribution of It+1 is the sum of It Geometric distributions -> NegBin with clumping It

Today’s expectation for match (2)

eg age-structured TSIR forecast for rubella (South Africa)

Metcalf et al 2013

* difference in heat intensity is due to underreporting

*

Yesteryear’s expectations

Kot and Schaffer (1985) JTB:

“One way of resolving the problem is to view the motion in phase space,i.e. in a vector space whose axes are the state independent variables.However, for most real world ecological and epidemiological systems, thisrequirement is not easily met. It is often difficult even to enumerate all ofthe state variables, much less to follow their magnitudes over time. Put another way, the variables studied in nature are generally embedded in more complex systems. As a practical matter, it is unlikely that population dynamicists will ever be able to write down the complete governing equations for any natural system.”

ID complexities: age-structured mixing, age-specific seasonality in transmission, spatial heterogeneity, heterogeneities in susceptibility, etc, etc

Journey from there to here (1)

-> Many ‘Obvious’ challenges were painstakingly resolved along the way

Journey from there to here (1)

-> Many ‘Obvious’ challenges were painstakingly resolved along the way

1) Perhaps models may have some qualitative relevance?- Nonparametric forecasting to distinguish cycles from chaos (Suigihara &c). - Nonparametric Lyapunov exponent estimators (Ellner &c).

Journey from there to here (1)

-> Many ‘Obvious’ challenges were painstakingly resolved along the way

1) Perhaps models may have some qualitative relevance?- Nonparametric forecasting to distinguish cycles from chaos (Suigihara &c). - Nonparametric Lyapunov exponent estimators (Ellner &c).

2) If we can somehow reconstruct the unobserved susceptible class, would it be egregiously ambitious to compare model simulations and data? - Semiparametric models with smart embedding (Ellner &c) - Susceptible reconstruction (Bobashev &c; Finkenstadt &c)

Journey from there to here (1)

-> Many ‘Obvious’ challenges were painstakingly resolved along the way

1) Perhaps models may have some qualitative relevance?- Nonparametric forecasting to distinguish cycles from chaos (Suigihara &c). - Nonparametric Lyapunov exponent estimators (Ellner &c).

2) If we can somehow reconstruct the unobserved susceptible class, would it be egregiously ambitious to compare model simulations and data? - Semiparametric models with smart embedding (Ellner &c) - Susceptible reconstruction (Bobashev &c; Finkenstadt &c)

3) A seasonal chain-binomial model can in fact be recast as a non-autonomous autoregressive regression: I dear you! - Time-series SIR ver 1 (Finkenstadt & Grenfell) and TSIR ver 2

Journey from there to here (1)

-> Many ‘Obvious’ challenges were painstakingly resolved along the way

1) Perhaps models may have some qualitative relevance?- Nonparametric forecasting to distinguish cycles from chaos (Suigihara &c). - Nonparametric Lyapunov exponent estimators (Ellner &c).

2) If we can somehow reconstruct the unobserved susceptible class, would it be egregiously ambitious to compare model simulations and data? - Semiparametric models with smart embedding (Ellner &c) - Susceptible reconstruction (Bobashev &c; Finkenstadt &c)

3) A seasonal chain-binomial model can in fact be recast as a non-autonomous autoregressive regression: I dear you! - Time-series SIR ver 1 (Finkenstadt & Grenfell) and TSIR ver 2

4) Why in the world does the TSIR seem to fit measles in E&W? - ‘Emergent simplicity’ (Grenfell); Dynamic homogeneity (Earn &c)

Journey from there to here (2)

5) We believe! Real dynamics can be predicted by simple mechanistic models (that incorporates key idiosyncrasies) - POMP et al (King &c)

- Hierarchical models with observation process (Cauchemez &c). - Age-structured TSIR (Metcalf &c).….. (cf Simon’s talk)

Journey from there to here (2)

5) We believe! Real dynamics can be predicted by simple mechanistic models (that incorporates key idiosyncrasies) - POMP et al (King &c)

- Hierarchical models with observation process (Cauchemez &c). - Age-structured TSIR (Metcalf &c).….. (cf Simon’s talk)

Lessons from last 20 years: - ‘All models are wrong …’ Some much less than we expected.- Emergent simplicity once key idiosyncrasies are identified - ?Tactical/strategical? models may be more relevant than we expected.

[The prevailing notion that computation was the important driver in the field is wrong (Cambridge MRCs BUGS has been around since 20 years)]

Some current challenges

Some critical issues are:

(i) use nonlinear stochastic modeling to identify all potentially undesirable side effects of intervention-induced reduction in circulation.

-Rubella and CRS (cf Jess’ talk)

-Chikenpox vaccine and increased shingles incidence

-Whooping cough and the role of natural antigen circulation in maintaining immune memory. The possibility of long-term vaccine failure.

More case law!

1920 1940 1960 1980 2000

Pertussis in Massachusetts

Years

Re

port

ed

pe

rtus

sis

case

s (/

yr.)

10

10

01

00

010

00

0

Cas

es (

/mo.

)

1990 1995 2000 2005

200

400 Recent cases

Vaccine introduced

The first decades of vaccine induced control was extremely successful …

Mass-vaccination introduced in most rich countries in mid ‘40s - early ’50s

1920 1940 1960 1980 2000

Pertussis in Massachusetts

Years

Re

port

ed

pe

rtus

sis

case

s (/

yr.)

10

10

01

00

010

00

0

Cas

es (

/mo.

)

1990 1995 2000 2005

200

400 Recent cases

Vaccine introduced

The first decades of vaccine induced control was extremely successful …… Then even in very high cover areas throughout the developed world (e.g. Massachusetts with consistent >95% cover) the disease re-emerged!

Mass-vaccination introduced in most rich countries in mid ‘40s - early ’50s

Re-emergence is associated with a completely different core group

age class

dens

ity0.

000

.05

0.1

00

.15

0-1 5-6 10-15 15-60

pre-vaccine era

age class

dens

ity0.

000

.05

0.1

00

.15

0-1 5-6 12-13 20-21 30-31 40-41 50-51

vaccine era

Massachusets age-incidence patterns

Lavine, King and Bjornstad. 2011. PNAS

The ‘anamnestic’ 4 compartment SIR model – re-exposure helps maintain immune memory

S – suceptible

I – Infected

R – Highly immune

W – Waning: resistant to infection and will get boosted or loose immunity depending on competing rates

- force of infection

- boosting coeffiecient

- recovery rate

- rate of loss of immunity

- rate of loss of immunity

Lavine, King and Bjornstad. 2011. PNAS

As long as the anamnestic response is at least 10x greater than the naïve response:

Pre-vaccination prediction

Age

Post-vaccination prediction:

Lavine, King and Bjornstad. 2011. PNAS

0.0 0.2 0.4 0.6 0.8 1.0

Vaccine coverage

Pro

po

rtio

n in

fect

ed Total incidence

0.0

008

0.0

011

0.0

014

? 0.5? 20? 5000

Natural immune boosting in pertussis dynamics and the potential for long-term vaccine failure

Inci

denc

e

Vaccine coverage

‘SIR’

‘SIS’

Pro

po

rtio

n ca

ses

0.0

00

.01

0.0

20.

03

0.0

4

Eff ect of booster v accine at a g e 15

1 3 5 7 9 11 13 15 17 19 21 23

Boost at 15No boost

Predicted public health consequences of a booster vaccine at age 15 …

… The booster may push circulation towards adults of childbearing age and increase perinatal infection and increase severe disease.

(cf CRS but different mechanism)

Some critical issues are:

(ii) robust forecasting in the face of rapidly changing demographies and vaccination schedules

(ii) robust forecasting in the face of rapidly changing demographies and vaccination schedules

e.g. Measles Incidence in China: 3 provinces

From Matt Ferrari

(ii) robust forecasting in the face of rapidly changing demographies and vaccination schedules; *

* Perreti et al PNAS 2013 110:5253-5257 (‘Model-free forecasting outperforms the correct mechanistic model for simulated and experimental data’) is not the way to go

(iii) probabilistically projecting possible/probable build-up of ‘susceptible pockets’ in the face of imperfect vaccination programs

(iii) probabilistically projecting possible/probable build-up of ‘susceptible pockets’ in the face of imperfect vaccination programs

Burkina Faso: 2009

Malawi: 2010

•>135,000 cases following 10 years of low incidence

France, 2011

0

200

400

600

800

1000

1200

1986

Jul-

86

1987

Jul-

87

1988

Jul-

88

1989

Jul-

89

1990

Jul-

90

1991

Jul-

91

1992

Jul-

92

1993

Jul-

93

1994

Jul-

94

1995

Jul-

95

1996

Jul-

96

1997

Jul-

97

1998

Jul-

98

MES / AÑO

CA

SOS

1986

-96

0

2000

4000

6000

8000

10000

12000

CA

SOS

1997

-98

CAMPAÑA MAYO 87SARAMPIÓN 9M - 14 A

CAMPAÑA JUNIO 92M-M-R 1 - 10 A

Sao Paolo: 1997

Eg Measles (from Matt Ferrari)

(iv) Important challenge:

We need accurate parametric anchoring of mechanistic models

-7000

-6000

-5000

-4000

-3000

-2000

-1000

2 e-04 4 e-04 6 e-04 8 e-04 1 e-03

2 e-04

4 e-04

6 e-04

8 e-04

1 e-03

intra.beta

inte

r.b

eta

Inference for mechanistic models usually reveal severe multicollinearity among parameters:-Various parameter combinations can fit observed data equally well,-but will not make the same out of sample predictions

Log-likelihood

Intrastage β

Inte

rsta

ge β

Eg 2-stage PDV model (cf Klepac et al. 2009)

- ‘All models are wrong …’ Some much less than we expected.- Emergent simplicity once key idiosyncrasies are identified. -?Tactical/strategical? models may be more relevant than we expected.

- We have an enormous arsenal of model fitting tools.- Multicollinearity makes anchoring of estimates a critical challenge.

Current modeling challenges:- Study unanticipated Public health consequences - Consequences of rapidly changing demographics- Understand build-up of susceptible pockets

Thank you!