Computational Sustainability March 12, 2010 at 1:25PM – 2:40PM

1150 Snee Hall

Optimizing Intervention Strategies in Food Animal Systems: modeling production, health

and food safety

Elva Cha, DVM, PhD candidate, Rebecca Smith, DVM, PhD candidate, Zhao Lu, PhD, Research Associate and Cristina Lanzas, DVM, PhD, Research Associate and Yrjö T. Gröhn,

DVM, MPVM, MS, PhDDepartment of Population Medicine and Diagnostic Sciences,

College of Veterinary Medicine, Cornell University

Perhaps I should start with “ a disclaimer”…

If I understood correctly, your approach is to assume the model is 'correct', then optimize the system. As veterinarians, our responsibility is to build a model based on subject matter representing reality.

Of course, our goal is to find the optimal way to control disease (not necessarily eradication). And sometimes we need to learn the economically optimal way to coexist with them.

Food Supply Veterinary Medicine….all aspects of veterinary medicine's involvement in food supply systems, from traditional agricultural production to consumption.

Modeling production, health and food safety:

1. Optimizing health and management decisions

2. Mathematical modeling of zoonotic infectious diseases (such as L. monozytogenes, E. coli, MDR salmonella and paratuberculosis).

Three examples …

1. Modeling production and health:Project 1. “Optimal Clinical Mastitis Management in Dairy cows.”

Elva Cha’s PhD researchProject 2. “Cost Effective Control Strategies for The Reduction of

Johne’s Disease on Dairy Farms.” Zhao Lu, Research Associate and Becky Smith’s PhD research

2. Modeling Food Safety:Project 3. “Food Animal Systems-Based Mathematical Models of

Antibiotic Resistance among Commensal Bacteria.” Cristina Lanzas, Research Associate

Let’s start with…

1. Modeling production and health:

Our overall goal is to develop a comprehensive economic model, dynamic model (DP), to assist farmers in making treatment and culling decisions.

Our 1st example: Elva Cha’s PhD research: “Optimal Clinical Mastitis Management in Dairy cows”

Mastitis (inflammation in mammary gland)

Common, costly disease (major losses: milk yield, conception rates, and culling).

The question we address is whether it is better to treat animals at the time CM is first observed or whether it is worthwhile collecting more information related to the nature of the pathogen involved (Gram-positive or Gram-negative, or even the actual pathogen), and then make a treatment decision.

More information would seem to be of benefit when deciding what to do with diseased cows, but unless the additional tests provide a different recommended outcome, they are only an extra cost.

Objective:

• We will build upon our existing Dynamic Programming model.

• This work requires us to determine the risk and consequences of CM (milk loss, delayed conception, mortality) as functions of cow characteristics, including the disease history and disease prognosis of the cow.

To do this

• To determine the economically optimal amount of information needed to make mastitis treatment decisions

How do we address our objective?

• We will compare 2 models– Model 1 which will identify cows based on generic

(non-specific) mastitis– And Model 2, whereby if a cow has mastitis, it will

be specified as gram-positive, gram-negative or other types of mastitis

– We will then compare the profit of each modelWhat is this model?

The models

• The models will calculate the optimal policy for each individual cow

• This can be done by ‘dynamic programming’

Dynamic programming

• Numerical method for solving sequential decision problems– Based on the ‘Bellman principal of optimality’– In our case, the ‘system’ is dairy cows, observed

over an infinite time horizon split into ‘stages’ i.e. months of a cow’s life

Dynamic programming

At each stage, the state of the system is observed• E.g. pregnancy status, disease status, milk yield

And a decision is made• Replace, keep (treat and inseminate) or sell

This decision influences stochastically the state to be observed at the next stage

Depending on the state and decision, a reward is gained

Dynamic programming

– Value function is the expected total rewards from the current stage until the end of the horizon

– Optimal decisions depending on stage and state are determined backwards step by step

Structure of the hierarchic Markov process optimization model

Process

Stage

State

Action1

a

a

a

a

a

b

b

2

3

4

5

6

7

8

0

1

2

3

4

5

6

Founder

Child

Grandchild

Lactation

Month in lactation c

Replace

Keep

Inseminate

c

c

c

c

c

a states of permanent milk yield

b states of mastitis in previous lactation

c states of temporary milk yield, pregnancy, and mastitis

Parameters in our model

• For each type of mastitis– Risk of mastitis– Repeated risk of mastitis– Effect on milk yield– Effect on conception– Effect on mortality and culling

Limitations

• Now we can only study 3 different types of mastitis, although we have data for very specific types of mastitis! (i.e. >3!)

Why is this a limitation? Can’t we simply expand the model?

Implications of a larger model

• By expanding the model, we will encounter the ‘curse of dimensionality’– An opportunity cost of including another disease,

and hence the parameters associated with it– The model increases as a power function, not by a

factor of 1, 2 or 3…– This makes computations even more challenging

and time consuming

The curse of dimensionalityexample: Houben et al. 1994

State variables:• Age (monthly intervals, 204 levels)

• Milk yield, present lactation (15 levels)

• Milk yield, previous lactation (15 levels)

• Length of calving interval (8 levels)

• Mastitis, present lactation (4 levels)

• Mastitis, previous lactation (4 levels)

• Clinical mastitis (yes/no)

Total state space 6,821,724 states

The curse of dimensionalityexample: Gröhn et al. 2003

State variables:• Parity (12 levels)

• Conception in month (10 levels)

• Stage of lactation (20 levels)

• Milk yield (5 levels)

• Month of calving (12 levels)

• Disease index (212 levels)

Total state space 144,000 x 212= 30,528,000

Structure of the hierarchic Markov process optimization model:

First level: 5 milk yield levelsSecond level: 8 possible lactations and 2 carry-over mastitis states from previous

lactation (yes/no).Third level: 20 lactation stages (max calving interval of 20 months).

5 temporary milk yield levels (relative to permanent milk yield),9 pregnancy status levels (0=open, 1-7 months pregnant, and 8=to be dried off), one involuntary culled state13 mastitis states: 0=no mastitis 1 = 1st occurrence of CM (observed at the end of the stage),2, 3, 4 = 1, 2, 3 and more months after 1st CM,5 = 2nd CM, 6, 7, 8 = 1, 2, 3 and more months after 2nd CM,9 = 3rd CM, 10, 11, 12 = 1, 2, 3 and more months after 3rd CM,CM events > 3 assigned same penalties as if they were 3rd occurrence.

After deleting impossible stage-state combinations, the model described 560,725 stage-state combinations.

If we were able to overcome the curse of dimensionality …

No longer only generic guidelines for the generic cow.

The DP recommendations could be tailored to the individual cow in real time according to her cow characteristics and economics of the herd.

Project 2: “Cost Effective Control Strategies for The Reduction of Johne’s Disease on Dairy Farms”

Zhao Lu, PhD, Research Associate, and Becky Smith, DVM, PhD student

Johne’s disease (paratuberculosis)• Johne’s disease is a chronic, infectious, intestinal disease

caused by infection with Mycobacterium avium subspecies paratuberculosis (MAP).

• Infection process of paratuberculosis on a dairy cow:

Transient shedding Latency Low shedding High shedding

Infection status

Disease status

No clinical signs Sub-clinical Clinical

Infection

Issues of Johne’s disease• Economic loss: > 200 million $ per year (Ott, 1999) due to the reduced

milk production, lower slaughter value, etc.

• Public health: a potential association between Johne’s disease and human Crohn’s disease has been debated.

• Control of Johne’s disease:– Test and cull strategies, i.e., to cull/remove infectious animals form herd by

test-positive results using diagnostic testing methods, such as culture and ELISA tests.

– Improved hygiene management;– Vaccination.

However, it is difficult to control JD spread:– Long incubation period;– Low diagnostic test sensitivity for animals shedding low levels of MAP;– Cross reactivity of Johne’s disease vaccines with tuberculosis (TB).

Modeling of MAP transmission on a dairy herd

• A deterministic compartmental model (Mitchell et al., 2008). The test-based culling rates of low and high shedders are denoted by δ1 and δ2, respectively.

X1

Susceptible

-

d

X2

Resistant

d

TrTransient

d

() HLatent

d

Y1

Low

d

Y2

High

d1 2

Evaluation of effectiveness of test-based culling in Johne’s disease control

• The reproduction ratio R0 was derived and a global parameter uncertainty analysis was performed to determine the effectiveness of test-based culling intervention (Lu et al., 2008)

A stochastic multi-group model(Evaluation of effectiveness of test-based culling)

Calves

Heifers

Cows

Optimal control of Johne’s disease:economic model

• Objective function (cost function): – Profit: selling milk, culling cows (meat);– Cost: raising first and second-year calves/heifers; diagnostic testing; – Lost: false-positive testing results.

• Control: various scenarios of test-based culling rates and

management

• Constraints: a system of dynamic equations.– Compartment model providing numbers of calves, heifers, and cows in

each compartment, which are needed in the objective functional (cost function).

• A deterministic, discrete time economic model has been developed to find optimal test-based culling rates with large (6 and 12 month) time steps.

What do we want?(Optimal control of test-based culling rates)

• A deterministic, continuous time economic model. – Analytical studies of linear controls (test-based culling rates);– Numerical search of optimal test-based culling rates.

• A stochastic economic model.

– Reasons: more realistic (variable prevalence and fadeout due to random events)

– Optimal culling rates using stochastic differential equations;– Numerical simulations of optimal culling rates for the mean of cost

function.

• Economic analysis of Johne’s disease vaccines.– Mathematical modeling of imperfect Johne’s vaccines is in progress.

Modeling the efficacy of an imperfect vaccine with multiple effects

• Vaccines are often imperfect – They may not prevent all infections– They may have effects other than decreasing susceptibility

• Efficacy can be considered as the proportional effect on a rate or probability parameter in a compartmental model

5 vaccine effects:1. Vertical transmission2. Horizontal transmission

i. Susceptibilityii. Infectiousness

3. Duration of latency4. Duration of low-infectious

period5. Progression of clinical

symptoms

Modeling vaccine efficacy against JD

X1

(1-p)(-) (1-p)

d

X2

Resistant

d

TrTransient

d

() HLatent

d

Y1

Low

d

Y2

High

d

Susceptible

bTrbYbYbbbb tVtVtVetTrtYtYt 31211132211

N

tVtVtVetTrtYtYt TrYY 322112132211

VX1

Susceptible

p(-) p

d

VX2

Resistant

d

VTrTransient

d

e20() VHLatent

d

e3 VY1

Low

d

e4 VY2

High

d e5

Estimating vaccine efficacy against JD with field data

• Known information:– birth date– death date – annual test dates and

results– vaccination status

• Missing information:– Date of infection– Onset of low-

shedding– Onset of high-

shedding– True infection status

(if all tests results were negative)To estimate vaccine efficacy,

missing information must also be estimated

Estimating vaccine efficacy with Markov Chain Monte Carlo models

MCMC models are Bayesian statistical models, useful for disease modeling because they

• Can account for nonlinear systems– parameters may be inter-related

• Can account for time-dependence – i.e. infectious pressure

• Have a mechanism for missing-data problems:– Missing information can be estimated probabilistically, given a set of

parameters drawn from a prior distribution– The full dataset can then be used to determine the relative likelihood

of a different set of parameters drawn from the prior• The new set of parameters may be accepted or rejected, based on its

relative likelihood– This process is iterated until it converges on a posterior distribution for

all parameters

Validating MCMC models

• In order to test that an MCMC model predicts the true parameter distribution, we feed it data simulated with known parameters

• In the case of the JD model, the full model requires individual animal data:– Infection status– Vaccination status– Dates of birth, compartment transitions, death

• We need an individual-animal stochastic model

Optimal control of Johne’s disease using an individual-based (agent-based) modeling approach

• Controls: test-based culling, farm management, JD vaccines

• Advantages of individual-based modeling (IBM)– Providing a general framework to model infectious disease transmission

in a dairy herd;– Integrating all individual information together to predict the dynamics on

farms;– Adding controls on farm level and/or individual animals easily. – Economic analysis based on IBM would be more accurate.

• Disadvantages of IBM– Individual information collection: a detailed profile for each animal in a

herd. (also an advantage)– Simulations: efficient algorithm and powerful computers are necessary.

Project 3: “Develop, evaluate and improve food animal systems-based mathematical models

of antimicrobial resistance among commensal bacteria”

Cristina Lanzas, DVM, PhD, Research Associate

Bergstrom and Feldgarden, 2008

• At least 200,000 people suffer from hospital acquired infection every year, and at least 90,000 die in US.

• Economical burden of antimicrobial resistance in clinical settings in US is estimated to be as high as $ 80 billion annually.

• Pathogens outside the hospital are also becoming progressively resistant to common antimicrobials.

• Antimicrobial resistance is also considered a food safety issue because infections with drug resistant foodborne pathogens (e.g. Salmonella) can be particularly serious.

• Reservoirs of resistant genes are found in commensal bacteria in the human and animal gastrointestinal tracts (small intestine supports ~ 1010 bacterial cells/g) .

• Commensal bacteria can transfer mobile genes coding antimicrobial resistance among themselves and to pathogen bacteria (e.g. plasmid transfer between Salmonella and E. coli)

Molecular mechanisms involved in the spread of antimicrobial resistance. Inter-cellular movement (horizontal spread) is the main cause of acquisition of resistance genes.

Boerlin, 2008Salyers et al., 2004

WITHIN HOST

R

S

Population dynamics of antibiotic-sensitive and –resistant bacteria

Linked to antibiotic exposure

Emergence of resistance during antibiotic treatment

Fitness cost linked to microbial growth

BETWEEN HOSTS

Transmission of resistant clones

Individuals colonized with either susceptible or resistant strains

The host population is divided according its epidemiological status (e.g. susceptible, infectious)

“Binary response”: Animal carries the bacteria carrying the resistance or not

S I

- +

Within host dynamics of antimicrobial resistance dissemination

HH

H

d

MIC

CEC

MIC

CE

CE

50

max

1)(

Microbial growth for sensitive and resistant strains with horizontal gene transfer

sdrs

srs

ss NCuE

N

NNN

K

NNrrN

dt

dN)(

)(

rrdrs

rrs

rr pNrNCuE

N

NNN

K

NNrNr

dt

dN)1()(

)()1()1(

Logistic Growth

Antibiotic effect Plasmid loss

during segregation

Plasmid transfer

1 2 3 4 50

10

20

30

40

50

60

70

80

90

100

Duration of treatment, days

% b

act

eri

a w

ith r

esi

sta

nce

Percentage of resistant bacteria 24 h after the end of the antimicrobial treatment

Between host dynamics of antimicrobial resistance dissemination

S

IS IR

ISR

S

susceptible

I

infectious

W

environment

γ

πη

λ

Integrating within and between host antimicrobial resistance dynamics

• Interventions to minimize the dissemination of antimicrobial resistance can be applied at different organizational levels (e.g. within host/between hosts and environment):

• Optimize antimicrobial dosage regimes to mitigate the dissemination of antimicrobial resistance within enteric commensal bacteria.

• Reduce the exposure of animals to antimicrobial resistant bacteria.

• Mathematical approaches that integrate within and between host dynamics are necessary to optimize mitigation strategies acting at different hierarchical scales:

• Agent-based/Individual-based models

• Dynamic nested models

Modeling On-farm Escherichia coli O157:H7 population dynamics

• Metapopulation models has allowed us to investigate the potential role of non-bovine habitats (i.e., water troughs, feedbunks, and the surrounding pen environment) on the persistence and loads of E. coli O157:H7 in feedlots.

• O157:H7 survive and reproduce in water troughs, feed, slurry, pen floors.

EmmpCmWmK

EEr

dt

dE

FmpCmK

FFr

dt

dF

pCmEmWmmK

WWr

dt

dW

FmWmEmCpK

CCr

dt

dC

ecewecewee

e

fcfcff

f

cwewwewcww

w

fcwceccc

c

1

1

1

1

Ayscue et al., Foodborne Pathog Dis, 6:461-470 (2009)

Monogastric calf

Growing ruminant heifer

Lactating cow

Bred ruminant heifer

Dry cow

Cull cow

Dairy beef

S, R S, R

S, R S, R

Animal patches

Selective

pressures

Environment

(S, R)

Water

(S,R)

This metapopulation approach is suitable for modeling the dynamics of antimicrobial resistance dissemination. Pharmacokinetics and pharmacodynamics and biological fitness of antimicrobial resistance can be integrated.

Assuming three types of ecological patches (water, environment and n animals) and assuming indirect transmission (bacteria are transmitted to animals through water and environment):

sjjejwsejswjjsdjj

jrjs

jsj

jrjs

jsjjjs NmmEmWmNCEu

N

NNN

K

NNrNr

dt

dN)()(

)(

rjjejwrejrwj

jrjrjdjj

rjsjjr

j

rjjs

jrjjjr

NmmEmWm

pNrNCEuN

NNN

K

NNrNr

dt

dN

)(

)1()()(

)1()1(

For the j animal:

Water patch:

sweswj

nj

jsewjsjw

nj

jsw

rs

sw

rs

wsw

sWmWmEmNmWu

W

WWW

K

WWrWr

dt

dW

11

)(

rwrwerwj

nj

jrewjrjw

nj

jrw

rs

rw

rs

wrwr

pWrWmWmEmNmWuW

WWW

K

WWrWr

dt

dW)1(

)()1()1(

11

Environmental patch:

sewsej

nj

jswejsje

nj

jse

rs

se

rs

ese

sEmEmWmNmEu

E

EEE

K

EErEr

dt

dE

11

)(

eerewrej

nj

jrwejrje

nj

jre

rs

re

rs

erer

pWrEmEmWmNmEuE

EEE

K

EErEr

dt

dE)1(

)()1()1(

11

Potential students projects

• Application of optimal control to evaluate strategies in metapopulation models

• Development of agent based models to address antimicrobial resistance dissemination.

• Optimization in agent based models• Optimization in hierarchical models