controlling systemic inflammation using nonlinear model...

Controlling Systemic InflammationUsing Nonlinear Model Predictive Control

with State Estimation

Gregory Zitelli, Judy Day

University of Tennessee, Knoxville

July 2013With generous support from the NSF, Award 1122462

Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Motivation

I We’re going to be interested in the acute inflammatoryresponse to biological stress in the form of a bacterialpathogen.

I The inflammatory response aims to remove the presence ofthe pathogen.

I However, an excessive response may lead to collateral tissuedamage, organ failure, or worse.


Motivation

I To recover from a high-inflammation state, it is necessary forthe inflammatory mechanisms to downregulate itself throughthe use of some anti-inflammatory mediator.

I We will consider a mathematical formulation of these ideas interms of a highly coupled, nonlinear ODE model, as well ascontrol applied to both the pro and anti-inflammatoryinfluences within the system.


Interaction Diagram


Nonlinear Model

I P represents the bacterial pathogen population.

I N∗ represents the concentration of pro-inflammatorymediators, such as activated phagocytes and their producedcytokines.

I D acts as a marker of tissue damage.

I CA represents the concentration of anti-inflammatorymediators.


Nonlinear Model

dP

dt= kpgP

(1− P

P∞

)− kpmsmP

µm + kmpP− kpnf(N∗)P

dN∗

dt=

snrR(P,N∗, D)

µnr +R(P,N∗, D)− µnN∗

dD

dt= kdn

f(N∗)6

x6dn + f(N∗)6− µdD

dCAdt

= sc + kcnf(N∗ + kcndD)

1 + f(N∗ + kcndD)− µcCA


Nonlinear Model

The system (P,N∗, D,CA) has three fixed points corresponding tothe three biologically relevant scenarios:

I P = N = D = 0 and CA = scµc

, where the patient is healthy.

I All states elevated, where the patient is septic.

I P = 0, and N∗, D,CA > 0, where the patient is aseptic.

For values of kpg (the pathogen growth rate) in the interval(0.5137, 1.755), all three states are stable. (Reynolds et al 2006)


???

So what does this model actually look like?


Septic Simulation

0 50 100 150 2000

2

4

6

8

10

P (

sta

te (

−))

Time

0 50 100 150 2000

0.5

1

1.5

Time

N*

(sta

te (

−))

0 50 100 150 2000

5

10

15

20

D (

sta

te (

−))

Time

0 50 100 150 2000.1

0.2

0.3

0.4

0.5

0.6

Ca (

sta

te (

−))

Time


Aseptic Simulation

0 50 100 150 2000

1

2

3

4

5

6

P (

sta

te (

−),

est(

−−

))

Time

0 50 100 150 2000

0.2

0.4

0.6

0.8

1

Time

N*

(sta

te (

−),

est(

−−

))

0 50 100 150 2000

5

10

15

20

D (

sta

te (

−),

est(

−−

))

Time

0 50 100 150 2000.1

0.2

0.3

0.4

0.5

Ca (

sta

te (

−),

est(

−−

))

Time


Healthy Simulation

0 50 100 150 2000

0.2

0.4

0.6

0.8

P (

sta

te (

−))

Time

0 50 100 150 2000

0.05

0.1

0.15

0.2

Time

N*

(sta

te (

−))

0 50 100 150 2000

0.5

1

1.5

D (

sta

te (

−))

Time

0 50 100 150 2000.1

0.2

0.3

0.4

0.5

0.6

Ca (

sta

te (

−))

Time


Goals

I We would like to apply some kind of control to this model, todirect septic and aseptic patients towards the healthy fixedstate.

I This is accomplished using Nonlinear Model PredictiveControl, or NMPC.

I The NMPC is applied to the pro and anti-inflammatorymediators, N∗ and CA.

I We only apply positive control, and there are restrictions onhow much can be introduced within certain time frames.

I We assume that the level of pro and anti-inflammatorymediators, N∗ and CA, are measurable with Gaussian noise.


Overview of NMPC


Overview of NMPC

I Our reference trajectory is zero levels of P and D, andminimal amounts of control.

J = minAI(t),P I(t)

‖ΓDD‖22 + ‖ΓPP‖22 + ‖ΓAIAI‖22 + ‖ΓPIPI‖22


Note on Observations

I We take noisy measurements of N∗ and CA at each timeinterval which update our predictive model.

I The levels of P and D are not measured directly, and areinstead estimated from the predictive model.

I Being unable to measure P often leads the NMPC to be tooaggressive. Many virtual patients were unnecessarily harmedunder this scheme.

I Since there are biologically relevant scenarios for loosemeasurements of P (indicators like body temperature or bloodpressure give an idea whether or not the infection persists), apathogen update is done every four timesteps.


Pathogen Update

I If the predictive model reads low (< 0.05) but the pathogenlevels are high (P > 0.5) then the level in the predictivemodel is reset to 0.5.

I If the predictive model reads high (> 0.5) but the pathogenlevels are low (P < 0.05) then the level in the predictivemodel is reset to 0.


Virtual Patient Pool

I 1,000 patients are randomly generated with differentparameters, including unique values of the pathogen growthrate kpg.

I 620 acquire elevated P values to suggest treatment.

I Of the 620, 251 (40%) will return to a healthy state on theirown.


Initial Results (Patient Snapshot)

0 50 100 150 2000

0.5

1

1.5

P (

sta

te (

−),

est(

−−

))

Time

0 50 100 150 2000

0.2

0.4

0.6

0.8

Time

N*

(sta

te (

−),

est(

−−

))

0 50 100 150 2000

1

2

3

4

D (

sta

te (

−),

est(

−−

))

Time

0 50 100 150 2000

0.2

0.4

0.6

0.8

Ca (

sta

te (

−),

est(

−−

))

Time


Initial Results (Day et al 2010)

Therapy Type: PlaceboMismatchkpg = 0.52

Mismatchkpg = 0.6

Mismatchkpg = 0.8

Percentage Healthy: 40% (251) 60% (369) 82% (510) 83% (513)

Percentage Aseptic: 37% (228) 19% (120) 8% (49) 17% (107)

Percentage Septic: 23% (141) 21% (131) 10% (61) 0% (0)

Percentage Harmed: na 0% (0/251) 1% (2/251) 6% (16/251)

Percentage Rescued: na 32% (118/369) 71% (261/369) 75% (278/369)


Particle Filter

I One way to go about improving these results is to implimentrobust state estimation for the unobserved variables P and D.

I This is accomplished using a particle filter, which tracks itsaccuracy by comparing its predictions of N∗ and CA to theobserved values.


Particle Filter

I The particle filter is initialized with 1,000 particles randomizednear our measurements for N and CA and initial predictionsfor P and D.

I Each particle pi = (Pi, N∗i , CA,i, Di) is simulated for one time

step.

I At the next time step, the particles pi are assigned weights qidepending on how close N∗i , CA,i are to the newmeasurements for N∗ and CA.


Particle Filter

I At the end of the time step, each of the 1,000 slots holding aparticle is randomly assigned a new particle pi according totheir weights qi.

I This causes bad particles with low weights qi to die off, whilegood particle with high weights replicate.


Particle Filter Results

Therapy Type:Originalkpg = 0.52

Originalkpg = 0.6

Originalkpg = 0.8

Percentage Healthy: 60% (369) 80% (497) 84% (519)Percentage Aseptic: 19% (121) 10% (60) 16% (101)Percentage Septic: 21% (130) 10% (63) 0% (0)

Therapy Type:Particleskpg = 0.52

Particleskpg = 0.6

Particleskpg = 0.8

Percentage Healthy: 60% (369) 80% (493) 82% (511)Percentage Aseptic: 19% (120) 10% (66) 18% (109)Percentage Septic: 21% (131) 10% (61) 0% (0)


Particle Filter Results

I The particle filter does slightly worse than our originalmismatched predictive model.

I On the other hand, the particle filter is self correcting, so itneeds no pathogen update to correct a misled pathogenprediction.


Controllability

Consider the general nonlinear system with affine control

y = F (y) +

m∑i=1

gi(x)ui

w = h(y)

where f : Rn → Rn and h : Rn → Rk. We typically takeu : [0,∞)→ Rm locally integrable.


Controllability Rank Condition

y = F (y) +

m∑i=1

gi(x)ui

w = h(y)

Definition (Controllability Rank Condition)

Given the same nonlinear system with affine control, we say thatthe system satisfies the controllability rank condition if a finitesub-matrix of the following matrix has rank n[

g1 . . . gm [F, g1] . . . [F, [F, g1]] . . .]


Controllability

I If the system is linear,

Controllability Rank Condition =⇒ Locally Controllable

I If the system is nonlinear,

Controllability Rank Condition =⇒ Locally Accessible

I If the system is nonlinear and has symmetric control (for eachcontrol u there is a u′ such that F (y, u) = −F (y, u′) forevery y ∈ Rn), then

Controllability Rank Condition =⇒ Locally Controllable


Controllability

I It can be shown that our system does satisfy thecontrollability rank condition in the first octant.

I This means that the system is locally accessible there.

I Unfortunately, our control does not satisfy the symmetryproperty, so this does not imply that the system is locallycontrollable.


What’s Next?

I Djouadi and Bara are working on the use of adaptive controlfor this model, to help overcome the mismatch between thepredictive model and patient model parameters like kpg.

I Although the pathogen update was necessary for the variableP , it appears that the damage variable D is well estimated byour predictive model under many different initial conditionsand model parameters. It would be nice to make this notionprecise.

I Are there elements of this model that can be exploited toincrease the effectiveness of our filter?


The End?

Thank you very much!


Model

dP

dt= kpgP

(1− P

P∞

)− kpmsmP


dN∗

dt=

snrR(P,N∗, D)


dD

dt= kdn

f(N∗)6


dCAdt




Model

Where

R(P,N∗, D) = f(knpP + knnN∗ + kndD)

f(x) =x

1 +(CAc∞

)2c∞ is chosen so that when CA is at its highest, f(x) ≈ 1

4x.


Pathogen

I The bacterial pathogen follows logistic growth.

I Healthy individuals have a baseline, non-specific, localimmune response. We assume through the initial analysis ofthis interaction that the local response reaches a quasi-steadystate value of

kpmµm+kmpP

.

I The pathogen are directly attacked by the phagocytic immunecells N∗, which may be inhibited by the presence of theanti-inflammatory CA.

dP

dt= kpgP

(1− P

P∞

)− kpmsmP



Pro-Inflammatory Mediator

I The first term comes from the assumed quasi-steady state ofa subsystem involving resting and activated phagocytes.

I The second term represents the gradual decay in theconcentration of pro-inflammatory mediators.

dN∗

dt=

snrR(P,N∗, D)



Damage

I Collateral tissue damage from activated phagocytes motivatedthe first term. The damage saturates, with the large Hillcoefficient necessary to produce a more realistic basin ofattraction for the healthy fixed state.

I The second term represents tissue regeneration.

dD

dt= kdn

f(N∗)6



Anti-Inflammatory Mediators

I The initial biological source term sc is augmented by theMichaelis-Menten term.

I The second term represents the gradual decay in theconcentration of anti-inflammatory mediators.

dCAdt




Essential Components of NMPC

(I) Reference TrajectoryDesired trajectory we want the system to approach.Our reference trajectory will be P = D = 0.

(II) Prediction of Process OutputA predictive model is used to provide an estimation of thestates which are unobserved.

(III) Objective FunctionAn objective function will describe how are current predictionsdeviate from the reference trajectory.Our objective function is a weighted sum of squares for theundesireable variables P and D, as well as for the controllevels.

J = minAI(t),P I(t)

‖ΓDD‖22 + ‖ΓPP‖22 + ‖ΓAIAI‖22 + ‖ΓPIPI‖22Controlling Systemic Inflammation Using NMPC University of Tennessee, Knoxville

Essential Components of NMPC

(IV) Sequence of Control Moves at Each Time StepA discrete control is designed for k future steps to minimizethe objective function over each time interval.

(V) Error Prediction UpdateThe measured values of N and CA from the patient modelare compared to the predictive model after the control hasbeen applied to both.It is at this step that implementations of the particle filter canbe refined through this comparison.


Controllability

Consider the general nonlinear system with affine control

y = F (y) +

m∑i=1

gi(x)ui

w = h(y)

where f : Rn → Rn and h : Rn → Rk. We typically takeu : [0,∞)→ Rm locally integrable. For us,

F (y) =

kpgy1

(1− y1

P∞

)− kpmsmy1

µm+kmpy1− kpnf(y2)y1

snrR(y1,y2,y3)µnr+R(y1,y2,y3)

− µny2kdn

f(y2)6

x6dn+f(y2)6 − µdy3

sc + kcnf(y2+kcndy3)

1+f(y2+kcndy3)− µcy4


Controllability

Definition

We say that a point a ∈ Rn is attainable from y if there exists anappropriate control uy (locally integrable) such that the trajectorygiven by

y = F (y) +

m∑i=1

gi(x)uyi y(0) = y

is such that y(T ) = a for some finite trajectory [0, T ].


Controllability

Definition

If y ∈ Rn and O is an open set containing y, we let AO(y, T )denote the set of all points in O which are attainable from y intime T , such that y(t) ∈ O for all t ∈ [0, T ].

Definition

We say that the nonlinear system is locally accessible at a pointy ∈ Rn if the sets AO(y, T ) has nonempty interior for every openset O 6= ∅ and every T > 0.


Controllability

Definition

Given the same nonlinear system with affine control, we say thatthe system satisfies the controllability rank condition if a finitesub-matrix of the following matrix has rank n[

g1 . . . gm [F, g1] . . . [F, [F, g1]] . . .]


Controllability

If our system was linear, then it turns out that the controllabilityrank condition implies that the system is controllable. In thenonlinear case, we have the following results.

Theorem

Given the same nonlinear system with affine control, if the systemsatisfies the controllability rank condition at a point y ∈ Rn then itis locally accessible there.

Theorem

If for a nonlinear system y = F (y, u) we have that for any locallyintegrable control u there exists another locally integrable controlu′ such that F (y, u) = −F (y, u′) for all y ∈ Rn and the systemsatisfies the controllability rank condition at a point y ∈ Rn, thenthe system is locally controllable at y.


Controllability

Recall our system is in the following form

y =

kpgy1

(1− y1

P∞

)− kpmsmy1

µm+kmpy1− kpnf(y2)y1

snrR(y1,y2,y3)µnr+R(y1,y2,y3)

− µny2 + u1

kdnf(y2)6

x6dn+f(y2)6 − µdy3

sc + kcnf(y2+kcndy3)

1+f(y2+kcndy3)− µcy4 + u2

+

0u100

+

000u2


controlling systemic inflammation using nonlinear model...

Documents