powerpoint presentationusers.iems.northwestern.edu/~hazen/medical markov mo… · ppt file · web...

Markov versus Medical Markov Modeling – Contrasts and Refinements

Gordon HazenFebruary 2012

Medical Markov Modeling

• We think of Markov chain models as the province of operations research analysts

• However …• The number of publications in medical journals

– using Markov models– to address medical cost-effectiveness– approaches 300 per year!

2


• Why the large buy-in from the medical community?– Easy-to-use software that

combines decision trees and Markov models (Data, TreeAge)

– Simplicity of models• Discrete time• Transient

3

Overview of this talk

1. Background on medical Markov modeling2. Population modeling versus individual-level

modeling3. Product structure in medical Markov models

Overview


modeling3. Product structure in medical Markov models.


• The kind of modeling that is typical

6

IHD = Ischemic heart diseaseMI = Myocardial infarction (heart attack)

A simplification of: Palmer S, Sculpher M, Phillips Z, Robinson M, Ginnelly L, Bakhai A et al. Management of non-ST elevation acute coronary syndrome: how cost-effective are glycoprotein IIb/IIIa antagonists in the U.K. National Health Service?. International J Cardiology 100 (2005) 229-40.

The kind of modeling that is typical

• Cohort analysis

7

0.9 0.7 0IHD Post MI Dead Alive QALY

1 0 0 1 0.9

0.9867 0.01 0.0033 0.9967 0.895

0.9735 0.0195 0.007 0.993 0.8898

… … … … …

0.9 0.7 0IHD Post MI Dead Alive QALY

1000 0 0 1000 900

986.67 10 3.3333 996.67 895

973.51 19.533 6.9556 993.04 889.83

… … … … …

Our preference: Continuous-time

• Cohort analysis in continuous time

8

pMI = lDtp0 = m0Dtp1 = m1Dt

𝑝 𝑗 (𝑡 )=𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑠𝑡𝑎𝑡𝑒 𝑗 𝑖𝑠𝑜𝑐𝑐𝑢𝑝𝑖𝑒𝑑𝑎𝑡 𝑡𝑖𝑚𝑒𝑡

Our preference – continuous time

• Discounted expected quality-adjusted life years:

9

Cohort analysis in continuous time

• Intervention: Post-MI mortality rate m1 = 0.1/yr is decreased by 75% and the MI incidence rate l = 0.12/yr is decreased by 50%.

10

0 5 10 15 200

0.2

0.4

0.6

0.8

1

(yr)

pIHD t( )

pPostMI t( )

t

0 5 10 15 200

0.2

0.4

0.6

0.8

1

(yr)

pIHD t( )

pPostMI t( )

t

6.62 QALY/patient 10.28 QALY/patient

Continuous-time version of cohort analysis

• Let dt 0 to obtain

11

𝑝 𝑗 (𝑡+𝑑𝑡 )=𝑝 𝑗 (𝑡 )+∑𝑖𝑝𝑖 (𝑡 ) 𝜆𝑖𝑗𝑑𝑡−∑

𝑘𝑝 𝑗 (𝑡)𝜆 𝑗𝑘𝑑𝑡

… the Kolmogorov differential equations.• The cohort analysis procedure

is merely the Euler method for solving the Kolmogorov equations.

Overview



Question: How to incorporate population issues?

• An intuitive approach: Restart following death

13

• Then compute steady-state probabilities in the resulting irreducible chain.

Open routing process Closed routing process

Question: How to incorporate population issues?

• Balance equations for steady-state probabilities:

14

• Intervention assumptions: – Post-MI mortality rate m1 = 0.1/yr is decreased by 75%– MI incidence rate l = 0.12/yr is decreased by 50%.

• Results: PostMI increases from 23.0% to 38.5%• The population is less healthy!• So what is wrong here?

Population issues: A more rigorous approach

• Observation: A population of non-interacting individuals is equivalent to a Jackson network of infinite-server queues.

15

Equilibrium results

• Jackson network balance equations

16

𝛼 𝑗=𝑚𝑒𝑎𝑛𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑖𝑛𝑑𝑖𝑣𝑖𝑑𝑢𝑎𝑙𝑠 𝑖𝑛h h𝑒𝑎𝑙𝑡 𝑠𝑡𝑎𝑡𝑒 (𝑎𝑡 𝑠𝑡𝑎𝑡𝑖𝑜𝑛 ) 𝑗

• Theorem (e.g. Serfozo 1999): The counts nj of individuals in health state j are, at equilibrium, independent Poisson variables with means j given by the solution to the balance equations.

Equilibrium results

• Solve balance equations with entrance rate n = 1000/yr

mean (sd)Status Quo 6250 (79) 1875 (43) 8125 (90)

Intervention 10000 (100) 6000 (77) 16000 (126)

IHD Post MI

More survivors under intervention!

The closed routing process again

• Convert the open routing process to a closed one in the following way

18

𝜆 ′ 𝑖𝑗❑=𝜆𝑖𝑗+𝜇𝑖

𝜈 𝑗

𝜈

𝜈=∑𝑗𝜈 𝑗

Open Closed

Open versus closed routing

• Theorem (Hazen and Huang 2011): One may obtain equilibrium means from steady state probabilities, and vice versa:

19

Equilibrium means j Steady-state probabilities j

jj

jj

j j

n m

.

• where is the total arrival rate, and is the equilibrium departure rate .

Open versus closed results

20

Equilibrium means j Steady-state probabilities j

.

IHD MIStatus Quo 77.0% 23.0%

Intervention 62.5% 37.5%

IHD MI TotalStatus Quo 6250 1875 8125

Intervention 10000 6000 16000

Example: Re-analysis of the preventive use of tamoxifen

• Original analysis: Col et al 2002• Tamoxifen

– an estrogen agonist/antagonist– an effective therapy against established breast cancer

• Evidence that it can reduce breast cancer incidence • But life-threatening side effects

– endometrial cancer – vascular events.

• Would the benefit of its prophylactic use in healthy women be worth the associated risks?

21

Preventive use of tamoxifen: Our model

• Cure rate models for breast and endometrial cancer treatment– Mortality decreases in time survived after cancer diagnosis. – This cannot be directly modeled as a Markov model – it is semi-Markov.– Cure rate model with unobserved states Cured/ Not Cured allows

implicit mortality to decrease over time survived.

22

mb

lb

1-pb

pb

No Disease

Breast Ca

Cured

Not Cured

Death

le

1- pe

pe

me

No Disease

Endometrial Ca

Not Cured

Cured

Death

Breast cancer incidence and treatment

Endometrial cancer incidence and treatment

Preventive use of tamoxifen: Our model• Overall model is the Cartesian product of the two factors

below and a third Background Mortality factor.• More on this later …

23

mb

lb

1-pb

pb

No Disease

Breast Ca

Cured

Not Cured

Death

le

1- pe

pe

me

No Disease

Endometrial Ca

Not Cured

Cured

Death

Breast cancer incidence and treatment

Endometrial cancer incidence and treatment

Preventive use of tamoxifen: Our model• Estimated parameters (max likelihood estimates)

24

mb

lb

1-pb

pb

No Disease

Breast Ca

Cured

Not Cured

Death

le

1- pe

pe

me

No Disease

Endometrial Ca

Not Cured

Cured

Death

Variable Description Valueλb0 incidence rate of br ca without

tamoxifen (high risk)0.0086/yr

RRb risk ratio of br ca with tamoxifen

0.4936

pb probability of cure for br ca 0.5631

μb mortality rate of br ca if not cured

0.0996/yr

Variable Description Valueλe0 incidence rate of endo ca

without tamoxifen0.00152/yr

RRe risk ratio of endo ca with tamoxifen

4.0132

pe probability of cure for endo ca 0.9019

μe mortality rate of endo ca if not cured

0.3159/yr

Variable Description Valueμ0 Background mortality (age 50) 0.03118/yr

Preventive use of tamoxifen: Our model

• Product structure for quality of life – Qjk = Qbj Qek

– more on this later

25

Endo CaQjk No Ca Cured Not Cured

No Ca 1 0.81 0.39

Cured 0.81 0.656 0.316

Not Cured 0.39 0.316 0.152

Br Ca

• Model entry rate n0 = 110,000/yr– 2.3 M women reaching age 50 each year x 4.8% at high risk

for breast cancer

Preventive use of tamoxifen: Results

• A more nuanced picture of the effects of this intervention than just incremental QALYs .

26

IncrementalEndo Ca

DQALYjk No Ca Cured Not CuredNo Ca 0.405 1.07 0.004

Br Ca Cured -0.791 0 0

Not Cured -0.032 0 0

IncrementalD jk Endo Ca

(1000s) No Ca CuredNot

CuredNo Ca -15.84 322.74 2.824 309.7

Br Ca Cured -262.79 35.31 0.131 -227.3Not

Cured-34.87 3.52 0.023 -31.3

-313.5 361.57 2.98 51.0

Endo Ca

Djk None Cured Not Cured

None -0.02 0.10 0.00 0.08Br Ca Cured -0.08 0.01 0.00 -0.07

Not Cured -0.01 0.00 0.00 -0.01-0.11 0.11 0.00

Incremental QALYs/woman

Incremental equilibrium means

Incremental equilibrium probabilities

Overview



Markov models with product structure

• Product structure is relatively common in medical Markov models

28

Roach PJ, Fleming C, Hagen MD, Pauker SG. Prostatic cancer in a patient with asymptomatic HIV infection: are some lives more equal than others? Med Decis Making. 1988 Apr-Jun;8(2):132-44.

Markov models with product structure

• Much simpler depiction of model structure: Independent factors

29

Product structure is relatively common

• Schousboe et al. considered 5 different types of fractures: – hip fracture– clinical vertebral (Cv) fracture– radiographic vertebral (Rv) fracture– distal forearm (Df) fracture– other fracture

• In principle this should allow 25 = 32 state combinations corresponding to 5 factors each at 2 possible levels.

• What their model actually did: 6 states– 5 states corresponding to a single fracture type– 1 other state corresponding to the combination of the worst two possible

fracture types– Disadvantage: Such a model “forgets” past fractures when a new fracture

occurs, which the 32-state model would not do.

30

Schousboe JT, Nyman JA, Kane RL, Ensrud KE. Cost-effectiveness of alendronate therapy for osteopenic postmenopausal women. Ann Intern Med. 2005 May 3;142(9):734-41.

Advantages of explicitly accounting for product structure

• Model formulation: Simpler to merely consider one factor at a time

• Model presentation: Simple factors easier to understand and critique.– Model is less likely to be perceived as a “black box”

• Computational advantages as well when factors are independent.

31

Computation of QALYs under product structure

• Product structure– Health state – Quality coefficient

• The latter assumption on is equivalent to standard gamble independence (Hazen 2003)

• Standard gamble independence:

32

p

(y i*,z i,t )

(y i*,z i,0 )(y i,z i,t )

1-p

~

True for one implies true for all - the indifference does not depend on what is.


• Product structure– Health state – Quality coefficient

• Theorem (Hazen and Li 2010): One may calculate mean QALYs as follows.

33

E[QALY/person] =

E [𝑄𝑅 (𝑡 ) ]=∏𝑖E [𝑄𝑅𝑖 (𝑡 )]

E [𝑄𝑅𝑖 (𝑡 ) ]=∑𝑥 𝑖

𝑣 𝑖(𝑥 𝑖)𝑃𝑥𝑖(𝑡)

(cohort analysis in factor i)


34

Cancer AIDSBkgd

MortalityHealthquality 1 0.6 0 1 0.5 0

YearNo

Cancer Cancer Death

E[QALYRate]/person

NoAIDS AIDS Death

E[QALYRate]/person

MortalityRate

/10,000SurvivalProb.

QALYRate/

person0 10000 0 0 1.0000 10000 0 0 1.0000 1.0000 0.51 9680 320 0 0.9872 9048 952 0 0.9524 99 0.9901 0.90382 9371 545 85 0.9697 8187 1212 601 0.8793 107 0.9796 0.78733 9071 700 229 0.9491 7408 1226 1366 0.8021 116 0.9682 0.67454 8781 804 415 0.9264 6703 1157 2140 0.7282 126 0.9561 0.57305 8500 872 628 0.9023 6065 1064 2870 0.6598 136 0.9431 0.48436 8228 912 859 0.8776 5488 970 3542 0.5973 148 0.9293 0.40807 7965 934 1101 0.8525 4966 880 4154 0.5406 160 0.9146 0.34278 7711 941 1349 0.8275 4493 797 4710 0.4892 173 0.8988 0.28729 7464 938 1598 0.8027 4066 721 5213 0.4426 188 0.8821 0.240210 7225 928 1847 0.7782 3679 653 5668 0.4005 203 0.8644 0.2005

38 2908 399 6693 0.3148 224 40 9737 0.0244 1894 0.0952 0.000239 2815 386 6799 0.3047 202 36 9762 0.0220 2051 0.0775 0.000240 2725 374 6901 0.2950 183 33 9784 0.0199 2221 0.0621 0.0001

Total 6.3331

E[QALY/person] =

E [𝑄𝑅 (𝑡 ) ]=∏𝑖E [𝑄𝑅 𝑖 (𝑡 )]

E [𝑄𝑅 𝑖 (𝑡 ) ]=∑𝑥 𝑖

𝑣 𝑖(𝑥 𝑖)𝑃𝑥 𝑖(𝑡)

𝑃𝑥 𝑖(𝑡 )=P( h𝐻𝑒𝑎𝑙𝑡 𝑠𝑡𝑎𝑡𝑒𝑖𝑛 𝑓𝑎𝑐𝑡𝑜𝑟 𝑖𝑎𝑡 𝑡𝑖𝑚𝑒𝑡 𝑖𝑠 𝑥𝑖)

Computation of expected cost under product structure

• Assumptions– Costs accrue in each factor as long as survival is maintained in other

factors– P(Survival through time t in factor i)– Costs add across factors

• Theorem (Hazen and Li 2010): Computational formulas are

35

E[Cost/person for factor i] =

E [𝐶𝑅𝑖 (𝑡 ) ]=∑𝑥 𝑖

𝑐(𝑥 𝑖)𝑃𝑥 𝑖(𝑡)

(cohort analysis in factor i)

Advantages of factored computation

• Computational work in cohort analysis is proportional to the number of state transitions

• Suppose the number of transitions in a factor with s non-death states is roughly s also.

• Then assuming s states in each factor and f factors,– sf transitions in the overall model under naïve cohort analysis – sf transitions in cohort decomposition– Big advantage for large f

• Caveat: s and f are not usually large.

36

Cohort decomposition issues

• How often are factors independent?– Ans: More often not probabilistically independent.– But one factor is almost always probabilistically independent:

Background mortality.• How reasonable is the product form for the quality

coefficient v(x)?– Empirical support for product form in HUI literature – additive

decomposition is not supported.– Often only one factor carries quality adjustments, in which case

product form holds by default.

37

Summary• These are just the basics

– Population modeling• Population model Jackson network• One can get at equilibrium population issues by solving the usual balance

equations for steady-state probabilities and scaling them up appropriately.– Product structure

• Common feature of medical Markov models• Recognizing it can assist in model formulation and presentation, as well as

computation.– Drawbacks for continuous-time models

• Medical researchers don’t “get” the models• Software not widely available

• There is more to do here …

Questions?

powerpoint presentationusers.iems.northwestern.edu/~hazen/medical markov mo… · ppt file · web...

Documents