health economics ii – 2010 health economic evaluations part vi lecture 4 decision analytic...
TRANSCRIPT
Health Economics II – 2010 Health Economic Evaluations Part VI
Lecture 4
Decision analytic modellingPresentation and use of economic evaluations
Nils-Olov Stålhammar
Measurement vs. decision analysis
Measurement
• Focus on estimating and testing hypotheses relating to particular parameters
• Concentration on relatively few parameters
• Focus on uncertainty in parameters
• Randomized trials as a vehicle for measurement
Decision analysis
• Focus on identifying an appropriate course of action
• Informing decisions • Identification of a preferred
option based on expected values of the alternatives
• Explicit acceptance that there always will be uncertainty
The need for comparison of all relevant options - modelling when relevant clinical trials do not exist
• Synthesizing head-to-head comparisons– Clinical trials sometimes compare new compound to
placebo or – at least – omit some relevant active comparators
• Informing decisions in the absence of hard data– Clinical trials may not be feasible because of
• Ethical considerations - for instance, when standard care is compared to less aggressive treatment
• Sample size requirements - screening
• Long follow-up – vaccination
Modelling in economic evaluationsalongside clinical trials
• Combining several data sources to reflect all appropriate evidence– Resource use, unit costs, utilities etc.
• Extrapolating from intermediate clinical endpoints to final outcomes– From, for instance, clinical events avoided to life years
gained (or, preferably, QALYs gained)
• Extrapolating beyond the period observed in a trial
Extrapolating beyond the clinical trial time period• Same time horizon as in clinical trial
– “Stop and drop”; this within trial measure will be an underestimate• If longer time horizon; should treatment be assumed to
continue?– If yes, should we assume same effect as in clinical trial or reduced
effect?– If no, should we assume
• same rate of death for everyone after trial, or,• higher rate of death in patients who received intervention, or,• that treatment gives a continued benefit resulting in lower rate
of death?• Can the shapes of the survival curves within the trial give
information?• The biology of the intervention?• Important to run alternative scenarios • Low compliance in the long run, discounting and whish to
minimise number of assumptions may to some extent justify assumption about stop of treatment
• Inappropriate use of clinical data* efficacy rather than effectiveness* not representative of appropriate setting and/or country* biased selection
• Uncertainty in extrapolations based on assumptions
• Sometimes a 'black box‘ – lack of transparency
Concerns about modelling
• Keep it simple
• Transparent presentation
• Be concerned about data quality
• Explore uncertainty through sensitivity analysis
• Try to validate the model– Let someone else build a copy of the model in another
programming language
– Validate (parts of) output against other analyses/data
Recommendations about modelling
Modelling
Tools for modelling
• Decision Trees - a diagram which illustrates all alternative courses of action in response to a specific problem
• Markov Models - describes several discrete states between which a person may move as time passes
Decision tree comparing omeprazole and ranitidine in reflux oesophagitis (Based on Lindberg & Jönsson, Läkartidningen 1992;89:2530-3)
Probability Drug cost Effect Expected P C E Cost Effect P x C P x E
0.14 2,280 0 319 0.00
0.17 2,280 1 388 0.17
0.12 1,192 1 143 0.12
0.57 596 1 340 0.57
0.38 772 1 293 0.38
0.33 1,860 1 614 0.33
0.29 1,860 0 539 0.00
1.00 1,190 0.86
1.00 1,446 0.71
Sum
Sum
RO
Ome 20 mg
Unhealed;Ome 20 mg
Unhealed;Ome 40 mg
Unhealed
Healed
Healed
Healed0.43
0.72
0.46
0.57
0.28
0.54
Healed
Healed
Unhealed;Ome 40 mg
Unhealed
0.38
0.54
0.62
0.46
Ran 150 mg x 2
Week 0 4 8 12
Decision tree comparing omeprazole and ranitidine in reflux oesophagitis (Based on Lindberg & Jönsson, Läkartidningen 1992;89:2530-3)
Probability Drug cost Effect Expected P C E Cost Effect P x C P x E
0.14 2,280 0 319 0.00
0.17 2,280 1 388 0.17
0.12 1,192 1 143 0.12
0.57 596 1 340 0.57
0.38 772 1 293 0.38
0.33 1,860 1 614 0.33
0.29 1,860 0 539 0.00
1.00 1,190 0.86
1.00 1,446 0.71
Sum
Sum
RO
Ome 20 mg
Unhealed;Ome 20 mg
Unhealed;Ome 40 mg
Unhealed
Healed
Healed
Healed0.43
0.72
0.46
0.57
0.28
0.54
Healed
Healed
Unhealed;Ome 40 mg
Unhealed
0.38
0.54
0.62
0.46
Ran 150 mg x 2
Week 0 4 8 12
Decision node
Chance node
Branch probability
A pathway
Pathway probabilities
Pathway ’values’
Expected values
Markov processes
• A convenient way of modelling problems with repetitive events
Well
Sick Dead
Markov processes• The patient is always in one of a finite number of states
- Markov states
• The time horizon is divided into equal increments of time - Markov cycles; the length of each cycle depends on the problem studied
• During each cycle the patient may make a transition from one state to another - this is how events are modelled
• States can be transient, temporary (just once) or absorbing
• Transition probabilities can be constant or time-dependent (no. of cycles)– When the transition probability is constant the Markov models
are referred to as Markov chains
Markov processes
• Each state is assigned a cost and an effect, the contribution to the overall result depends on the time spent in the state
• The Markovian assumption (a key limitation): – The transition probabilities depend only on the
current health state – disease history is not taken into account• Can be circumvented by more health states
Markov processes
• If probability of recurrence and/or death changes after first occurrence of illness
Well 1
Sick 1 Dead
Well 2
Sick 2
A 2nd ‘Well’ state to allow for a different set of probabilities
Markov cohort simulation - an example
Well
Sick Dead
P=0.2P=0.3
P=0.2
P=0.1
QALY-weight of the Well state = 1
QALY-weight of the Sick state = 0.7
Markov cohort simulationThe Markov trace
Cycle Well Sick Dead Cycle Total
Sum Sum
Start 1 0 0 - -
1 0.70 0.20 0.10 0.84 0.84
2 0.55 0.24 0.21 0.72 1.56
3 0.46 0.23 0.31 0.62 2.18
4 0.39 0.21 0.41 0.53 2.71
5 0.33 0.18 0.48 0.46 3.17
6 0.29 0.16 0.55 0.40 3.57
. . . . . .
The Markov model is typically run until all of the cohort are in the dead state – at least when life threatening diseases are studied
Markov cohort simulationThe Markov trace
Cycle Well Sick Dead Cycle Total
Sum Sum
Start 1 0 0 - -
1 0.70 0.20 0.10 0.84 0.84
2 0.55 0.24 0.21 0.72 1.56
3 0.46 0.23 0.31 0.62 2.18
4 0.39 0.21 0.41 0.53 2.71
5 0.33 0.18 0.48 0.46 3.17
6 0.29 0.16 0.55 0.40 3.57
. . . . . .
The Markov model is typically run until all of the cohort are in the dead state – at least when life threatening diseases are studied
70% of those who are well remain well30% of those who are sick get well
0.70*0.70+0.30*0.20=0.49+0.06=0.55
50% of those who are sick remain sick20% of those who are well get sick
0.50*0.20+0.20*0.70=0.10+0.14=0.25
Half-cycle correction in Markov models
• It’s assumed that all transitions occur at the end of each cycle – in reality, most transitions occur gradually (on average, half-way through)
• Rewards will be overestimated since a transitioning individual will have received a full cycle’s worth at the beginning of the cycle
• Most correct would be to implement a half cycle correction as a toll at each transition
• Instead, as an approximation, a) subtracting a half –reward at the beginning of the process (cycle 0), b) members remaining in the state when the process terminates, are given back the half-reward
Half-cycle correction in Markov models
• Two states with utility 1 and 0.5 respectively; 10 cycles; all individuals start in state 1
• The utility for an individual transitioning in the middle of cycle 4 will be overestimated ; 7.5 (=5*1 + 5*0.5) vs. 7.25 (=4.5*1 +5.5*0.5)
• Correct solution would be to assign only a half-reward at the start of each cycle, and then at each transition a half-reward corresponding to the jump state (the state the individual will be in during the next cycle)
• The approximation: a) subtract a half –reward at the beginning of the process (cycle 0), b) give back a half-reward to individuals remaining in the state at the end of the process (Do it for all states!)
1
0.5Cycles
0.5 1 1 1 1 0.5 0.5 0.50.5 0.5 0.25 =7.25
Micro Simulation / 1st order Monte Carlo Simulation / Random walk / Random trial
• One individual at a time transition through the model
• For each transition a random number is drawn from a Uniform [0,1] distribution – for the first transition:– If in range [0, 0.1], move to Death
– If in range [0.1, 0.3], move to Sick
– If in range [0.3, 1], stay in Well
• With a large number of individuals, the means will be the same as for Markov cohort simulation
• The main advantage is the ability to capture clinical history– A binary variable can be defined to distinguish between first
occurrence of a disease and recurrences
– Transition probabilities can depend on the value of the binary variable
Monte Carlo simulation
Cycle Well Sick Dead
Start
1
2
3
4
5
6
X
X
X
X
X
X
X
Binary variable for history of illness would be set to 1
1st and 2nd order uncertainty
• Overall variability between patients– 1st order uncertainty– Reflected in standard deviations associated with a mean
value– Micro simulation (one individual at a time with
‘known’ transition probabilities) will illustrate this uncertainty
• Parameter uncertainty– 2nd order uncertainty– Uncertainty in mean parameter values– Reflected in standard error of the mean– Probabilistic Sensitivity Analysis (PSA) increasingly
used to illustrate this uncertainty (clear advantages compared to simple sensitivity analysis)
Probabilistic sensitivity analysis (PSA)• A probability distribution is assigned to uncertain parameters
– In practice only a relatively small number of distributions are relevant to consider
– Parameters are sampled from assigned distributions – N samples • For each sample the model can be evaluated with
– Markov Cohort simulation (Expected value calculation)
or– Micro simulation/1st order Monte Carlo Simulation/Random
walk/Random trial; X patients are sent through the model and means are calculated
• The distribution of the results from the N samples reflects the uncertainty
• Present as– Confidence interval around ICER or incremental net benefit– Scatter-plot in the C-E plane– CEAC
Handling uncertainty in modelling based analyses
• Methodological– Reference case, sensitivity analysis
• Parameter uncertainty– Probabilistic sensitivity analysis
• Modelling uncertainty– Sensitivity analysis
• Generalizability– Sensitivity analysis
QALY league tableIntervention £/QALY at 1990 pricesCholesterol testing and diet therapy (all adults aged 40–69) 220Neurosurgical intervention for head injury 240GP advice to stop smoking 270Neurosurgical intervention for subarachnoid haemorrhage 490Antihypertensive treatment to prevent stroke (ages 45–64) 940Pacemaker implantation 1,100Hip replacement 1,180Valve replacement for aortic stenosis 1,410Cholesterol testing and treatment (all adults aged 40–69) 1,480Docetaxel (as opposed to paclitaxel) in treatment of recurrent metastatic breast cancer 1,890CABG (left main-vessel disease, severe angina) 2,090Kidney transplantation 4,710Breast cancer screening 5,780Heart transplantation 7,840Cholesterol testing and treatment incrementally (all adults aged 25–39) 14,150Home haemodialysis 17,260CABG (one-vessel disease, moderate angina) 18,830Hospital haemodialysis 21,970Erythropoietin treatment for anaemia in dialysis patients (assuming 10% reduction in mortality) 54,380Addition of interferon-α2b to conventional treatment in newly diagnosed multiple myeloma 55,060Neurosurgical intervention for malignant intracranial tumours 107,780Erythropoietin treatment for anaemia in dialysis patients (assuming no increase in survival) 126,290
Adapted from Hutton J et al. PharmacoEconomics 1996; 9(Suppl 2): 8–22.; Maynard A. The
Economic J 1991; 101: 1277–86.; Nord E, et al. PharmacoEconomics 1997; 12: 89–103.
Caveats re. QALY league table
• Is the methodology of the studies sound and homogenous?– Rate of discount
– The method for estimating health state preferences
– Range of costs and consequences considered
– The setting
– Choice of comparison programme
Caveats re. cost-effectiveness threshold• Society’s WTP for an additional QALY will depend on
the size of the program– Would like to know the true opportunity cost
• Ethical reasons may influence society’s WTP • Evidence that people are concerned with the
distribution:– Unwilling to discriminate between patients on the grounds of
the size of the QALY benefit– Tendency to distribute resources to equalise outcomes – prefer
to treat severely ill patients even if the benefit is low– Health benefits to younger people are valued higher than
similar benefits to older people– Many whish to give lower priority to treatment of illness
caused by life style– Whish to discriminate in favour of those with dependants
Cost per QALY gained thresholds Acceptable additional cost per QALY gained by using a
more effective treatment strategy
No official thresholds, but…– UK1: £ 30,000 (≈ € 45,000)
– US2: USD 50,000 -100,000 (≈ € 40,000 – 80,000)
– Sweden3: SEK 500,000 (≈ € 55,000)
1. Raftery J. NICE: faster access to modern treatments? Analysis of guidance on health technologies. British Medical Journal 2001;323:1300-3.
2. Ubel P, Hirth R, Chernew M, Fendrick M. What is the price of life and why doesn’t it increase at the rate of inflation? Arch Intern Med 2003:163;1637-41.
3. Socialstyrelsens riktlinjer för hjärtsjukvård. Artikelnummer 2004-102-2. Available from: http://www.sos.se.
0
50
100
150
200
250
PBAC Decisions: Some Correlation With Cost-Effectiveness
Cost/life year gained (AUS$)
Recommended at Price
Recommended at Lower
Price
Reject
Source: George et al, 1999
0
50
100
150
200
250
Decisions Are Not Driven Only By Cost-Effectiveness
Cost/life year gained (AUS$)
Recommended at Price
Recommended at Lower
Price
Reject
Similar cost-effectiveness but different
outcome
Source: George et al, 1999
Probability of rejection by NICE
From: Devlin N et al. Health Economics 2004; 13:437-52
Model 1 – ICER
Model 2 – ICER, UNCERTAINTY
Model 3 – ICER, UNCERTAINTY, BURDEN
Model 4 – ICER, UNCERTAINTY, BURDEN, OTHER THERAPY
Transferability of results from economic evaluations affected by
• Basic demography and epidemiology
• Availability of health care reosurces
• Treatment traditions
• Incentives to health care givers• Relative prices• Population values for WTP and/or
utilities
Old exam question
Assume that you are planning for a modelling analysis of a choice between two treatments.
a) You consider to set up a Markov model but you realise that the ‘Markov assumption’ is too restrictive, i.e. you would like to build in memory allowing for an individuals disease history to affect transition probabilities. Describe (conceptually) two ways in which this can be done.
b) The main source of uncertainty is the uncertainty around the estimates of the treatment effects. Describe the main elements of a probabilistic sensitivity analysis (PSA) when it is used to illustrate this uncertainty in the modelling analysis.
c) The result from a PSA can be presented as a scatter plot in the so called cost-effectiveness plane. Illustrate how a cost effectiveness acceptability curve is derived from such a scatter of points.
Old exam question
Regarding the transferability of economic evaluation results from one country to another.
Describe at least four factors/aspects which usually differ between countries and which are likely to affect the cost-effectiveness of various interventions, thereby limiting the transferability of economic evaluation results from one country to another.
Old exam question
a) Regarding assessment of uncertainty.
a) What are the main drawbacks of a conventional sensitivity analysis?
b) What is a threshold analysis?
c) Describe the main elements of a probabilistic sensitivity analysis (PSA) when used to illustrate parameter uncertainty in a model.
d) What is a cost-effectiveness acceptability curve (CEAC) and how can it be derived from a CE-scatter (i.e., a scatter of points on the so called cost-effectiveness plane)?
Old exam question
A mechanistic use of a cost-effectiveness threshold to allocate health care resources would imply that all QALYs have the same value regardless of who the gainer is. But several studies have shown that the general public are concerned with the distribution of health gains and that there is an unwillingness to discriminate between patients on the grounds of the size of the QALY benefit only. Describe a situation (a patient group) where the general public – according to studies – consider the social value of a QAYL-gain to be higher than what otherwise would be regarded as normal. Similarly, describe a situation (a patient group) where the general public – according to studies – consider the social value of a QAYL-gain to be lower than what otherwise would be regarded as normal.
Old exam question A decision tree is presented below. The following terms represent different parts of the decision tree: Pathway Probabilities, Decision Node, Expected Values, Branch Probability, Chance Node, Pathway, Pathway Values.Associate each term with a number 1-7, as shown in the second figure, to indicate which part each term represents.
Decision tree comparing omeprazole and ranitidine in reflux oesophagitis (Based on Lindberg & Jönsson, Läkartidningen 1992;89:2530-3)
Probability Drug cost Effect ExpectedP C E Cost Effect
P x C P x E
0.14 2,280 0 319 0.00
0.17 2,280 1 388 0.17
0.12 1,192 1 143 0.12
0.57 596 1 340 0.57
0.38 772 1 293 0.38
0.33 1,860 1 614 0.33
0.29 1,860 0 539 0.00
1.00 1,190 0.86
1.00 1,446 0.71
Sum
Sum
RO
Ome 20 mg
Unhealed;Ome 20 mg
Unhealed;Ome 40 mg
Unhealed
Healed
Healed
Healed0.43
0.72
0.46
0.57
0.28
0.54
Healed
Healed
Unhealed;Ome 40 mg
Unhealed
0.38
0.54
0.62
0.46
Ran 150 mg x 2
Week 0 4 8 12
Decision tree comparing omeprazole and ranitidine in reflux oesophagitis (Based on Lindberg & Jönsson, Läkartidningen 1992;89:2530-3)
Probability Drug cost Effect ExpectedP C E Cost Effect
P x C P x E
0.14 2,280 0 319 0.00
0.17 2,280 1 388 0.17
0.12 1,192 1 143 0.12
0.57 596 1 340 0.57
0.38 772 1 293 0.38
0.33 1,860 1 614 0.33
0.29 1,860 0 539 0.00
1.00 1,190 0.86
1.00 1,446 0.71
Sum
Sum
RO
Ome 20 mg
Unhealed;Ome 20 mg
Unhealed;Ome 40 mg
Unhealed
Healed
Healed
Healed0.43
0.72
0.46
0.57
0.28
0.54
Healed
Healed
Unhealed;Ome 40 mg
Unhealed
0.38
0.54
0.62
0.46
Ran 150 mg x 2
Week 0 4 8 12
5
1
2
6
3
7
4