estimating outcomes in decision analysis miriam kuppermann, phd, mph associate professor departments...
Post on 28-Dec-2015
223 Views
Preview:
TRANSCRIPT
Estimating Outcomes Estimating Outcomes in Decision Analysisin Decision AnalysisMiriam Kuppermann, PhD, MPHMiriam Kuppermann, PhD, MPH
Associate ProfessorAssociate Professor
Departments of Obstetrics, Departments of Obstetrics, Gynecology & Reproductive Gynecology & Reproductive Sciences and Epidemiology & Sciences and Epidemiology & BiostatisticsBiostatistics
University of California, San FranciscoUniversity of California, San Francisco
Review—Last LectureReview—Last Lecture
• Formulate an explicit questionFormulate an explicit question• Make a decision treeMake a decision tree• Estimate probabilitiesEstimate probabilities• Estimate utilitiesEstimate utilities• Compute expected utility of each branchCompute expected utility of each branch• Perform sensitivity analysisPerform sensitivity analysis
Preview—Where We Are Preview—Where We Are GoingGoing
Recall Ms. Brooks and her aneurysm.Recall Ms. Brooks and her aneurysm.
We want to: We want to: • Determine her utility: clip, and not clipDetermine her utility: clip, and not clip• Compare incremental utility and costCompare incremental utility and cost• Compare cost-per-unit of utility across Compare cost-per-unit of utility across
private and public uses of funds. private and public uses of funds.
Overview – Today’s Overview – Today’s LectureLecture
• OutcomesOutcomes• Measuring UtilitiesMeasuring Utilities• Quality Adjusted Life Years (QALYs)Quality Adjusted Life Years (QALYs)• Discounting Discounting • Prenatal testing examplePrenatal testing example
Quantifying Health OutcomesQuantifying Health Outcomes• Mortality Mortality • Life yearsLife years
Number of expected years of life Number of expected years of life • Significant morbiditySignificant morbidity
Paralysis, loss of sightParalysis, loss of sight• Quality-Adjusted Life Years (QALYs)Quality-Adjusted Life Years (QALYs)
Expected life years adjusted for the Expected life years adjusted for the valuation of the possible states in each yearvaluation of the possible states in each year
• Financial valuation of outcomesFinancial valuation of outcomesCosts to patient, payor, or societyCosts to patient, payor, or societyWillingness to pay to avoid outcomes, Willingness to pay to avoid outcomes,
obtain treatment obtain treatment
Health Outcomes – MortalityHealth Outcomes – Mortality• MortalityMortality
Death from disease/accident/procedureDeath from disease/accident/proceduree.g. If Ms. Brooks undergoes surgery, one of e.g. If Ms. Brooks undergoes surgery, one of the possible outcomes is mortalitythe possible outcomes is mortality
• Life YearsLife Years Calculate an expected value of life years Calculate an expected value of life years using a probabilistically weighted average of using a probabilistically weighted average of expected life expected life e.g.e.g. If Ms. Brooks does not undergo surgery, If Ms. Brooks does not undergo surgery, her life expectancy is less than if she did not her life expectancy is less than if she did not have aneurysm, these outcomes are have aneurysm, these outcomes are measured in expected life yearsmeasured in expected life years
Health Outcomes – MorbidityHealth Outcomes – Morbidity• MorbidityMorbidity
Some health state that is less than Some health state that is less than perfectperfecte.g. disability from stroke, chronic paine.g. disability from stroke, chronic pain
• Comparison of morbidities Comparison of morbidities Difficult – apples and oranges problem Difficult – apples and oranges problem e.g. which is worse:e.g. which is worse:Blind vs. deafBlind vs. deafDeaf vs. paraplegiaDeaf vs. paraplegiaParaplegia vs. blindParaplegia vs. blind
Outcomes - UtilityOutcomes - UtilityUtility is the currency of outcomesUtility is the currency of outcomes• Scaled from 0 to 1Scaled from 0 to 1• Commonly Death = 0Commonly Death = 0• Perfect Health = 1Perfect Health = 1
Sounds good, but how can this be Sounds good, but how can this be measured?measured?
Utility MeasurementUtility Measurement
Utilities are commonly estimated using Utilities are commonly estimated using comparisons to the 0 and 1 anchorscomparisons to the 0 and 1 anchors• Methods are:Methods are:• Visual Analog ScaleVisual Analog Scale• Standard GambleStandard Gamble• Time Trade-offTime Trade-off
Utility MeasurementUtility MeasurementClinical Scenario:Clinical Scenario:• Patient in the hospital has infection of the leg Patient in the hospital has infection of the leg • Two options: BKA v. Medical ManagementTwo options: BKA v. Medical Management• BKA – below the knee amputation (1% mortality BKA – below the knee amputation (1% mortality risk)risk)• Medical – 20% chance of infection worseningMedical – 20% chance of infection worsening
If worse, AKA – above the knee amputation If worse, AKA – above the knee amputation (10% mortality risk) (10% mortality risk) • How can we value these outcomes?How can we value these outcomes?
Utility MeasurementUtility Measurement
Visual Analog ScaleVisual Analog Scale
Linear Scale from 0 to 1Linear Scale from 0 to 1
(death 0-----------------------------1.0 cure)(death 0-----------------------------1.0 cure)Where is AKA?Where is AKA?
(death 0-----------------------------1.0 cure)(death 0-----------------------------1.0 cure)Where is BKA?Where is BKA?
Utility MeasurementUtility MeasurementVisual Analog ScaleVisual Analog Scale
Advantages: Advantages: quantitative, easy to quantitative, easy to understand, visual understand, visual
Disadvantages: Disadvantages: may bias values to the may bias values to the middle, seems disconnected from medical middle, seems disconnected from medical realityreality
Utility Measurement--Utility Measurement--Standard GambleStandard Gamble
Method relies on patients choosing between:Method relies on patients choosing between: 1) a certain outcome1) a certain outcome 2) a gamble between a better outcome and a 2) a gamble between a better outcome and a worse outcomeworse outcome
How it works:How it works: Choice 1Choice 1: You live with a : You live with a BKABKA Choice 2Choice 2: The gamble – you might have a : The gamble – you might have a curecure; ;
you might you might diedie..
Goal of method is to find the break-even point.Goal of method is to find the break-even point. What probability of death would you accept to avoid What probability of death would you accept to avoid
living with the BKA? living with the BKA?
Utility Measurement – Utility Measurement – Standard GambleStandard Gamble
Choose BKA?
Yes
No
BKA (intermediate outcome)
Perfect health
Death
Live?
p %
(100-p) %
Utility Measurement – Utility Measurement – Standard GambleStandard Gamble
Standard gamble measurement involves questioning patients Standard gamble measurement involves questioning patients to determine the p at which the two outcomes are equivalent to determine the p at which the two outcomes are equivalent
Using expected utilities, the value of p [p = P(cure)] gives the Using expected utilities, the value of p [p = P(cure)] gives the utility:utility: Utility (BKA) x Prob (BKA) = Utility (BKA) x Prob (BKA) = Utility(cure) x (p) + Utility(death) x (1-p)Utility(cure) x (p) + Utility(death) x (1-p) [Note that here, P(BKA) = 1.][Note that here, P(BKA) = 1.]
You can demonstrate that the utility of BKA = p:You can demonstrate that the utility of BKA = p:Utility (BKA) = [Utility(cure) x (p) + Utility(death) x (1-p)] Utility (BKA) = [Utility(cure) x (p) + Utility(death) x (1-p)] = [1.0 * p + 0 * (1-p)] = p= [1.0 * p + 0 * (1-p)] = p
Utility Measurement – Utility Measurement – Standard GambleStandard Gamble
In utility estimation utilizing the standard gamble In utility estimation utilizing the standard gamble measurement, using the simple comparison described, the measurement, using the simple comparison described, the value of p, the probability of perfect health needed to make value of p, the probability of perfect health needed to make the patient indifferent between the two choices is the the patient indifferent between the two choices is the estimated utility of the outcome.estimated utility of the outcome.
Advantages: Advantages: the incorporation of risk into the model, the incorporation of risk into the model, comparison or choice between different outcomes.comparison or choice between different outcomes.
Disadvantages: Disadvantages: possible nonrealistic choices patients may possible nonrealistic choices patients may be asked to make, the difficulty of understanding the be asked to make, the difficulty of understanding the question (especially for non-gamblers), and the difficulty for question (especially for non-gamblers), and the difficulty for some in understanding a math equation.some in understanding a math equation.
Utility Measurement – Utility Measurement – Time Trade-offTime Trade-off
Time Trade-off involves patients choosing between: Time Trade-off involves patients choosing between: quality of life vs. length of time alivequality of life vs. length of time alive
We want to determine when patients are equivocal between We want to determine when patients are equivocal between choice:choice: Time A * Utility A = Time B * Utility BTime A * Utility A = Time B * Utility B e.g. -- If you have a life expectancy of 30 years with a BKA, e.g. -- If you have a life expectancy of 30 years with a BKA, how much time would you give up to live in your current state?how much time would you give up to live in your current state?
Would you give up 5 years? 3 years? 1 year?Would you give up 5 years? 3 years? 1 year? 30 years * Utility (BKA) = (30-x) years * 1.0 If you’re willing 30 years * Utility (BKA) = (30-x) years * 1.0 If you’re willing to give up 3 years, that means the utility of BKA is: to give up 3 years, that means the utility of BKA is: = [(30-3)*1/ 30] = 27/30 = 0.9= [(30-3)*1/ 30] = 27/30 = 0.9
Utility Measurement – Utility Measurement – Time Trade-offTime Trade-off
Time trade off can be used to measure the utility of Time trade off can be used to measure the utility of an outcome in the fashion described. Like an outcome in the fashion described. Like standard gamble, patients need to understand standard gamble, patients need to understand subtlety of questions being asked.subtlety of questions being asked.
Advantages: Advantages: the simplicity of the choice between the simplicity of the choice between different outcomes, consideration of long-term different outcomes, consideration of long-term outcomes.outcomes. Disadvantages: Disadvantages: fails to incorporate risk, lack of fails to incorporate risk, lack of clarity of when time traded occurs, different clarity of when time traded occurs, different valuation of time during life, and the theoretical valuation of time during life, and the theoretical lack of realism of the choice.lack of realism of the choice.
Utility Measurement – Utility Measurement – Health State Excluding Health State Excluding
Clinical ProblemClinical Problem 0 to 1 utility scaling is simple and useful for determining the 0 to 1 utility scaling is simple and useful for determining the utility of different outcomes for a single patient.utility of different outcomes for a single patient.
However, comparisons across patients or across programs However, comparisons across patients or across programs raises the issue of ‘Is a utility of 1 really available?’raises the issue of ‘Is a utility of 1 really available?’
Consider: (a) polio vaccinations for children, and (b) hip Consider: (a) polio vaccinations for children, and (b) hip replacement surgery for 85-year old patients.replacement surgery for 85-year old patients.
Typically (on average) the health state of a polio-free 5-year old Typically (on average) the health state of a polio-free 5-year old will be very close to 1, whereas the health state of an 85-year old will be very close to 1, whereas the health state of an 85-year old even with a perfectly functioning hip will typically be less than even with a perfectly functioning hip will typically be less than one.one.
Utility Measurement – Utility Measurement – ChainingChaining
• The utility of the health state of the 85-year old can The utility of the health state of the 85-year old can be determined by “chaining”.be determined by “chaining”.
• Consider first the clinical situation under review: what Consider first the clinical situation under review: what is the utility, on a 0 to 1 scale, of a hip with reduced is the utility, on a 0 to 1 scale, of a hip with reduced function and some pain? Let us say utility = .8.function and some pain? Let us say utility = .8.
• Consider next the utility of an 85-year old with no hip Consider next the utility of an 85-year old with no hip problems, but some other reduced function or chronic problems, but some other reduced function or chronic pain. Here, perhaps overall utility = .9.pain. Here, perhaps overall utility = .9.
• Total utility for the bad-hip case = .8 * .9 = .72Total utility for the bad-hip case = .8 * .9 = .72 Here, the perfect-hip case utility = 1 * .9 = .9Here, the perfect-hip case utility = 1 * .9 = .9
Utility Measurement – Utility Measurement – Additional InformationAdditional Information
• Multi-Attribute Health Status Classification Multi-Attribute Health Status Classification SystemSystem
• Developed by Health Utilities, Inc.Developed by Health Utilities, Inc.
•Available at: Available at: http://www.healthutilities.com/overview.htmhttp://www.healthutilities.com/overview.htm
Utilities in Decision Utilities in Decision AnalysisAnalysis
• Now that we have methods to estimate Now that we have methods to estimate utilities, these can be used in the DAutilities, these can be used in the DA• However, our outcomes often include both However, our outcomes often include both mortality and morbiditymortality and morbidity• Want a way to add in life expectancyWant a way to add in life expectancy• Quality Adjusted Life-Years (QALYs)Quality Adjusted Life-Years (QALYs)
QALYsQALYs• QALYs are generally considered the QALYs are generally considered the standard unit of comparison for outcomes standard unit of comparison for outcomes
• QALYs = time (years) x quality (utils)QALYs = time (years) x quality (utils)
• e.g. 30 years after BKA, util(BKA) = 0.9e.g. 30 years after BKA, util(BKA) = 0.9
= 30 x 0.9 = 27 QALYs= 30 x 0.9 = 27 QALYs
QALYsNo aneurysm rupture0.9825
No surgery34.86 Die
Aneurysm rupture 0.450.0175 Survive
0.55
No aneurysm ruptureDifference 1
_ QALYs -2.85 Survive surgery0.902 Die
Aneurysm rupture 0.45Clipping 0 Survive
32.01 0.55Key Inputs Surgery-induced disabilityRupture risk/yr 0.0005 0.075Expected life span 35RR rupture w/ surgery 0 Surgical deathSurgical mortality 0.023 0.023Surg morb (disability) 0.075
0.0
Ms. Brooks
17.5
35.0Normal survival
Disability, shorter survival
5.8
Immediate death
Normal survival 35.0
Normal survival
Normal survival
Early death
Early death
35.0
17.5
35.0
QALYsQALYs
QALYsQALYsAneurysm ExampleAneurysm Example
• We said life expectancy is reduced by 2/3, We said life expectancy is reduced by 2/3, so instead of 35, it is = 35 * .333 = 11.67so instead of 35, it is = 35 * .333 = 11.67
• Here, we have assigned a utility of .5 to Here, we have assigned a utility of .5 to surgery-induced disability, so QALYs = surgery-induced disability, so QALYs =
years * utils = 11.67 * .5 = 5.8years * utils = 11.67 * .5 = 5.8
Outcomes - DiscountingOutcomes - Discounting• However, are all years considered However, are all years considered equal?equal?
• Consider:Consider: Favorite MealFavorite Meal
Extreme PainExtreme Pain
Lifetime IncomeLifetime Income
Outcomes - DiscountingOutcomes - Discounting• Generally, present > futureGenerally, present > future• One common way to value the different times One common way to value the different times is discounting is discounting • Essentially this year is worth Essentially this year is worth δδ more than more than next yearnext year• δδ is commonly set at 0.03 or 3% is commonly set at 0.03 or 3%• In order to compare values of all future times, In order to compare values of all future times, a calculation, net present value, is often useda calculation, net present value, is often used• NPV = 1 / (1 + NPV = 1 / (1 + δδ))t t Where t is number of years Where t is number of years in the futurein the future
Outcomes - DiscountingOutcomes - Discounting• Aneurysm ExampleAneurysm Example• If utility is 0.5 and life expectancy is 3 If utility is 0.5 and life expectancy is 3 yearsyears• NPV would be: NPV would be: Utility / (1 + Utility / (1 + δδ))t t
• However, when is year 1? Often, since However, when is year 1? Often, since events in year one occur on average half events in year one occur on average half way through, we use 0.5 for year 1way through, we use 0.5 for year 1• NPV = 0.5 / (1.03)NPV = 0.5 / (1.03)0.50.5 + 0.5 / (1.03) + 0.5 / (1.03)1.51.5 + 0.5 / + 0.5 / (1.03)(1.03)2.52.5
• NPV = 0.5 * {(1.03)NPV = 0.5 * {(1.03)-0.5 -0.5 + (1.03) + (1.03) -1.5-1.5 + (1.03) + (1.03) --
2.52.5}}
QALYsdisc
No aneurysm rupture0.9825
No surgery21.37 Die
Aneurysm rupture 0.45
0.0175 Survive0.55
No aneurysm ruptureDifference 1
Δ QALYs -1.63 Survive surgery0.902 Die
Aneurysm rupture 0.45
Clipping 0 Survive19.74 0.55
Key Inputs Surgery-induced disabilityRupture risk/yr 0.0005 0.075
Expected life span 35RR rupture w/ surgery 0 Surgical deathSurgical mortality 0.023 0.023
Surg morb (disability) 0.075
0.0
Ms. Brooks
13.3
21.4Normal survival,
worry
Disability, shorter survival
4.8
Immediate death
Normal survival,worry
21.4
Normal survival
Normal survival
Early death,worry
Early death
21.5
13.4
21.5
Outcomes - DiscountingOutcomes - Discounting
Outcomes - DiscountingOutcomes - Discounting• Another issue is partial yearsAnother issue is partial years• Can use similar adjustment:Can use similar adjustment:• Exponent is half way through time periodExponent is half way through time period• Numerator is multiplied by fraction of yearNumerator is multiplied by fraction of year• e.g. disability for 1.6 years, utility of 0.5e.g. disability for 1.6 years, utility of 0.5• NPV = NPV = Utility / (1 + Utility / (1 + δδ))t t
• NPV = 0.5 / (1.03)NPV = 0.5 / (1.03)0.50.5 + 0.5*0.6 / (1.03) + 0.5*0.6 / (1.03)1.31.3
Discounting – Special TopicDiscounting – Special Topic
• Issues with exponential discounting:Issues with exponential discounting:• Relatively easy to manipulateRelatively easy to manipulate• However, assumes same difference However, assumes same difference between any two time periods is the same between any two time periods is the same value difference value difference • ConsiderConsider
today today : : vs. tomorrow and vs. tomorrow and 10 yrs vs. 10 yrs + 1 day 10 yrs vs. 10 yrs + 1 day
• Essentially, “today” versus all other time Essentially, “today” versus all other time periods is valued higher for many outcomesperiods is valued higher for many outcomes
Discounting – Special TopicDiscounting – Special Topic• Importance of discount rate chosen:Importance of discount rate chosen:• Imagine a benefit of $1,000,000 to be Imagine a benefit of $1,000,000 to be received in 30 yearsreceived in 30 years• Present value of this benefit at different Present value of this benefit at different discount rates:discount rates:
1% -- $741,9231% -- $741,923
3% -- $411,9873% -- $411,987
8% -- $99,3778% -- $99,377
17% -- $9,00417% -- $9,004
Overall ReviewOverall Review• OutcomesOutcomes
MortalityMortalityMorbidityMorbidity
• Measuring UtilitiesMeasuring UtilitiesVisual AnalogVisual AnalogStandard GambleStandard GambleTime Trade-offTime Trade-off
• Quality Adjusted Life Years (QALYs)Quality Adjusted Life Years (QALYs) QALYs = time (years) x quality (utils)QALYs = time (years) x quality (utils)• Discounting Discounting
NPV = NPV = Utility / (1 + Utility / (1 + δδ))t t
Informed Decision-Making in Prenatal Testing
Miriam Kuppermann, PhD, MPHAssociate Professor
Decision and Cost-Effectiveness Analysis EPI 213
January 20, 2005Department of Obstetrics, Gynecology, & Reproductive Sciences
Prenatal Tests for Chromosomal Disorders
Screening Tests (non-invasive)Expanded AFP Nuchal translucency
Diagnostic Tests (invasive)Amniocentesis Chorionic villus sampling (CVS)
Guidelines For Prenatal Testing Are Dichotomized By Maternal Age
Women < 35 Screening first Invasive testing only
if “positive” results
Women > 35 Invasive testing
offered Screening as an
option
Rationale for GuidelinesNeed to limit access to invasive testing
Inherent risk of procedure Limited availability of providers,
laboratoriesAge 35 selected as the threshold
Increasing risk with advancing maternal age
Threshold set where risks equal Cost/benefit considerations
Kuppermann, Nease, Goldberg, Washington. Who should be offered prenatal diagnosis? The 35-year-old question. Am J Public Health 1999; 89:160-3
Equal Risk Threshold
Risk of Miscarriage = Risk of Down Syndrome
Implicit assumption: equally burdensome outcomes
Procedure-related miscarriage
Down-syndromeaffected infant
First Challenge to Guidelines
Do women find procedure-related miscarriage and Down-syndrome-affected birth to be equally burdensome?
How Do Women Value Testing Outcomes?
1082 socioeconomically and age-diverse women
English-, Spanish- or Chinese-speaking
Interviewed <20 weeks pregnant Preferences for testing outcomes Demographic/attitudinal questions Testing behavior
Simplified Decision Tree for Prenatal Testing
Preference Score
“Utility” Ranges from 0 to 1 1 = ideal outcome, 0=death Measured using “time tradeoff””
Time Tradeoff Preference Elicitation
Choice A Choice B
Which do you prefer?
40 years of life remaining with affected child
40 years of life remaining with unaffected child (give up 0 years of life)
Time Tradeoff Preference Elicitation
40 years of life remaining with affected child
30 years of life remaining with unaffected child
(give up 10 years of life)
Which do you prefer?
Choice A Choice B
Calculation of Time Tradeoff Scores
reduced life expectancy with unaffected child (30 years)UTTO = __________________________________________
full life expectancy with affected child (40 years)
= 0.75
Median utility for procedure-related
miscarriage= 0.86
Median utility for Down-syndromeaffected infant
= 0.73
On average, women do not equally weight the outcomes of procedure-related
miscarriage and Down syndrome-affected birth
P<0.001 by Wilcox sign rank test
Kuppermann, Nease, Learman, Gates, Blumberg, Washington. Procedure-related miscarriages and Down syndrome-affected births: implications for prenatal testing based on women’s preferences. Obstet Gynecol 2000; 96:511-6.
Preference Difference Score
One way to look at the relative value women assign to procedure-related miscarriage and DS-affected birth
Pref score misc – Pref score DS
Higher score = greater preference for miscarriage over DS
0
25
50
75
100
125
150
175
200
Num
ber
-1 -.75-.5 -.25 0 .25 .5 .75 1
Preferences Vary Substantially
Utility misc - Utility DS
First Conclusion
Current guidelines do not adequately reflect the preferences of pregnant women.
Second Challenge to GuidelineSecond Challenge to Guideline
Old paradigm: COST BENEFITBenefits (in $$ terms) of program should exceed costs. Costs of offering testing should be offset by savings accrued by averting the birth of Down-syndrome-affected infants
New paradigm: COST EFFECTIVENESSNo $$ value assigned to outcomes. Cost of offering testing should be “worth” the gain in quantity and quality of life.
How is Cost Effectiveness Measured?
Δ CC
Δ EE
Δ Costs
Δ QALY==
QALY = Quality–Adjusted Life Year
Cost Effectiveness of Prenatal Diagnosis QALYs Lifetime cost Cost-utility ratio
Age 20
Amniocentesis 24·16 $54,080 $14,200
No testing 24·08 $52,940
Age 35
Amniocentesis 20·39 $61,490 $12,600
No testing 20·30 $60,360
Age 44
Amniocentesis 17·08 $59,020 $11,300
No testing 16·98 $57,890
Harris, Washington, Nease, Kuppermann. Cost utility of prenatal diagnosis and the risk-based threshold. Lancet 2004; 363:276-82.
Second Conclusion
Offering invasive testing to women of all ages and risk levels can be cost effective.
Implications
Guidelines should be changed to enable all women to make informed choices about which prenatal tests, if any, to undergo.
Tools that help women and partners make informed choices should be implemented.
top related