lecture 3 2015
DESCRIPTION
economic evaluationTRANSCRIPT
-
Deakin Health Economics1
HSH719 Economic Evaluation 1
Topic 3:CONTEXT: Methods and Stages:
The Key Stages of an Economic Evaluation
-
Deakin Health Economics2
Economic Evaluation: Context, Method and Stages
Learning objectives:1. Know the logical stages of an economic evaluation2. Be familiar and know how to use quality assessment criteria for economic evaluations3. Understand the different ways in which the results of economic evaluations can be presented. Know how to calculate a simple ICER
-
Deakin Health Economics3
Fig 1: The Steps in Undertaking an Economic Appraisal
Step One: Deciding upon the study question
Step Two: Clear statement of alternatives to be appraised
Step Three: Assessment of costs and benefits of both alternatives
a) Identification of the appropriate costs and benefits to include in the
appraisal
b) Measurement of resources used and saved by the program alternatives,
and the outcomes produced by each
c) valuing resources used (and saved) and valuing outcomes
Step Four: Adjusting for timing
Discounting for the time stream of costs and outcomes
Step Five: Adjusting for risk and uncertaintyModeling and sensitivity analysis
Step Six: Making a decisionCalculating and using decision rules
a) Net present value of programme
b) Comparing cost-effectiveness ratios
-
Deakin Health Economics4
Figure 2: Basic Types of Economic Evaluation
COSTS CONSEQUENCES
HEALTH CARE
PROGRAM
Patient and family
(C2 )
Other sectors (C3 )
Healthcare sector
(C1 )
Identification Measurement Valuation
Health state
changed
Other value
created
Resources saved
Effects (E)
Health state preferences
(U)
Healthcare sector (S1 )
Patient and family
(S2 )
Other sectors (S3 )
Resources saved
Other value created
(V)
Willingness-to-pay
(W)
or
Global
willingness
-to-pay
(W)
Resources
consumed
-
Deakin Health Economics5 5
Update of Figure 2
COSTS
CONSEQUENCES
HEALTH CARE
PROGRAM
Patient & family (C3)
Other sectors (C2)
Healthcare sector
(C1 )
Identification Measurement Valuation
Health state changed
Other value created
Resources
saved
Effects (E)
Health state preferences (U)
Health sector (S1 )
Patient & family (S3)
Other sectors (S2)
Other value created (V)
Willingness-to-pay (W)
or
Global
willingness-
to-pay (W)
Resources
consumed
Drummond textbook p19 Figure 2.1: Components of economic evaluation in health care
Productivity
losses (C4)
Productivity gains
(S4)
-
Deakin Health Economics6 6
Key Questions
Is the methodology employed in the study appropriate and are the results valid?
If the results are valid, would they apply to my setting?
Reference: Drummond et al (2005) CHEERS checklist
-
Deakin Health Economics7 7
Study Frame
Gold et al 1995 (Washington Panel) outline two key aspects on an economic appraisal The Study Frame and the Study Design
The Study Frame covers the vital conceptualization of the evaluation purpose and methods this is really about a GOOD study question Objectives; decision context; intended audience
Perspective and choice of evaluation techniques (CEA; CUA; CBA; CCA; PBMA: etc) and ICERs
Choice of intervention(s) and comparators
Target population
Study boundaries
Time horizon (for intervention and tracking costs/outcomes)
-
Deakin Health Economics8 8
Study Design
The Study Design specifies the key elements of data collection and analysis
There are four basic decisions Appropriateness of the analyses used to describe the intervention and
its effects on health outcomes (modelled versus a within trial design)
Appropriateness of data collections for activities, costs and outcomes for the intervention and comparators
Appropriateness of the analysis used to combine the information (modelling assumptions; uncertainty analysis; sensitivity analysis; discounting; shadow pricing)
Appropriateness of the interpretation/ conclusions, given the above
-
Deakin Health Economics9
How can we decide if the study is any good?
Criteria checklists
There are a few in the literature
Best known are: Drummond 10 point Checklist (Chapter 3
textbook)
CHEERs checklist (in your readings)
Fundamentally cover similar ground CHEERS includes more explicit criteria around
modelling studies
-
Deakin Health Economics10 10
Drummonds Checklist
Did study examine both costs and effects of the service(s) or programme(s)?
Did the study involve a comparison of alternatives?
Was a viewpoint for the analysis stated and was the study placed in any particular decision-making content?
1. Was a well-defined question posed in answerable
form?
-
Deakin Health Economics11 11
Drummonds Checklist
Were any important alternatives omitted?
Was (Should) a do-nothing alternative (be) considered?
2. Was a comprehensive description of the competing
alternatives given?
(ie can you tell who did what to whom, where and how
often?)
-
Deakin Health Economics12 12
Drummonds Checklist
Was this done through a RCT? If so, did the trial protocol reflect what would happen in regular practice?
Was effectiveness established through an overview of clinical studies?
Were observational data or assumptions used to established effectiveness? If so, what are the potential biases in results?
3. Was the effectiveness of the programmes or services
established?
-
Deakin Health Economics13 13
Drummonds Checklist
Was the range wide enough for the research question at hand?
Did it cover all relevant viewpoints?
Were capital costs, as well as operating costs, included?
4. Were all the important and relevant costs and
consequences for each alternative identified?
-
Deakin Health Economics14 14
Drummonds Checklist
Were any of the identified items omitted from measurement? If so, does this mean that they carried no weight in the subsequent analysis?
Were there any special circumstances (eg joint use of resources) that made measurement difficult? Were these circumstances handled appropriately?
5. Were costs and consequences measured accurately in appropriate physical units?(eg hours of nursing time, number of physician visits, lost work-days, gained life-years?)
-
Deakin Health Economics15 15
Drummonds Checklist
Were the sources of all values clearly identified? (Possible sources include market values, patient or client preferences and views, policy-makers views and health professionals judgements).
Were market values employed for changes involving resources gained or depleted?
Where market values were absent (eg volunteer labour) what valuation sources were used?
Was the valuation of consequences appropriate for the question posed. CEA vs CUA vs CBA
6. Were costs and consequences valued credibly?
-
Deakin Health Economics16 16
Drummonds Checklist
Were costs and consequences which occur in the future discounted to their present values?
Was any justification given for the discount rate used?
7. Were costs and consequences adjusted for differential
timing?
-
Deakin Health Economics17 17
Drummonds Checklist
Were the additional (incremental) costs generated by one alternative over another compared to the additional effects, benefits or utilities generated?
More on this latter!!
8. Was an incremental analysis of costs and
consequences of alternatives performed?
-
Deakin Health Economics18 18
Drummonds Checklist
If data on costs or consequences were stochastic, were appropriate statistical analyses performed?
If a sensitivity analysis was employed, was justification provided for the ranges of values (for key study parameters)?
Were study results sensitive to changes in the values (within the assumed range for sensitivity analysis, or within the confidence interval around the ratio of costs to consequences)?
9. Was allowance made for uncertainty in the estimates
of costs and consequences?
-
Deakin Health Economics19 19
Drummonds Checklist
Were the conclusions of the analysis based on some overall index or ratio of costs to consequences (eg cost-effectiveness ratio)? If so, was the index interpreted intelligently or in a mechanistic fashion?
Were the results compared with those of others who have investigated the same question? If so, were allowances made for potential differences in study methodology?
Did the study discuss the generalisability of the results to other settings and patient/client groups?
10. Did the presentation and discussion of study results
include all issues of concern to users?
-
Deakin Health Economics20 20
Drummonds Checklist
Did the study allude to, or take account of, other important factors in the choice or decision under consideration (eg distribution of costs and consequences, or relevant ethical issues)?
Did the study discuss issues of implementation, such as the feasibility of adopting the preferred programmegiven existing financial or other constraints, and whether any freed resources could be redeployed to other worthwhile programmes?
10. Did the presentation and discussion of study results
include all issues of concern to users? cont/d..
-
Deakin Health Economics21 21
Economic Evaluation: Traps for Beginners
1. No comparatorThe essential question economic appraisal asks is what difference does it make?
2. Study perspective not specified1 and 2 lead to poorly specified research question, and problems with identification/measurement of costs and benefits.
3. Inadequate description of programme and comparatorWho does what to who, when and where?Leads to inadequate measurement of costs and benefits, and poorly identified data sources.
-
Deakin Health Economics22 22
4. Failure to specify inclusion/exclusion criteria for costs and benefits
Leads to internal and external validity issues. Financial costing rather than economic approach. Failure to consider if outcome measure captures all relevant benefits.
5. Failure to undertake marginal analysis
Reliance on average C/E results can hide important information.
6. Failure to undertaken sensitivity analysis
No provision for uncertainty.
Economic Evaluation: Traps for Beginners
-
Deakin Health Economics23 23
7. Failure to incorporate discounting
No provision for when costs and benefits are experienced.
8. Failure to consider evaluability
Is the project ready to be evaluated from output/outcome perspective?
9. Insufficient thought given to time period of the study
10. Double counting of benefits
(Life years saved plus forgone productivity).
Economic Evaluation: Traps for Beginners
-
Deakin Health Economics24
Presentation of Results
Economic Evaluation has become a lot more sophisticated over the last 15 years
When I first started:
ICER =12
12
ICER = Comparative, costs, benefits
Now also have: Cost-Effectiveness Planes (CEP)
Cost-Effectiveness Acceptability Curves (CEAC)
Net-Monetary Benefits (NMB)
-
Deakin Health Economics25
Example a simple ICER
Societal Cost of Drug 1 = $10,000
Societal Cost of Drug 2 = $5,000
Benefit of Drug 1 = 10 life years
Benefit of Drug 2 = 8 life years
ICER = 10,0005,000
108
ICER = 5000
2
ICER = $2,500/LY saved
Is this good value for money???
-
Deakin Health Economics26
MORE SAME LESS
MORE ? X
SAME X
LESS X ?
Incremental
cost of
Program B
compared
with
Program A
X = study reduces from CEA to a cost-minimisation analysis.
Incremental benefit of Program B vis-
-vis Program A
Dominant
Dominant
Dominated
Dominated
Decision rules
-
27
-
Deakin Health Economics28
Problems with League Tables
Comparability of methods, for example:
Perspective
Time Horizon
Settings (USA vs Aust)
Outcomes
Uncertainty in results
Rarely just one single point estimate!
-
Deakin Health Economics29
Example
Mihalopoulos et al, 2011, ANZJP, 45, 36-44 This study evaluated 2 interventions designed to
prevent depression in adults Brief bibliotherapy Group based psychological therapy
Comparator was treatment as usual for both Perspective was health sector Time horizon was 5 years Modelled economic evaluation Part of a larger project called ACE-Prevention
which had a detailed protocol of methods
-
Deakin Health Economics30
Mihalopoulos et al, 2011 (ANZJP, 45, 36-44)
Median point estimate (95% uncertainty interval)
Brief bibliotherapy
intervention
Group-based psychological
treatment
DALYs averted 2,600 (-1,500 6,700) 1,700 (518 3,000)
Cost of the intervention
Govt: $760,000 ($520,000 -
$1,000,000)
Private patient :$21M ($13M -
$31M)
Private other: $15M ($10M-
20M)
Govt: $20M ($13M - $30M)
Private patient :$3M ($2M -
$5M)
Private other: $14M ($9M-
20M)
Total costs $37M ($24M - $52M) $38M ($26M - $53M)
Cost offsets $6M ($-5M - $15M) $4M ($1M - $8M)
ICER (with cost-offsets)1 $8,600$20,000 ($12,000 -
$66,000)
ICER (without cost-offsets) 1 $11,000$23,000 ($14,000 - $
68,000)
Median point estimate (95% uncertainty interval)
Brief bibliotherapy intervention Group-based psychological treatment
DALYs averted 2,600 (-1,500 6,700) 1,700 (518 3,000)
Cost of the intervention
Govt: $760,000 ($520,000 -$1,000,000)
Private patient :$21M ($13M -$31M)
Private other: $15M ($10M-20M)
Govt: $20M ($13M - $30M)
Private patient :$3M ($2M - $5M)
Private other: $14M ($9M-20M)
Total costs $37M ($24M - $52M) $38M ($26M - $53M)
Cost offsets $6M ($-5M - $15M) $4M ($1M - $8M)
ICER (with cost-offsets)1 $8,600 $20,000 ($12,000 - $66,000)
ICER (without cost-offsets) 1 $11,000 $23,000 ($14,000 - $ 68,000)
Results from paper
-
Deakin Health Economics31
Box 3.2: The Cost-effectiveness
PlaneIn the diagram the horizontal axis represents the difference in effect between the intervention of interest (A)
and the relevant alternative (O), and the vertical axis represents the difference in cost. The alternative (O)
could be the status quo or a competing program.
If point A is in quadrants II or IV the choice between the programs is clear. In quadrant II the intervention of
interest is both more effective and less costly than the alternative. That is, it dominates the alternative. In
quadrant IV the opposite is true. In quadrants I and III the choice depends on the maximum cost-
effectiveness ratio one is willing to accept. The slope of the line OA gives the cost-effectiveness ratio.
-
+
+O
-
A
Intervention less effective and
more costly than O
Intervention more effective and
more costly than O
Intervention more effective and
less costly than O
Intervention less effective and less
costly than O
EFFECT DIFFERENCE
IV I
IIIII
COST DIFFERENCE
-
Deakin Health Economics32
Previous example as a CE Plane
Mihalopoulos et al, 2011
http://basecase.com/articles/cost-effectiveness-plane-explanation/
-
Deakin Health Economics33
Thresholds
Threshold values to denote value for money E.g. $50,000/QALY, 30,000/QALY in UK These can change in different jurisdictions
PBAC does not have a stated threshold NICE does McKie et al (2011) found that in Australia societal values changed
according to constructs such as hope. (Health Economics, (20), 945-957)
Are largely subjective value judgements What is something worth
WHO Commission on Macroeconomics developed a rule of thumb 1 DALY for less than the average per capita GDP for a given country is
very cost-effective Even up to 3 times per GDP is still cost-effective Aust per capita GDP is: $37,000
-
Deakin Health Economics34
Acceptability Curve
http://bjp.rcpsych.org/content/187/2/106
Excellent description of CEAC written for non-economists
Shows the probability that an intervention is cost-effective over a range of cost-effectiveness value for money thresholds
Are constructed using the same data used in the CEP
-
Deakin Health Economics35
Acceptability Curves
-
Deakin Health Economics36
Net Monetary Benefits
This is a composite measure (part CEA and part CBA)
Uses a threshold value for money E.G. $50,000/QALY
W>
E.g. ig C = $5000 & B = 2QALYs
= 50,000*2>5,000 Therefore intervention is deemed cost-effective Studies will often adopt different thresholds and
calculate probability that intervention is CE
-
Deakin Health Economics37
Further Reading
Drummond et al (2005), Chapter 3
Weblinks in the lecture
CHEERS Checklist in Unit Readings