Economic evaluation of health programmes

Department of Epidemiology, Biostatistics and Occupational Health

Class no. 22: Applying the net benefit framework, assessing the value of information, reporting

economic evaluations

Nov 17, 2008

Plan of class

Net benefit frameworkCost-effectiveness acceptability curves‘Marrying econometrics and cost-

effectiveness analysis’

Why are p-values for the NMB regression coefficient estimates equal to those

obtained with effect as the dependent variable?

When λ goes to infinity in

the variation in Ci becomes less and less important compared to that in Ei, so that the regression:

is mostly explaining variation in Ei: the Ci part represents a smaller and smaller share of the variation in NMBi. Hence, while the coefficients are different (NMBi is becoming larger and larger), the p-values are the same.

Additional points concerning Hoch et al. 06

Nature of outcome (1)

If a fainting episode (syncope) occurs, and the device that measures heart rhythm is activated on time, it is possible to see whether the syncope is or is not associated with heart arrhythmia.

Success occurs if : a) a syncope occurs b) the device is activated.

Nature of outcome (2)

Loop recorder: 63.27% of patients have symptom reccurrence and successful activation, vs. 23.53% of Holter group

The loop recorder is more expensive but has a higher chance of detecting whether arrhythmia is the cause of syncope

How the dependent variable (net benefit) is constructed

Review: why the p-value on the intervention dummy from the net benefit regression can be

used to construct the CEAC (1)

The CEAC shows, for a given value of λ, the probability that the intervention is cost-effective.

In the regression:

statistical software computes a t-statistic for the estimate of δ to test the null hypothesis that δ=0; if the p-value is small enough (usually less than 0.05), we reject the null.

(Under what condition does this become the probability density function of a t-distribution?)

Review: why the p-value on the intervention dummy from the net benefit regresion can be

used to construct the CEAC (2)

But we want to test the null hypothesis that δ <= (negative or equal to) 0.

If the estimate of δ is positive, then the associated t-statistic is also positive – so we locate it on the right-hand side of the t-distribution. Half of the corresponding p-value gives us the area to the right of that value. This is the significance level at which we can reject the null that δ <= (negative or equal to) 0. In other words, the probability that δ>0 is 1- half of the p-value.

Review: why the p-value on the intervention dummy from the net benefit regresion can be

used to construct the CEAC (3)

If the estimate of δ is negative, then the associated t-statistic is also negative – so we locate it on the left-hand side of the distribution. Half of the corresponding p-value gives us the area to the left of that value. This is the significance level at which we can reject the null that δ  >= (positive or equal to) 0. In other words, the probability that δ>0 is half of the p-value.

Regression estimates for different values of λ

Calculating probabilities of cost-effectiveness

Note how similar probabilities of cost-effectiveness derived from regression and from bootstrapping are

CEAC from probabilities of cost-effectiveness derived from one-sided p-values in previous table

Comments

Here, no variation in cost across individuals with same Tx

When there is variation in cost, skewness or heteroskedasticity may make p-values less valid – then use bootstrapping

NBRF enables estimates of mean net benefit of : Usual care (β0) New treatment (β0+ β1) as well as incremental net

benefit (β1)

Using models to assess value of additional research

Concept of decision uncertainty

Methodological uncertainty Sampling variation/

parameter uncertainty Modelling uncertainty Generalizability

Decision uncertainty

Is additional research necessary?

Expected value of perfect information (EVPI)

Probabilistic sensitivity analysis (PSA) to yield expected costs and effects of alternative options: identify preferred option

Determine probability of making wrong decision = 1 - probability that this is indeed best option (use CEAC)

Use PSA to determine cost of making wrong decision: Foregone health Wasted resources

Calculate expected cost of uncertainty by multiplication (in terms of health and dollars)

Multiply by number of patients to get population EVPI

Example of EVPI analysis

Implications of EVPI analysis

Additional research must cost less than EVPI

Can also use EVPI analysis to assess value of information to be yielded by alternative research designs

Frontier area of investigation EVPI absolutely not taken into account by

CIHR

Presentation and use of economic evaluation results

Economic evaluation is widely used…

Oregon Medicaid experiment Combined with public deliberative process

Requirement for formal economic evaluation for drugs to be approved for reimbursement Australia Several Canadian provinces U.K.

Wider use in England (NICE)

…but has many limitations

Validity can be hard to assess => Standardize reporting

Generalizability may be issueVery method of funding interventions

meeting $/QALY threshold has been criticized

Common reporting format Alleged benefits:

Transparency Comparability across studies

• Limits of league tables

Improve quality of evaluations?• Stifle innovation?

US Public Health Services Panel on Cost-effectiveness in Health and Medicine Include reference case in report

British Medical Journal Working Party on Economic Evaluation (1996) Too many methodological controversies, so just focus on common

reporting standards

Some common recommendations

To include in reporting of economic evaluation

Background: Question(s) to be addressed and its(their) importance

Viewpoint/perspective(s) of analysis

Justification for type of analysis

For whom is this relevant?

Comparators being assessed

Source and quality of medical evidence

Range of costs and how measured

Measure of benefits

Methods for dealing with uncertainty

Incremental analysis of costs and benefits

Overall study results and limitations

Concept of league table

Rank interventions in order of $/QALYWhy?

Place findings in broader context Inform decisions about which interventions to

fund• Use of league tables for this purpose has been

criticized

Examples from published studies (1998 US$)

Source: http://www.hsph.harvard.edu/cearegistry/comprehensive-revised.pdf