Types of analysis

Simulation rationale

Each patient’s natural history is random, but guided by underlying parameters.

With sufficiently large number of patients, Monte Carlo variability can be made as small as possible.– In this case, the SPM essentially serves as a

“counting machine” to estimate expected outcomes.

Analysis plan To compare two stroke treatments, set the

natural history parameters for the first treatment and run the simulation to obtain expected outcomes.

Then, reset the natural history parameters to correspond to the second treatment and rerun the simulation to obtain a second set of expected outcomes.

Finally, compare the two sets of outcomes.

Example

To assess the cost-effectiveness of an acute stroke drug for 70-year old males with ischemic stroke…

Group Cost Effectiveness Usual care170,000 3.67 QALY Intervention 180,000 4.17 QALY

ICER= 10,000 / .50 = 20,000 $/QALY

Types of analysis

Base case Sensitivity Bootstrapping Stochastic sensitivity ……..

Base case analysis

1 SPM run 1 patient type (e.g., 50,000 simulated

patients, all with the same characteristics) 1 set of fixed input parameters (e.g., fix

the natural history parameters, utilities, cost parameters, efficacy of intervention, etc.)

Sensitivity analysis

1 patient type Multiple SPM runs

– Each SPM run applies a separate set of pre-specified parameters.

– One or more parameters could be changed at a time.

One-way sensitivity analysis

Discount Rate ICER 0% 24,576 3% 21,864 5% 17,987 7% 13,747

As discount rate increases, intervention becomes increasingly cost effective.

Two-way sensitivity analysis

Discount Efficacy ICER0% 1.30 305,9875% 1.30 865,4830% 1.40 5,076

5% 1.40 12,946 Discount rate doesn’t matter, but intervention’s

efficacy does: small changes in efficacy imply very different conclusions about cost -

effectiveness.

Bootstrapped analysis

1 patient type 1 SPM run

– SPM parameters remain the same– Resampling of patients (i.e., conceptually, the RCT

is repeated a large number of times, and the ICER is estimated for each replication; the variability of the ICER describes the precision of the results)

Stochastic sensitivity analysis

1 patient type Multiple SPM runs

– Multiple parameters changed simultaneously– Parameters obtained by random sampling from

prior distributions (in comparison with sensitivity analysis, more

emphasis on estimating overall precision of results)

Example

Run Discount Efficacy ICER* 1 3.21% 1.52 21,056 2 4.56% 1.67 29,059 3 3.12% 1.34 22,356 4 2.18% 1.68 12,967 … … … …

*Mean ICER = 20,000; s.d. = 5,000

Combined bootstapped and stochastic sensitivity analysis

For each bootstrapped sample, rerun the SPM using input parameters randomly selected from prior distributions. Bootstrapping accounts for first-order

uncertainty (i.e., patient-level). Sampling from parameters accounts for

second-order uncertainty (i.e., in SPM parameters).

Standard of practice The current standard of practice is to use

modeling to attach expected values for long-term outcomes to each patient in the trial. One and multi-way sensitivity analyses are performed. Bootstrapping (perhaps combined with stochastic sensitivity analysis) is the state-of-the science, in order to assess the precision associated with the CEA.

Comment

Precision is very important to consider, as it is critical in determining the strength of the CEA’s conclusions. ICER = 20,000 with s.d. = 5,000 is strong

evidence in favor of the treatment. ICER = 20,000 with s.d. = 500,000 is very

weak evidence in favor of the treatment.

SPM structure

MI

IS

TIA

ASY DTH

HS

Bleed

SPM Structure

States, events, and transitions States are asymptomatic (ASY), transient ischemic

attack (TIA), ischemic stroke (IS), hemorrhagic stroke (HS), myocardial infarction (MI) and death (DTH).

An event is a transition between states (e.g., a TIA in a previously asymptomatic patient moves the patient from ASY to TIA).

Recurrent events are allowed (e.g., a second IS for a patient in the IS state).

The intervention language can also count other complications of treatment.

Sample patient history from a patient with ischemic stroke*

Month State Event Cost Utility 1 IS None C(I,1) U(I) 2 IS None C(I,2) U(I) 3 IS IS C(I,1) U(I) 4 MI MI C(M,1) U(M) 5 DT DT 0 0

*Note: if U(M)<U(I) then use U(I)

Modules

Natural history module -- generates patient histories

Cost module -- attaches costs to patient histories

Utility module -- attaches QOL to patient histories

Intervention module -- modifies natural history parameters

Basic philosophy

Use each data source to its best purpose. For example, administrative files are used to

estimate utilization (and thus costs), but not treatment efficacy.

Expert judgement is minimized, but used when other information is insufficient or implausible.

Data sources Natural history -- Framingham; Rochester,

Minnesota / Mayo Clinic; US life tables Costs -- most categories from Medicare Utilities -- from national patient survey and

literature Intervention effects -- meta-analysis / synthesis of

RCTs Expert judgement -- as needed

Natural history module Natural history module reflects the epidemiology

of stroke. All information presented as transition functions.

– The traditional survival curve is an example of a transition function,(outcome=death).

– Transition functions use proportional hazards model (i.e., baseline curve + effect of covariates).

Default cycle time is 1 month.

Cost module

The basic idea is that each new event places the patient at “month 0” of a cost curve (reflecting medical costs, over time, after an event such as IS).

Costs can be attached to patient histories either deterministically or stochastically.

Costs are currently in 1996 US dollars.

Cost categories Direct medical -- acute care hospital, physician,

outpatient, home health, skilled nursing facility, durable medical equipment, outpatient drugs, rehabilitation units, rehabilitation hospitals, nursing home (non-SNF)

Direct non-medical -- caregiver, modifications to environment

Indirect -- lost earnings, lost non-market productivity

Cost sources Medicare -- institutional costs (acute care

hospitals, rehabilitation, some skilled nursing), home health, hospital-based outpatient, physician, durable medical equipment

Medicare plus imputation -- skilled and other nursing home

UHC -- under 65s, drugs Literature -- caregiver, environmental

modifications, indirect costs

Utility module

The basic idea is that each event leads to a change (typically, a decrease) in QOL.

• Utilities are one measure of QOL Utilities can be attached to patient

histories either deterministically or stochastically.

Utility sources PORT patient survey + literature

613 AMCC inpatients 321 CHS population-based aged 65+ 319 UHC managed care, inpatients and outpatients,

mostly aged <65–Oversampling ensured sufficiently large numbers of

patients in asymptomatic, TIA, and minor stroke categories

–TTO and CS for current health state and hypothetical major stroke

SPM structure -- intervention module

Interventions can change natural history, costs, and/or utilities.

Parameters are obtained by meta-analysis / literature synthesis

Intervention specification (example)

Carotid endarterectomy has the following effects: 1-cycle decrement in QOL of .xx 1-time cost of $xxx Probability of stroke, MI, and death in next cycle

increased by xx%, xx%, and xx% Risk of stroke and MI multiplied by .xx in

subsequent cycles Duration of benefit of xx years

Covariates

Patient characteristics (covariates) affect natural history, cost, and QOL.

– For example, in the natural history module, the effect of covariates is described by terms in the proportional hazards model.

Users select degree of detail / complexity.

Extrapolation Epidemiologic cohorts had reliable follow-up for

approximately 6 years; Medicare files include 24-36 months per patient. However, the pattern of hazards and costs was nearly

linear by the conclusion of follow-up. We used constrained linear extrapolation technique.

–Constraints -- hazard for symptomatic patients can never fall below US population life table, monthly costs can never fall below those of a comparison sample...

Discounting

Discount rate can be varied. Default is to discount both life years

and costs by 3% risk-less rate.

SPM outputs

Survival Quality-adjusted survival Event-free survival Costs Costs by category Patient histories