Time to event analytic methods for health economic evaluation

Bohdan Nosyk, PhD

Associate Professor

St. Paul’s Hospital Canfar Chair in HIV/AIDS Research

Faculty of Health Sciences, Simon Fraser University

Research Scientist, Health Economics

Michael Smith Foundation for Health Research Scholar

BC Centre for Excellence in HIV/AIDS

Disclosure of conflicts of interest

• None to declare

Outline

• Context & key challenge

• State of the science for TTE analysis in CEA

• Three techniques to address methodological challenges of TTE data

• Next steps

Context• Model-based health economic evaluation

– Sculpher et al., Health Economics. 2006; 15: 677-87.

• Applications in chronic disease– recurrent course– Multiple comobidities/outcomes

• Problem: How do we estimate accurate transition probabilities in the presence of:– Competing risks– Recurrent events– Time-varying exposure, measured/unmeasured confounding

• Cohort-based models: Allowance for time-dependence in transition probabilities– Markov model:

• According to time in model • Within initial health state

– Semi-Markov model:• According to time in model • Within initial health state • Within subsequent health states

State of the science for TTE analysis in CEA

• CEA ‘Best Practices’ guidelines:– Generally not specific in describing methods to deal with TTE data

• Latimer et al (2014) review for TTE analysis:– Specific to extrapolation of (single episode, single outcome) RCT data

• “Analyst should demonstrate that all standard parametric models (exponential, Weibull, Gompertz, log-logistic) have been considered and compared”.

• Briggs, Sculpher and Claxton (2006) text:– Parametric (Weibull) regression model

• Allows for time-dependence in transition probability• Regression-based approach can handle heterogeneity• Underlying theoretical distribution can be sampled from in PSA

Data sources for CEA

• Clinical trial data: – Limited duration, chronicity, recurrence likely to be inadequately

captured

• Published literature: – Sparse, incompatible outputs for CEA, limited external validity

• Observational data: – Prospective cohort studies– Disease registries– Retrospective studies, based on health administrative databases (ie.

treatment utilization)• Comparison of effectiveness of competing treatment regimens may be

biased (endogeneity/selection bias)

TTE methods for different forms of data

• Standard: single, non-recurrent event– Semi-parametric: Cox Proportional Hazards (CPH) model– Parametric: Accelerated Failure Time models (exponential, Weibull, Gompertz)

• Multiple outcomes – Semi-parametric: CPH Competing risks models (Cause-specific hazards,

subdistribution hazards) – Semi-parametric: Multi-state Markov Models

• Non-terminal, recurrent events– Semi-parametric: CPH frailty models

• Time varying exposure, confounding– Semi-parametric: CPH model with time-varying covariates– Marginal structural models

An important distinction among competing risks models

Lau et al, Am J Epidemiol., 2009; 170: 244-56

cause-specific hazard: outcome: {duration, censor}

subdistribution hazard: outcome: {duration, multinomial}

csHR: (strong) assumption of independent competing risks required for valid inference

1. Capturing competing risks

• Nosyk et al, J Acquir Immune Defic Syndr. 2013;63(5):653-659.• Craig and Sendi, Health Econ. 2002; 11: 33-42.• R code: msm package: http://cran.r-project.org/web/packages/msm/index.html

CD4: 350-499 Death

CD4: 200-349

Off ART: CD4: 350-499

CD4>500Multi-state Markov modeling:

• Regression-based approach; • Can handle heterogeneity

• Designed for multiple outcomes, continuous-time data

• Outputs represent subdistribution hazards

• Semi-parametric method: no direct means of handling time-dependence; not useful for PSA• High dimensionality – problems with model convergence

2. Adjusting the hazard of TTE of successive ‘recurrent’ episodes

CPH frailty model form:

hij(t) = h0(t)vj exp(β’Zij)where

vj ~ γ(1/θ, 1/θ)

%

tTime in health state

Nosyk et al., Am J Epidemiol., 2009; 170 (6): 783-92.Nosyk et al., CMAJ, 2012; 84 (6): E317-28.R code: surv package: https://stat.ethz.ch/R-manual/R-patched/library/survival/html/Surv.html

1st episode:

2nd episode:

3rd episode:

• Mixed effects model; can adjust for unmeasured confounding that is fixed over time• Semi-parametric method; no accounting for time-dependence*• Cannot account for multiple outcomes

3. Handling time-dependent exposure, confounding

• Marginal Structural Models

• Context: outcome can also be a predictor of exposure; other time-varying confounding

• Effect of OST in standard GEE with time-varying covariates:

– OR: 1.91 (1.68, 2.19)

• Effect of OST in MSM model: – OR: 1.68 (1.48, 1.92)

Robins JM, Hernan MA, Brumback B. Epidemiology 2000;11:550 –560.Nosyk et al., AIDS. 2015. IN PRESS

• Assumes no unmeasured confounding• Cannot account for multiple outcomes

Next Steps

• Continued methodological development in TTE analytic methods needed:– Parametric competing risks model

• Jeong and Fine, Biostatistics 2007; 8(2): 184-96.

– Competing risks CPH Frailty model • Kauermann & Khomski, 2006. R package ‘CompetingRiskFrailty’ no longer

functional

– Competing risks Marginal structural model • None available to date

– Improved optimization algorithms for high-dimensional multi-state Markov models

• How do differences, limitations in existing methods affect CEA results?

Acknowledgements

• BC-CfE Health Economics Team: Emanuel Krebs, Jeong Min, Michelle Olding, Batool Yazdani

• UCLA Integrated Substance Abuse Programs; Centre for Advancing Longitudinal Drug Abuse (CALDAR) : Elizabeth Evans, Libo Li, Yih-Ing Hser

– [NIDA Grant No. P30 DA016383]

• Dr. Lei Liu, Northwestern University

• An empirical investigation into recovery from illicit drug abuse using recurrent event analytic methods

– [NIDA Grant No. R01-DA033424]

• Addiction and Urban Health Research Unit (UHRI), BC-CfE – [NIDA Grant No. R01-DA011591, R01-DA021525, R01-DA028532]

• Seek and Treat for Optimal Prevention of HIV/AIDS (STOP-HIV/AIDS) Program Evaluation– [BC Ministry of Health, Healthy Living and Sport]

Questions, Comments?

Thank you for your attention.

Bohdan Nosyk: bnosyk@cfenet.ubc.ca

State of the science for TTE analysis in CEA?Source Recommendation

Guidelines for the economic evaluation of health technologies: Canada [3rd Edition]. CADTH (2006).

Re.: extrapolation of short-term RCT data. The duration and the magnitude of the clinical benefit beyond the trial is often a critical judgment to make regarding extrapolation.Describe the strength of the evidence for extrapolating data and assess uncertainty through a sensitivity analysis. Describe the method of elicitation to establish parameter values, and the results of the exercise. Any limitations of the methods, potential biases in the parameter estimates, and caveats about the interpretation of results, should be reported.

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement:

Guidelines for Reporting Observational Studies. Annals of Internal Medicine (2007).

For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe all statistical methods.

State of the science for TTE analysis in CEA?Source Recommendation

Modeling Good Research Practices—Overview: A Report of the ISPOR-SMDM Modeling Good Research

Practices Task Force-1. Value Health (2012).

Transition probabilities and intervention effects should be derived from the most representative data sources for the decision problem.

Conceptualizing a Model: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-

2. Value Health (2012).

The problem conceptualization should be used to identify key uncertainties in model structure where sensitivity analyses could inform their impact.The conceptual structure should be driven by the decision problem or research question and not determined by data availability.

State-transition modeling: a report of the ISPOR-SMDM modeling good research practices task force-3.

Value Health (2012).

All methods and assumptions used to derive transition probabilities should be described. STMs should provide clear justification for estimates of transition probabilities and state values and their ranges for sensitivity analysis.As STMs allow deriving the time at which particular transitions occur, the results can be represented as (modeled) probability or survival curves and directly compared with survival curves from empirical studies.

State of the science for TTE analysis in CEA?Source Recommendation

Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and Elaboration: A

Report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task

Force. Value Health (2013).

For model-based economic evaluations, authors should describe and report how they estimated parameters, for example, how they transformed transition probabilities between events or health states into functions of age or disease severity. Regardless of study design, the handling of uncertainty and the separation of heterogeneity from uncertainty should be consistent themes.

Survival Analysis For Economic EvaluationsAlongside Clinical Trials - Extrapolation With

Patient-level Data. NICE DSU Technical Support Document 14 (2011 - Updated 2013).

The analyst should demonstrate that a range of parametric models [for survival analysis] have been considered and compared. The fit of alternative models should be assessed systematically.PH modelling should only be used if the proportional hazards assumption can be clearly justified. Scenario based sensitivity analysis should assess the importance of duration of treatment effect assumptions.

State of the science for TTE analysis in CEA?Source Recommendation

Cost-Effectiveness Analysis Alongside Clinical Trials II—An ISPOR Good Research Practices Task Force Report.

Value Health (2015).

When modeling beyond the follow-up period for the trial, it is important to project costs and outcomes over the expected duration of treatment.Parametric survival models estimated on trial data are generally recommended for such projections, unless models based on other data or methods are justified.

Applying Dynamic Simulation Modeling Methods in Health Care Delivery Research—The SIMULATE

Checklist: Report of the ISPOR Simulation Modeling Emerging Good Practices Task Force. Value Health

(2015).

The checklist identifies time-dependent and dynamic transitions that characterize simulation modeling methods and differentiate them from other modeling approaches such as Markov models and decision trees.

State of the science for TTE analysis in CEA?Source Recommendation

Guidelines for the economic evaluation of health technologies: Canada [3rd Edition]. CADTH (2006).

Describe the method of elicitation to establish parameter values, limitations of the methods, and potential biases.

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE). Annals of Internal Medicine (2007).

Describe all sources and statistical methods.

ISPOR-SMDM Modeling Good Research Practices. Value Health (2012).

All methods and assumptions used to derive transition probabilities should be described. The conceptual structure should be driven by the decision problem and not determined by data availability.

Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—ISPOR. Value Health (2013).

Describe how transition probabilities are estimated. The handling of uncertainty and the separation of heterogeneity from uncertainty should be consistent themes.

Survival Analysis For Economic Evaluations. NICE DSU Technical Support Document 14 (2011 - Updated 2013).

A range of parametric models [for survival analysis] should be considered and fit should be assessed systematically.

Cost-Effectiveness Analysis Alongside Clinical Trials II—ISPOR. Value Health (2015).

When modeling beyond the follow-up period, project costs and outcomes over the expected duration of treatment.Parametric survival models estimated on trial data are generally recommended for such projections.

Applying Dynamic Simulation Modeling Methods in Health Care Delivery Research—The SIMULATE Checklist. ISPOR. Value

Health (2015).

The checklist identifies time-dependent and dynamic transitions that characterize simulation modeling methods and differentiate them from other modeling approaches.

Nosyk et al, CMAJ. 2012; 184(6):E317-28.

The primary finding of this study was that patients experiencing multiple treatment episodes tended to stay in treatment for progressively longer periods in later episodes.

Nosyk et al, Am J Epidemiol. 2009; 170(6):783-92.

Decision Analytic ModelAllocated to MMT Allocated to DAM

Cycle, j=4-6

Cycle=3 Death

MMTj

MMT3

AbstinenceRelapse3

MMT-PD3

Death

DAMj-2

DAM1

AbstinenceRelapse3

Model parameterization• Supplemented trial TTE data with external data: BC linked health database

• Estimated weibull regressions to extrapolate TTE data

• Implemented by making use of R’s functionality with multi-dimensional arrays described by Hawkins, Sculpher and Epstein (2005) in order to program time dependence within each of the model states

• Multiplied adjusted hazards for episode j with adjusted hazard ratios, drawn from frailty models, for durations of episodes j+k

• Adjusted for age, gender, state-specific mortality risks

• Assumed dirichlet distribution for transitions to multiple states

Impact of changing distributional assumption on TTE curves

Cost, $1000CDNMean (95% CI)

QALYsMean (95% CI)

ICERMean (95% CI)

Baseline Formulation*

DAM 1096 (724, 1707) 7.9 (7.3, 8.5) CS (CS, 122)

MMT 1137 (737, 1777) 7.5 (6.9, 8.0)

Exponential distributions set for TTE curves

DAM 1104 (729, 1712) 7.8 (7.1, 8.4) CS (CS, 331)

MMT 1145 (738, 1812) 7.4 (6.8, 8.1)

*Gamma distributions set for time to discontinuation of each health state.

CPH Frailty models demonstrated durations of daily use diminished in Successive episodes over time. MSM models revealed primary stimulantusers had more erratic longitudinal patterns of drug use, transitioning more Rapidly between periods of treatment, abstinence, non-daily and daily use.

Nosyk et al., Drug Alcohol Depend. 2014; 140: 69-77.

1. Capturing competing risks (option b)

Cumulative incidence functions for 2 competingrisks:

Probability of having Syncitium-inducing (SI) HIV phenotype

Probability of developing AIDS

Putter H, Fiocco M, Geskus B. Stat Med. 2007; 26:2389-2430.Jeong J-H, Fine JP. Biostatistics. 2007; 8(2): 184-96.

Parametric regression methods for competing risks:

AIDS+SI

• Allows for time-dependent transition probabilities, heterogeneity and competing risks; • Only basic control of baseline confounders