sensitivity analysis of randomized trials with missing data daniel scharfstein department of...
TRANSCRIPT
Sensitivity Analysis of Randomized Trials with
Missing Data
Daniel Scharfstein Department of Biostatistics
Johns Hopkins [email protected]
ACTG 175 ACTG 175 was a randomized, double bind trial
designed to evaluate nucleoside monotherapy vs. combination therapy in HIV+ individuals with CD4 counts between 200 and 500.
Participants were randomized to one of four treatments: AZT, AZT+ddI, AZT+ddC, ddI
CD4 counts were scheduled at baseline, week 8, and then every 12 weeks thereafter.
Additional baseline characteristics were also collected.
ACTG 175 One goal of the investigators was to compare the
treatment-specific means of CD4 count at week 56 had all subjects remained on their assigned treatment through that week.
The interest is efficacy rather than effectiveness. We define a completer to be a subject who stays
on therapy and is measured at week 56. Otherwise, the subject is called a drop-out.
33.6% and 26.5% of subjects dropped out in the AZT+ddI and ddI arms, respectively.
ACTG 175 Completers-only analysis
TreatmentMean CD4 SE 95% CI
AZT+ddI 385 8.5
ddI 360 7.7
Difference 25 11.5 (3,48)p=0.0027
ACTG 175 The completers-only means will be valid
estimates if, within treatment groups, the completers and drop-outs are similar on measured and unmeasured characteristics.
Missing at random (MAR), with respect to treatment group.
Without incorporating additional information, the MAR assumption is untestable.
It is well known from other studies that, within treatment groups, drop-outs tend to be very different than completers.
Goal
Present a coherent paradigm for the presentation of results of clinical trials in which it is plausible that MAR fails (i.e., NMAR). Sensitivity Analysis Bayesian Analysis
Sensitivity Analysis
For each treatment group, specify a set of models for the relationship between the distributions of the outcome for drop-outs and completers.
Index the treatment-specific models by an untestable parameter (alpha), where zero denotes MAR.
alpha is called a selection bias parameter and it indexes deviations from MAR.
Pattern-mixture model
Step 1: Models
Treatment-specific imputed distributions of CD4 count at week 56 for drop-outs
Step 1: Models Selection model The parameter alpha is interpreted as the log
odds ratio of dropping out when comparing subjects whose log CD4 count at week 56 differs by 1 unit.
alpha>0 (<0) indicates that subjects with higher (lower) CD4 counts are more likely to drop-out.
alpha=0.5 (-0.5) implies that a 2-fold increase in CD4 count yields a 1.4 increase (0.7 decrease) in the odds of dropping out.
Sensitivity Analysis
Step 2: Estimation
For a plausible range of alpha’s, estimate the treatment-specific means by taking a weighted average of the mean outcomes from the completers and drop-outs.
Sensitivity Analysis
Treatment-specific imputed distributions of CD4 count at week 56 for drop-outs
Treatment-specific estimated mean CD4 at week 56 as function of alpha
Step 3: Testing
Test the null hypothesis of no treatment effect as a function of treatment-specific selection bias parameters.
For each combination of the treatment-specific selection bias parameters, form a Z-statistic by taking the difference in the estimated means divided by the estimated standard error of the difference.
Sensitivity Analysis
If the selection bias parameters are correctly specified, this statistic is normal(0,1) under the null hypothesis.
Reject the null hypothesis at the 0.05 level if the absolute value of the Z-statistic is greater than 1.96.
Step 3: Testing
Sensitivity Analysis
Contour Plot of Z-statistic
Contour Plot of Z-statistic
Bayesian Analysis Think of all model parameters as random. Place prior distributions on these parameters. Informative prior on alpha (e.g., normal with
mean -0.5 and standard deviation 0.25). Non-informative priors on all other parameters
(e.g., the distribution of the outcome). Results are summarized through posterior
distributions.
Posterior Distributions368 (342,391)
348 (330,365)
Posterior Distribution of Mean Difference
20 (-11,49); 91%
Likelihood-based Inference A parametric model for the outcome and a
parametric for the probability of being a completer given the outcome.
For example, the outcome is log normal. Inference proceeds by maximum likelihood
(ML). ML inference can be well approximated
using Bayesian machinery.
Maximum Likelihood Distributions
303 (278,331)
297 (271,324)
368 (342,391)
348 (330,365)
-2.6 (-3.0,-2.1)
-2.8 (-3.3,-2.2)
Treatment-specific imputed distributions of CD4 count at week 56 for drop-outs
Maximum Likelihood Distribution of Mean Difference
20 (-11,49)7 (-31,44)
Incorporating Auxiliary Information
MAR (with respect to all observable data) Sensitivity analysis with respect alpha. Bayesian methods under development.
Longitudinal/Time-to-Event Data
Same underlying principles.
LOCFBad idea Imputing an unreasonable value. Results may be conservative or anti-
conservative. Uncertainty is under-estimated.
Conjecture There is information from previously
conducted clinical studies to help in the analysis of the current trials.
Data from previous trials may be able to restrict the range of or estimate alpha.
Summary We have presented a paradigm for
reporting the results of clinical trials where missingness is plausibly related to outcomes.
We believe this approach provides a more honest characterization of the overall uncertainty, which stems from both sampling variability and lack of knowledge of the missingness mechanism.
[email protected] Scharfstein, Rotnitzky A, Robins JM, and Scharfstein DO: “Semiparametric
Regression for Repeated Outcomes with Non-ignorable Non-response,” Journal of the American Statistical Association, 93, 1321-1339, 1998.
Scharfstein DO, Rotnitzky A, and Robins, JM.: “Adjusting for Non-ignorable Drop-out Using Semiparametric Non-response Models (with discussion),” Journal of the American Statistical Association, 94, 1096-1146, 1999.
Rotnitzky A, Scharfstein DO, Su TL, and Robins JM: “A Sensitivity Analysis Methodology for Randomized Trials with Potentially Non-ignorable Cause-Specific Censoring,” Biometrics, 57:30-113, 2001
Scharfstein DO, Robin JM, Eddings W and Rotnitzky A: “Inference in Randomized Studies with Informative Censoring and Discrete Time-to-Event Endpoints,” Biometrics, 57: 404-413, 2001.
Scharfstein DO and Robins JM: “Estimation of the Failure Time Distribution in the Presence of Informative Right Censoring,” Biometrika 89:617-635, 2002.
Scharfstein DO, Daniels M, and Robins JM: “Incorporating Prior Beliefs About Selection Bias in the Analysis of Randomized Trials with Missing Data,” Biostatistics, 4: 495-512, 2003.