Quantifying the clinical measure of interest in the presence of missing data:

choosing primary and sensitivity analyses in neuroscience clinical trials

Sept 26, 2016

Elena Polverejan, Ph.D.Statistical Modeling & Methodology

Janssen R&D, Johnson & Johnson

His wif_ is not working today.

Impact of Missing Data

2

i or e?

Outline

• Background: Selection of the primary estimand and

statistical methods for a clinical trial

• Neuroscience trial simulation example:

• Assumptions

• Statistical methods

• Derived properties

• Role of simulations

3

Primary and Sensitivity Analyses

4

Estimand

Primary

Analysis

Sensitivity

Analysis 1

Sensitivity

Analysis k

Sensitivity

Estimate k

Sensitivity

Estimate 1

Primary

Estimate

…

…End of Trial

Challenges in Selection of Statistical Methods

Unique characteristics of each clinical trial:

Indication

Study population

Study design

Efficacy response

Likelihood of subjects remaining on treatment or in the trial etc.

Variety of statistical methods

Regulatory requirements that evolve over time

5

Simulation Example: Depression Efficacy Trial

Screening

Drug

Placebo

6

1:1 Randomization

N=66 Subjects/Group

4-week DB Efficacy Phase

Follow-Up

Phase

Primary time point

Primary efficacy measure:

depression scale

(MADRS) collected over

time in the DB phase

Primary efficacy endpoint:

change from baseline to

Week 4 in MADRS Total

Score

Simulation Evaluation Process

• Simulate full datasets (without any missing data)

• Create over time missing values based on various

assumptions (missing data cases)

• Apply considered statistical methods

• Determine operating characteristics of the considered

methods:

• Power (or Type I error rate)

• Estimated treatment difference and its variability

7

Simulate Full Datasets

Key Assumption: Over Time Efficacy Response

8

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

0 1 2 3 4 5 6 7 8 910111213141516171819202122232425262728

Day

Me

an

Re

sp

on

se

trt

Drug

Placebo

-6.5

Simulated Missing Data Cases

9

Averaged Distribution of Discontinuations for the Evaluated Cases

Case Group Mean Total %DC Mean %DC Other Mean %DC LOE

1 drug 30.6 25.1 5.4

1 control 26.0 14.7 11.2

2 drug 25.5 20.1 5.4

2 control 26.0 14.7 11.2

3 drug 18.8 15.1 3.7

3 control22.2 14.6 7.6

30.6

26.0

25.5

18.8

22.2

25.1

14.7

5.4

11.2

Statistical Methods?

26.0

DC = Discontinuation

LOE = Lack of Efficacy

Other discontinuation reasons include adverse events, lost to follow-up etc.

MAR vs MNAR

1

0

MAR = Missing at Random MNAR = Missing Not At Random

After withdrawal subjects would

tend to have similar efficacy to

subjects who remain in the

study after accounting for

observed characteristics

Assumptions need to be made on

the potential “trajectory” or

distribution of efficacy after

withdrawal, which will be different

from the one of subjects

remaining in the trial

Multiple Imputation Based Methods

Multiple imputation is a statistical technique for analyzing

incomplete datasets.

Application of this technique requires three steps:

c1_oc = observed change at Visit 1, c2_oc = observed change at Visit 2 etc.

11

DC c1_oc c2_oc c3_oc c4_oc

OTHER -1.06215 -5.17018 . .

_Imputation_ c1_mi c2_mi c3_mi c4_mi

1 -1.06215 -5.17018 -3.59436 -6.26150

2 -1.06215 -5.17018 -2.95910 -0.40440

3 -1.06215 -5.17018 -3.93861 -5.58729

4 -1.06215 -5.17018 -5.05197 -4.65008

5 -1.06215 -5.17018 -4.02463 -1.85226

6 -1.06215 -5.17018 -5.29770 -1.28843

7 -1.06215 -5.17018 -3.45573 -3.67338

8 -1.06215 -5.17018 -1.64735 -2.77245

9 -1.06215 -5.17018 -8.62956 -1.87126

10 -1.06215 -5.17018 -6.19145 -4.04486

_Imputation_ Diff vs Pbo StdErr

1 -0.6467 0.2717

2 -0.5798 0.2729

3 -0.4925 0.2800

4 -0.7350 0.2754

5 -0.4891 0.2722

6 -0.7668 0.2758

7 -0.7032 0.2699

8 -0.5977 0.2748

9 -0.6303 0.2709

10 -0.6180 0.2686

Pooled

Diff

Pooled

StdErr Pvalue

-0.625909 0.290099 0.0313

Imputation with multiple values

Analysis

Pooling

12September,

2014

Control-Based MI

Delta AdjustmentANCOVA_LOCF

ANCOVA_BOCF

Evaluated Statistical Methods

MMRM: Mixed Model for Repeated

Measures

MI_REG: MI with Regression Option

Missing at Random (MAR)

Based:

ANCOVA_FULL

MMRM_FULL

Single-Imputation (MNAR):

Multiple-Imputation (MI)

Under Missing Not at Random

(MNAR):

For Reference:

MI methods: MMRM for analysis

Joint Control-Based MI Methods

13

MIJOINT_J2R

MIJOINT_CR

Methodology and SAS macros developed by James Roger and shared through DIA missing data working group

site at http://www.missingdata.org.uk; Slide from O’Kelly & Davis short course at the 2015 ASA Biopharmaceutical

Workshop

MAR Methods

Delta Adjustment and Tipping Point Analysis

Analysis assuming that subjects who discontinue would,

on average, have their unobserved efficacy outcome

worse by the amount Delta compared to the observed

efficacy outcome of subjects who remain in the study.

Tipping Point Analysis: find the assumption at which

conclusions change from favorable to drug (statistically

significant) to unfavorable.

Recommended by the NRC 2010 report* and by FDA**

Delta adjustment may be applied:

only on the experimental groups (of regulatory interest)

on all treatment groups

for certain reasons of discontinuation.

14

* National Research Council report on the Prevention and Treatment of Missing Data in Clinical Trials

(2010)

**Thomas Permutt “Sensitivity analysis for missing data in regulatory submissions” Statist. Med. 2015

Delta Adjustment Types

Types of delta adjustment:

sequential (visit-by-visit)

marginal (multiple imputations first, then apply delta

adjustment)

In simulation exercise:

Marginal delta adjustment: MIDELTAMAR

first, MAR-based multiple imputation analysis (MI_REG)

apply delta mean worsening in active arm only

same delta adjustment across visits

sequence of delta of increasing severity: 0 to 10 by 1

15

Estimated Power

16

Case % DC

Control

% DC

Active

1 26% 31%

2 26% 26%

3 22% 19%

Assumption:

91% power for the “full dataset”,

before missing values applied

ancova_bocf

mijoint_j2r

mijoint_cr

ancova_locf

mi_reg

mmrm_oc

mmrm_full

ancova_full

60 70 80 90

Power

Me

tho

d

case

Case 1

Case 2

Case 3

Estimated Sample Size to Maintain 90% Power

17

Case % DC

Control

% DC

Active

1 26% 31%

2 26% 26%

3 22% 19%

Increasing the sample size

using expected amount of

drop-outs might not be

sufficient to maintain power:

Case 2: 26% adjustment to 66

subjects =~ 89 subjects ancova_bocf

mijoint_j2r

mijoint_cr

ancova_locf

mi_reg

mmrm_oc

mmrm_full

ancova_full

60 90 120 150 180

Number subjects per group to maintain 90% power

Me

tho

d

case

Case 1

Case 2

Case 3

mideltamar_0_10

mideltamar_0_9

mideltamar_0_8

mideltamar_0_7

ancova_bocf

mideltamar_0_6

mijoint_j2r

mideltamar_0_5

mideltamar_0_4

mideltamar_0_3

mijoint_cr

mideltamar_0_2

mideltamar_0_1

ancova_locf

mideltamar_0_0

mmrm_oc

mmrm_full

ancova_full

40 60 80

Power

Me

tho

d

case

Case 1

Case 2

Case 3

Estimated Power With Increasing Delta Adjustments

18

Case % DC

Control

% DC

Active

1 26% 31%

2 26% 26%

3 22% 19%

Probability(%) of Tipping From MAR MI Regression

19

Case % DC

Control

% DC

Active

1 26% 31%

2 26% 26%

3 22% 19%

0

5

10

15

20

25

30

35

40

45

50

55

1 2 3 4 5 6 7 8 9 10

Delta Adjustment

Pro

babili

ty o

f T

ippin

g f

rom

MA

R M

I_R

EG

Analy

sis

(%

)

case

Case 1

Case 2

Case 3 Tipping point analysis =

finding the delta worsening

that changes conclusions

from favorable to drug

(statistically significant) to

unfavorable

Role of Simulations

Understand at the study design stage:

the impact of amount and reason of missing data

the performance of the statistical analysis methods

considered for the primary and sensitivity analyses

potential sample size adjustments.

Incorporate clinical feedback

Communicate with regulatory agencies

Communicate the impact of missing data and the

importance to reduce the amount of missing

data through study design and conduct

20

21

Back-Up

ancova_bocf

mmrm_full

ancova_locf

ancova_full

mijoint_cr

mmrm_oc

mijoint_j2r

mi_reg

1.85 1.90 1.95 2.00 2.05 2.10

Mean SE

Me

tho

d

case

Case 1

Case 2

Case 3

ancova_full

mmrm_full

mi_reg

mmrm_oc

ancova_locf

mijoint_cr

mijoint_j2r

ancova_bocf

-6.5 -6.0 -5.5 -5.0 -4.5 -4.0

Mean Difference vs Control in LSMeans

Me

tho

d

case

Case 1

Case 2

Case 3

Estimated Mean Treatment Difference and

Mean Standard Error

23