finding studies adaptive designs in dose- · 2007. 11. 14. · webinar 2: adaptive designs in...
TRANSCRIPT
Fall 2007 web-based training series
Introduction (Alex Dmitrienko, Lilly)
Web-based training program
• http://www.amstat.org/sections/sbiop/webinarseries.html
Webinar 2: Adaptive designs in dose-finding studies
• Brenda Gaydos (Lilly), José Pinheiro (Novartis), Chris Coffey (University of Alabama-Birmingham)
Presentation slides
• http://www.biopharmnet.com/doc/doc03002.html
Discussion thread
• http://biopharmnet.com/forum/viewtopic.php?t=102
Sponsors: Biopharmaceutical Section of ASA ,
Adaptive Designs in Dose-
Finding Studies
Four-Part Web-based Training Series
Webinar 2
Adaptive Design Working Group
Adaptive Dose Ranging Studies Working Group
Sponsors: Biopharmaceutical Section of ASA ,
PRESENTERS
Christopher S. Coffey, PhD
University of Alabama at BirminghamEmail: [email protected]
Brenda Gaydos, PhD
Eli Lilly and CompanyEmail: [email protected]
José Pinheiro, PhD
Novartis PharmaceuticalsEmail: [email protected]
Sponsors: Biopharmaceutical Section of ASA ,
• Co-Chairs:
Michael Krams
Brenda Gaydos
• Member Authors:
Keaven Anderson
Suman Bhattacharya
Alun Bedding
Don Berry
Frank Bretz
Christy Chuang-Stein
Sylva Collins
Vlad Dragalin
Paul Gallo
Brenda Gaydos
Michael Krams
Qing Liu
Jeff Maca
Inna Perevozskaya
Jose Pinheiro
Judith Quinlan
• Members:
Zoran Antonijevic
Roy Baranello
Michael Branson
Carl-Fredrik Burman
Nancy Burnham
Daniel Burns
Bob Clay
Chris Coffey
David DeBrota
Alex Dmitrienko
Jennifer Dudinak
Greg Enas
Richard Entsuah
Parvin Fordipour
Sam Givens
Ekkehard Glimm
Andy Grieve
Shu Han
• Members (cont.):
David Henry
Melissa Himstedt
Tony Ho
Roger Lewis
Gary Littman
Cyrus Mehta
Wili Maurer
Allan Pallay
Michael Poole
Bob Parker
Yili Pritchett
Jerry Schindler
Jonathan Smith
Don Stanski
Joel Waksman
Bill Wang
Gernot Wassmer
ADAPTIVE DESIGNS WORKING GROUP
Sponsors: Biopharmaceutical Section of ASA ,
• Co-Chairs:
José PinheiroRick Sax
• Member Authors:
Björn Bornkamp
Frank Bretz
Alex Dmitrienko
Greg Enas
Brenda Gaydos
Chyi-Hung Hsu
Franz König
Michael Krams
Qing Liu
Beat Neuenschwander
Tom Parke
Amit Roy
Frank Shen
• Members:
Zoran Antonijevic
Vlad Dragalin
Parvin Fordipour
Marc Gastonguay
Bill Gillespie
Frank Miller
Inna Perevozskaya
Ashish Sanil
Jonathan Smith
ADAPTIVE DOSE-RANGING STUDIES WG
Sponsors: Biopharmaceutical Section of ASA ,
OUTLINE
I. Introduction and Methods for Early Exploratory Studies (~35 min.)
II. Methods for Late-Stage Exploratory Development(~50 min.)
III. Simulations and Conclusions (~35 min.)
Sponsors: Biopharmaceutical Section of ASA ,
I. Introduction and Methods for Early
Exploratory Studies
Outline:
• Overview of adaptive designs
• Summary of major philosophies regarding definition of maximum tolerated dose (MTD)
• Conventional 3+3 designs
• Model-based designs
Sponsors: Biopharmaceutical Section of ASA ,
OVERVIEW
Infinite number of adaptive design possibilities:
• Many aspects of the study can be changed:
- sample size - final test statistic
- primary endpoint - inclusion/exclusion criteria
- number of treatment arms - randomization procedure
- Number of interim looks - goal: superiority to non-inferiority
• Define objective of the adaptation and the design elements to adapt.
Sponsors: Biopharmaceutical Section of ASA ,
OVERVIEW
For this course, we focus on adaptive dose-response methods.
Such adaptive designs:
• Offer more efficient ways to learn about dose response
• Provide more information on dose-response profile earlier in development.
• Guide decision making on whether to continue program and, if so, which dose to select for further development
• Aim to increase probability of technical success by taking correct choice of dose forward for further study.
Sponsors: Biopharmaceutical Section of ASA ,
MTD - DEFINITION
Phase I clinical trials typically want to determine some maximum tolerated dose (MTD).
Accurate determination of the MTD is very important since the dose established as the MTD will be used for further testing in later phases.
Passing on too low of a dose may jeopardize a potentially useful drug
Passing on too high of a dose puts patients in later phase trials at risk
Sponsors: Biopharmaceutical Section of ASA ,
MTD - DEFINITION
Two major philosophies regarding MTD definition:
Dose that, if exceeded, would put patients at ‘unacceptable risk’ of toxicity.
• Treat the MTD as being observed from the data
• Vague from statisticians point of view since ‘unacceptable risk’may not be defined quantitatively
Specifying ‘unacceptable risk’ as a probability.
• Treat the MTD as an unknown parameter of a monotonic dose response curve.
• The MTD is estimated corresponding to a specified probability.
Sponsors: Biopharmaceutical Section of ASA ,
MTD - DEFINITION
1) Conventional up-and-down designs
• Such as 3+3 designs for cancer
2) Model-based designs where MTD is a quantile to be estimated
• Random walk rule
• Bayesian methods
These two definitions lead to two different approaches for designing phase I clinical trials:
Sponsors: Biopharmaceutical Section of ASA ,
CONVENTIONAL 3+3 DESIGNS
Conventional 3+3 methods employ an ad-hoc approach to screen dose levels and identify the MTD.
Toxicity is defined as a binary event and patients are treated in groups of three, starting with the initial dose.
Algorithm iterates moving dose up or down depending on the number of toxicities observed.
No estimation in a traditional sense is involved.
The MTD is a statistic identified from the data - highest dose studied with less than, say 1/3 toxicities (i.e., 0 or 1 dose-limiting toxicities out of six patients).
Sponsors: Biopharmaceutical Section of ASA ,
0 2 or moreCount
Events
Treat 3 patients at dose
Start at the lowest
reasonable dose
Increase dose to next level
Treat 3 additional patients
at dose
Count
Events
Decrease dose or stop and
select lower dose
1
0 1 or more
CONVENTIONAL 3+3 DESIGNS
Sponsors: Biopharmaceutical Section of ASA ,
CONVENTIONAL 3+3 DESIGNS
0%
20%
40%
60%
80%
100%
0.0 0.2 0.4 0.6 0.8 1.0
True "p"
Ch
an
ce o
f "S
tep
pin
g U
p"
Even with a
30% chance of
an “event”
there is still a
50% chance of
stepping up!
Sponsors: Biopharmaceutical Section of ASA ,
CONVENTIONAL 3+3 DESIGNS
Strengths:
Simple to implement and understand
Requires no computer program
Familiar to many clinicians
Drawbacks:
Tend to treat many patients at low, ineffective doses
Often provide poor estimates of MTD (i.e., large uncertainty) -probability of stopping at incorrect dose higher than perceived
Hence, unsafe or non-efficacious doses may be advanced to Phase III trials.
Sponsors: Biopharmaceutical Section of ASA ,
CRM
Continual Reassessment Method (CRM):
Originated as a Bayesian method for phase I cancer trials of cytotoxic agents.
For a pre-defined set of doses and a binary response, estimates MTD as the dose level that yields a particular target proportion of responses (e.g., TD20).
Assumes a particular model (such as logistic function)
Assignment of doses converges to the MTD.
See Garrett-Moyer (2006) for an excellent tutorial.
Sponsors: Biopharmaceutical Section of ASA ,
CRM
The method assumes that the probabilities of both efficacy and toxicity increase with increasing dose.
The method also assumes that toxicity can be defined as a binary outcome.
The “acceptable” toxicity rate is explicitly defined and the MTD is the highest (most efficacious) dose with acceptable toxicity.
Similar designs can be used to explore dose-efficacy relationships (for agents that are non-cytotoxic).
Sponsors: Biopharmaceutical Section of ASA ,
CRM
The method begins with an assumed a priori dose-toxicity curve and a chosen target toxicity rate.
The first patients are assigned the dose most likely to be associated with the target toxicity level.
The estimated dose-toxicity curve is refit (i.e., the posterior distribution of the model is updated) after each patient’s outcome has been observed.
Hence, the updated curve is shifted slightly up or down depending on whether the patient experienced a dose-limiting toxicity.
Sponsors: Biopharmaceutical Section of ASA ,
CRM
The next patient is assigned the dose closest to the MTD based on the updated dose-toxicity curve (posterior distribution).
Patients continue to be treated until some pre-defined level of certainty is achieved or pre-defined stopping criteria are met.
Once the stopping criteria is achieved, the final dose is selected as the MTD.
Sponsors: Biopharmaceutical Section of ASA ,
CRM
For example, consider the following curve:
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 5 10 15 20
Dose
Even
t R
ate
If target level of toxicity is 10%, then dose level 5 would be the optimal starting dose.
Sponsors: Biopharmaceutical Section of ASA ,
CRM
An example of how the CRM might work:
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 5 10 15 20
Dose
Even
t R
ate
Sponsors: Biopharmaceutical Section of ASA ,
CRM
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 5 10 15 20
Dose
Even
t R
ate
An example of how the CRM might work:
Final Dose
Sponsors: Biopharmaceutical Section of ASA ,
CRM
The implementation of a CRM requires a substantial collaboration between the investigator and statistician.
This collaboration is important in order to determine:
The dose-toxicity model to use
• one parameter logistic, two parameter logistic, etc.
The target rate of toxicity (or response)
Stopping rules
• Fixed # of patients
• Fixed # of patients treated at a dose
Sponsors: Biopharmaceutical Section of ASA ,
CRM
When publishing results from a CRM trial, typical to display:
The recommended dose (MTD) for a future trial, along with some estimate of the variability surrounding the MTD estimate.
A table that shows how the CRM progressed, including:
• Number of dose-limiting toxicities for each cohort
• Estimated dose at end of each cohort
Sponsors: Biopharmaceutical Section of ASA ,
CRM
Strengths:
“Learns” from information gained at early time points in the study – all patients studied contribute to the estimated dose.
Less likely to treat patients at toxic doses – tends to incur fewer dose-limiting toxicities.
More likely to treat patients at efficacious doses
Can more accurately estimate the MTD as compared to standard 3+3 designs
Sponsors: Biopharmaceutical Section of ASA ,
CRM
Drawbacks:
Mathematical and statistical complexities make it difficult for many clinical investigators to understand.
Properties must be assessed via simulation.
Early on, large dose escalations can occur based on little information which may cause more patients to be treated at unsafe doses.
Dosing first patients at level deemed appropriate by a priori curve may be worrisome due to uncertainty surrounding this curve.
Sponsors: Biopharmaceutical Section of ASA ,
MODIFIED CRM’s
To address some of the concerns with the original CRM, Several modified CRM approaches have been developed and implemented:
Always start at the lowest dose level under consideration
Enroll 2-3 patients in each cohort
Proceed as a standard 3+3 dose escalation design in the absence of dose-limiting toxicities.
Any given dose escalation cannot increase by more than one level.
Sponsors: Biopharmaceutical Section of ASA ,
MODIFIED CRM’s
Strengths:
Mathematical model is not solely responsible for determining dosage increases – restricted by design.
Starting dose can be chosen as with a traditional design – start dosing at the lowest level
Drawbacks:
Mathematical and statistical complexities make it difficult for many clinical investigators to understand.
Properties must be assessed via simulation.
Sponsors: Biopharmaceutical Section of ASA ,
OTHER BAYESIAN DESIGNS
1) Escalation with overdose control
• Similar to CRM, but addresses ethical need to control probability of overdosing
2) Designs based on Bayesian decision theory
• Next dose assignment is determined by maximizing the gain function, which is sequentially updated after each response
3) Bayesian D-Optimal designs
• Introduces formal optimality criterion (D-optimality) minimizing the determinant of the variance-covariance matrix of the model parameter estimates
Sponsors: Biopharmaceutical Section of ASA ,
SUMMARY
Standard 3+3 designs were not designed with the intention of producing accurate estimates of a target quantile.
Bayesian model-based methods (such as the CRM) provide better estimates of the MTD and dose-response curve.
However, such methods are complicated to explain to non-statisticians and computationally challenging to implement.
The key to their usefulness lies in the packaging of these methods in user-friendly software that runs quickly and is well-documented.
Sponsors: Biopharmaceutical Section of ASA ,
1. Multiple Comparison (MC) Approaches
Frequentist based
Some approaches imbed Bayesian methods within study stages
2. Model Based Approaches
Bayesian
Frequentist (D-optimal Criterion)
II. Adaptive Dose-Response Methods for
Late-Stage Exploratory Development
Sponsors: Biopharmaceutical Section of ASA ,
PART 1: MC APPROACHES
Objectives
Identify a dose-response relationship (trend test)
Select dose(s) effective relative to a control (pairwise comparisons)
Hypothesis tests based on contrasts between doses
Dose is a categorical variable
Advantages
Easy to implement
Limited assumptions needed
Useful if too few doses to enable modeling
Sponsors: Biopharmaceutical Section of ASA ,
DISADVANTAGES
Designed for estimation and testing of doses used in the trial
Hypothesis testing typically requires high sample sizes per dose group
Feasibility limits number of doses explored
May identify if dose response exists BUT
Does not provide information on the dose response profile
Does not provide for quantitative estimates and precision of targeted dose(s) of interest such as MED or ED95
Sponsors: Biopharmaceutical Section of ASA ,
Approaches
Extending the classical group sequential framework
Combination function approaches (foundation in meta-analysis)
Types of adaptations:
Add or drop doses
Sample size reassessment for future stages
Early stopping for futility or efficacy
Seamless shift across development phases
MC FRAMEWORK FOR ADAPTIVE METHODS
Sponsors: Biopharmaceutical Section of ASA ,
Stallard & Todd (2003)
Extended classical group sequential designs to multiple treatment arms
Identify best treatment based on a maximum standardized test statistic
GROUP SEQUENTIAL DESIGNS
Sponsors: Biopharmaceutical Section of ASA ,
Approach
Trial analyzed in a series of independent stages
Very flexible
– Do NOT have to define what you will adapt in advance
– Bayesian decision theoretical approach can be used
– DO have to define a-priori how you will combine the test statistics from the stages to make inference
Controls family-wise type I error rate
Adjustments needed for inference (Posch et al. 2005)
Multiplicity adjusted p-values for dose-control comparisons
Point estimates and CI adjusted for early stopping and treatmentselection
ADAPTIVE TREATMENT SELECTION BASED ON
COMBINATION TESTS
Sponsors: Biopharmaceutical Section of ASA ,
Stage wise tests
Independent observations between stages
Let p1, p2 be p-values from stages 1 and 2 respectively
A-priori define at minimum
Combination function
Stage wise and overall alpha levels
More than 18 different combination functions (Becker, 1994)
Commonly used functions
Fisher’s: C(p1,p2) = p1*p2
Inverse Normal: C(p1,p2) = -w1N-1(1-p1) - w2N
-1(1-p2)
THEORETICAL BACKGROUND
Sponsors: Biopharmaceutical Section of ASA ,
COMBINING P-VALUES: FISHER’S METHOD
Under the null: Pi ~ U(0,1) i.i.d.
Let ; then i.i.d.
and
To test the Global Null Hypothesis: H0: H01 H02
Note: Pr(H01 H02) = P1·P2
Hence; compare -2(ln p1 + ln p2) to critical value
2lni iX P2
2~i dfX
2
2
1
~n
i n df
i
X
2
4 (1 )df
Sponsors: Biopharmaceutical Section of ASA ,
Note P may only be approximately uniform [0,1] under the Null:
IF individual hypotheses are composite, or if responses are discrete
Jennison & Turnbull (2005); Robins, et al. (2000)
Two-stage procedure Bauer & Kieser (1999):
Weaker condition:
Distribution of P1 and conditional distribution of P2|P1
stochastically larger than or equal to the uniform distribution on [0,1]
ON DISTRIBUTION OF P-VALUE
Sponsors: Biopharmaceutical Section of ASA ,
Jennison & Turnbull (2005)
Inverse Normal: C(p1,p2) = -w1N-1(1-p1) - w2 N-1(1-p2)
Historically
Mosteller & Bush (1954): generalization based on fixed weights
– If
– Then
Interpretation concern– Weighting patient information unequally based on stage
INVERSE NORMAL FUNCTION
1
1
1( ... ) (1 ) ~ (0,1)k k kZ Z where Z N p N
k
2
1
1K
k
k
w
1
~ (0,1)K
k k
k
w Z N
Sponsors: Biopharmaceutical Section of ASA ,
Let
Then the z-statistic for pooled data equals:
Combination test statistic equals z-statistic for pooled data
– Invariant to partitioning of the data
– A function of the sufficient statistic (efficient)
– Sample size of stages must be fixed
Note: In general, the number of stages & weights can be adapted for K > 2
Fisher (1998) “variance spending”
– Spend of variance of Z statistic: study ends when sum is 1
INVERSE NORMAL FUNCTION (cont.)
1
K
k k
k
Z w Z
kk
nw
N
2
kw
Sponsors: Biopharmaceutical Section of ASA ,
Bauer & Kohne (1994)
The pre-specified combination test needs to be followed
Properties only hold if followed
e.g. Cannot decide to treat Stage 1 as internal pilot (even if no adaptation is made for Stage 2)
“Protocol has to describe which types of adaptation are
intended.”
Conclusions depend on types of adaptations
Ad-hoc adaptations (even if family-wise Type I error preserved) can make interpretation difficult
Estimates may be biased or intractable
POTENTIAL ABUSES
Sponsors: Biopharmaceutical Section of ASA ,
CombinesClosed testing procedure
A multiplicity adjustment procedure (e.g. Bonferroni-Holm min P, Simms)
A combination test procedure (Fishers, Inverse Normal)
Same approach can be used for seamless designsStages can span Phase II/III
APPLICATION TO DOSE FINDING
Sponsors: Biopharmaceutical Section of ASA ,
Assess design options/power via simulations
Power is a function of unknown dose response
In two stage approach with fixed sample sizes, inverse normal combination function is efficient
Model based approaches may be more efficient (but also more complex)
Resulting estimates can be biased
Recommend assessing via simulation
Last resort, use the last stage for estimation purposes
DO NOT ABUSE
Follow required pre-specified rules
Describe possible adaptations in protocol
RECOMMENDATIONS
Sponsors: Biopharmaceutical Section of ASA ,
Objectives:
Estimate dose-response
Identify optimal (target) dose(s)
ASSUMES a functional relationship between the dose and response (Parametric & Non-parametric model-based approaches)
Advantages
Quantitative estimates, such as ED95, can be made from the model
Provides quantitative information on the dose response profile useful in planning future studies
PART 2: MODEL BASED APPROACHES
Sponsors: Biopharmaceutical Section of ASA ,
More sensitive to model assumptions (parametric model)
Estimation and analysis more complex (e.g. non-linear models)
Sample size calculation usually requires simulations
Typically requires more doses to implement
DISADVANTAGES
Sponsors: Biopharmaceutical Section of ASA ,
Focus
Efficient learning: More observations at doses that better inform the dose-response and or the target dose(s)
Effective treatment of patients in the trial: maximize patient exposure to effective doses/Tx
Approaches
Bayesian (predictive probabilities)
Frequentist (optimal experimental design)
ADAPTIVE MODEL BASED APPROACHES
Sponsors: Biopharmaceutical Section of ASA ,
Dose-response model: parametric or non-parametric
Longitudinal models: predict final patient outcome data at interims
Prior distributions on model parameters
Decision making components may include:
Dose allocation rules
Stopping rules
Decision rules can be highly flexible:
Minimize expected loss function
Maximize expected utility function ( - loss function)
BAYESIAN APPROACH: COMPONENTS
Sponsors: Biopharmaceutical Section of ASA ,
West and Harrison (1997): Bayesian Forecasting and Dynamic Models
A piece-wise linear model
Smoothed transitions in the dose-response slope across the doses
Does not restrict the shape of the dose response curve
Developed for analysis and forecasting of time series data
Other non-parametric models
Splines, Kernel Methods
NORMAL DYNAMIC LINEAR MODEL
Sponsors: Biopharmaceutical Section of ASA ,
Assumptions
Response at each dose normally distributed about a mean
Change in mean between adjacent doses can be predicted by a simple linear model
Variability decomposed into two components
Observational variability for the patient response about the mean for the given dose
System variability around the linear model that relates the
adjacent means
NDLM (cont.)
Sponsors: Biopharmaceutical Section of ASA ,
NDLM (cont.)
Priors placed on:
µk (mean at dose k) H (smoothing parameter)
(slope parameters) 2 (observational variance)
Let Rik be the ith patient response at dose k, and Dk
represent the kth dose with mean µk.
Observation Equation:
System Equations:
2| ~ (0, )ik k k ik ikR D where N
2
1 1 ~ (0, )k k k k kwhere N H
2
1 ~ (0, )k k k kwhere N H
Sponsors: Biopharmaceutical Section of ASA ,
4-PARAMETER LOGISTIC MODEL
4
1 22
3
1 ( )i i
i
RD
Patient indicator
Patient response
Level of drug
Response at 0 drug
Max. attributable effect of drug + 1
Dose producing response half way between 1 and 2
Related to steepness of slope
Random error for patient I {often iid N(0,1) }
1
2
3
4
i
i
i
i
R
D
1-1 Comparison to Emax Model
(Di)-1= Di
4 = Hill Coef- 4 = Hill Coef
( 3)-1 = ED503 = ED50
1 - 2 = Emax1 - 2 = Emax
2 = E02 = E0
4> 0
4< 0
50
max0
ii i
i
D ER E
D ED
Sponsors: Biopharmaceutical Section of ASA ,
Requires monotonicity: increasing or decreasing
Minimum of 5 doses desirable (4-parameter model)
If highest dose < ED95
Estimates of Emax, ED50, and Hill Coefficient (gamma) impacted: high coefficient of variation & bias
Fit in data range usually good
Bayesian approach
Strong priors might be assumed for Emax if highest dose thought to be less than ED95
4-PARAMETER LOGISTIC MODEL (cont.)
Sponsors: Biopharmaceutical Section of ASA ,
Targets estimation of the overall dose-response
Formal optimality criterion (D-optimal)
Minimize determinant of the variance covariance matrix of the model parameter estimates (maximizes information)
Allocates patients (sequentially or group sequentially) to provide the most information
Typically keep allocation of placebo constant
Wide class of dose-response models are applicable
e.g. Four-parameter logistic model
D-OPTIMAL: AN OPTIMAL DESIGN APPROACH
Sponsors: Biopharmaceutical Section of ASA ,
One approach (ADRS WG white paper)
Allocate equally across doses for first cohort
Fit model
Based on this model, determine optimal allocation ratio for nextcohort of patients to maximize information
Bayesian D-Optimal
Place a prior distribution on the model parameters
After each cohort, calculate the posterior distribution
Similarly, update allocation ratio to minimize determinate of the variance co-variance matrix of model parameters
D-OPTIMAL (cont.)
Sponsors: Biopharmaceutical Section of ASA ,
Neuropathic Pain
Minimum Clinical Significance:
Average Daily Pain Score (ADPS)
Ranges (0 no pain, 10 worst pain)
1.5 difference from placebo change from baseline
Design PoC study to select future dose(s) Phase III
12 fold dose range
Dose-response unknown…may be inverted-U shaped
Positive control desirable for assay sensitivity
Too costly to explore dose-range?
CONSIDER
Sponsors: Biopharmaceutical Section of ASA ,
Resp
on
se
Dose
NotInformative
Informative
FIXED DOSE DESIGN
Sponsors: Biopharmaceutical Section of ASA ,
• Pfizer: Smith, Jones, Morris, Grieve, Tan (2006)
1 wk
Lead-In
4 wks
Double Blind Treatment
1 wk
Follow-up
7 Doses
Positive Control
Placebo
Max n=35 per arm
Type I error < 5%
Power ~ 80%
ADAPTIVE PoC CASE STUDY
Sponsors: Biopharmaceutical Section of ASA ,
Objective: select doses for Phase 3
Dose-Response Model: NDLM
Decision Making Components:Dose Allocation Rule: equal allocation
Stopping Rules (2 interims)
– Continue the study if at least one dose X has:
Pr (Effect at dose X > 1.5) > 0.80
– Drop up to 2 non-efficacious doses at each interim where:
Pr ( Effect at dose X < 1.5 ) > 0.80
ADAPTIVE FEATURES
Sponsors: Biopharmaceutical Section of ASA ,
Trial stopped at first interim (flat dose-response)
Approximately $2M saved due to stopping early
Rough comparison to fixed design with pairwise comparison of each dose to placebo
Approximately 3-4 times larger
No early stopping
Controlling for multiple comparisons
Type I error, 1-sided, 10%
RESULTS
Sponsors: Biopharmaceutical Section of ASA ,
Objectives:
Identify target dose (ED95)
Estimate dose response
Dose-Response Model: NDLM
Decision Making Components:
Dose Allocation Rule (maximized utility function)
Stopping Rules (decision analytic)
EXAMPLE: ASTIN (Krams et al. 2003)
Sponsors: Biopharmaceutical Section of ASA ,
Maximize Utility Function: Allocate patients to maximize information about ED95
Minus the variance of the predicted mean response at the ED95
Includes uncertainty in ED 95 dose & in the dose response
Function of future patient data
Determining next patient assignment
Calculate the expected utility for each possible dose assignment
– Expectation over the posterior predictive distribution for the data yet to be observed
– Ongoing patient data predicted from earlier data using a longitudinal model (that gets updated during the study)
Assume next patient is last patient
Assign dose that is expected to result in the smallest variance
Maintain blind by selecting randomly within 5% of optimal dose
DOSE ALLOCATION RULE
Sponsors: Biopharmaceutical Section of ASA ,
Trial stopped for futility
Savings
Dollars: $3.5M direct grant costs
Time: termination decision at least 6 months earlier
RESULTS
Sponsors: Biopharmaceutical Section of ASA ,
Phase II dose ranging study
Schizophrenia
Objective
Confirm positive POC study
– 3 arm: High dose, Active (assay sensitivity), Placebo
Explore lower doses
Determine dose(s) Phase III
Dose range 8 fold
4 doses
Primary Measure
PANSS total score at 6 weeks
EX: ADAPTIVE DESIGN NOT RECOMMENDED
Sponsors: Biopharmaceutical Section of ASA ,
Subjective primary measure / 20 sites
Significant effect due to site
Desirable to stratify by site
Long term outcome relative to expected enrollment rate (no biomarker)
Narrow dose range well covered by 4 doses
High dose effective, but may not be near Emax
STUDY CHARACTERISTICS
Sponsors: Biopharmaceutical Section of ASA ,
Fixed design with equal allocation
Adaptive allocation
Bayesian D-Optimal Criterion (4 parameter logistic model)
Allocation adapts to increase efficacy of estimates of model parameters
Stopping Rules (4 interim analyses)
Stop for Futility: If predicted mean difference high dose vs placebo > -5, with 95% confidence
Stop for Efficacy: If predicted mean difference low dose vsplacebo < 0, with 95% confidence
DESIGNS COMPARED
Sponsors: Biopharmaceutical Section of ASA ,
No compelling advantage to adaptive randomization over fixed allocation
Adaptive randomization favored slightly more subjects randomized to placebo and approximately equal allocation to other doses
Fixed design would be slightly more powerful for pairwise comparisons with unequal allocation & not effect dose-response estimation adversely (2:1:1:1:1)
Efficiencies over-estimated since perfect information was assumed in simulations for adaptive allocation
Use of parametric dose-response model (unknown dose-response)
Additional resources/complexity not warranted
RECOMMENDATION
Sponsors: Biopharmaceutical Section of ASA ,
Consider routinely assessing appropriateness of adaptive designsin exploratory development
Asses potential gains against those of standard fixed designs
Balance complexity with potential gains
Trial simulations typically needed
Fine tune design
Assess operating characteristics
Recommended even when ONLY considering a fixed trial design
Consider Seamless PoC/Phase 2 dose-response studies
Recommend model based approaches
More informative of dose response profile than MC
Critical to assess model assumptions
Non-parametric models less restrictive
CONCLUDING REMARKS
Sponsors: Biopharmaceutical Section of ASA ,
III. Simulations Comparing Adaptive and
Non-Adaptive DF Methods
Outline:
• Evaluating statistical operational characteristics of complex DF designs and methods
• Comparing DF designs and methods: PhRMA’s Adaptive Dose Ranging Studies working group simulation study
• Conclusions from ADRS WG simulations
Sponsors: Biopharmaceutical Section of ASA ,
MOTIVATION
Evaluation of operational characteristics (OCs) of proposed statistical methods is a critical step in designing a clinical trial – comparison of methods
The OCs include the power to detect signals of interest, the precision of estimates for quantities of interest, expected duration, etc in particular, used to determine sample size and number of arms
Complexity of adaptive dose finding designs and other non-traditional dose finding methods typically no closed form expressions for OCs metrics
Simulation-based evaluation needs to be employed
Sponsors: Biopharmaceutical Section of ASA ,
KEY GOALS OF DF TRIALS
Typical goals of Phase II trials:
Determine evidence of dose response (DR) signal, i.e., if average response changes with dose level – proof-of-concept (PoC)
Select target dose(s) for confirmatory phase – typically MED; other targets also used (e.g., maximum useful dose)
Estimate DR profile – usually for efficacy, but safety of increasing interest
These goals determine the design of the study and the operational characteristics that need to be evaluated
Sponsors: Biopharmaceutical Section of ASA ,
SIMULATING DF TRIALS
Trial simulation is the main tool for evaluating study OCs; itneeds to properly incorporate multiple factors in study:
Type: parallel groups, cross-over, titration, etc
Available doses, inclusion of active control(s)
Dose allocation scheme (fixed vs. adaptive)
If adaptive, frequency and timing of adaptations (and algorithm for recalculating allocation ratios)
Dose response profile(s):
more than one should be used to assess sensitivity
flat dose response should be included to assess Type I error andimpact on dose selection
Sponsors: Biopharmaceutical Section of ASA ,
SIMULATING DF TRIALS (CONT.)
Response variables: type (e.g., continuous, binary, count, ordinal); distribution (e.g., normal, Poisson)
Variance and covariance parameters (e.g., within- and between patient variances, between-site variances)
Sample size (e.g., expected, maximum)
Sensitivity analysis: impact of changes in assumed parameters/models/design on OCs (highly recommended)
Drop-out and missing data models (e.g., time to drop-out)
Patient accrual process (e.g., rates, uniformity over time)
Stopping rules, if any (e.g., futility, efficacy)
Sponsors: Biopharmaceutical Section of ASA ,
PhRMA ADRS WG
Adaptive Dose Ranging Studies (ADRS) WG: formed, with others, asresult of survey to identify key drivers of poor performance in pharmaindustry poor understanding of DR indicated as a leading cause for high attrition in late development.
Investigate and develop designs and methods for efficiently learningabout efficacy and safety DR profiles
Evaluate statistical OCs of alternative designs and methods (adaptive and fixed) to make recommendations on their use
Increase awareness about ADRS, promoting their use, when advantageous
Comprehensive simulation study comparing ADRS to other DF methods, quantifying potential gains
Sponsors: Biopharmaceutical Section of ASA ,
SUMMARY OF DESIGN AND ASSUMPTIONS
Proof-of-concept + dose-finding trial, motivated by neuropathic pain indication (conclusions and recommendations can be generalized)
Key questions: Whether there is evidence of dose response and, if so, which dose level to bring to confirmatory phase and how welldose response (DR) curve is estimated.
Primary endpoint: Change from baseline in VAS at Week 6 (continuous, normally distributed)
Dose design scenarios (parallel arms):
- 5 equally spaced dose levels: 0, 2, 4, 6, 8
- 7 unequally spaced dose levels: 0, 2, 3, 4, 5, 6, 8
- 9 equally spaced dose levels: 0, 1, …, 8
Significance level: one sided FWER = 0.05
Sample sizes: 150 and 250 patients (total)
Sponsors: Biopharmaceutical Section of ASA ,
DOSE RESPONSE PROFILES
Sponsors: Biopharmaceutical Section of ASA ,
DF METHODS USED IN SIMULATONS
Traditional ANOVA based on pairwise comparisons and multiplicity adjustment (Dunnett)
MCP-Mod combination of multiple comparison procedure (MCP) and modeling (Bretz, Pinheiro, and Branson, 2005)
MTT: novel method based on Multiple Trend Tests
Bayesian Model Averaging: BMA
Nonparametric local regression fitting: LOCFIT
GADA: Dynamic dose allocation based on Bayesian normal dynamic linear model (Krams, Lee, and Berry, 2005)
D-opt: adaptive dose allocation based on D-optimality criterion
Sponsors: Biopharmaceutical Section of ASA ,
SELECTED SIMULATION
RESULTS
More detailed results given in the ADRS WG’s White Paper, available at http://biopharmnet.com/doc/doc12005.html
Sponsors: Biopharmaceutical Section of ASA ,
POWER TO IDENTIFY DR
Sponsors: Biopharmaceutical Section of ASA ,
DOSE SELECTION UNDER FLAT DR
Sponsors: Biopharmaceutical Section of ASA ,
DOSE SELECTION UNDER ACTIVE DR
Sponsors: Biopharmaceutical Section of ASA ,
CORRECT TARGET DOSE INTERVAL
Sponsors: Biopharmaceutical Section of ASA ,
DOSES SELECTED – LOGISTIC, N=150
Sponsors: Biopharmaceutical Section of ASA ,
SAMPLE PRED. DR – LOGISTIC, N=150
Sponsors: Biopharmaceutical Section of ASA ,
ADRS WG CONCLUSIONS
Detecting DR is considerably easier than estimating it
Current sample sizes for DF studies, based on power to detect DR, are inappropriate for dose selection and DR estimation
None of methods had good performance in estimating dose in the correct target interval: maximum observed percentage of correct interval selection – 60% larger N
needed
Adaptive dose-ranging methods (i.e., ADRS) lead to gains in power to detect DR, precision to select target dose, and to estimate DR – greatest potential in the latter two
Sponsors: Biopharmaceutical Section of ASA ,
ADRS WG CONCLUSIONS
Model-based methods have superior performance compared to methods based on hypothesis testing
Number of doses larger than 5 does not seem to produce significant gains (provided overall N is fixed) trade-offbetween more detail about DR and less precision at each dose
In practice, need to balance gains associated with adaptive dose ranging designs against greater methodological and operational complexity
Sponsors: Biopharmaceutical Section of ASA ,
IV. Overall Conclusions and
Recommendations
Sponsors: Biopharmaceutical Section of ASA ,
CONCLUSIONS & RECOMMENDATIONS
Adaptive, model-based dose finding designs should be routinely considered for use in drug development (Early Development, PoC, Dose Ranging) can lead to substantial gains in efficiency over traditional methods
Dose assignment algorithm should be prospectively andclearly specified in study protocol
Trial simulations should be used to fully evaluate operational characteristics of design prior to study start
Seamless approaches should be considered to improve efficiency, especially between PoC (Ph. I/IIa) and dose ranging (Ph. IIb)
Sponsors: Biopharmaceutical Section of ASA ,
CONCL. & RECOMMENDATIONS (CONT.)
Sample size calculations for adaptive DF designs should take into account the precision of target dose estimates and, more broadly, the accuracy of the decision(s) to be made from the study
Early stopping rules, for efficacy and safety, should be implemented, when feasible, to allow greater efficiency gains in adaptive design
Potential gains associated with adaptive approaches should always be contrasted to additional complexity and costs related to their implementation – not a panacea
Sponsors: Biopharmaceutical Section of ASA ,
CONCL. & RECOMMENDATIONS (CONT.)
Greater usage of these adaptive DF designs should be encouraged and will require:
• Good quality software packages with well documented code and examples for implementing approaches and conducting simulations needed to evaluate operating characteristics of these methods.
• A greater understanding of the strengths and weaknesses of these approaches (hopefully, this course has helped out along this regard)
• More published examples of studies that have utilized these methods.
Sponsors: Biopharmaceutical Section of ASA ,
Bauer P and Kieser M (1999). Combining different phases in the development of medical treatments within a single trial. Statistics in Medicine, 18:1833-1848.
Bauer P and Kohne K (1994). Evaluation of experiments with adaptive interim analysis. Biometrics, 50:1029-1041.
Fisher LD (1998). Self-designing clinical trials. Statistics in Medicine, 17:1551–1562
Gaydos B, Krams M, Perevozskaya I, Bretz F, Liu Q, Gallo P, Berry D, Chuang-Stein C, Pinheiro J, Bedding A (2006). Adaptive Dose-Response Studies. Drug Information Journal, 40(6): 451-461.
Garrett-Moyer E (2006). The continual reassessment method for dose-finding studies: A Tutorial. Clinical Trials, 3: 57-71.
Jennison C and Turnbull BW (2005). Meta-analyses and adaptive group sequential designs in the clinical development process. Journal of Biopharmaceutical Statistics, 15: 537-558.
SELECT REFERENCES
Sponsors: Biopharmaceutical Section of ASA ,
Krams M, Lees KR, Hacke W, Grieve AP, Orgogozo JM, Ford GA for the ASTIN Study investigators. (2003). ASTIN: An adaptive dose-response study of UK-279,276 in acute ischemic stroke. Stroke, 34: 2543-254.
Posch M, Koenig F, Branson M, Brannath W, Dunger-Baldauf C, and Bauer P (2005). Testing and estimation in flexible group sequential designs with adaptive treatment selection. Statistics in Medicine, 24: 3697-3714.
Robins JM, van der Vaart A, and Ventura V (2000). Asymptotic Distribution of P Values in Composite Null Models. Journal of the American Statistical Association, 95: 1143-1156.
Smith MK, Jones I, Morris MF, Grieve AP, and Tan K (2006). Implementation of a Bayesian adaptive design in proof of concept study. Pharmaceutical Statistics,5:39-50.
Stallard N and Todd S (2003). Sequential designs for phase III clinical trials incorporating treatment selection. Statistics in Medicine, 22: 689-703.
West M, Harrison PJ. (1997). Bayesian Forecasting and Dynamic Models. Springer-Verlag, New York.
SELECT REFERENCES (CONTINUED)
Sponsors: Biopharmaceutical Section of ASA ,
BACK-UP
NOTE: The following slides will not be covered in the presentation, but are added to provide
additional details.
Sponsors: Biopharmaceutical Section of ASA ,
Following example taken from PhRMA Adaptive Design Working Group training presentation
Titled: Adaptive Seamless Designs for Phase IIb/III
Clinical Trials
Author: Jeff Maca, Ph.D., Novartis
Full set of this and other training slides can be found at the following open access WEB site:
http://biopharmnet.com/doc/doc12004.html
ADDITIONAL READING MATERIAL
Sponsors: Biopharmaceutical Section of ASA ,
Closed test procedure
• n null hypotheses H1, …, Hn
• Closed test procedure considers all intersection hypotheses.
• Hi is rejected at global level ifall hypotheses HI formed by intersection with Hi arerejected at local level
H1 can only be rejected
at =.05 if H12 is also
rejected at =.05
Source: Jeff Maca
CLOSED TESTING
Sponsors: Biopharmaceutical Section of ASA ,
• A typical study with 3 doses 3 pairwise hypotheses.
• Multiplicity can be handled by adjusting p-values from each stage using Simes procedure
iSi
S pi
Sq min S is number of elements in Hypothesis,
p(i) is the ordered P-values
Source: Jeff Maca
CLOSED TESTING (cont.)
Sponsors: Biopharmaceutical Section of ASA ,
Stage sample sizes: n1 = 75, n2 =75
Unadjusted pairwise p-values from the first stage:
p1,1= 0.23, p1,2 = 0.18, p1,3 = 0.08
Dose 3 selected at interim
Unadjusted p-value from second stage: p2,3 = .01
Source: Jeff Maca
SCENARIO: DOSE FINDING
3 DOSES & CONTROL
Sponsors: Biopharmaceutical Section of ASA ,
q1,123 = min( 3*.08, 1.5*.18, 1*. 23)= .23
q2,123 = p2,3 = .01
C(q1,123, q2,123) = 2.17 P value = .015
Source: Jeff Maca
THREE-WAY TEST
Sponsors: Biopharmaceutical Section of ASA ,
q1,13 = min( 2*.08, 1*. 23)= .16
q1,23 = min(2*.08,1*.18) = .16
q2,13 = q2,23 = p2,3 = .01
C(q1,13, q2,13) = C(q1,23, q2,23) = 2.35 P.value = .0094
Source: Jeff Maca
TWO-WAY TEST
Sponsors: Biopharmaceutical Section of ASA ,
q1,3 = p1,3 = .08
q2,3 = p2,3 = .01
C(q1,3, q2,3) = 2.64 P.value = .0042
Conclusion: Dose 3 is effective
Source: Jeff Maca
FINAL TEST
Sponsors: Biopharmaceutical Section of ASA ,
SOFTWARE IMPLEMENTATIONStable, documented software is available for the CRM and its variations. Can be downloaded from:
• http://www.cancerbiostats.onc.jhmi.edu/Software.cfm (Johns Hopkins Comprehensive Cancer Center)
• http://biostatistics.mdanderson.org/SoftwareDownload (M.D. Anderson)
Computations for other adaptive dose finding methods, such as the NDLM and D-Optimal approaches, are more complex and intensive – software implementation is often based on customized code (e.g., Fortran programs)
Because of computational intensity, generally use lower level programming language (e.g., C++, Fortran) – R and SAS implementations would be too slow
Sponsors: Biopharmaceutical Section of ASA ,
SOFTWARE IMPLEMENTATION (CONT.)
Commercial software currently in prototype phase and under testing – not yet available for general use.
• GADA implementations of NDLM (Tessella)
• Bayesian D-Optimal and variations (CytelSim)
Different pharma companies are developing their own adaptive dose finding software, in combination with simulation software for evaluation of Op. Characteristics
Opportunity for pre-competitive collaboration via PhRMAto develop more general purpose software – possible consortium sponsored by PhRMA member companies
Sponsors: Biopharmaceutical Section of ASA ,
Practical Considerations
Sponsors: Biopharmaceutical Section of ASA ,
PRACTICAL CONSIDERATIONS
Assessing projects for an adaptive design:
• Rapid acquisition of data relative to enrollment rate
- Outcome is more immediate (and accurately) observable
- Trials of longer duration, with relatively slow recruitment, canbe good candidates
• Existence of predictive biological models and/or prior information in patient population of interest
- Predictive models for longer term outcome
Sponsors: Biopharmaceutical Section of ASA ,
PRACTICAL CONSIDERATIONS
Assessing projects for an adaptive design (cont.):
• Ethical considerations as driver for adaptation
• A high exploratory aspect may indicate greater efficacy gains
- Uncertainty relative to e.g., dose, variability, effect size
- Wide dose range
• Caution: Assuming the patient population remains constant over time
- Trials of long duration
- Selection bias due to unblinding of information
Sponsors: Biopharmaceutical Section of ASA ,
PRACTICAL CONSIDERATIONS
Considerations for simulations:
• Leverage information from disease state and exposure-response models
- Selecting dose-response model
- Defining prior distributions for model parameters
- Development of adaptive algorithm (decision criteria)
- Trial simulations to assess design performance
• Optimize over PD response model (best guess of truth)
• Assess sensitivity to using other response models (include models different from the design dose-response model)
Sponsors: Biopharmaceutical Section of ASA ,
PRACTICAL CONSIDERATIONS
Considerations for simulations (cont.):
• Understand impact of enrollment rate
- Include different rates in simulations
- Consider controlling rate if simulations indicate gains
• Assess impact across different dropout models
• Include information lag (e.g. batches) in simulations
• Demonstrating control of Type I error rate
- Simulate over a grid of scenarios in the null space
- Simulate across various dropout & enrollment rate models
Sponsors: Biopharmaceutical Section of ASA ,
PRACTICAL CONSIDERATIONS
Practical Issues for implementation:
• Additional time to develop the design & protocol
- May need to run extensive simulations to understand operating characteristics
- Communicate with the primary investigator’s about the design, receive feedback, and address concerns
• Clinical trial material needs
- Dosage strengths, quantity, packaging
• Additional resources for modeling & data analysis
- Interim data preparations & analyses
- Final analysis more complex
Sponsors: Biopharmaceutical Section of ASA ,
PRACTICAL CONSIDERATIONS
Practical Issues for implementation (cont.):
• Increase in site communication
- Design changes / Patient treatment assignments
- Fax, Interactive Voice Response Systems, WEB interface
- Additional site training
• Determine type of committee needed to monitor trial
- Ensure protocol is followed (no programming errors)
- Unanticipated safety signals not accounted for in the adaptive algorithm
- Engage committee early in scenario simulations (prior to protocol approval)
Sponsors: Biopharmaceutical Section of ASA ,
PRACTICAL CONSIDERATIONS
Practical Issues for implementation (cont.):
• Determine what data will be needed
• How data will be collected?
- Electronic data capture, Expedited report forms (not monitored), Voice Response system, Excel Spreadsheet
- eDC systems not friendly for interim data extraction
• How clean data needs to be?
- Fully verified (locked) data is not typical
- Use latest data in modeling/analysis (continually clean data)
• Document, Document, Document!!!