a general framework for model-based drug development using probability metrics for quantitative...
TRANSCRIPT
A General Framework for Model-Based Drug Development Using Probability
Metrics for Quantitative Decision Making
Ken Kowalski, Ann Arbor Pharmacometrics Group (A2PG)
2
Outline Population Models
Basic Notation and Key Concepts Basic Probabilistic Concepts General Framework for Model-Based Drug
Development (MBDD) Examples Final Remarks/Discussion Bibliography
PaSiPhIC 2012 A2PG
PaSiPhIC 20123
Population ModelsBasic Notation
A2PG
General Form of a Two-Level Hierarchical Mixed Effects Model:
Definitions:
,0~ ),or (e.g., ,
,0~ ,,
Neg
Nhfy
iiiiii
iiiii
i
effects random individual-intra ofmatrix covariance
effects random individual-inter ofmatrix covariance
individualfor effects random individual-intra of vector
individualfor effects random individual-inter of vector
parameters effects fixed of vector
individualfor parameters specific-subject of vector
individualfor nsobservatio of vector
i
i
i
iy
i
i
i
i
PaSiPhIC 20124
Population ModelsKey Concepts
A2PG
Typical Individual Prediction:
Easy to compute, same functional form as f
Population Mean Prediction:
Integral is often intractable when f is nonlinear Typically requires Monte-Carlo integration (simulation)
The typical individual and population mean predictions are not the same when f is nonlinear Cannot observe a ‘typical individual’ Can observe a sample mean
fyE ii 0
iiiii dpyEyE
PaSiPhIC 20125
Basic Probabilistic Concepts Statistical intervals (i.e., confidence and
prediction intervals) Statistical power Probability of achieving the target value (PTV) Probability of success (POS) Probability of correct decision (POCD)
A2PG
PaSiPhIC 20126
What’s the difference between a confidence interval and prediction interval?
A2PG
A confidence interval (CI) is used to make inference about the true (unknown) quantity (e.g., population mean) Reflects uncertainty in the parameter estimates Typically used to summarize the current state of
knowledge regarding the quantity of interest based on all available data used in the estimation of the quantity
A prediction interval (PI) is used to make inference for a future observation (or summary statistic of future observations) Reflects both uncertainty in the parameter estimates
as well as the sampling variation for the future observation
PaSiPhIC 20127
Relationship Between CIs and PIs
A2PG
Confidence Limits for
X
X
Prediction Limits Recognizing Uncertainty
in E( )X
Distribution of sampling variation
Prediction Limits if E( ) Located Here
X
Note: Prediction intervals are always wider than confidence intervals.
PaSiPhIC 20128
Confidence interval for the mean based on a sample of N observations
A2PG
N
stX N 1,1 2
Sample mean (parameter estimate)
Standard error of the mean (parameter uncertainty)
PaSiPhIC 20129
Prediction interval for a single future observation
A2PG
22
1,1 2s
N
stX N
Sample mean (parameter estimate)
Sample variance of the mean (parameter
uncertainty)
Sample variance of a future observation (sampling variation)
Note: The sample mean based on N previous observations is the best estimate for a single future observation.
PaSiPhIC 201210
Prediction interval for the mean of M future observations
A2PG
M
s
N
stX N
22
1,1 2
Sample mean (parameter estimate)
Sample variance of the mean (parameter
uncertainty)
Sample variance of the mean of M future
observations (sampling variation)
Note 1: The sample mean based on N previous observations is the best estimate for the mean of M future observations.Note 2: A prediction interval for M=∞ future observations is equivalent to a confidence interval (see Slide 8). This will also be referred to as ‘averaging out’ the sampling variation.
PaSiPhIC 201211
A Conceptual Extension of Confidence and Prediction Intervals to Population Modeling
A2PG
Measure/Quantity Simple Mean Model Population Model
Parameters , , Ω,
Prediction
Sampling Variation
Parameter Uncertainty*
Confidence Interval See Slide 8Stochastic
simulations with sufficiently large M
Prediction Interval See Slide 10Stochastic
simulations with finite M
X ˆ fy
s
ˆ,ˆ,Cov
I2ˆˆ,ˆ
N
ssX
* Note for the simple mean model the standard error of the mean does not take into account uncertainty in the sampling variation (s) whereas in population models we typically take into account the uncertainty in Ω and .
PaSiPhIC 201212
Quantifying Parameter Uncertainty in Population Models – Nonparametric Bootstrap
A2PG
Randomly sample with replacement subject data vectors to preserve within-subject correlations to construct bootstrap datasets
Re-estimate model parameters for each bootstrap dataset to obtain an empirical (posterior) distribution of the parameter estimates (, Ω, )
May require stratified-resampling procedure (by design covariates) for a pooled-analysis with very heterogeneous study designs E.g., limited data at a high dose in one study may
be critical to estimation of Emax
PaSiPhIC 201213
Quantifying Parameter Uncertainty in Population Models – Parametric Bootstrap
A2PG
Draw random samples from multivariate normal distribution with Mean vector = [ ] Covariance matrix = Cov( )
Obtained from Hessian or other procedure (e.g., COV step in NONMEM) Based on Fisher’s theory (Efron, 1982)
Assumes asymptotic theory (large sample size) that maximum likelihood estimates converge to a MVN distribution See Vonesh and Chinchilli (1997)
Based on Wald’s approximation that likelihood surface can be approximated by a quadratic model locally around the maximum likelihood estimates Approximations are dependent on parameterization Improved approximations may occur by estimating the natural
logarithm of the parameter for parameters that must be positive
ˆ,ˆ, ˆ,ˆ,
PaSiPhIC 201214
Non-parametric Versus Parametric Bootstrap Procedures
A2PG
The non-parametric bootstrap procedure is widely used in pharmacometrics Often used as a back-up procedure to quantify parameter
uncertainty when difficulties arise in estimating the covariance matrix (eg., NONMEM COV step failure) In this setting issues with a large number of convergence
failures in the bootstrap runs may call into question the validity of the confidence intervals (i.e., Do they have the right coverage probabilities?)
This form of parametric bootstrap procedure is less computationally intensive than other bootstrap procedures that require re-estimation Requires successful estimation of the covariance matrix
(NONMEM COV step) but can lead to random draws outside the feasible range of the parameters unless appropriate transformations are applied
PaSiPhIC 201215
Non-parametric Versus Parametric Bootstrap Procedures (2)
A2PG
Developing stable models that avoid extremely high pairwise correlations (>0.95) between parameter estimates and have low condition numbers (<1000) can help Ensure successful covariance matrix estimation Reduce convergence failures in non-parametric
bootstrap runs Choice of bootstrap procedure should focus on
the adequacy of the parametric assumption Random draws from MVN versus the more
computationally intensive re-estimation approaches (e.g., non-parametric bootstrap)
PaSiPhIC 201216
Simulation Procedure to Construct Statistical Intervals for Population Model Predictions
A2PG
Obtain random draw of , Ω, from bootstrap
procedure for kth trial
Simulate subject level dataYi | , Ω,
for M subjects
Summarize predictions (e.g.,
mean) stratified by
design (dose ,time, etc.)
Repeat for
k=1,…,K trials
Use percentile method to
obtain statistical interval from K
predictions
k<K
k=KNote 1: To construct confidence interval consider sufficiently large M (e.g., ≥2000 subjects) to average out sampling variation in Ω and . Note 2: For prediction intervals, M is chosen based on observed or planned sample size.
PaSiPhIC 201217
To describe other probabilistic concepts we need to define some additional quantities
A2PG
True (unknown) treatment effect or quantity ()
Target value (TV) A reference value for
Data-analytic decision rule (e.g., Go/No-Go criteria) Based on an observed treatment effect or quantity
(T)
PaSiPhIC 201218
Treatment Effect ()
A2PG
is the true (unknown) treatment effect Models provide a prediction of
Uncertainty in the parameter estimates of the
model provides uncertainty in the prediction of P( ) denotes the distribution of predictions of
ˆ,ˆ,ˆˆ g
PaSiPhIC 201219
Example of Model-Predicted Dose-Response Model for Fasted Plasma Glucose (FPG)
A2PG
Semi-mechanistic model of inhibition of glucose production
tK
out
inoutin
outeDED
DRFPG
K
KR ,FPGK
DED
DK
dt
dFPG
11
1
500
050
Mean Model Fit of FPG Versus Dose
(integrates data across dose and time)
Delt
a F
PG
(m
g/d
L)
Dose (mg)
Week 0 Week 2
Week 4
Week 8
Week 6
Week 12
Observed MeanTypical Individual Prediction (PRED)
Model-Predicted Placebo-Corrected
FPG Versus Dose at Week 12
Dose (mg)Pla
ceb
o-C
orr
ect
ed
Delt
a F
PG
(m
g/d
L)
Population Mean Prediction5th Percentile (90% LCL)95th Percentile (90% UCL)
PaSiPhIC 201220
Target Value (TV)
A2PG
Suppose we desire to develop a compound if the true unknown treatment effect () is greater than or equal to some target value (TV) TV may be chosen based on:
Target marketing profile Clinically important difference Competitor’s performance
If we knew truth we would make a Go/No-Go decision to develop the compound based on: Go: ≥ TV No-Go: < TV
PaSiPhIC 201221
Data-Analytic Decision Rule
A2PG
But we don’t know truth… So we conduct trials and collect data to obtain an
estimate of the treatment effect (T) T can be a point estimate or confidence limit on the
estimate or prediction of (e.g., placebo-corrected change from baseline FPG)
We might make a data-analytic Go/No-Go decision to advance the development of the compound if: Go: T ≥ TV No-Go: T < TV
PaSiPhIC 201222
Statistical Power
A2PG
Power is a conditional probability based on an assumed fixed value of the treatment effect () Power = P(T ≥ TV | ) where P(T ≥ TV | = TV) =
(false positive) TV=0 for statistical tests of a treatment effect
Power is an operating characteristic of the design based on a likely value of No formal assessment that the compound can
achieve the assumed value of
PaSiPhIC 201223
Simulation Procedure to Calculate Power Based on a Population Model-Predicted
A2PG
Use the same final estimates
(, Ω, ) for each
simulated trial
Simulate subject level dataYi | , Ω,
for M subjects
Analyze simulated data
as per SAP to test
Ho: = TVHa: TV
Repeat for
k=1,…,K trials
Power is calculated as the fraction of the K trials in
which Ho is rejected
k<K
k=K
Note 1: Typically TV=0 when assessing whether the compound has an overall treatment effect.Note 2: When using simulations to evaluate power it is good practice to also simulate data under the null (e.g., no treatment effect or placebo model) to verify that the Type 1 error () is maintained.
PaSiPhIC 201224
Probability of Achieving the Target Value (PTV)
A2PG
Probability of achieving the target value is defined as the proportion of trials where the true ≥ TV PTV = P( ≥ TV)
Does not depend on design or sample size Based only on prior information through the model(s)
and its assumptions
PTV is a measure of confidence in the compound at a given stage of development Can change as compound progresses through
development PTV can be calculated from the same set of
simulations used to construct confidence intervals of the predicted treatment effect ()
PaSiPhIC 201225
Simulation Procedure to Calculate PTV Based on Population Model Predictions
A2PG
Obtain random draw of , Ω, from bootstrap
procedure for kth trial
Simulate subject-level
data Yi | , Ω, for arbitrarily
large M
Summarize simulated data to obtain population mean predictions
of
Repeat for
k=1,…,K trials
Calculate PTV as proportion of K trials in which
≥ TV
k<K
k=KNote: To calculate PTV use an arbitrarily large M (e.g., ≥2000 subjects) to average out sampling variation in Ω and . PTV should only reflect the parameter uncertainty based on all available data used in the model estimation.
PaSiPhIC 201226
Probability of Success (POS)
A2PG
Probability of success is defined as the proportion of trials where a data-analytic Go decision is made POS = P(Go) = P(T ≥ TV)
POS is an operating characteristic that evaluates both the performance of the compound and the design In contrast to Power = P(T ≥ TV | ) which is an
operating characteristic of the performance of the design for a likely value of
POS is sometimes referred to as ‘average power’ where a Go decision is based on a statistical hypothesis test
dPTVTPTVTPGoP
PaSiPhIC 201227
Simulation Procedure to Calculate POS Based on a Population Model-Predicted
A2PG
Obtain random draw of , Ω, from bootstrap
procedure for kth trial
Simulate subject-level
data Yi | , Ω, for planned
sample size (M)
Summarize simulated data to obtain estimate of (T) and perform hypothesis test
Repeat for
k=1,…,K trials
Calculate POS as proportion of K trials in which
T ≥ TV
k<K
k=KNote: POS integrates the conditional probability of a significant result over the distribution of plausible values of reflected through the uncertainty in the parameter estimates for , Ω, and .
PaSiPhIC 201228
Probability of Correct Decision (POCD)
A2PG
A correct data-analytic Go decision is made when T ≥ TV and ≥ TV
A correct data-analytic No-Go decision is made when T < TV and < TV
Probability of a correct decision is calculated as the proportion of trials where correct decisions are made POCD = P(T ≥ TV and ≥ TV) + P(T < TV and <
TV) POCD can only be evaluated through
simulation where the underlying truth () is known based on the data-generation model used to simulate the data
29
Simulation Procedure to Calculate POCD Based on a Population Model-Predicted
A2PGPaSiPhIC 2012
Obtain random draw of , Ω, from bootstrap
procedure for kth trial
Simulate subject-level
data Yi | , Ω, for planned
sample size (M)
Summarize simulated data
to obtain estimate of
(T)
Repeat for
k=1,…,K trials
Calculate POS as proportion of K trials in which
T ≥ TV
k<K k=K
ClassifyGo: ≥TV
No Go: <TV Under Truth
ClassifyGo: T≥TV
No Go: T<TV Under Trial Data
Compare Truth Versus
Data-Analytic Decision
ClassifyGo: ≥TV
No Go: <TV Under Truth
Note: Classification of trial under truth is obtained from the PTV simulations.
PaSiPhIC 201230
General Framework for MBDD Basic assumptions of MBDD Six components of MBDD Clinical trial simulations (CTS) as a tool to
integrate MBDD activities Table of trial performance metrics Improving POCD Setting performance targets Comparing performance targets between
early and late stage clinical drug development
A2PG
PaSiPhIC 201231
Basic Assumptions of MBDD
A2PG
Predicated on the assumptions: That we can and should develop
predictive models That these models can be used in
CTS to predict trial outcomes Think of MBDD as a series of
learn-predict-confirm cycles Update models based on new data
(learn) Conduct CTS to predict trial
outcomes (predict) Conduct trial to obtain actual
outcomes and evaluate predictions (confirm)
Learn
Predict
Confirm
PaSiPhIC 201232
Six Components of MBDD
A2PG
MBDD
PK/PD & Disease Models
Meta-Analytic Models (Meta-Data from Public Domain)
Design & Trial Execution Models
Data-Analytic Models
Quantitative Decision Criteria
Trial Performance Metrics
Explicitly and quantitatively defined
criteria “draw line in the sand”
Leverage understanding of
pharmacology/disease – useful for
extrapolation
Understand competitive landscape from a dose-response
perspective
Evaluate designs and dose selection; incorporate trial
execution models such as dropout models
Implement SAP, evaluate alternative analysis methods – ANCOVA, MMRM, regression, NLME
Evaluate probability of achieving
target value (PTV), success (POS),
correct decisions (POCD)
PaSiPhIC 201233
Clinical Trial Simulations (CTS)
A2PG
Just as a clinical trial is the basic building block of a clinical drug development program, clinical trial simulations should be the cornerstone of an MBDD program
CTS allows us to assume (know) truth for a hypothetical trial Based on simulation model we know Mimic all relevant design features of a proposed clinical trial
Sample size, treatments (doses), covariate distributions, drop out rates, etc. Analyze simulated data based on the proposed statistical analysis
plan (SAP) Calculate T (test statistic for treatment effect) and apply data-
analytic decision rule CTS should be distinguished from other forms of stochastic
simulations E.g., CIs for dose predictions, PTV calculations, etc.
CTS can be used to integrate the components of MBDD and the various probabilistic concepts (including POS and POCD)
PaSiPhIC 201234
Table of Trial Performance Metrics
A2PG
Trial No Go Trial Go Total
“True” No Go
“True” Go
Total
Correct No Go
P(Trial Go)
Incorrect Go P(True No Go)
Incorrect No Go
P(Trial No Go) 1.0
P(True Go)Correct Go
PTVPOCD POS
PaSiPhIC 201235
Improving the probability of making correct decisions
A2PG
Change the design n/group Regression-based designs ( # of dose groups,
n/group) Consider other design constraints (cross-over,
titration, etc.) Change the data-analytic model
Regression versus ANOVA Longitudinal versus landmark analysis
Change the data-analytic decision rule Alternative choices for T
Point estimate, confidence limit, etc.
All of the above can be evaluated in a CTS
PaSiPhIC 201236
Setting Performance Targets
A2PG
PTV will change over time as model is refined and new data emerge Bring forward compounds/treatments with higher
PTV as compound moves through development PTV may be low in early development
Industry average Phase 3 failure rate is approximately 50% It is difficult to improve on this average unless we can
routinely quantify PTV Strive to achieve PTV>50% before entering Phase 3
Strive to achieve high POCD in decision-making throughout development
PaSiPhIC 201237
Comparing performance targets between early and late stage clinical drug development
A2PG
Low Low Low
Low High High
Low High 100%Total
Total
True No Go
Trial No Go
True Go
Trial Go
Late Development POCD should be high
PTV should be high
Advance good compounds / treatments to registration
High Low High
Low Low Low
Low Low 100%Total
Total
Trial No Go
Trial Go
Early Development POCD should be high
PTV may be low
Kill poor compounds / treatments early
True No Go
True Go
PaSiPhIC 201238
Examples
A2PG
Rheumatoid Arthritis Example Phase 3 development decision
Urinary Incontinence Example Potency-scaling for back-up to by-pass Phase 2a
POC trial and proceed to a Phase 2b dose-ranging trial
Acute Pain Differentiation Case Study Decision to change development strategy to
pursue acute pain differentiation hypothesis
PaSiPhIC 201239
Example – Rheumatoid Arthritis
A2PG
Endpoints: DAS28 remission (DAS28 < 2.6) ACR20 response (20% improvement in ACR score)
Models developed based on Phase 2a study: Continuous DAS28 longitudinal PK/PD model with Emax direct-
effect drug model ACR20 logistic regression PK/PD model with Emax drug model
Both direct and indirect-response models evaluated
Conducted clinical trial simulations for a 24-week Phase 2b placebo-controlled dose-ranging study (placebo, low, medium and high doses) At Week 12 non-responders assigned to open label extension
at medium dose level Primary analysis at Week 24; Week 12 responses for non-
responders carried forward to Week 24 Evaluated No-Go/Hold/Go criteria for Phase 3
development
PaSiPhIC 201240
Example – Rheumatoid Arthritis (2)
A2PG
DAS28-CRP Remission (Difference from placebo)
ACR20 (Difference from placebo)
<20% 20-25% 25-30% 30%
<10% No Go No Go Hold Hold
10-16% No Go Hold Hold Hold
16-20% Hold Hold Hold Go
20% Hold Hold Go Go
No Go: Stop development Hold: Wait for results of separate Phase 2b active
comparator trial Go: Proceed with Phase 3 development without waiting
for results from comparator trial
PaSiPhIC 201241
Example – Rheumatoid Arthritis (3)
A2PG
TreatmentProbability (%)
No Go Hold Go
Low Dose 96.1 3.9 0.0
Medium Dose 28.1 62.9 9.0
High Dose 18.4 65.6 16.0
CTS results suggest a high probability that the team will have to wait for results from the Phase 2b active comparator trial before making a decision to proceed to Phase 3. Very low probability of taking low dose into Phase 3.
PaSiPhIC 201242
Example – Urinary Incontinence
A2PG
Endpoint: Daily micturition (MIC) counts
Models developed: Longitudinal Poisson-Normal model developed for daily MIC counts
for lead compound Time-dependent Emax drug model using AUC0-24 as measure of exposure
Potency scaling for back-up based on: In vitro potency estimates for lead and back-up (back-up more potent than
lead) Equipotency assumption between lead and back-up
Conducted CTS to evaluate Phase 2b study designs for back-up compound (placebo and four active dose levels) Evaluated various dose scenarios of low (L), medium #1 (M1),
medium #2 (M2) and high (H) doses levels Implemented SAP (constrained MMRM analysis with step down
trend tests) Quantified POS for the L, M1, M2 and H doses for the various dose
scenarios and potency assumptions
PaSiPhIC 201243
Example – Urinary Incontinence (2)
A2PG
Note: Low (L) dose was selected to be a sub-therapeutic response. Design was not powered to detect a significant treatment effect at this dose.
Dose Scenario
L M1 M2 H Comment
1 1X 2.5X 12.5X 25X Doses selected favor in vitro potency assumption (i.e., back-up more potent than lead compound)
2 1X 2.5X 12.5X 37.5X
3 1X 5X 25X 50X
4 2.5X 5X 25X 75X
5 2.5X 12.5X 37.5X 75X Doses selected favor equipotent assumption6 5X 12.5X 50X 100X
PaSiPhIC 201244
Example – Urinary Incontinence (3)
A2PG
CTS results: High POS (>95%) demonstrating statistical significance at the H
dose for all 6 dose scenarios Insensitive to potency assumptions
High POS (>88%) demonstrating statistical significance at the M2 dose for all 6 dose scenarios Insensitive to potency assumptions
POS varied substantially for demonstrating statistical significance of the M1 dose Depending on dose scenario and potency assumption
POS < 60% for demonstrating statistical significance at the L dose Except for dose scenarios 4 – 6 for the in vitro potency assumption
CTS results provided guidance to the team to select a range of doses that would have a high probability of demonstrating dose-response while being robust to the uncertainty in the relative potency between the back-up and lead compounds. Provided confidence to bypass POC and move directly to a Phase 2b trial for the back-up.
PaSiPhIC 201245
Case Study – Acute Pain DifferentiationBackground
A2PG
SC-75416 is a selective COX-2 inhibitor Capsule dental pain study conducted
Poor pain response relative to active control (50 mg rofecoxib)
Lower than expected SC-75416 exposure with capsule relative to oral solution evaluated in Phase 1 PK studies
PK/PD models developed to assess whether greater efficacy would have been obtained if exposures were more like that observed for the oral solution Pain relief scores (PR) modeled as an ordered-categorical
logistic normal model Dropouts due to rescue therapy modeled as a discrete
survival endpoint dependent on the patient’s last observed PR Assumes a missing at random (MAR) dropout mechanism
PaSiPhIC 201246
Case Study – Acute Pain DifferentiationBackground (2)
A2PG
PK/PD modeling predicted greater efficacy with oral solution relative to capsules A 6-fold higher SC-75416 dose (360 mg) than previously
studied predicted to have clinically relevant improvement in pain relief relative to active control (400 mg ibuprofen)
Model extrapolates from capsules to oral solution and leverages in-house data from other COX-2s and NSAIDs
Project team considers change in development strategy to pursue a high-dose efficacy differentiation hypothesis Original strategy was to determine an acute pain dose that
was equivalent to an active control and then scale down the dose for chronic pain (osteoarthritis) Based on well established relationships that chronic pain doses
for NSAIDs tend to be about half of the acute pain dose
PaSiPhIC 201247
Case Study – Acute Pain DifferentiationProposed POC Dental Pain Trial
A2PG
Proposed conducting a proof of concept oral solution dental pain study Demonstrate improvement in pain relief for 360 mg
SC relative to 400 mg ibuprofen Primary endpoint is TOTPAR6 (SC vs. ibuprofen) TOTPAR6 = 3 (TV) is considered clinically relevant
Perform ANOVA on observed LOCF-imputed TOTPAR6 response and calculate LS mean differences T = LS mean (SC) – LS mean (ibuprofen) LCL95 = 2-sided lower 95% confidence limit on T
Compound and data-analytic decision rule: Truth: Go if ≥3, No-Go if <3 Data: Go if T≥3 and LCL95>0, No-Go if T<3 or LCL95≤0
PaSiPhIC 201248
Case Study – Acute Pain DifferentiationSimulation Procedure to Calculate PTV
A2PG
Simulate PR Model
Parameters
(PR,2) ~ MVN
Simulate Dropout Model Parameters
DO ~ MVN
Simulate Dropout Times
M=2,000 patients
per treatment
Simulate PR Scores
M=2,000 patientsper treatment Perform LOCF
Imputation and Calculate
TOTPAR6
Calculate Population Mean
TOTPAR6 & TOTPAR6
Across M=2,000 pts
Determine True Decision
Go: 3No Go: <3
Summarize Distribution of TOTPAR6 ()
k=K Repeat for k = 1,
…,K=10,000 trials
k<K
PaSiPhIC 201249
Case Study – Acute Pain DifferentiationPosterior Distribution of TOTPAR6
A2PG
0
500
1000
1500
2000
2500
Fre
quency
0 1 2 3 4 5Delta-TOTPAR6
360 mg SC-75416 vs 400 mg Ibuprofen360 mg SC-75416 vs 400 mg Ibuprofen
PTV = P( 3) = 67.2%
Mean Prediction = 3.2
PTV = 67.2% sufficiently high to warrant recommendation to conduct oral solution dental pain study to test efficacy differentiation hypothesis.
PaSiPhIC 201250
Case Study – Acute Pain DifferentiationCTS Procedure to Evaluate POC Trial Designs
A2PG
Simulate PR Scores for k-th
Trialn pts / treatment
Simulate Dropout Times
for k-th Trialn pts / treatment
Perform LOCF Imputation &
Calculate TOTPAR6
Calculate Mean TOTPAR6 (T),
SEM & 95% LCL
Apply Decision Rule
Go: LCL>0 and T3No Go: LCL0 or
T<3
Compare Truth vs. Data-Analytic
Decision
Calculate MetricsPOS
POCD
k=KRepeat for k=1,
…,K=10,000 trials
k<K
PaSiPhIC 201251
Case Study – Acute Pain DifferentiationCTS Trial Performance Metrics
A2PG
TrialTruth
Trial No GoLCL95 0 or T<3
Trial GoLCL95> 0 and T3
Total
<3 20.81% 11.99% 32.80%
3 17.29% 49.91% 67.20%
Total38.10% 61.90%
100%(out of 10,000 trials)
POCD = 70.72% POS = 61.90%
PTV = 67.20%
A sufficiently high POCD and POS supported the recommendation and approval to proceed with the oral solution dental pain study.
PaSiPhIC 201252
Case Study – Acute Pain DifferentiationComparison of Observed and Predicted (About 9 months later…)
A2PG
0
100
200
300
400
Fre
quency
-12 -8 -4 0 4 8DELTA
-12 -8 -4 0 4 8DELTA
0
100
200
300
400
Fre
quency
-12 -8 -4 0 4 8DELTA
-12 -8 -4 0 4 8DELTA
PlaceboPlacebo 60 mg SC-7541660 mg SC-75416
180 mg SC-75416180 mg SC-75416 360 mg SC-75416360 mg SC-75416
Pred = 3.2Pred =
2.0
Pred = -0.9Pred = -7.0Obs = -9.6
Obs = -1.8
Obs = 3.3Obs =
2.6
PaSiPhIC 201253
Case Study – Acute Pain DifferentiationSummary of Results
A2PG
360 mg SC-75416 met pre-defined Go decision criteria Confirmed model predictions Demonstrated statistically significant improvement relative
to 400 mg ibuprofen MBDD approach provided rationale to pursue acute
pain differentiation strategy that might not have been pursued otherwise
Allowed progress to be made while reformulation of solid dosage form was done in parallel
Validation of model predictions provided confidence to pursue alternative pain settings for new formulations without repeating dental pain study Model could be used to provide predictions for new
formulations
PaSiPhIC 201254
Final Remarks/Discussion
A2PG
Some thoughts on implementing MBDD Challenges to implementing MBDD
PaSiPhIC 201255
Final Remarks/DiscussionSome thoughts on implementing MBDD
A2PG
We need to clearly define objectives What questions are we trying to address with our models?
We need explicit and quantitatively defined decision criteria It’s difficult to know how to apply the models if decision
criteria are ambiguous or ill-defined We need complete transparency in communicating
model assumptions Entertain different sets of plausible model assumptions Evaluate designs for robustness to competing assumptions
We need to routinely evaluate the predictive performance of the models on independent data Modeling results should be presented as ‘hypothesis
generating’ requiring confirmation in subsequent independent studies
PaSiPhIC 201256
Final Remarks/DiscussionSome thoughts on implementing MBDD (2)
A2PG
Conduct CTS integrating information across disciplines Implement key features of the design and trial
execution (e.g., dropout) Implement statistical analysis plan (SAP)
Provide graphical summaries of CTS results for recommended design prior to the release of the actual trial results Perform quick assessment of predictive performance
when actual trial reads out Update models and quantification of PTV after
actual trial reads out i.e., Begin new learn-predict-confirm cycle
PaSiPhIC 201257
Final Remarks/DiscussionChallenges to implementing MBDD
A2PG
Focus on timelines of individual studies and a ‘go-fast-at-risk’ strategy (i.e., minimizing gaps between studies) can be counter-productive to a MBDD implementation M&S (learning phase) is a time-consuming effort
Integration of MBDD activities in project timelines will require focus on integration of information across studies Not just tracking of individual studies
May need processes to allow modelers to be un-blinded to interim results to begin modeling activities earlier to meet aggressive timelines
Insufficient scientific staff with programming skills to perform CTS Pharmacometricians and statisticians with such skills should be
identified CTS implementation often requires considerable customization to
address the project’s needs (i.e., no two projects are alike)
PaSiPhIC 201258
Final Remarks/DiscussionChallenges to implementing MBDD (2)
A2PG
Insufficient modeling and simulation resources to implement MBDD on all projects
Reluctance to be explicit in defining decision rules (i.e., reluctance to ‘draw line in the sand’) Due to complexities and tradeoffs in making decisions Can be difficult to achieve consensus
http://www.ascpt.org/Portals/8/docs/Meetings/2012%20Annual%20Meeting/2012%20speaker%20presentations/ASOP%20TUE%20CHERRY%20BLOS%20SESSION%201.pdf
Reluctance to use assumption rich models We make numerous assumptions now when we make
decisions…we’re just not very explicit about them MBDD can facilitate open debate about explicit
assumptions
PaSiPhIC 201259
Bibliography
A2PG
Neter, J., and Wasserman, W. Applied Linear Statistical Models, Irwin Inc., IL, 1974, pp. 71-73.
Efron, B. The Jackknife, the Bootstrap, and Other Resampling Plans, Society for Industrial and Applied Mathematics, PA, 1982, pp. 29-30.
Vonesh, E.F., and Chinchilli, V.M. Linear and Nonlinear Models for the Analysis of Repeated Measurements, Marcel Dekker, Inc., NY, 1997, pp. 245-246.
Kowalski, K.G., Ewy, W., Hutmacher, M.M., Miller, R., and Krishnaswami, S. “Model-Based Drug Development – A New Paradigm for Efficient Drug Development”. Biopharmaceutical Report 2007;15:2-22.
Lalonde, R.L., et al. “Model-Based Drug Development”. Clin Pharm Ther 2007;82:21-32.
Chuang-Stein, C.J., et al. “A Quantitative Approach to Making Go/No Go Decisions in Drug Development”. DIJ 2011;45:187-202.
Smith, M.K., et al. “Decision-Making in Drug Development – Application of a Model-Based Framework for Assessing Trial Performance”. Book chapter in Clinical Trial Simulations: Applications and Trends, Kimko H.C. and Peck C.C. eds. , Springer Inc., NY, 2011, pp. 61-83.
Kowalski, K.G., Olson, S., Remmers, A.E., and Hutmacher, M.M. “Modeling and Simulation to Support Dose Selection and Clinical Development of SC-75416, a Selective COX-2 Inhibitor for the Treatment of Acute and Chronic Pain”. Clin Pharm Ther, 2008; 83: 857-866.