bayesian modeling in clinical trials: from early ... · bayesian modeling in clinical trials: from...

Jouni Kerman

Statistical Methodology Group / Novartis, Switzerland

May 27, 2011 / MBDD Conference, Stockholm

Bayesian modeling in clinical trials: from early development to Phase III

Acknowledgements

Beat Neuenschwander (Oncology)

Michael Branson (Translational Sciences)

Roland Fisch (Statistical Methodology)

Björn Bornkamp (Statistical Methodology)

Heinz Schmidli (Statistical Methodology)

Amy Racine (Modeling and Simulation)

Marc Vandemeulebroecke (Modeling and Simulation)

2 | MBDD Stockholm | Jouni Kerman | 27 May 2011 | Bayesian modeling in clinical trials | Business Use Only

About the Novartis Statistical Methodology group

Role of “the Methods” group

• 11 members in the Basel headquarters and in the U.S.

• Consult statisticians on actual projects

• Promote innovative methods (adaptive/seamless designs, Bayesian methods, ...)

• Also an external focus: conferences, papers

• Keep up dialogue / scientific discussions with regulators


Contents

Introduction: a need for better statistical methods

• Shortcomings of conventional statistics at Pharma

The Bayesian approach – what and why

• And, how

Use of Bayesian statistics at Novartis

• Cases: From Phase 1 to Phase 3

Summary and conclusion



Introduction:

A need for better statistical methods

Step back and ask: what‟s our business?

Our challenge, as a business

• Make informed decisions in the face of uncertainty

• This involves taking all relevant information into account

Our reason for existence, as statisticians

• Help making informative decisions by quantifying the uncertainty affecting decision making

• This involves incorporating all relevant information into our statistical analyses


Example: disconnect of statistics and reality

TeGenero TGN1412 First-in-Man Trial (2006)

• 8 healthy volunteers: 2 on placebo, 6 on TGN1412 (a monoclonal antibody)

• all 6 TGN patients had severe adverse reactions from a cytokine storm; neither of the 2 placebo patients had any AEs


But Fisher‟s

exact test gave

a p-value of

3.5% so it‟s not

significant at

2.5% level

Cytokine storm?!

I am 100% sure

that this is due to

the drug!

Example: disconnect of statistics and reality

TGN1412 data analysis: what was missing?

• Cytokine storm is very rare; the clinician took this into account but the statistician didn‟t for the sake of “objectivity.” Who is right?

• Suppose we only had data from ONE patient – what can we say?

Cytokine storm?!

I am 100% sure

that this is due to

the drug!

Sorry,

insufficient

data!


Does traditional statistics deliver?

Reality vs. Hypotheses: Success ≠ Power

• Language of statistics ≠ language of clinicians (or business)

• Traditional hypothesis testing framework is awkward and “misses the point”

What is our

chance of

success??

I can‟t say, but you‟ve

got 80% probability

to reach a statistically

significant result,

given that your

alternative

hypothesis is true



Significance ≠> Success

• P-value by itself is meaningless

• We are always interested in the magnitude of the effect as well • If ignored, this has implications to sample size...

We got a p-

value of

0.04, great!

Hold it!

The point estimate

was 0.2, while the

alternative was 0.5.

Do you think we got

a successful study?



Meaningless tests

• Do we really need a study to test μ=0 vs μ≠0 ?

The result

was not

significant.

What does

it mean?

We can only

conclude that the

sample size was

too small.



The Bayesian way:

complete modeling of uncertainty

Bayesian modeling is not just “modeling” ...

Bayesian modeling = modeling of uncertainty

• Not just modeling of curves / shapes / time series / dependencies ...

• Bayesians model uncertainty using probability models – involving all relevant information

Uncertainty is quantified by a probability distribution

• All quantities (parameters) that have uncertainty are modeled to have a distribution: treatment effect, responder rate...

• Allows you to incorporate external information


-20 0 20 40 60 80 100 120

How the Bayesian approach helps

Better confidence in the trial results

• Quantify uncertainty (or, „confidence‟) properly

• Don‟t ignore available information – and don‟t ignore lack of it!

• Use all available external information in the design and analysis

Better communication by direct focus on the scientific / business questions

• Formulate criteria and scientific questions directly in terms of quantities of interest

• No need for meaningless null hypotheses that make no sense in clinical trials


Example of a Bayesian approach to CTs

“What is the clinical definition of success?”

• “Treatment difference Δ must be better than placebo, and of clinically relevant magnitude”

Δ >0 and Δ ≥ δ

“What is the model for the data?”

y ~ N(Δ, s2)

“What do we know?”

• We have information on Δ and s, based on past trials and publications: introduce uncertainty distributions for Δ and s


Example of a Bayesian approach to CTs

“How many subjects do we need?”

• We recruit as many as needed to satisfy our requirements for precision:

Pr( Δ > 0 | data ) ≥ 95%

Pr( Δ ≥ δ | data ) ≥ 50%

• We can also compute Type 1 and 2 errors given some relevant scenarios

• The starting point of the design should however always be the definition of clinical success


-20 0 20 40 60 80 100 120

δ 0

Posterior of Δ

Bayesians and frequentists: a peaceful coexistence

Role of Bayesian statistics / modeling

• Sponsor / Study-level trial design

• Provides for a sound framework to defining a well-behaved statistical procedure that takes into account all relevant information and its uncertainty as well

Role of frequentist statistics

• Regulatory perspective

• Provides a framework for evaluating the false positive error rate and the power of the trial



Bayes at Novartis:

Some examples


Proof-of-Concept studies

Early phase: Proof-of-Concept (PoC) trials

PoC: a translational step from “research” to “clinic”

• Early answer to key scientific questions:

• “Does the mechanism of the drug work”?

• “Does the drug work in this indication”?

Key decision point within the development strategy

• “Do we have enough confidence to invest further in the development of the candidate drug?”

Obvious platform for Bayesian models

• Trials are always quite small– it will pay off including external information in the model


Historical controls in PoC (and elsewhere)

Reuse information about the effect of a known drug (control)

• Combine information from many sources of information: in-house trials, publications

• Quantify the existing effect via a meta-analytic (hierarchical) Bayesian model

• Predict the effect for the future study

• Down-weight the information appropriately taking the size of the future trial into account

• Take this distribution as the prior information and incorporate it into the (Bayesian) analysis model

Fewer

patients in

control arm


Example: PoC in Crohn‟s disease (An inflammatory bowel disease)

Population:

• Male and female patients with moderate to severe active CD

Design:

• Multicenter, double-blind, randomized, 2 parallel groups (placebo or high dose), immediate readout after 2 iv infusions 3 weeks apart

Primary endpoint:

• Crohn„s Disease Activity Index (CDAI) • Gold standard, composite disease activity score, low scores are good

• Comprises assessments of stool, pain, well-being, signs and symptoms, treatment for diarrhea, abdominal mass, hematocrit and body weight

• Clinically meaningful scores:

• <150 (remission), <220 (mild), <450 (moderate)

• decrease by 70 or 100 points (response)


Example: PoC in Crohn‟s disease

Model

• CDAI change from baseline ~ N(q, s2)

• Quantity of interest: Δ = – (qActive – qPlacebo)

Prior information

• qActive – noninformative (prior with very little information)

• qPlacebo ~ N(50, 882/20), based on 671 placebo patients from 6 studies, “discounted” to a prior with 20 patients‟ worth of information


Example: PoC in Crohn‟s disease

PoC criteria

• CDAI change from baseline ~ N(q, s2)

• Quantity of interest: Δ = – (qActive – qPlacebo)

-20 0 20 40 60 80 100 120

Positive PoC if

P(Δ ≥ 50 | data ) ≥ 50%

and

P(Δ > 0 | data ) ≥ 95%

Thresholds

Levels of proof


-20 0 20 40 60 80 100 120

Example: PoC

Points to note

• We take into account both “significance” (Δ > 0) and magnitude of the effect (Δ ≥ 50)

• Thresholds are the important clinically relevant ones

• Levels of proof are adjusted to match required precision of estimates => sample size follows

• False negative/positive errors are controlled at acceptable levels

• Note: in PoC, false positive of 20% may be acceptable!



Phase I dose finding in oncology

Phase I dose finding in Oncology

Phase I dose escalation cancer trials

• Goal: identify the Maximum Tolerated Dose (MTD) while monitoring for dose-limiting toxicity (DLT)

• Small: often 15 – 30 patients

• Adaptive: dose escalations depend on data from past cohorts

• Large uncertainty during and at end of trial – external information may help



Traditional dose escalation schemes

• Algorithmic (e.g. 3+3) • Simplistic: does not take into account of all past

information

• Used to be “the gold standard”

• Not used anymore at Novartis

• Continual Reassessment Method (CRM) • Bayesian, but not without problems...



“Are model-based designs too aggressive?”

• Muler et al. (2004) JCO

• One-parameter Continual Reassessment Method (CRM)

• MTD recommendation from CRM: 50 mg!

• Is it justified? No!


20mg 30mg 40mg 50mg

Data: DLT/n 0/5 0/5 4/8


Bad Bayesian modeling is still bad

• One-parameter CRM model is inappropriate!

• Too simplistic models are too constraining, possibly leading to bad decisions within a trial

• Even though the operating characteristics (Type 1, 2 error control) may look fine, this does not guarantee that the “on-study characteristics” make sense

• On-study performance is also important!



Modeling the probability of a DLT at a given dose

• Logit(p(dose)) = log α + β log(dose/dose*) • with α, β > 0. dose* = reference dose

• A more realistic representation of the dose-toxicity curve

• The Bayesian model yields posterior distributions conditioned on the dose


p(dose)


Decisions are based on posterior summaries of the probability of DLT

• Three main regions of interest: under-dosing, targeted dosing, overdosing

• The model yields probabilities for each possibility

• Dose recommendation relies on maximizing probability of targeted dosing while keeping the probability of overdosing at < 25%



Improved performance


20mg 30mg 40mg 50mg

Data: DLT/n 0/5 0/5 4/8

overdosing

target

under-dosing


“But are you actually using Bayesian methods in your cancer trials?”

Yes. 100% of the Phase 1 trials in Oncology

at Novartis are now Bayesian



Probability of success in Phase III

Probability of success in Phase 3

“What are our chances of success in Phase 3?”

• This is a question that calls for a probability that is essentially Bayesian: not a long-run frequency!

• This is a conditional probability: probability of success later given what we know today (at end of Phase 2)

• This is needed for an internal decision – no regulatory constraints (and no Type 1 error control) here!

• We are obliged to use the prior information that we believe reflects the uncertainty appropriately

Probability of Success

• = the (posterior) predictive probability of a success criterion being fulfilled at the end of a future Phase 3 study


Probability of success in Phase 3 Case study: a compound for acute heart failure

Calculating the probability of success

• Sampling distributions were set up to simulate a data-generating process and unknown parameters were modeled as prior distributions – derived from the Phase II data

• A Bayesian probability model was used to predict the data that could be obtained in a future study of 800, 1000, 1200, 1400, or 1600 patients

• Outcomes such as “Study success”, “Dyspnea demonstrated”, and “Early submission” were computed based on the actual analyses that would be used in Phase III (e.g. chi-square test, log-rank test)


Bayesian clinical trial simulation Case study: a compound for AHF


Simulation of trial outcomes

• Set up a joint prior distribution of the unknowns (θ) given Phase 2 data, capturing their uncertainty

• Generate a sequence of S draws θ(i) from the distribution– then for each θ(i) a data set of N patients was simulated.

Computing the PoS

• Compute a Success / No success decision di

(0 or 1) for each trial outcome

• Obtain the PoS by averaging over the di

• Uncertainty in the parameters is then propagated to the simulated trial data points

y(1), ...,y(S)

Σdi/S = 0.89

0, 1, 1, ..., 0, 1

θ

θ(1), ..., θ(S)


Challenges and opportunities

Challenges in adopting the Bayesian approach

• Ignorance / lack of acceptance • Misconceptions

• Lack of regulatory guidelines / acceptance

• Mathematical expertise required • Some distribution calculus / likelihood formulations...

• Modeling expertise required • Biostatistics: lots of opportunities for creative modeling – lots of variety

• Every problem has its own special features – few models fit all problems

• Best solutions are not necessarily trivial, but some trivial solutions may be good enough in many cases!

• Computational expertise required • No standard “tests” available! – prepare to do lots of programming

• Simulation methods are used almost for every trial design

• MCMC / convergence issues / speed

• WinBUGS, R, SAS


Challenge => opportunity

Trends

• Increasing competition and outsourcing

• Statisticians must strive to offer higher and higher value

• Statisticians must evolve - and offer better and better solutions

A chance for industry statisticians!

• Bayesian expertise is in demand



Conclusion

Conclusion Take-home points

Bayesian modeling approach offers ...

• Better informed decisions via more realistic and precise modeling of sources of uncertainty

• Better communication to clinicians and stakeholders via intuitive probabilistic formulation of scientific / business questions



Tack! Kiitos! Thank you!


References

References: textbooks / general

Textbooks

• Berry, Carlin, Lee, Müller (2011) Bayesian Adaptive Methods for Clinical Trials. Chapman & Hall / CRC Press.

• Spiegelhalter, Abrams, Myles (2004) Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Wiley (Statistics in Practice)

• Gelman, Hill (2007). Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press

General discussion about Bayes in clinical trials

• Clinical Trials Special Issue 2005, 2:271-378


References: some papers on specific issues

Papers

• Neuenschwander, Branson, Gsponer (2008) Critical aspects of the Bayesian approach to phase I cancer trials. Statistics in Medicine 27:2420-2439

• Neuenschwander, Capkun-Niggli, Branson, Spiegelhalter (2010) Summarizing historical information on controls in clinical trials. Clinical Trials 7:5-18.

• Muler, McGinn, Normolle et al. Phase I trial using a time-to-event continual reassessment strategy for dose escalation of Cisplatin combined with Gemcitabine and radiation therapy in pancreatic cancer. Journal of Clinical Oncology 2004.

• Bailey, Neuenschwander, Laird, Branson. A Bayesian case study in oncology phase I combination dose-finding using logistic regression with covariates. JBS 2009.


bayesian modeling in clinical trials: from early ... · bayesian modeling in clinical trials: from...

Documents