setting the cut point bar an industry perspective...–cut point analysis out put: no statistical...
TRANSCRIPT
Setting the cut point bar
An industry perspective
16th May 2016
Manoj Rajadhyaksha
Bioanalytical Sciences
Introduction
What are most of us observing with
respect to ADA assay cut points (ACPs)?
Mock study specific cut point case study:
For illustration
What is relevant?: For discussion
Background
Provided by Dr. Ron Bowsher
ACP history and evolution
– Clinical Immunogenicity Risk Experience Phase
– Stratification of products in Risk Assessment Categories Phase High, Moderate, Low Association of bioanalytical strategies with the expected risk
of the clinical outcome to the patient
– Initial Proposals for establishing ADA assay cut points Use of statistics in context of data obtained Balanced experimental design and elimination of outliers Taking into account the analytical and biological variability To ensure application of objective criteria (not subjective eg.
50% for CCP)
– Industry experience of bioanalytical outcome using these ACPs.
General observations in clinical studies in industry
(Personal Survey)
– With respect to Human monoclonal antibody drugs
– Validation screen and confirmation cut points are established using
normal human sera or commercial patient sera
Validation screen cut point is established with a 5% FP rate
– Early clinical phase false positive rate observed ~4-8% – FP: defined as % ADA positives in baseline samples of clinical
study population
Less apparent, small “n”
– Later phase clinical false positive rate observed ~25-50%
Reverse observation: Clinical FP rate drops to < 1%
If observed in only 1 study, data comparison across studies is problematic
Mock Case study for illustration:
– Human monoclonal antibody drug
– Assay cut point values established as recommended Shankar G et al. 2008. Journal of Pharmaceutical and Biomedical Analysis
– Validation screen cut point established with a 5% FP rate Bridging ADA assay on MSD platform
screen and confirmation cut point established using normal human subjects
(n=50)
Floating Screen CP multiplicative factor ~1.5 (Parametric/Non-parametric)
1% FP rate Confirmation CP ~20% Inh
Floating Titer CP factor ~ 1.4 (Parametric)
– Late stage clinical analysis Bridging ADA assay on MSD platform (No change in Life Cycle Management)
Observed false positive rate in clinical study baseline samples ~15-20%
Mock in-study cut point analysis
– In study cut point analysis: screen and confirmation cut point established using baseline patient samples
~400 values from ~ >100 baseline study samples (For ACP: Screen, Confirmation and Titer)
– Cut point analysis out put: No statistical difference in least square means for the analyst effect
statistically significant difference detected among run means
Floating Screen CP multiplicative factor ~ 1.8
1% FP rate Confirmation CP ~26% Inh
Floating Titer CP multiplicative factor ~ 1.9
Validation Clinical Study
Screen CP ~1.5 ~1.8
Confirmation CP ~20% ~26%
Titer CP ~1.4 ~1.9
Obsvd FP rate ~23% ~4.7%
Observations: Mock study
Comm. NHV
Comm. Indication A; Clinical Indication A
Comm. Indication B; Clinical Indication B
NHV Comm
A
Comm
B
Clin
A
Clin
B
Observations: Mock study
Pop 1 Pop 2 P Use
HNV Clin A < 0.001 ×
HNV Clin B < 0.001 ×
Comm
B
Clin B < 0.001 ×
Comm
A
Clin A 0.5237 √
Published Observations
Modified from Gerry Kolaitis et al poster. BMS
This direct comparative analysis shows that depending on the indication, the pre-
study validation CP v/s the population specific in-study CP, provide different FP
rates for a given study population.
Contributing Factors
– Validation ACP established without target clinical population
Disease state matrix is very rare or very expensive
Availability of a surrogate matrix – not representative of target population
– NHV or commercial samples heterogeneous in composition
sub- populations of TN, TP and FP
No patient stratification or criteria
Stage of disease at which sample is taken
– Multimeric targets or receptors fluctuate through course of disease
– Clinical trial (target) population little less heterogeneous
Inclusion criteria and exclusion criteria
Stratification based on study objective (Baseline characteristics)
– Drug development franchise
Multiple indications pursued for the same drug and same MOA
– Patient related factors
Age group (Pediatric and geriatric populations)
Concomitant medications
Sex
Pre-existing antibodies
Impact of “clinically not-relevant” cut point
– Impact on resources
several rounds of re-analysis
– Impact on timelines
Factors to be considered, tested, mitigated – Any potential matrix interference? – Any Target interference? – Any pre-existing antibodies?
Larger amount of samples going into confirmation assay
For very large long term studies (> 5000 patients) the issue may be realized mid-way in the trial – forcing lot of rework
– Impact on overall study costs
Bioanalysis/re-analysis if done at CROs– larger amount of samples going into confirmation assay
Statistical analysis done at CROs
Impact of “clinically not-relevant” cut point
– Impact on product distinction
Two products with similar MOA Equivalent efficacy, safety profile, dosing regimens
Dramatic difference in reported immunogenicity incidence in label
– Baseline FP rate: 0% Drug A v/s 8% Drug B
– Treatment Emergent: 1% Drug A v/s 20% Drug B
Does study specific cut point resolve all
issues?
– Significant interference in baseline matrix needs to be resolved Target interference by multimeric targets
Anti-allotype, anti-carbohydrate, anti-frame work, pre-existing antibodies
– Subsequent study populations may be significantly different Different indications pursued for the same drug
Different geographical sectors included/excluded in a given trial
Different study designs
– Monotherapy, Combination Therapy +/- immune modulators
– These are bioanalytical cut points that reflect analytical and biological
variability (May not be clinically relevant) “conservative” and “non-conservative” cut point
– A sample positive in the assay does not mean it has necessary
“levels” of ADA to cause a meaningful clinical impact (PK, safety,
efficacy)
– Concept of a clinically relevant cut point
14
Variable outcome of different methods
(Mock Data)
Non-parametric
Simple Parametric
Robust Parametric
Quantile lower bound
Least Conservative
Most Conservative
What is relevant? (Mock Data)S
ign
al/
No
ise
1
2
3
Points to Ponder
– Should assay validation cut points be “fit for purpose” to assess
validation parameters? Used for FIH or FIP trials with small “n”
– As applicable, can relevant (in study) cut points be set during pivotal
studies when the “target population is accessible”?
– Is application of the most conservative statistical method for ACPs for
all drug products valid? Can cut point rules be established based on “risk assessments”?
– Less conservative for low risk
– More conservative for high risk
– Is ADA impact on PK clinically relevant?
– Is there a need to build a consensus approach based more on
“biological” data and assisted by “statistical” approach? Various sponsors developing and presenting their own cut point assessment
rules
Acknowledgements
REGN BAS Team
Al Torri
Thomas Daly
Giane Sumner
Matthew Andisik
Michael Partridge
Ching Ha Lai
Weiping Shao
B2S Team
Wendell Smith
Rocco Brunelle
Ron Bowsher