challenges of bridging studies in biomarker driven clinical trials may 19, 2015. mbsw conference....

Challenges of Bridging Studies in Biomarker Driven Clinical Trials

Szu-Yu Tang, Chang Xu, Bonnie LaFleurMay 19, 2015. MBSW Conference. Muncie, IN.

1

Bridging Studies in diagnostic device:

Bridge the “efficacy” in the subpopulation tested from the clinical trial assay (CTA or old assay) to the subpopulation tested by a companion in vitro diagnostic device (CDx or new assay)

2

Clinical bridging studies : A bridging study is defined as a study performed in the new region to provide pharmacodynamic or clinical data on efficacy, safety, dosage and dose regimen in the new region that will allow extrapolation of the foreign clinical data to the population in the new region (ICH E5 guideline).

Outline

• Bridging study examples

• The impact of diagnostic accuracy to treatment efficacy in enrichment trail• Single assay • Old assay to new assay

• Conclusion

3

Confidential and proprietary to Ventana Medical Systems, Inc. For internal use only. Do not copy. Do not distribute.

4

Companion Diagnostic Test (CDx)

DISCOVERY PRECLINICAL

Phase 1 Phase 2 Phase 3

CUT-OFF DETERMINATION

ANALYTICAL VALIDATION

CLINICAL VALIDATION

EARLY RESEARCH PROTOTYPE & DEVELOPMENT

EARLY PHASE ALGORITHM DEVELOPMENT

VALIDATION STUDIES

Companion diagnostic test (CDx): the use of a diagnostic assay as a test, assay, or test system to screen, or select patients who may be candidates for a specific drug therapy.


Bridging Study Examples

• No CDx available in clinical trial• A laboratory-development test (LDT) is used in

the clinical trial instead of CDx Note: The FDA defines a Laboratory Developed Test (LDT) as an in vitro

diagnostic test that is manufactured by and used within a single laboratory (i.e. a laboratory with a single CLIA certificate). LDTs are also sometimes called in-house developed tests, or “home brew” tests.

• Multiple CDx products• Multiple (competing) CDx products can be driven

by improved efficiency, cost, novel/new technologies, or updated techniques over time.

5


Bridging Study Challenges

Verification Bias: • Lack of specimen material or lack of consent for

re-testing from patients

• Lack of efficacy outcome for negative group from old assay in enrichment trial

Old Assay

+ -

New Assay

+ a c

- b d

6

No clinical Outcome!!


Impact of diagnostic accuracy on treatment efficacy (single assay)

Goal: establish quantitative relationship between response rate and components of CDx accuracy in an enrichment trial.

All screened patients

+

-

Stratified by marker

Marker (+)

T

NT

Enrichment TrialRandomize

Response

No Response

7

Marker (-)


• Analytical versus clinical accuracy Analytical accuracy is sensitivity/ specificity but clinical accuracy

needs to link the assay performance to treatment efficacy.

• Method comparison studiesMethod comparison studies are used to compare the new assay with the one currently in use to see whether their measurements are indeed comparable.

• Issues with method comparison studies: Reference standard, non-identifiability, conditional independent

assumptions (CIA)…etc

Impact of diagnostic accuracy on treatment efficacy (single assay)

8


The impact of diagnostic accuracy on treatment efficacy (single assay)

We establish the relationship between response rate in an

enrichment trial () and CDx accuracy as follows:

Response rate (r) CDx test sensitivity () CDx test specificity () Prevalence of marker (+) patients () Difference in efficacy between treated and control populations () Response rate in marker (+) treated group adjust for device accuracy ()

Assumptions:

(1) Marker (+) patients are 1: 1 randomized to treatment and control group.

(2) Response rate of true marker (–) patients and also patients in the control

group ()

(3) Response rate of true marker (+) patients ()

(Ref[1]) Maitournam A, Simon R: On the efficiency of targeted clinical trials. Statistics in Medicine 2005; 24:329–

339.

9


Simulation Results

• Prevalence of marker (+) = 20%; response rate of marker (-) = 40%; response rate of marker (+) = 60%

• will be between 40% and 60%

• Fix sensitivity at 80% and 90% and plot vs. specificity; Fix specificity at 80% and 90% and plot vs. sensitivity

10


The impact of diagnostic accuracy on treatment efficacy (single assay)

What we see What it means

Dotted curves ( versus specificity for given sensitivity) are highest when sensitivity or specificity is over 80%

Dotted curves are also higher than the solid (RR versus specificity) at these levels

The upper limit of the response rate is determined by specificity, not sensitivity

[ When specificity = 100% reaches maximum regardless of sensitivity. When sensitivity = 100%, does not necessarily reach maximum]

Dotted curves have steeper slope than solid curves when sensitivity or specificity is large

Specificity improves response rate faster than sensitivity

11


Method comparison studies

The first step is to consider the “accuracy” of both assays in establishing true marker positivity and negativity

Probability of patients(+) in the enrichment trial tested by new assay is conditional on patients have been tested as (+/-) from old assay:P(new assay=+| old assay=+) = ( /( 𝜋 𝜋 (Ref [2])P(new assay=+| old assay=-) = (𝜋/(𝜋 (Ref [2])

Old Assay

+ -

New Assa

y

+ a c

- b d

Old Assay

+ -

New Assa

y

+ a´ c´

- b´ d´

True Biomarker Positive True Biomarker Negative

12(Ref [2]) Lu Y, Dendukuri N, Schiller I, Joseph L: A Bayesian approach to simultaneously adjusting for

verification and reference standard bias in diagnostic test studies.Stat Med 2010, 29(24):2532-2543.


Link of assay performance to efficacy

We establish the relationship between response rate in

enrichment trial using new assay () and old assay accuracy as

follows:

Old assay test sensitivity () / new assay test sensitivity ()

Old assay test specificity () / new assay test specificity ()

Prevalence of marker (+) patients ()

Difference in efficacy between treated and control populations ()

Assumption: New assay and old assay are conditionally independent given

the true disease status.

13


Assumptions

Conditional independence Assumption (CIA)

The CIA means that the test and the standard are

not prone to misdiagnose the same patients:

Old Assay

+ -New Assay

+ 64 80

-

80 100

True Positive ExampleWhen assumption holds= 80% * 80%= 64% (if CIA is satisfied)

When assumption is violated> 64%

(Ref [3]) Zhou XH, Obuchowski N, McLish D. Statistical Methods in Diagnostic Medicine, 2 nd ed. Hoboken, New Jersey: John Wiley and Sons; 2011.

14


Assay performance affect on efficacy

We establish the relationship in enrichment trial

between response rate using new assay () and old

assay accuracy without CIA as follows:

• Covp~U(0, min(S1, S2)-S1*S2)• Covn~U(0, min(C1, C2)-C1*C2)

15

(Ref [2]) Lu Y, Dendukuri N, Schiller I, Joseph L: A Bayesian approach to simultaneously adjusting for verification and reference standard bias in diagnostic test studies.Stat Med 2010, 29(24):2532-2543.


Simulation parameters

• Prevalence of marker (+) = 20%; response rate of marker (-) and control group = 40%; response rate of marker (+) = 60%

• / will be between 40% and 60%

• Fix sensitivity/specificity for old assay at 30%/ 50%/ 90% and plot / vs. specificity/ sensitivity 30%/ 50% / 90% for old assay.

• The impact of / also provides for different combinations of (,

• For correlated scenario ( / ), 80% dependency is defined as:

Covp=(min(S1, S2)-S1*S2)*0.8

Covn=(min(C1, C2)-C1*C2)*0.8

16


Simulation results CIA only

• Conditional on the test result from old assay is positive:

17


Summary of resultsCIA only


• When S1, C1 are low, the are low.

• The slope of specificity effect on is steeper than sensitivity effect from old assay .

• The top three lines in each panel are C2=0.9.

• Low sensitivity or specificity of old assay reduces .

• The effect of old assay’s specificity is larger than sensitivity.

• Specificity of new assay improves

• Conditional on the test result from old assay is positive:

18


• Conditional on the test result from old assay is negative:


19



• When S1, C1 are high, the are low.

• The slope of sensitivity effect onis steeper than specificity effect from old assay .

• The top three lines in each panel are C2=0.9.

• High sensitivity or specificity of old assay reduces .

• The effect of old assay’s sensitivity is larger than specificity.

• Specificity of new assay improves .

• Conditional on the test result from old assay is negative:


20


Simulation results dependency of 80% compared to CIA

• Conditional on the test result from old assay is positive: Diff ( - )

21



• When S1, C1 are low, the diff ( - ) are mostly above 0 except S2=0.9.

• When S1, C1 are high, the diff ( - ) are mostly below 0 except S2=0.3 and C1=0.9.

• Compared to CIA, high dependency between two assays enhances efficacy when old assay’s sensitivity or specificity is low except when the new assay has high sensitivity.

• Compared to CIA, high dependency between two assays reduces efficacy when old assay’s sensitivity or specificity is high except for the scenario of low sensitivity in new assay and high specificity in old assay.


• Conditional on the test result from old assay is positive: Diff ( - )

22



• Conditional on the test result from old assay is negative:diff ( - )

23



• Conditional on the test result from old assay is negative:diff ( - )


• When S1, C1 are low, the diff ( - ) are mostly above 0 except S2=0.3 and S1=0.3.

• When S1, C1 are high, the diff ( - ) are mostly below 0 except S2=0.9 and C1=0.9.

• Compared to CIA, high dependency between two assays enhances efficacy when old assay’s sensitivity or specificity is low except when both assays have low sensitivity.

• Compared to CIA, high dependency between two assays reduces efficacy when old assay’s sensitivity or specificity is high except for the scenario of high sensitivity in new assay and high specificity in old assay.

24


Conclusion

• We established a statistical framework that describes how assay accuracy affects clinical efficacy in biomarker driven clinical trials

• Inaccurate CDx diminishes treatment efficacy but specificity is the driving factor to improve response rate.

• We describe the processes for methods comparison studies between two assays under the following conditions

• Conditional independence assumption is satisfied:

• Depending on positive or negative result of old assay, sensitivity or specificity of old assay has opposite effect on new assay’s efficacy. However, high specificity of new assay improves efficacy no matter what test result of old assay is.

• Dependency is about 80%, compared with CIA:

• In general, when old assay’s sensitivity or specificity is low, high dependency enhances new assay’s efficacy. On the contrary, high dependency reduces the efficacy when old assay’s sensitivity or specificity is high.


Future Work

• Generalize the simulation to cover the range of sensitivity/ specificity, marker prevalence and dependency.

• Use measurement error models to incorporate impact of assay methods comparison to efficacy evaluation

• Evaluate and compare impact of clinical study design on assay bridging studies (e.g., all-comers versus enrichment studies)

26


References

[1] Maitournam A, Simon R: On the efficiency of targeted clinical trials. Statistics in Medicine 2005; 24:329–339.

[2] Lu Y, Dendukuri N, Schiller I, Joseph L: A Bayesian approach to simultaneously adjusting for verification and reference standard bias in diagnostic test studies.Stat Med 2010, 29(24):2532-2543.

[3] Zhou XH, Obuchowski N, McLish D. Statistical Methods in Diagnostic Medicine, 2 nd ed. Hoboken, New Jersey: John Wiley and Sons; 2011.

27

Doing now what patients need next

challenges of bridging studies in biomarker driven clinical trials may 19, 2015. mbsw conference....

Documents

diagnostic assay

clinical trial assay

clinical bridging studies

assay perf

new assay conclusion

clinical outcome

clinical trials

diagnostic device cdx