bios 6648: design & conduct of clinical...

Date: 25 Nov 2013

5. Special topics anddesigns5.1 Biomarker validationstudies

Bios 6648- pg 1

Bios 6648: Design & conduct of clinical researchSection 4 - Documenting the study

5. Special topics and designs

5.1 Design and analysis of biomarker validation studies5.2 Design and analysis of crossover studies5.3 Design and evaluation of factorial studies

Date: 25 Nov 2013


Bios 6648- pg 2

5.1 Design and analysis of biomarker validation studies

Context

I The search for new biomarkers has been accelerated bynew molecular and genetic technologies.

I Potential uses:I Improved diagnostic testI Elucidating disease biologyI Risk stratificationI Targeted therapies

I Example (see also lecture 1.2): Molecular guided care

Date: 25 Nov 2013


Bios 6648- pg 3


Molecular guided therapy for heart failure

I Background: Patients who present with heart failure arestarted on beta-blocker therapy. If their ejection fractionhas not improved to greater than 35% after one month oftherapy, then are often given an implantable defibrillator.This procedure is used in about 90% of heart failurepatients. After 12 months most of those patients have notneeded the defibrillator, and most (80%) have ejectionfraction above 35%.

I Clinical question: Can we use molecular expressions topredict the patients who should and should not receiveimplantable defibrillators?

I Initial studies evaluate gene expression from heart biopsytissue:

I Cases: Heart failure patients given beta blockade who fail torespond by 30 days, but respond by 1 year.

I Controls: Heart failure patients given beta blockade who fail torespond by 30 days, but do not respond by 1 year.

* Response = Eject fraction > 35%.I Can molecular expression discriminate between cases and

controls?

Date: 25 Nov 2013


Bios 6648- pg 4


Recall the Pepe phases

I Phase I: Preclinical explorationI Phase II: Clinical assay and validation (prevalent

case-control study)I Phase III: Retrospective longitudinal (incident case-control

study)I Phase IV: Prospective screening (extend and type of

disease detected; false referral rate estimated)I Phase V: Disease control (screening with the biomarker

reduces disease mortality).

Date: 25 Nov 2013


Bios 6648- pg 5

Example: Molecular guided therapy for heart failure

Pepe phases I&II

I Thousands of molecular markers evaluated:I Computational biology tools:

I Thousands of t-tests on each marker separately (pick themost significant).

I Machine learning.I Leave-one-out cross-validation.I Considering documented pathways from other settings

(natural language processing).I Results are used to select ≈ 50 key molecular markers.I Custom chip constructed to measure expression of the 50

markers.I Analytic questions:

I How do we predict "cases" from molecular markers?I Does this predictor add anything to standard clinical

predictors?

Date: 25 Nov 2013


Bios 6648- pg 6


Pepe phases I&II

I Logistic regression gives risk score:

logit(p) = β0 + β1M1 + β2M2 + ...+ β50M50

where p = Pr(case), and M1,M2, ...,M50 denoteexpression magnitudes for markers 1 through 50.

I Risk is often scored by the fitted value for theright-hand-side of the logistic regression model:

Score = β̂0 + β̂1M1 + β̂2M2 + ...+ β̂50M50

I Larger values for the Score indicate greater chance forbeing a case.

I Predictive ability of the Score is often summarized in aROC curve.

Date: 25 Nov 2013


Bios 6648- pg 7

Receiver Operator Characteristic Curves

ROC curve

I Steps to construct:1. Calculate the sensitivity and specificity for all possible

thresholds:I For threshold T :

If score > T , then diagnose “case".If score ≤ T , then diagnose “non-case".

I Repeat for all possible thresholds.

2. Plot sensitivity versus 1−specificity(i.e., True positive probability versus False positiveprobability).

I Accuracy of the diagnostic test is sometimes summarized bythe area under the ROC curve.

I Best possible test has area = 1.0.I Flipping a coin has area = 0.5.I There is a statistical test for whether 2 diagnostic tests have a

significant difference in the area under the ROC curve.

I Problem: depending on the clinical situation sensitivitymay be more important than specificity (or vice versa).

I Area under ROC curve does not consider relativeimportance of sensitivity and specificity.

Date: 25 Nov 2013


Bios 6648- pg 8


Pepe phase III

I Prospective cohort of heart failure patients with:I Initially presenting with ejection fraction < 35%.I Treated with beta-blockadeI Still had ejection faction < 35% after 1 month.I All patients receive heart biopsy so that molecular risk can

be calculated.I All patients followed for 2 years:

I Cases = patients with ejection fraction > 35% at 1 year (lateresponders).

I Controls = patients with EF < 35% at 1 year(non-responders).

I Upon study completion:I Threshold chosen to give 100% sensitivity for cases (late

responders).I Specificity estimated using this threshold.

Specificity = proportion of patients who could potentiallyforego implantable defibrillator.

Date: 25 Nov 2013


Bios 6648- pg 9


Pepe phase IV

I Patients with EF < 35% after 1 month of beta-blockaderandomized to:

I Molecular guided care:I Risk score larger than threshold then do not receive ICD

(implantable defibrillator).I Risk score less than threshold then receive ICD.I Patients without ICD get defibrillator vest for safety.

I Standard care: Use of ICD determined by physicianjudgement (no molecular risk score provided).

I Outcome: probability that EF > 35% at 12 months (orcardiac event).

I Hypothesize that patients with high score will haveEF > 35% at 12 months, so it was safe to forego the ICD.

Date: 25 Nov 2013


Bios 6648- pg 10


Pepe phase V

I RCT as in phase IVI Follow for long-term outcomes (mortality and morbidity).

Date: 25 Nov 2013


Bios 6648- pg 11


Summary/overview

I Molecular markers identified from heart tissue biopsiesI Development of predictor (phase I/II):

I Logistic regression (in prevalent case/controls) givesmolecular risk score.

I Logistic regression evaluates whether molecular risk scoreadds to prediction based on Seattle Heart Failure RiskScore.

I ROC curves allow comparisons across all possible cut-offs.I Testing risk equation (Pepe phase III):

I Risk equation validated in prospective cohort of heart failurepatients.

I Diagnostic criteria:I Diagnostic threshold selected to give 100% sensitivity.I Specificity estimated (low specificity is more acceptable than

low sensitivity)I Sample size selected to give sufficient precision for estimated

sensitivity and specificity

Date: 25 Nov 2013


Bios 6648- pg 12


Summary/overview (con’t)

I Prospective screening (Pepe phase IV):I Randomize to standard care versus molecular guided careI Defibrillation vest assures safety

I Long-term outcomes (Pepe phase V):I Randomize to standard care versus molecular guided care

I Note: subsequent care and follow-up are likely to differdramatically between arms.

I Endpoint: all-cause mortality (my preference).

bios 6648: design & conduct of clinical...

Documents