data-driven optimization of treatment outcomes in …...vs placebo comparison for at least 6 weeks,...

Data-driven optimization of treatment outcomes in depressionTreatment Selection Idea Lab @ U. Penn Adam Chekroud Yale University 04/06/16

$9.6 Billion U.S. spending on antidepressants

70%do not fully recover

from initial treatment

OUTLINE

1. Objective: Prospectively identify antidepressant responders

2. Statistical modeling procedures

3. Performance generalizes to independent clinical trial

4. Implementation at scale

5. Recent progress

www.thelancet.com/psychiatry Published online January 20, 2016 http://dx.doi.org/10.1016/S2215-0366(15)00471-X 1

Articles

Cross-trial prediction of treatment outcome in depression: a machine learning approachAdam Mourad Chekroud, Ryan Joseph Zotti, Zarrar Shehzad, Ralitza Gueorguieva, Marcia K Johnson, Madhukar H Trivedi, Tyrone D Cannon, John Harrison Krystal, Philip Robert Corlett

SummaryBackground Antidepressant treatment effi cacy is low, but might be improved by matching patients to interventions. At present, clinicians have no empirically validated mechanisms to assess whether a patient with depression will respond to a specifi c antidepressant. We aimed to develop an algorithm to assess whether patients will achieve symptomatic remission from a 12-week course of citalopram.

Methods We used patient-reported data from patients with depression (n=4041, with 1949 completers) from level 1 of the Sequenced Treatment Alternatives to Relieve Depression (STAR*D; ClinicalTrials.gov, number NCT00021528) to identify variables that were most predictive of treatment outcome, and used these variables to train a machine-learning model to predict clinical remission. We externally validated the model in the escitalopram treatment group (n=151) of an independent clinical trial (Combining Medications to Enhance Depression Outcomes [COMED]; ClinicalTrials.gov, number NCT00590863).

Findings We identifi ed 25 variables that were most predictive of treatment outcome from 164 patient-reportable variables, and used these to train the model. The model was internally cross-validated, and predicted outcomes in the STAR*D cohort with accuracy signifi cantly above chance (64·6% [SD 3·2]; p<0·0001). The model was externally validated in the escitalopram treatment group (N=151) of COMED (accuracy 59·6%, p=0.043). The model also performed signifi cantly above chance in a combined escitalopram-buproprion treatment group in COMED (n=134; accuracy 59·7%, p=0·023), but not in a combined venlafaxine-mirtazapine group (n=140; accuracy 51·4%, p=0·53), suggesting specifi city of the model to underlying mechanisms.

Interpretation Building statistical models by mining existing clinical trial data can enable prospective identifi cation of patients who are likely to respond to a specifi c antidepressant.

Funding Yale University.

IntroductionAs few as 11–30% of patients with depression reach remission with initial treatment, even after 8–12 months.1–4 One factor reducing eff ectiveness of treatment is the inability to personalise pharmacotherapy.5 Clinicians match patients with specifi c antidepressants via a prolonged period of trial and error, delaying clinical improvement and increasing risks and costs of treatment. The absence of clinical prediction tools in psychiatry starkly contrasts with other areas of medicine, such as oncology, cardiology, and critical care, where algorithmic models often have important roles in medical decision making,6–8 and routinely outperform judgment of individual clinicians.9–11

A challenge when developing predictive clinical tools is to establish what information should be used. Genetic and brain imaging measures are possible sources of information, and have generated interest.12,13 However, even if eff ective, the cost and time of collecting and processing data might not be practical. By contrast, behavioural (eg, patient-reported) data are already collected but perhaps underused. Clinical experience guides what information is used in treatment decisions;14 however, early-stage clinicians

have little experience, and even experienced clinicians might overlook useful information or overweight salient clinical examples.15 Previous attempts to identify clinical predictors of treatment outcome have generally identifi ed a few predictors based on clinical experience, and have investigated their overall eff ect in a stepwise manner.16 One important study took a slightly diff erent approach, quantifying the eff ect of nine symptom dimensions derived from a factor analysis.17 However, examination of all potential predictors simultaneously in an unbiased manner (sometimes called data mining) provides an opportunity for discovery. Machine-learning methods are especially well suited for this challenge.18,19 Rather than separately considering the eff ect of one variable on an outcome of interest, machine-learning methods identify patterns of information in data that are useful to predict outcomes at the individual patient level. Modern machine-learning approaches off er key benefi ts over traditional statistical approaches (generalised linear models, and even non-linear regression models [generalised additive models]), because (when present) they can detect complex (non-linear) high-dimensional interactions that might inform predictions.

Lancet Psychiatry 2016

Published OnlineJanuary 20, 2016http://dx.doi.org/10.1016/S2215-0366(15)00471-X

See Online/Commenthttp://dx.doi.org/10.1016/S2215-0366(15)00542-8

Department of Psychology (A M Chekroud MSc, Z Shehzad MSc, M K Johnson PhD, T D Cannon PhD), Department of Biostatistics (R Gueorguieva PhD), and Department of Psychiatry (T D Cannon, J H Krystal MD, P R Corlett PhD), Yale University, New Haven, CT, USA; Capital One, McLean, VA, USA (R J Zotti BSc); Department of Psychiatry, UT Southwestern, Dallas, TX, USA (M H Trivedi MD); and Centre for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA (A M Chekroud)

Correspondence to:Mr Adam M Chekroud, Department of Psychology, Yale University, New Haven, CT 06511, [email protected]

STAR*D - DESIGN

Largest clinical trial of MDD (N= 4,041)

Archival dataset - freely-available from NIMH

Prospective, randomized clinical trial; realistic sampling

Level 1: 12-week SSRI treatment

Amongst Level 1 treatment completers, label patients that reached remission (QIDS ≤ 5)

Citalopram responder

Citalopram non-responder

STAR*D - DESIGN

STAR*D Cohort

STAR*D - DATA

Extract all readily-available data

• Demographic

• Diagnostic criteria (DSM-IV)

• Questionnaire responses about symptoms

• Psychiatric History

• Family history

Chekroud et al. (2016) Lancet Psychiatry

“Wide” Feature Set

FEATURE SELECTION“Wide” Feature Set

Elastic NetRegularization

Best 25 Questions


Zou & Hastie, J.R. Stat. Soc. (2005)

FEATURE SELECTION“Wide” Feature Set


Best 25 Questions


Zou & Hastie, J.R. Stat. Soc. (2005)

Ordinary Least Squares

!"#(!!,!)∈ℝ!!!

12!

!

!!!(!! − !! − !!!!)! + !

Ridge/L2

! !)! + ![(1− !)||!||!!/2+ !!||!!/2+ !||!||!],!

Lasso/L1

BEST PREDICTIVE ITEMS

β Item

0.078 Initial Depressive Severity (QIDS-C)

-0.069 Currently Employed

0.069 Feeling Restless (Psychomotor Agitation)

0.059 Low Energy Levels (Fatigue)

0.056 Ethnicity = Black/African American





TRAIN PREDICTIVE MODEL

Train GBM using only 25 features

Relative Variable Importance

All InformationAvailable Before

Treatment


Best 25 Predictors





MODEL PERFORMANCE

Test-fold Performance

Gradient Boosting Machine

Accuracy 65%

p-value [acc>NIR] 9.8 x10-33

Sensitivity 63%

PPV 64%

PPV = Positive Predictive Value





WHAT NEXT?





5. Recent progress

STAR*D COMED

A) External Generalization of STAR*D Model

Acc 65% ***AUC 0.70Sens 63%Spec 66%

Sequenced Treatment Alternatives to Relieve Depression Combination Medication to Enhance Depression Outcomes

CIT onlyN = 1,949 ESC + PLACEBO

N = 151

VEN + MIRN = 140

ESC + BUPN = 134

Acc 51% [n.s]Sens 39%, Spec 65%

Acc 60% *

Sens 56%, Spec 63%

Acc 60% *

Sens 49%, Spec 71%


WHAT NEXT?





5. Recent progress

www.spring.care

WE RECOMMEND

Progress

OUTLINE





5. Recent progress

REVIEW

Antidepressant Drug Effectsand Depression SeverityA Patient-Level Meta-analysisJay C. Fournier, MARobert J. DeRubeis, PhDSteven D. Hollon, PhDSona Dimidjian, PhDJay D. Amsterdam, MDRichard C. Shelton, MDJan Fawcett, MD

ANTIDEPRESSANT MEDICATION(ADM) represents the cur-rent standard of treatment formajor depressive disorder

(MDD).1 Antidepressant medication hasbeen shown to be superior to placeboin thousands of controlled clinical trialsover the past 5 decades.2,3 The extentto which ADM outperforms placebo(which controls for nonpharmacologi-cal aspects of ADM) can be used to in-dex the “true” pharmacological effectof ADM in clinical settings.

The randomized, double-blind,placebo-controlled trial is the goldstandard for testing treatment efficacyand affords the opportunity to identifypatient characteristics that predict dif-ferential pharmacological response.Baseline symptom severity is onedimension that may affect treatmentoutcome. Kirsch et al4 and Khan et al5

presented independent meta-analysesof randomized placebo-controlledtrials based on data from the Food andDrug Administration (FDA) clinicaltrial database. Using mean scores andstandard deviations on the HamiltonDepression Rating Scale (HDRS)6 fromeach study, they examined the effect

of baseline symptom severity on therelative efficacy of ADM vs placebo.Kirsch et al found that as the meanbaseline HDRS score increased, themagnitude of HDRS change decreasedfor placebo but remained unchangedfor ADM. Khan et al did not find a sig-nificant relationship between baselinescores and symptom change for the

Author Affiliations: Departments of Psychology (MrFournier and Dr DeRubeis) and Psychiatry (Dr Amster-dam), University of Pennsylvania, Philadelphia; De-partments of Psychology (Dr Hollon) and Psychiatry (DrShelton), Vanderbilt University, Nashville, Tennessee;Department of Psychology, University of Colorado atBoulder (Dr Dimidjian); and Department of Psychia-try, University of New Mexico School of Medicine, Al-buquerque (Dr Fawcett).Corresponding Author: Jay C. Fournier, MA, Depart-ment of Psychology, University of Pennsylvania, 3720Walnut St, Philadelphia, PA 19104 ([email protected]).

Context Antidepressant medications represent the best established treatment for ma-jor depressive disorder, but there is little evidence that they have a specific pharma-cological effect relative to pill placebo for patients with less severe depression.

Objective To estimate the relative benefit of medication vs placebo across a widerange of initial symptom severity in patients diagnosed with depression.

Data Sources PubMed, PsycINFO, and the Cochrane Library databases were searchedfrom January 1980 through March 2009, along with references from meta-analysesand reviews.

Study Selection Randomized placebo-controlled trials of antidepressants ap-proved by the Food and Drug Administration in the treatment of major or minor de-pressive disorder were selected. Studies were included if their authors provided therequisite original data, they comprised adult outpatients, they included a medicationvs placebo comparison for at least 6 weeks, they did not exclude patients on the basisof a placebo washout period, and they used the Hamilton Depression Rating Scale(HDRS). Data from 6 studies (718 patients) were included.

Data Extraction Individual patient-level data were obtained from study authors.

Results Medication vs placebo differences varied substantially as a function of base-line severity. Among patients with HDRS scores below 23, Cohen d effect sizes for thedifference between medication and placebo were estimated to be less than 0.20 (astandard definition of a small effect). Estimates of the magnitude of the superiority ofmedication over placebo increased with increases in baseline depression severity andcrossed the threshold defined by the National Institute for Clinical Excellence for a clini-cally significant difference at a baseline HDRS score of 25.

Conclusions The magnitude of benefit of antidepressant medication compared withplacebo increases with severity of depression symptoms and may be minimal or non-existent, on average, in patients with mild or moderate symptoms. For patients withvery severe depression, the benefit of medications over placebo is substantial.JAMA. 2010;303(1):47-53 www.jama.com

©2010 American Medical Association. All rights reserved. (Reprinted) JAMA, January 6, 2010—Vol 303, No. 1 47

Downloaded From: http://jama.jamanetwork.com/ by a University of Pennsylvania User on 06/03/2016

Less than 50% of people with depression respond to the firstprescribed antidepressant, but the majority eventually respondto a different treatment.1,2 The rate and magnitude of responseappear to be similar for tricyclic antidepressants and selectiveserotonin reuptake inhibitors (SSRIs).3–5 Psychiatrists are unableto predict which drug will work for whom and the choice of firstand subsequent treatments has to progress by trial and error. Thepresent study addresses two major methodological challenges thatmay have precluded identification of drug-specific effects inprevious studies: symptomatic heterogeneity and statistical power.

Although depression is conceived as a single condition, itsdefining symptoms do not necessarily co-occur and individualsymptoms may differ in their distribution across individuals andtheir response to treatments.6 This heterogeneity of depressivesymptoms complicates exploration of drug effects. For example,the early improvement of sleep with tricyclic antidepressantsmay be unrelated to sustained response, but early improvementin anxiety precedes and predicts overall improvement.7 Suchcross-sectional and longitudinal dissociations between symptomdimensions decrease the correlations between items of scales thatcombine mood, anxiety and sleep items in a single score, i.e.impair their internal consistency, to a degree where a summed testscore is uninformative.8,9 We have sought to remediate this problemand, using categorical item factor analysis, we identified threedimensions of depressive symptoms with good psychometric prop-erties: observed mood, cognitive and neurovegetative symptoms.10

The present study tests the hypothesis that escitalopram andnortriptyline differ in their effects on these dimensions.

A second challenge concerns the effectiveness of statisticalanalysis. Most previous trials were powered to compare activemedication with placebo, but differences between active anti-depressants are likely to be smaller.11 To maximise the power fora specified sample size, it is essential that all information onoutcome is used in the analysis. Many previous investigations useddichotomised outcomes (e.g. responder/non-responder).However, response to antidepressants is a matter of degree ofchange rather than a yes/no qualitative transformation, anddichotomising a continuous outcome is associated with asubstantial loss of power.12,13 Furthermore, temporal charac-teristics of antidepressant response are lost in end-point analysisand the commonly used last observation carried forwardprocedure for missing data produces biased results.14–16 In thepresent report, we apply mixed-effect modelling that permits theuse of data measured at multiple time points, and providesunbiased estimates in the presence of missing data.14,16,17 Thisapproach also separates inter-individual variation in anti-depressant response from measurement error and unmeasuredcentre differences. This partitioning allows estimation of theproportion of variance attributable to unmeasured individual-specific characteristics, including genes.

Method

Study design

Genome Based Therapeutic Drugs for Depression (GENDEP) is apartially randomised multicentre clinical and pharmacogenetic

252

Differential efficacy of escitalopramand nortriptyline on dimensional measuresof depressionRudolf Uher, Wolfgang Maier, Joanna Hauser, Andrej Marusic, Christine Schmael, Ole Mors,Neven Henigsberg, Daniel Souery, Anna Placentino, Marcella Rietschel, Astrid Zobel,Monika Dmitrzak-Weglarz, Ana Petrovic, Lisbeth Jorgensen, Petra Kalember, Caterina Giovannini,Mara Barreto, Amanda Elkin, Sabine Landau, Anne Farmer, Katherine J. Aitchison and Peter McGuffin

BackgroundTricyclic antidepressants and serotonin reuptake inhibitorsare considered to be equally effective, but differences mayhave been obscured by internally inconsistent measurementscales and inefficient statistical analyses.

AimsTo test the hypothesis that escitalopram and nortriptylinediffer in their effects on observed mood, cognitive andneurovegetative symptoms of depression.

MethodIn a multicentre part-randomised open-label design (theGenome Based Therapeutic Drugs for Depression (GENDEP)study) 811 adults with moderate to severe unipolardepression were allocated to flexible dosage escitalopram ornortriptyline for 12 weeks. The weekly Montgomery–AsbergDepression Rating Scale, Hamilton Rating Scale forDepression, and Beck Depression Inventory were scoredboth conventionally and in a more novel way according todimensions of observed mood, cognitive symptoms andneurovegetative symptoms.

ResultsMixed-effect linear regression showed no difference betweenescitalopram and nortriptyline on the three original scales,but symptom dimensions revealed drug-specific advantages.Observed mood and cognitive symptoms improved morewith escitalopram than with nortriptyline. Neurovegetativesymptoms improved more with nortriptyline than withescitalopram.

ConclusionsThe three symptom dimensions provided sensitivedescriptors of differential antidepressant response andenabled identification of drug-specific effects.

Declaration of interestS.L. owns shares in GlaxoSmithKline. N.H. participated inclinical trials sponsored by pharmaceutical companiesincluding GlaxoSmithKline and Lundbeck. A.F., P.M. and K.J.A.have received consultancy fees and honoraria forparticipating in expert panels from pharmaceuticalcompanies including Lundbeck and GlaxoSmithKline. Fundingdetailed in Acknowledgements.

The British Journal of Psychiatry (2009)194, 252–259. doi: 10.1192/bjp.bp.108.057554

“DEPRESSIVE SEVERITY”

Chekroud, Gueorguieva, Krumholz, Trivedi, Krystal & McCarthy. (in prep)

1. Low mood

2. Insomnia

3. Feelings of guilt/worthlessness

4. Cognitive impairments

5. Weight/appetite changes

6. Suicidal ideation

STAR*D Clustering Solution

Sleep onset insomniaMid−nocturnal insomnia

Early morning insomnia

Hypersomnia

Mood (sad)

Concentration/decision making

Outlook (self)

Suicidal ideation

Involvement

Energy/Fatigability

Psychomotor SlowingPsychomotor Agitation

SUB-SYMPTOM STRUCTURE

QIDS


REPLICABLE STRUCTURE?

Difficulties falling asleep

Waking during the night

Waking up too early

Cannot get out of bed

Feeling sad

Poor concentration/decision making

Low self-worth

Suicidal ideation

No interest in people/activities

Low energy levels

Slowed movement and thinking

Restlessness/”Fidgeting”

Difficulties falling asleep

Waking during the night

Waking up too early

Cannot get out of bed

Feeling sad

Poor concentration/decision making

Low self-worth

Suicidal ideation

No interest in people/activities

Low energy levels

Slowed movement and thinking

Restlessness/”Fidgeting”

STAR*D COMED

SLEEP

ATYPICAL

COGNITIVE / AFFECTIVE

SLEEP

ATYPICAL

COGNITIVE / AFFECTIVE


SUB-SYMPTOM TRAJECTORIES

0.5

1.0

1.5

2.0

0 2 4 6 8 10 12Duration of Treatment (Weeks)

Aver

age

Seve

rity

Citalopram

Atypical

Core Depressive

Sleep/insomnia



Chekroud, Gueorguieva, Krystal & McCarthy. (in prep)

Atypical

Core Depressive

Sleep/insomnia

0.5

1.0

1.5

2.0


Aver

age

Seve

rity

Escitalopram + Placebo

0.5

1.0

1.5

2.0


Aver

age

Seve

rity

Venlafaxine + Mirtazapine

0.5

1.0

1.5

2.0


Aver

age

Seve

rity

Escitalopram + Buproprion

0.5

1.0

1.5

2.0


Aver

age

Seve

rity

Citalopram



Atypical Core Sleep

0.5

1.0

1.5

0 2 4 6 8 10 12 0 2 4 6 8 10 12 0 2 4 6 8 10 12Duration of Treatment (Weeks)

Aver

age

Seve

rity

Citalopram Escitalopram...Bupropion Escitalopram...Placebo Venlafaxine...Mirtazapine

12−week symptom cluster response trajectories − QIDS



Atypical Core Depressive Sleep

0.6

0.9

1.2

1.5

0 2 4 6 8 0 2 4 6 8 0 2 4 6 8Duration of Treatment (Weeks)

Aver

age

Seve

rity

DuloxetinehighDuloxetinelow

EscitalopramFluoxetine

ParoxetinePlacebo

8−week symptom cluster response trajectories − HAM−D

SUMMARY

1. Statistical models might guide medication choice in Depression

2. Statistical models are not the end goal!

3. Extensions:

A. Finer predictions? Better predictions?

B. Other treatments, other mental illnesses

SUMMARY

• Antidepressants don’t work on all symptoms

• Core symptoms yes; sleep symptoms sometimes, atypical not much

• Pick the most effective AD for *your symptoms*

• core/atypical ~ 80-120mg BID duloxetine, 20mg paroxetine QD• sleep ~ 37-300mg venlafaxine + 15-45 mirtazapine QD

• Streamline and optimize care pathways for mental illness• efficient screening ~ iPad in waiting room• treatment selection ~ predictive modeling• outcome tracking ~ is it working!?! (& more data, more models, better tmt)

ACKNOWLEDGEMENTSJohn KrystalGregory McCarthy

Phil Corlett

Ralitza Gueorguieva

Madhukar Trivedi

Tyrone Cannon

Marcia Johnson

Harlan Krumholz

data-driven optimization of treatment outcomes in …...vs placebo comparison for at least 6 weeks,...

Documents