EHR-BasedPhenotyping:BulkLearningandEvalua;on(with

Infec;ousDiseases) Po-Hsiang (Barnett) Chiu

Phenotypesandphenotyping

Physicallyobservabletraitsofgenotypes(andtheirinterac;onswithenvironments)

Diseases(anddiseasesubtypes)

AFribu;onsofdiseases(e.g.suscep;bility)

Biochemicalorphysiologicalproper;es,behavior,andproductsofbehavior

Data-DrivenPhenotyping•  Data-drivenphenotyping

–  Twomainmethodologies•  Rule-basedapproach(e.g.eMerge,hFps://emerge.mc.vanderbilt.edu)•  Predic3veAnaly3cs

–  Datasources:•  EHRs/EMRs:Medicinaltreatments,diagnoses,labmeasurements,etc.•  Genomicdata:SNParrays,copynumbervaria;on(CNVs),etc.

–  Phenotypes•  Diseases,subtypes,orvariablesaFributedtodiseasepredic;ons

Diagnos;cConceptUnits•  Variousdiseasessharingthesamesetofdiagnos;cconceptunits

•  Infec;ousdiseases–  Labtests

•  Microorganism,blood,urine,body;ssues,stool– Medica;ons

•  An;bio;c,an;virus,anthelmin;c

•  Buildsta;s;calmodelsforeachdiagnos;ccomponentandcombinethemappropriately–  Ensemblelearning

BulkLearninginaNutshell…Bulk Learning is a batch-phenotyping framework that uses multiple diseases collectively (i.e. bulk learning set) as a substrate for model learning and evaluation wherein (a given) medical ontology is used to perform feature selection and model stacking is used to construct abstract feature representation of low sample complexity in order to reduce training requirements.

KeyConcepts:

1.Buildphenotypingmodelsontopofmul;plediseases

3.Modelsarecombinedviamodelstacking(aformofensemblelearning)4.Abstractfeatures

Dimensionalityreduc;on

2.Automa;cfeatureselec;onusinganexis;ngontology

5.Lesslabeleddatarequiredformodelevalua;ons

PhenotypingviaBulkLearning•  Undermodelstacking,wethenarriveattheno;onof“concept-drivenphenotyping”–  Asubsetorcombina;onsoflabtestsaremoreaFributabletosomediseaseswhiletheothersarebeFerexplainedbymedica;ons

•  Inthisstudy,infec;ousdiseasesassociatedwith100ICD-9codesasthedomainofstudyforbulklearning–  Forsimplicity,considerdifferentdiagnos;ccodesasdifferentdiseases…

– Why100codes?–  Codeselec;onstrategy?

BulkLearningBasicsI

•  Addressestwocentralissuesinpredic;veanaly;calapproachtocomputa;onalphenotyping– Featureengineering

•  Medicalontologyforfeaturedecomposi;on•  MedicalEn;;esDict(hFp://med.dmi.columbia.edu)

– Dataannota;on•  Ensemblelearning(e.g.stackedgeneraliza;on[Wolpert1992])

•  Featureabstrac;onfordimensionalityreduc;on

MedicalOntologyforGroupingFeatures

•  SnapshotofMedicalEn;;esDic;onary(hFp://med.dmi.columbia.edu)

ModelStacking•  Whyinspec;ngmul;ple(infec;ous)diseases?

–  Usingmul3plediseasesassubstrateandiden;fytheircommonelements–  Examplestackingarchitecture(understackedgeneraliza;onmethod)

Level 1

Level 0

Antibiotic Measure

Urinary Chemistry Measure

Intravenous Chemistry Measure

Microbiology Measure

Level 2

Attributes: Level-0 Probabilities and IndicatorsTarget: Diagnostic Codes (Silver Standard)

Other Phenotypic Measures (e.g. Antiviral)

Attributes: Level-1 Probabilities and ICD-9Target: True Labels (Gold Standard)

SurrogateLabelsvsTrueLabels

•  Modelstackingisusedtoachieve:–  Improveuponbasemodelperformances–  TransformEHRdatatoadenserform

•  Usesdiagnos;ccodes(e.g.ICD-9)assurrogatelabelstoestablish“approximatepredic;vemodels.”

•  Whysurrogatelabels(e.g.ICD-9)?–  FeaturesextractedfromEHRcanbelarge–  Usedtoderivecompactrepresenta;onofthetrainingdata–  “Free”supervisedsignalsthataresufficientlyclosebutcanbeobtained

withoutextrawork•  Objec;ve:Buildsta;s;calmodelsinabstractfeaturespace

–  Createasparseannota;onset(i.e.goldstandard)thatservesaproxydatasetfordownstreammodelevalua;ons

–  83annotatedcases

⌃

⌃

⌃

⌃

⌃

m1

a1

b1

u1

m1(1)

m1g a1g b1g u1g

global2

(i)

(i-1)

(i+1)

a1(1) b1

(1) u1(1)

logistic unitsraw

features

microbiology

antibioticblood test

urine test

f11

f12

f1j

f21

f2j

f31

f41

f3j

Four Example Base Models

PerformanceEvalua;ons

•  HowwelldoesthemodelpredictICD-9s(usingaseparatetestdata)?

•  Howwelldoesthemodelpredictannotateddata(assoc.with“truelabels”)?–  (Binarized)ICD-9becomesacandidatefeatureamongabstractfeatures(e.g.probabilityscores,indicators)

•  AnnotatedsampleconsistsofrandomlyselectedcasesinwhicherrorsofICD-9codingarecorrected

•  Dataannota;onsandcodingproceduresaretwoindependentprocesses

BaseLevelPerformances

127.4Enterobiasis

047.8(Other)viralmeningi;s

009.1Gastroenteri;s...

053.9Herpezzoster

117.9Mycoses

OtherComponents

•  Semi-supervisedlearningandvirtualannota;onset

•  The3rd;erinmodelstackinghierarchy–  Trade-offbetweenlearnedabstractfeaturesandtheICD-9codesassurrogatelabels.

–  Performanceevalua;ononpredic;ngannotatedlabels•  Ontology-basedfeatureengineering•  Properdesignoftreatmentandcontrol(training)data

ModelingPerspec;ve•  EHRdataconsistofobserva;onsandlatentvariables

–  Observa;onscanbedirectlyansweredviasimplequeries•  Didthepa;enthavetestsonE.Coli?•  Didthepa;enttakeCekriaxon?

•  Latentvariablesrepresentquan;;esthatcannotbedirectlyobservedinEHRorcomputedviasimplequeries–  Doesthepa;enthaveaninfec;on?–  Diagnos;cques;ons:specificallywhichinfec;onsdothepa;enthave?

•  Learnclassifierstopredictlatentvariables(withonlyaccesstoobserva;ons)

MedicalPerspec;ve•  Seeminglydifferentinfec;ousdiseasesmaysharesimilarsetsoflabtestsandmedica;ons–  Staph.aureus

•  Skininfec;ons,pneumonia,bloodpoisoning–  Cekriaxone

•  Meningi;s•  Infec;onsatdifferentsitesofthebody(e.g.bloodstream,lungs,urinarytracts)

•  Mul;pleclassifiersforthesamedisease–  4classifiersperICD-9code,eachofwhichisbinaryclassifier

•  400classifiersatbaselevel

DataDistribu;onPerspec;ve

“Canwebuildajointmodelapplicabletoalldiseases?”

AbstractFeatureRepresenta;on:DesignChoices

•  Relatedworkinconstruc;nghigh-levelfeatures–  PCA,unsupervisedfeaturelearning,manifoldlearning,etc.

•  Designchoices–  Datacharacteris;cs–  Interpretability

•  DeepNeuralNetwork–  Linearcombina;on–  Non-lineartransforma;on(e.g.sigmoid,rec;fier,etc.)

•  Featureset:con;nuous,dense,and“homogeneous”–  Imagepixels–  Timesseriesoflabmeasurements–  word2vec

•  EHRdatahoweverareverydifferent–  sparseandincomplete–  consistofmanydifferenttypes(binary,categorical,con;nuous,etc.)–  Featuresassociatedwithmul;pleconcepts

MovingForward…•  Summary

–  Bulklearningisaframeworkwithatleastthefollowingsystemchoices•  Thebulklearningset(oftargetcondi;ons)=>basemodels•  Classifica;onalgorithms(guideline:probabilis;cclassifiers+well-calibrated)•  Stackingarchitecture(mul;ple;ers=>levelsofabstrac;ons)•  Strategyforcombiningindividual(local)diseasemodelstoaglobalmodel

–  Advantage:Canuseasmallannotatedsampleformodelconstruc;onandevalua;onwithintheabstractfeaturespace(e.g.level-1data)

•  83clinicalcaseswerelabeledinthisstudy–  Challenge:Themodelinvolvingtheinterac;onbetweenabstractfeaturesand

ICD-9donotgeneralizewellintotheregionofthedatawheretheICD-9codingwasincorrect

•  Mul;pletypesofsurrogatelabels⌃

⌃

⌃

⌃

⌃

m1

a1

b1

u1

m1(1)

m1(i)

a1(i)

b1(i)

u1(i)

⌃

m1g a1g b1g u1g

local2(i)

global2

(i)

(i-1)

(i+1)

a1(1) b1

(1) u1(1)

(i-1)(i)

(i+1)

Semi-supervisedlearningAc3velearning

Complexdecisionboundary?

Othersurrogatelabels

•  Ongoingandfuturework

Reference[1]D.H.Wolpert,Stackedgeneraliza;on,NeuralNetworks.5(1992)241–259.[2]K.M.Ting,I.H.WiFen,Issuesinstackedgeneraliza;on,J.Ar;f.Intell.Res.10(1999)271–289.[3]J.JinChen,C.ChengWang,R.RunshengWang,UsingStackedGeneraliza;ontoCombineSVMsinMagnitudeandShapeFeatureSpacesforClassifica;onofHyperspectralData,IEEETrans.Geosci.RemoteSens.47(2009)2193-2205.[4]DavidBaorto,JamesCimino,etal.Available:hFp://med.dmi.columbia.edu.Accessdate:Oct20,2016.[5]T.A.Lasko,J.C.Denny,M.A.Levy,Computa;onalPhenotypeDiscoveryUsingUnsupervisedFeatureLearningoverNoisy,Sparse,andIrregularClinicalData,PLoSOne.8(2013)e66341.

T H A N K

Y O U ⌃

⌃

⌃

⌃

m1

a1

b1

u1

f11f12

f1j

f21

f2j

f31

f41

f3j

⌃

⌃

⌃

⌃

m1

a1

b1

u1

logistic unitsraw

featuresf11

f12

f1j

f21

f2j

f31

f41

f3j

m1

a1

b1

u1

⌃

Level 0 Level 1

Microbiology

An;bio;c

Bloodtest

Urinetest

ExampleFeatures

⌃

⌃

⌃

⌃

⌃

m1

a1

b1

u1

m1(1)

m1(i)

a1(i)

b1(i)

u1(i)

⌃

m1g a1g b1g u1g

local2(i)

global2

(i)

(i-1)

(i+1)

a1(1) b1

(1) u1(1)

(i-1)(i)

(i+1)

logistic unitsraw

features

microbiology

antibioticblood test

urine test

2. Compute Base Models

Level-1 Global Unit

Individual Level-1 Local Units

Level-1 abstractfeatures

f11

f12

f1j

f21

f2j

f31

f41

f3j

Four Example Base Models

3. Compute Meta Models (via Ensemble Learning)1. Define Feature Groups Using Medical Ontology

1a. Gather EHR data according to medical concepts

1b. Use Medical Entities Dictionary to delineate feature scopes

1c. Apply feature selection within each

concept group

3a. Per-disease ensembles:compute local level-1 models

3b. Cross-disease ensemble: compute a global

level-1 model

Global level-1 features