automatic formalization of clinical practice guidelines
DESCRIPTION
Automatic Formalization of Clinical Practice Guidelines. Matthew S. Gerber and Donald E. Brown Department of Systems and Information Engineering University of Virginia. James H. Harrison Department of Public Health Sciences University of Virginia. Clinical Practice Guidelines. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/1.jpg)
Automatic Formalization of Clinical Practice Guidelines
Matthew S. Gerber and Donald E. BrownDepartment of Systems and Information Engineering
University of Virginia
James H. HarrisonDepartment of Public Health Sciences
University of Virginia
![Page 2: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/2.jpg)
2
• Many treatment options – what to do?
Clinical Practice GuidelinesB
enef
its /
cos
ts
Evidence quality
Randomized clinical trial: beneficial
Expert opinion: might be beneficial
Meta-analysis: usually beneficial
Strength
Recommended
Should consider
Might consider
![Page 3: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/3.jpg)
3
Clinical Practice Guidelines
• Development– Expert synthesis of current evidence– Example from heart failure:
![Page 4: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/4.jpg)
4
Clinical Practice Guidelines
• Expected outcomes– Evidence-based clinical decision aid– Reduction in cost and treatment/outcome variation– Improvement in patient health
• Challenges– A guideline for any occasion
– Guidelines change periodically – Lengthy (HFSA CPG is 259 pages)
Total (NGC) 2,269
Cardiovascular diseases 486
Heart failure 152
![Page 5: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/5.jpg)
5
Clinical Decision Support Systems
• Goal: deliver CPG knowledge at point of care• Alleviate burden on clinician• Problem: CPGs contain minimally structured text
Formalization is required
![Page 6: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/6.jpg)
6
Traditional CPG Formalization
Knowledge engineersMedical experts
Knowledge representation
Knowledge management software (e.g., Protégé)
CDSS
CPGCPG
Automatic formalization
![Page 7: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/7.jpg)
…
Retrospective analyses
The Big Picture
7
Structured knowledge
Medical decision support
?
EndocrineInfections
…Cardiovascular
NLP
![Page 8: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/8.jpg)
8
Data Collection
• Yale Guideline Recommendation Corpus– Hussain et al. (2009)– 1,275 recommendations– Representative sample of domains and rec. types
“Oral antiviral drugs are indicated within 5 days of the start of the episode and while new lesions are still forming.”
– Simplifications• Delimited recommendations• No inter-recommendation dependencies
• Random sub-sample of YGRC (n=200)
![Page 9: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/9.jpg)
9
Recommendation Representation
• SNOMED-CT– Medical concept ontology– Broad coverage
Keywords ? Asbru, etc.
Fidelity: Low High
Automation: Trivial Impossible
![Page 10: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/10.jpg)
10
Recommendation Representation
(Sundvalls et al., 2012)
![Page 11: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/11.jpg)
11
Recommendation RepresentationPrimary Secondary Example recommendation
ACTION EVALUATION [computed tomography CT] should be used
PROCEDURE [red blood cell transfusion] is appropriate
…
EVIDENCE STRONG [it has been shown to reduce the occurrence of NTDs]
WEAK/NONE [there is insufficient evidence]
MODALITY OBLIGATORY computed tomography CT [should] be used
NEVER oral risedronate [should not] be used
OPTIONAL physician [may] choose
AGENT [physician] may choose
MORBIDITY prevent [preeclampsia]
POPULATION [obese women with gestational diabetes mellitus]
PURPOSE is used [to prevent osteoporotic fractures]
TEMPORAL [initial] treatment
SNOMED-CT CONCEPT: 129265001
![Page 12: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/12.jpg)
12
Recommendation Annotation
• Task: manually identify representational elements within recommendations
• Example
Diuretics are recommended for patients with heart failure.
[DRUG Diuretics] are recommended for [POPULATION patients with [MORBIDITY heart failure]].
![Page 13: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/13.jpg)
13
Methods
• Natural language processing• Supervised classification• Per-recommendation pipeline
1. Syntactic parsing
2. Parse node classification
3. Post-processing
![Page 14: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/14.jpg)
14
Methods: (1) Syntactic Parsing
• Constituency parser (Charniak and Johnson, 2005)
![Page 15: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/15.jpg)
15
Methods: (2) Parse Node Classification
• Unit of classification: node• Multi-class logistic regression• Example: 1 positive, 17 negative• Actual
– 12K nodes– 10 classes (primary)
![Page 16: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/16.jpg)
16
Methods: (2) Parse Node Classification
• Linguistic features– Word stems under node– Syntactic configuration of node– …
![Page 17: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/17.jpg)
17
Methods: (2) Parse Node Classification
• Learning– Forward feature selection– Per-class costs (LibLinear)
![Page 18: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/18.jpg)
18
Methods: (3) Post-processing
• Remove duplicates• Other possible issues
– Conflicts– Embedding
![Page 19: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/19.jpg)
19
Evaluation Results
Element # Precision (%) Recall (%) F1 (%)
ACTION 301 74.5 26.2 38.7
MODALITY 158 71.9 73.0 72.4
POPULATION 140 83.7 56.6 67.5
TEMPORAL 53 28.6 1.1 2.2
TRIGGER 45 81.7 93.3 87.1
PURPOSE 43 58.3 16.3 25.5
EVIDENCE 38 83.3 13.2 22.7
AGENT 37 76.0 47.6 58.5
MORBIDITY 19 50.0 10.5 17.4
Overall 834 75.3 41.7 53.7
• 10-fold cross-validation
![Page 20: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/20.jpg)
20
Discussion
• High variance across classes• Alternative strategies
– Identify more informative features– Change the model formulation– Annotate more data
![Page 21: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/21.jpg)
21
Conclusions
• CPGs are an important knowledge source• Difficult to use within CDSS• Prior CPG formalization
– Manual– Automatic for specific domains / recommendations
• Our contributions– SNOMED-CT representation– Manually annotated recommendation sample– Statistical NLP model / evaluation
![Page 22: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/22.jpg)
22
Future Work
• Refined representation
• Model formulation
• Feature engineering
• Controlled natural language
![Page 23: Automatic Formalization of Clinical Practice Guidelines](https://reader035.vdocument.in/reader035/viewer/2022062321/56812a5e550346895d8dca38/html5/thumbnails/23.jpg)
23
Questions?
• References1. Charniak, E. & Johnson, M. Coarse-to-fine n-best parsing and MaxEnt
discriminative reranking. Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, 2005, 173-180.
2. Hussain, T.; Michel, G. & Shiffman, R. N. The Yale Guideline Recommendation Corpus: A representative sample of the knowledge content of guidelines. I. J. Medical Informatics, 2009, 78, 354-363.
3. Fan, R.-E.; Chang, K.-W.; Hsieh, C.-J.; Wang, X.-R. & Lin, C.-J. LIBLINEAR: A Library for Large Linear Classification. Journal of Machine Learning Research, 2008, 9, 1871-1874.
4. Sundvall, E.; Nystrom, M.; Petersson, H. & Ahlfeldt, H. Interactive visualization and navigation of complex terminology systems, exemplified by SNOMED CT. Studies in health technology and informatics, IOS Press; 1999, 2006, 124, 851.