cui tao, phd - semantic-web based annotation
TRANSCRIPT
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
1/55
Semantic-Web based Annotationand Reasoning for Temporal
Information in Clinical Narratives
Cui Tao, PhD
Division of Biomedical Statistics and InformaticsDepartment of Health Sciences Research
Mayo Clinic, Rochester, MN
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
2/55
Acknowledgement
Christopher Chute, MD,DrPH
Kim Clark Piet de Groen, MD
Guergana Savova, PhD
Deepak Sharma
Harold Solbrig
This research issupported by theNational ScienceFoundation underGrant #0937060 tothe ComputingResearch
Association for theCIFellows Project
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
3/55
Introduction
Time is essential in clinical research
Uncover temporal pattern
Disease level Patient level
Explain past events
Predict future events
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
4/55
Introduction
Vast amount of data
Data embeded in narratives
Many temporal relations are notexplicitlystated in the clinical narratives,but rather needs to be inferred
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
5/55
Temporal Relation Reasoning
(Example) Patients INR value is below normal(Event 1)
today. (note date: 01/26/07)
He has had the chills
and body
aches (
Event 2)before the abnormal test. (Event 3) (note date:
01/26/07)
On Jan. 30, 2007, patient started Coumadindosing plan of 1.0 mg(Event 4).(note date:02/09/07)
Question: did the patient experience bodyachesbefore he started the Coumadin dosing plan?(was Event 2 before Event 4?)
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
6/55
Temporal Relation Reasoning
(Example) Patients INR value is below normal
(Event 1) today. (note date:01/26/07)
He has had the chills and bodyaches (Event 2) before the abnormaltest. (Event 3) (note date: 01/26/07)
On Jan. 30, 2007, patient started
Coumadin dosing plan of 1.0 mg(Event 4).(note date: 02/09/07)
Question: did the patient experiencebodyaches before he started theCoumadin dosing plan?
Event1 = Event3
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
7/55
Temporal Relation Reasoning
(Example) Patients INR value is below normal
(Event 1) today. (note date:01/26/07)
He has had the chills and bodyaches (Event 2) before the abnormaltest. (Event 3) (note date: 01/26/07)
On Jan. 30, 2007, patient started
Coumadin dosing plan of 1.0 mg(Event 4).(note date: 02/09/07)
Question: did the patient experiencebodyaches before he started theCoumadin dosing plan?
Event1 = Event3
+
Event2 before Event3
Event2 before Event1
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
8/55
Temporal Relation Reasoning
(Example) Patients INR value is below normal
(Event 1) today. (note date:01/26/07)
He has had the chills and bodyaches (Event 2) before the abnormaltest. (Event 3) (note date: 01/26/07)
On Jan. 30, 2007, patient started
Coumadin dosing plan of 1.0 mg(Event 4).(note date: 02/09/07)
Question: did the patient experiencebodyaches before he started theCoumadin dosing plan?
Event1 = Event3
+
Event2 before Event3
Event2 before Event1
Event1 01/26/07 +
Event4 01/30/07
Event1 before Event4
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
9/55
Temporal Relation Reasoning
(Example) Patients INR value is below normal
(Event 1) today. (note date:01/26/07)
He has had the chills and bodyaches (Event 2) before the abnormaltest. (Event 3) (note date: 01/26/07)
On Jan. 30, 2007, patient started
Coumadin dosing plan of 1.0 mg(Event 4).(note date: 02/09/07)
Question: did the patient experiencebodyaches before he started theCoumadin dosing plan?
Event1 = Event3
+
Event2 before Event3
Event2 before Event1
Event1 01/26/07 +
Event4 01/30/07
Event1 before Event4
Event2 before Event4
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
10/55
Introduction
The Semantic Web provides a suitableenvironment for temporal data
representation and reasoning: Standard mechanism with explicitand formal semantic definition
OWL DL
SWRL
Reasoning tools, querying andstorage mechanisms
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
11/55
System Overview
CNTRO
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
12/55
CNTRO Ontology
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
13/55
Clinical Narrative Temporal
Relation Ontology (CNTRO) Event
Time
Time Instant
Time Interval
Time Phase
Time Period
Temporal Relation
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
14/55
CNTRO Overview
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
15/55
Temporal Information
Representation (Instant) Example Sentence:
the second cycle ofchemotherapy was onJune 10, 2004
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
16/55
Temporal Information
Representation (Interval) Example Sentence:
monitor patients heartrate for 72 hoursstarting from today(note date:2004-06-01)
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
17/55
Temporal Information
Representation (Period & Phase) Example Sentence:
take antibiotics every8 hours for 10 daysstarting from today(note date:2004-06-01)
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
18/55
Temporal Information
Representation (Uncertainty) Example Sentence:
see the patient backin approximately twoweeks prior to histhird cycle ofchemotherapy (notedate:2004-06-10)
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
19/55
Temporal Information
Representation (Statement) Example Sentence:
patients bilirubin iselevated 2 weeksafter the second cycleof chemotherapy
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
20/55
CNTRO Evaluation
153 sentences with temporal information
178 events
98 time instants 10 time intervals
53 time phases
170 temporal relations 142/153 sentences were represented
without losing any time-relatedinformation
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
21/55
CNTRO 1.0
provides a formal semantic foundationto leverage the semantic-web
techniques still necessary to arrive at a shared set
of semantics and operational rules withcommonly used ontologies for the timedomain
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
22/55
CNTRO 2.0: A HarmonizedSemantic Web Ontology for
Temporal RelationInferencing in Clinical Narratives
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
23/55
Semantic Alignments
Initial Alignments
Time Ontology
SWRL Temporal Ontology Basic Formal Ontology
Measurement Units Ontology
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
24/55
Entity Definition
Time-related classes harmonization
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
25/55
Entity Definition
Event Class: any sort of occurrence, state,perception, procedure, symptom or situationthat occurs on a time line in clinical narratives
Subclasses:
Process (BFO)
Fiat_process_part (BFO)
Process_boundary (BFO) Process_observable
Process_repeat
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
26/55
Entity Definition
Process (BFO)
a clinical event that associates with a timeinterval
patients chemotherapy started from Jan.10 and ended on Mar. 19
Process_boundary (BFO)
Clinical events that associate with a timeinstant
the patient checked in at 3pm
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
27/55
Entity Definition
Fiat_process_part (BFO)
events with unknown or unspecified starting andending time
patient experienced headache during last week
Process_observable
observable states, conditions, or qualities
Process_repeat
clinical events recur periodically
take antibiotics every 8 hours for 10 days startingfrom today
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
28/55
Temporal Relation Harmonization
Allens time operators: between timeintervals
SWRL Temporal Ontology: betweentime intervals and instants
Time Ontology: between intervals,
additional general properties for instants CNTRO: between two events, or event
and time
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
29/55
Temporal Relation Harmonization
Define CNTRO properties using SWRLtemporal ontology properties:
Leverage the SWRL Temporal Built-Ins library to
perform temporal operations
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
30/55
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
31/55
Unit
Mapped the Measurement Units Ontology
muo:QualityValue muo:UnitOfMeasurementmuo:measuredIn
xsd:float
rdf:value
cntro:Duration cntro:Unit
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
32/55
Semantic Annotation
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
33/55
Semantic Annotation Tool
Knowtator extension
RDF exporter
Connects to the temporal reasoningframework API
Currently a manual process
Will connect with Mayos cTAKESpipeline
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
34/55
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
35/55
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
36/55
Temporal Reasoning Framework
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
37/55
Temporal Relation Reasoning
Temporal Representation Normalization
OWL DL Reasoning
SWRL-based Reasoning
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
38/55
Temporal Representation
Normalization Normalize to xsd DateTime Data Type format
BYUOntos information extraction technique
Mayo Clinic cTAKES (clinical Text Analysisand Knowledge Extraction System)
Jan 23rd, 01/23/2010, 1/23/10, 1-23-10,etc
Today, yesterday, in 2 days, 2 hours ago, etc
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
39/55
Logical Characteristics of Properties
Transitive properties: i.e., before, after
Inverse properties, beforeafter
Symmetric properties, i.e., equal
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
40/55
Operators for Offsets
inverse operators and
Temporal relation between e1 and e4?
(3 days) (2days) = (1day)
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
41/55
Restriction Assertions
a cancer chemotherapy must happen after a cancer
diagnosis based on clinical evidence
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
42/55
Semantic Definition of Concepts With
OWL DL
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
43/55
SWRL Temporal Built-In Library
Works with Protg SWRL Tag
Implemented Allens temporal operators
Serves as basic functional blocks in ourtemporal relation reasoning framework
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
44/55
SWRL RuleML
Rule-based Definition forConsistency
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
45/55
SWRL RuleML
Rule-based Definition for properties
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
46/55
SWRL RuleML
Rule-based Definition for concepts
premature labor after 22 weeks but before 37
completed weeks of gestation without delivery
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
47/55
Implementation Status
findEvent(searchText)
returns a list of events that match thesearching criteria. Currently we look forevents based on text search.
GetEventFeature(event, featureflag)
returns a specific time feature for a givenevent.
Sample query: When was the patient diagnosed with
diabetes?
When did the patient start hischemotherapy?
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
48/55
Implementation Status
getDurantionBetweenEvents(event1, event2)
returns the time interval between two events.
Sample query: How long after the patient was
diagnosed colon cancer did he start thechemotherapy?
getDuration(event)
returns the duration of a given event.
Sample query: How long did the symptoms ofrectal bleeding last?
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
49/55
Implementation Status
getTemporalRelationType(event1, event2)
returns the temporal relations between twoevents if it can be retrieved or inferred.
Sample query: Was the CT scan after thecolonoscopy?
getTemporalRelationType(event1, time)
returns the temporal relations between an
event and a specific time if it can be inferredor retrieved.
Sample query: Is there any behavior changewithin a week of the test?
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
50/55
Implementation Status
sortEventsByTemporalRelationsOrTimeline(events)
returns the order(timeline) of a set of events.
sample query:
What is the tumor status timeline asindicated in the patients radiology note?
What is the treatment timeline as recordedin oncology notes?
When was the first colonoscopy done? When was the most recent glucose test?
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
51/55
Pilot Study
Late stent thrombosis adverse event
Complaint files from Manufacturer and
User Facility Device Experience(MAUDE) database
Detect potential temporal patterns within
complaint files of similar adverse events
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
52/55
Pilot Study
MAUDE Database
Confirmed Late Stent
Thrombosis Reports
15 Reports
CNTRO Reasoning
System-generated
answersCompare
Human Expert
Human-generated
answers (gold
standard)
Temporal QuestionsTemporal Questions
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
53/55
Pilot Study
Evaluation
System answered 65 out of 73
questions correctly Accuracy: 89.04%
Errors caused by ambiguities in
annotations
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
54/55
Summary
A more robust and semantically crisp model
A semi-automatic annotation system
A temporal relation reasoning framework
A user friendly querying system
A GUI where users can view and query
information of interested in its original context
CNTRO.ORG
-
8/7/2019 Cui Tao, PhD - Semantic-WEb Based Annotation
55/55