knowledge enabled information and services science semantic web: promising technologies, current...

Knowledge Enabled Information and Services Science

Semantic Web: Promising technologies, Current Applications & Future Directions

Australia, July-August 2008

Amit P. [email protected]

Thanks Kno.e.sis team and collaborators

http://knoesis.wright.edu/amit/

http://www.knoesis.org/research/


Outline

• Semantic Web – very brief intro of key capabilities and technlologies

• Real-world Applications demonstrating benefit of semantic web technologies

• Exciting on-going research


Evolution of the Web

Web of pages - text, manually created links - extensive navigation

2007

1997

Web of databases - dynamically generated pages - web query interfaces

Web of services - data = service = data, mashups - ubiquitous computing

Web of people - social networks, user-created content - GeneRIF, Connotea

Web as an oracle / assistant / partner - “ask to the Web” - using semantics to leverage text + data + services + people


What is Semantic Web?

• Associating meaning with data: labeling data so it is more meaningful to the system and people. Formal description increases automation. Common interpretation increases interoperability.

• TBL – focus on data: Data Web (“In a way, the Semantic Web is a bit like having all the databases out there as one big database.”)

• Others focus on reasoning and intelligent processing


• Ontology: Agreement with Common Vocabulary & Domain Knowledge; Schema + Knowledge base

• Semantic Annotation (metadata Extraction): Manual, Semi-automatic (automatic with human verification), Automatic

• Reasoning/computation: semantics enabled search, integration, complex queries, analysis (paths, subgraph), pattern finding, mining, hypothesis validation, discovery, visualization

Semantic Web Enablers and Techniques


Open Biomedical Ontologies

Open Biomedical Ontologies, http://obo.sourceforge.net/

Many ontologies exist


Drug Ontology Hierarchy (showing is-a relationships)

owl:thing

prescription_drug

_ brand_na

me

brandname_unde

clared

brandname_comp

osite

prescription_drug

monograph_ix_cla

ss

cpnum_ group

prescription_drug

_ property

indication_

property

formulary_

property

non_drug_

reactant

interaction_proper

ty

property

formulary

brandname_indivi

dual

interaction_with_prescriptio

n_drug

interaction

indication

generic_ individua

l

prescription_drug_ generic

generic_ composit

e

interaction_ with_non_ drug_react

ant

interaction_with_monograph_ix_class


N-Glycosylation metabolic pathway

GNT-Iattaches GlcNAc at position 2

UDP-N-acetyl-D-glucosamine + alpha-D-Mannosyl-1,3-(R1)-beta-D-mannosyl-R2 <=>

UDP + N-Acetyl-$beta-D-glucosaminyl-1,2-alpha-D-mannosyl-1,3-(R1)-beta-D-mannosyl-$R2

GNT-Vattaches GlcNAc at position 6

UDP-N-acetyl-D-glucosamine + G00020 <=> UDP + G00021

N-acetyl-glucosaminyl_transferase_VN-glycan_beta_GlcNAc_9N-glycan_alpha_man_4


WWW, EnterpriseRepositories

METADATA

EXTRACTORS

Digital Maps

NexisUPIAP

Feeds/Documents

Digital Audios

Data Stores

Digital Videos

Digital Images. . .

. . . . . .

Create/extract as much (semantics)metadata automatically as possible;

Use ontlogies to improve and enhanceextraction

Information Extraction for Metadata Creation


Metadata and Ontology: Primary Semantic Web enablers

Shallow semantics

Deep semantics

Expr

essi

vene

ss,

Rea

soni

ng


Automatic Semantic Metadata Extraction/Annotation


Characteristics of Semantic Web

SelfDescribing

Machine &Human

Readable

Issued bya TrustedAuthority

Easy toUnderstand

ConvertibleCan be

Secured

The Semantic Web:XML, RDF & Ontology

Adapted from William Ruh (CISCO)


Application Example 1:

• Status: In use today• Where: Athens Heart Center• What: Use of semantic Web technologies

for clinical decision support


Operational since January 2006


Goals:• Increase efficiency with decision support

• formulary, billing, reimbursement

• real time chart completion

• automated linking with billing

• Reduce Errors, Improve Patient Satisfaction & Reporting• drug interactions, allergy, insurance

• Improve Profitability

Technologies:• Ontologies, semantic annotations & rules

• Service Oriented Architecture

Thanks -- Dr. Agrawal, Dr. Wingeth, and others. ISWC2006 paper

Active Semantic Electronic Medical Records (ASEMR)

http://iswc2006.semanticweb.org/items/in_use_8.php


Demonstration

http://lsdis.cs.uga.edu/projects/asdoc/index.php?page=1


Opportunity: exploiting clinical and biomedical data

text

Health Information

Services

Elsevier iConsult

Scientific Literature

PubMed300 Documents

Published Online each day

User-contributed Content (Informal)

GeneRifs

NCBI Public Datasets

Genome, Protein DBs

new sequencesdaily

Laboratory Data

Lab tests, RTPCR,

Mass spec

Clinical Data

Personal health history

Search, browsing, complex query, integration, workflow, analysis, hypothesis validation, decision support.

binary


Application Example 2

• Status: Completed research• Where: NIH/NIDA • What: Understanding the genetic basis of

nicotine dependence. • How: Semantic Web technologies (especially

RDF, OWL, and SPARQL) support information integration and make it easy to create semantic mashups (semantically integrated resources).


Entrez Gene

ReactomeKEGG

HumanCyc

GeneOntology HomoloGene

Genome and pathway information integration

• pathway

• protein

• pmid

• pathway

• protein

• pmid

• pathway

• protein

• pmid

• GO ID • HomoloGene

ID


BioPAXontology

EntrezKnowledge

Model(EKoM)


Biological Significance:

• Understand the role of genes in nicotine addiction

• Treatment of drug addiction based on genetic factors

• Identify important genes and use for pharmaceutical productions

Gene-Pathway Data Integration–Understanding the Genetic-basis of Nicotine DependenceCollaborators: NIDA, NLM


Scenario 3

• Status: Completed research• Where: NIH • What: queries across integrated data

sources– Enriching data with ontologies for integration,

querying, and automation– Ontologies beyond vocabularies: the power of

relationships


Use data to test hypothesis

gene

GO

PubMed

Gene name

OMIM

Sequence

Interactions

Glycosyltransferase

Congenital muscular dystrophy

Link between glycosyltransferase activity and congenital muscular dystrophy?

Adapted from: Olivier Bodenreider, presentation at HCLS Workshop, WWW07


In a Web pages world…

Congenital muscular dystrophy,type 1D

(GeneID: 9215)

has_associated_disease

has_molecular_function

Acetylglucosaminyl-transferase activity

Adapted from: Olivier Bodenreider, presentation at HCLS Workshop, WWW07


With the semantically enhanced data

MIM:608840Muscular dystrophy,

congenital, type 1D

GO:0008375

has_associated_phenotype

has_molecular_function

EG:9215LARGE

acetylglucosaminyl-transferase

GO:0016757glycosyltransferase

GO:0008194isa

GO:0008375acetylglucosaminyl-

transferase

GO:0016758

From medinfo paper.Adapted from: Olivier Bodenreider, presentation at HCLS Workshop, WWW07

SELECT DISTINCT ?t ?g ?d { ?t is_a GO:0016757 . ?g has molecular function ?t . ?g has_associated_phenotype ?b2 . ?b2 has_textual_description ?d .FILTER (?d, “muscular distrophy”, “i”) . FILTER (?d, “congenital”, “i”) }


Scenario 5

• Status: Research prototype and in progress• Workflow withSemantic Annotation of Experimental

Data already in use

• Where: UGA• What:

– Knowledge driven query formulation– Semantic Problem Solving Environment (PSE)

for Trypanosoma cruzi (Chagas Disease)


Knowledge driven query formulation

Complex queries can also include:- on-the-fly Web services execution to retrieve additional data- inference rules to make implicit knowledge explicit


T.Cruzi PSE Query Interface

Figure 4: Semantic annotation of ms scientific data


N-Glycosylation Process (NGP)

Cell Culture

Glycoprotein Fraction

Glycopeptides Fraction

extract

Separation technique I

Glycopeptides Fraction

n*m

n

Signal integrationData correlation

Peptide Fraction

Peptide Fraction

ms data ms/ms data

ms peaklist ms/ms peaklist

Peptide listN-dimensional arrayGlycopeptide identificationand quantification

proteolysis

Separation technique II

PNGase

Mass spectrometry

Data reductionData reduction

Peptide identificationbinning

n

1


Storage

Standard FormatData

Raw Data

Filtered Data

Search Results

Final Output

Agent Agent Agent Agent Biological Sample Analysis

by MS/MS

Raw Data to

Standard Format

DataPre-

process

DB Search

(Mascot/Sequest)

Results Post-

process

(ProValt)

O I O I O I O I O

Biological Information

SemanticAnnotationApplications

Semantic Web Process to incorporate provenance


830.9570 194.9604 2

580.2985 0.3592

688.3214 0.2526

779.4759 38.4939

784.3607 21.7736

1543.7476 1.3822

1544.7595 2.9977

1562.8113 37.4790

1660.7776 476.5043

parent ion m/z

fragment ion m/z

ms/ms peaklist data

fragment ionabundance

parent ionabundance

parent ion charge

ProPreO: Ontology-mediated provenance

Mass Spectrometry (MS) Data


<ms-ms_peak_list><parameter instrument=“micromass_QTOF_2_quadropole_time_of_flight_mass_spectrometer”

mode=“ms-ms”/><parent_ion m-z=“830.9570” abundance=“194.9604” z=“2”/>

<fragment_ion m-z=“580.2985” abundance=“0.3592”/><fragment_ion m-z=“688.3214” abundance=“0.2526”/><fragment_ion m-z=“779.4759” abundance=“38.4939”/><fragment_ion m-z=“784.3607” abundance=“21.7736”/><fragment_ion m-z=“1543.7476” abundance=“1.3822”/><fragment_ion m-z=“1544.7595” abundance=“2.9977”/><fragment_ion m-z=“1562.8113” abundance=“37.4790”/><fragment_ion m-z=“1660.7776” abundance=“476.5043”/>

</ms-ms_peak_list>

OntologicalConcepts

ProPreO: Ontology-mediated provenance

Semantically Annotated MS Data


Problem – Extracting relationships between MeSH terms from PubMed

Biologically active substance

LipidDisease or Syndrome

affects

causes

affectscauses

complicates

Fish Oils Raynaud’s Disease???????

instance_of instance_of

UMLS Semantic Network

MeSH

PubMed9284 documents

4733 documents

5 documents


Background knowledge used

• UMLS – A high level schema of the biomedical domain– 136 classes and 49 relationships– Synonyms of all relationship – using variant lookup

(tools from NLM)– 49 relationship + their synonyms = ~350 mostly verbs

• MeSH – 22,000+ topics organized as a forest of 16 trees– Used to query PubMed

• PubMed – Over 16 million abstract– Abstracts annotated with one or more MeSH terms

T147—effect T147—induce T147—etiology T147—cause T147—effecting T147—induced


Method – Parse Sentences in PubMed

SS-Tagger (University of Tokyo)

SS-Parser (University of Tokyo)

(TOP (S (NP (NP (DT An) (JJ excessive) (ADJP (JJ endogenous) (CC or) (JJ exogenous) ) (NN stimulation) ) (PP (IN by) (NP (NN estrogen) ) ) ) (VP (VBZ induces) (NP (NP (JJ adenomatous) (NN hyperplasia) ) (PP (IN of) (NP (DT the) (NN endometrium) ) ) ) ) ) )

• Entities (MeSH terms) in sentences occur in modified forms• “adenomatous” modifies “hyperplasia”• “An excessive endogenous or exogenous stimulation” modifies

“estrogen”• Entities can also occur as composites of 2 or more other entities

• “adenomatous hyperplasia” and “endometrium” occur as “adenomatous hyperplasia of the endometrium”


Method – Identify entities and Relationships in Parse Tree

TOP

NP

VP

S

NPVBZ

induces

NPPP

NPINof

DTthe

NNendometrium

JJadenomatous

NNhyperplasia

NP PP

INby

NNestrogenDT

the

JJexcessive ADJP NN

stimulation

JJendogenous

JJexogenous

CCor

MeSHIDD004967MeSHIDD006965 MeSHIDD004717

UMLS ID

T147

ModifiersModified entitiesComposite Entities


• What can we do with the extracted knowledge?

• Semantic browser demo

http://knoesis.wright.edu/projects/SemanticBrowser/index.html


PubMed

Complex Query

SupportingDocument setsretrieved

Migraine

Stress

Patient

affects

isaMagnesium

Calcium Channel Blockers

inhibit

Keyword query: Migraine[MH] + Magnesium[MH]

Evaluating hypotheses


Workflow Adaptation: Why and How

• Volatile nature of execution environments– May have an impact on multiple activities/ tasks in the

workflow• HF Pathway

– New information about diseases, drugs becomes available

– Affects treatment plans, drug-drug interactions• Need to incorporate the new knowledge into

execution– capture the constraints and relationships between

different tasks activities


Workflow Adaptation Why?

New knowledge abouttreatment found duringthe execution of the pathway

New knowledge about drugs,drug drug interactions


Workflow Adaptation: How

• Decision theoretic approaches– Markov Decision Processes

• Given the state S of the workflow when an event E occurs– What is the optimal path to a goal state G– Greedy approaches rely on local optimization

• Need to choose actions based on optimality across the entire horizon, not just the current best action

– Model the horizon and use MDP to find the best path to a goal state


Conclusion

• semantic web technologies can help with:– Fusion of data: semi-structured, structured,

experimental, literature, multimedia– Analysis and mining of data, extraction,

annotation, capture provenance of data through annotation, workflows with SWS

– Querying of data at different levels of granularity, complex queries, knowledge-driven query interface

– Perform inference across data sets


Take home points

• Shift of paradigm: from browsing to querying

• Machine understanding: – extracting knowledge from text– Inference, software interoperation

• Semantic-enabled interfaces towards hypothesis validation


References

1. A. Sheth, S. Agrawal, J. Lathem, N. Oldham, H. Wingate, P. Yadav, and K. Gallagher, Active Semantic Electronic Medical Record, Intl Semantic Web Conference, 2006.

2. Satya Sahoo, Olivier Bodenreider, Kelly Zeng, and Amit Sheth, An Experiment in Integrating Large Biomedical Knowledge Resources with RDF: Application to Associating Genotype and Phenotype Information WWW2007 HCLS Workshop, May 2007.

3. Satya S. Sahoo, Kelly Zeng, Olivier Bodenreider, and Amit Sheth, From "Glycosyltransferase to Congenital Muscular Dystrophy: Integrating Knowledge from NCBI Entrez Gene and the Gene Ontology, Amsterdam: IOS, August 2007, PMID: 17911917, pp. 1260-4

4. Satya S. Sahoo, Olivier Bodenreider, Joni L. Rutter, Karen J. Skinner , Amit P. Sheth, An ontology-driven semantic mash-up of gene and biological pathway information: Application to the domain of nicotine dependence, submitted, 2007.

5. Cartic Ramakrishnan, Krzysztof J. Kochut, and Amit Sheth, "A Framework for Schema-Driven Relationship Discovery from Unstructured Text", Intl Semantic Web Conference, 2006, pp. 583-596

6. Satya S. Sahoo, Christopher Thomas, Amit Sheth, William S. York, and Samir Tartir, "Knowledge Modeling and Its Application in Life Sciences: A Tale of Two Ontologies", 15th International World Wide Web Conference (WWW2006), Edinburgh, Scotland, May 23-26, 2006.

• Demos at: http://knoesis.wright.edu/library/demos/

http://knoesis.wright.edu/library/resource.html?id=00004

http://www2007.org/workshops/paper_149.pdf

http://www2007.org/workshops/paper_149.pdf

http://www2007.org/workshop-W2.php







More about the Relationship Web

Relationship Web takes you away from “which document” could have information I need, to “what’s in the resources” that gives me the insight and knowledge I need for decision making.

Amit P. Sheth, Cartic Ramakrishnan: Relationship Web: Blazing Semantic Trails between Web Resources. IEEE Internet Computing July 2007 (to appear) [.pdf]

http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/s/Sheth:Amit_P=.html

http://knoesis.wright.edu/students/cartic/

http://knoesis.wright.edu/library/publications/w4sem.pdf


Events: 3 Dimensions – Spatial, Temporal and Thematic

Spatial

Temporal

Thematic


Events and STT dimensions

• Powerful mechanism to integrate content– Describes the Real-World occurrences– Can have video, images, text, audio all of the same event– Search and Index based on events and STT relations

• Many relationship types– Spatial:

• What events happened near this event? • What entities/organizations are located nearby?

– Temporal: • What events happened before/after/during this event?

– Thematic:• What is happening?• Who is involved?

• Going further– Can we use What? Where? When? Who? to answer Why? /

How?– Use integrated STT analysis to explore cause and effect

Knowledge Enabled Information and Services Science50

High-level Sensor

Low-level Sensor

How do we determine if the three images depict …

• the same time and same place?

• the same entity?

• a serious threat?

Example Scenario: Sensor Data Fusion and Analysis


Sensor Data Pyramid

Raw Sensor (Phenomenological) Data

Feature Metadata

Entity Metadata

Ontology Metadata

Expr

essi

vene

ss

Data

Information

Knowledge

Data Pyramid


What is Sensor Web Enablement?

http://www.opengeospatial.org/projects/groups/sensorweb


GeographyML (GML)

TransducerML (TML)

Observations &

Measurements (O&M)

Information Model for Observations and Sensing

Sensor and Processing Description Language

Real Time Streaming Protocol

Common Model for Geographical

Information

SensorML (SML)

Sam Bacharach, “GML by OGC to AIXM 5 UGM,” OGC, Feb. 27, 2007.

SWE Components - Languages


CatalogService

SOS

SAS

SPS

Clients

Sensor Observation

Service: Access Sensor

Description and Data

Sensor Planning Service:

Command and Task Sensor

Systems

Sensor Alert Service Dispatch Sensor Alerts to registered Users

Discover Services, Sensors,

Providers, Data

Accessible from various types of

clients from PDAs and Cell Phones to

high end Workstations

Sam Bacharach, “GML by OGC to AIXM 5 UGM,” OGC, Feb. 27, 2007.

SWE Components – Web Services


Semantic Sensor Web


Data

• Raw Phenomenological Data

Semantic Sensor Data-to-Knowledge Architecture

Information

• Entity Metadata

• Feature Metadata

Knowledge

• Object-Event Relations

• Spatiotemporal Associations

• Provenance/Context

Feature Extraction and Entity Detection

Data Storage(Raw Data, XML, RDF)

Semantic Analysis and Query

Sensor Data Collection

Ontologies• Space Ontology

• Time Ontology

• Domain Ontology

SemanticAnnotation


57

Ontology & Rules

• Weather

• Time

• Space

OracleSensorDB

Get Observation

Describe Sensor

Semantic Sensor Observation Service

Collect Sensor Data

BuckeyeTraffic.org

Get Capabilities

Semantic Annotation Service

S-SOS Client

SWE Annotated SWE

HTTP-GET Request

O&M-S or SML-S Response

Semantic Sensor Observation Service


Standards Organizations

OGC Sensor Web Enablement• SensorML

• O&M

• TransducerML

• GeographyML

Web Services• Web Services Description

Language

• REST

National Institute for Standards and Technology

• Semantic Interoperability Community of Practice

• Sensor Standards Harmonization

W3C Semantic Web• Resource Description

Framework

• RDF Schema

• Web Ontology Language

• Semantic Web Rule Language

• SAWSDL*

• SA-REST

• SML-S

• O&M-S

• TML-S

Sensor Ontology

* SAWSDL is now a W3C Recommendation

Sensor Ontology


Current Research Towards STT Relationship Analysis• Modeling Spatial and Temporal data using SW standards (RDF(S))1

– Upper-level ontology integrating thematic and spatial dimensions

– Use Temporal RDF3 to encode temporal properties of relationships

– Demonstrate expressiveness with various query operators built upon thematic contexts

• Graph Pattern queries over spatial and temporal RDF data2

– Extended ORDBMS to store and query spatial and temporal RDF

– User-defined functions for graph pattern queries involving spatial variables and spatial and temporal predicates

– Implementation of temporal RDFS inferencing– Extended SPARQL for STT queries

1. Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS '06), Arlington, VA, November 10 - 11, 2006

2. Matthew Perry, Amit Sheth, Farshad Hakimpour, Prateek Jain. “Supporting Complex Thematic, Spatial and Temporal Queries over Semantic Web Data", Second International Conference on Geospatial Semantics (GeoS ‘07), Mexico City, MX, November 29 – 30, 2007

3. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman. “Temporal RDF”, ESWC 2005: 93-107


Example Domain Ontology


Temporal RDF: Incorporating Temporal Information

Student

Undergraduate Graduate

rdfs:subClassOfrdfs:subClassOf

Student1

rdf:type : [2004, 2008]rdf:type : [2002, 2004]

rdf:type[?, ?]

Temporal InferencingInterval Union: (Student1, rdf:type, Student) : [2002, 2008]

1. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman. “Temporal RDF”. ESWC 2005: 93-107

Associate temporal label with a statement that represents the valid time of the statement

(Student1, rdf:type, Graduate) : [2004, 2008]


E1:Soldier

E2:Soldier

E3:Soldier

E5:Battle

E4:Address

E6:Address

E7:Battle

occurred_at

occurred_at

located_at located_at

lives_at

lives_at

assigned_to

E8:Military_UnitE8:Military_Unit

assigned_to

participates_in

participates_in

Georeferenced Coordinate Space

(Spatial Regions)

Dynamic EntitiesSpatial OccurrentsNamed Places

Contexts Linking Non-Spatial Entities to Spatial Entities

ResidencyBattle Participation

E1:Soldier


Querying in the STT dimensions

• Define a notion of context based on a graph pattern– Query about entities w.r.t. a given context

• Associate spatial region with an entity w.r.t. a context• Associate temporal interval with an entity w.r.t. a context• How are entities related in space and time w.r.t. a given

context


An Example: Battlefield Intelligence

?Person?Symptom

Chemical_X

?Military_Event?Location_1

Enemy_Group_Y

?Location_2

?Enemy

participated_in

has_symptom

induces

located_at

spotted_at

member_of

How close are these locations in space?

How are these eventsrelated in time?

SELECT ?pFROM TABLE(spatial_eval(‘(?p has_symptom ?s)(Chemical_X induces ?s) (?p participated_in ?m)(?m located_at ?l1)’, ‘?l1’, ‘(?e member_of Enemy_Group_y)’); )(?e spotted_at ?l2)’, ‘?l2’, ‘geo_distance(distance=2 unit=mile)’);


SPARQL-ST – Spatio-Temporal SPARQL

Politician_123

Committee_456

District_789

Polygon_1

Linear_Ring_1

NAD83

-85.32 34.1, -85.33 34.2, …, -85.32 34.1

on_committee : [1990, 2000]

represents : [1984, 1992]

located_at : [1990, 2000]

uses_crs : [-∞, + ∞]

exterior : [-∞, + ∞]

lrPosList : [-∞, + ∞]

SELECT ?c, %s, #t1WHERE { <Politician_123> on_committee ?c #t1 . <Politician_123> represents ?d #t2 . ?d located_at %s #t3 }

Maps to single URI

Maps to a set of triplesMaps to a time interval


The Machine Factor

Formal representation of knowledge

– RDF(S), OWL, etc.Statistical analysis

– Similarity– Cooccurrence– Clustering

Intelligent aggregation of knowledge

– Collaboration/Problem Solving Environments– Decision support tools


Putting the man back in Semantics

The Semantic Web focuses on artificial agents

“Web 2.0 is made of people” (Ross Mayfield)

“Web 2.0 is about systems that harness collective intelligence.” (Tim O’Reilly)

The relationship web combines the skills of humans and machines


Putting the man back in Semantics

“Web 2.0 is made of people” (Ross Mayfield)“Web 2.0 is about systems that harness collective intelligence.”

(Tim O’Reilly)The Semantic Web focuses on artificial agentsThe relationship web combines the skills of humans and machines


Going places …

Formal

Social,Informal

Implicit

Powerful

knowledge enabled information and services science semantic web: promising technologies, current...

Documents

data web

web web of pages

services science semantic

knowledge enabled information

web of databases

connotea web

services science ontology

services science evolution