Value Domain and Pick List Support in

LexEVS 5.1

Sridhar Dwarkanath

Mayo ClinicDwarkanath.sridhar@mayo.edu

CaBIG Architecture/VCD Joint Workspace F2FAtlanta, GA USA

October 2009

Session Goal

• Familiarize with Value Domain Definition and Pick List Definition part of LexGrid model and

• Their functions supported in LexEVS 5.1 release.

LexEVS – Lexical Enterprise Vocabulary Services

LexEVS

• Developed to allow federated, open source system for delivering vocabulary content.

• Strategic EVS interface, replacing the legacy EVS API and the EVS 3.2 model.

• Key infrastructure tool for caBIG ®.

• Available through caCORE distribution.

• Represents the next generation of NCI Enterprise Vocabulary Services.

LexEVS – Lexical Enterprise Vocabulary Services

LexEVS combines LexGrid and LexEVS Java API

• LexGrid :• A standard model of controlled vocabularies and ontologies. • Defines how vocabularies should be formatted and represented

programmatically.• Rich enough to represent vocabularies provided in numerous source formats

including:• Open Biomedical Ontologies (OBO)

• Web Ontology Language (OWL), e.g., NCI Thesaurus

• Unified Medical Language System (UMLS) Rich Release Format (RRF), e.g., NCI MetaThesaurus

• LexEVS Java API :• Applies LexGrid vision and technologies to requirements of the caBIG®

community.• Provide a flexible implementation for vocabulary storage and persistence.• Provide standard tooling for load and distribution of vocabulary content.

LexEVS – Lexical Enterprise Vocabulary Services

LexEVS includes the following components:

• Java API - A Java interface based on the LexGrid Object Model

• Distributed LexBIG (DLB) API - A Java interface based on the LexGrid data model and relies on a LexEVS Proxy and Distributed LexEVS Adapter to provide remote clients access to the native LexEVS API

• REST/HTTP Interface - Offers an HTTP based query mechanism. Results are returned in either XML or HTML formats

• SOAP/Web Services Interface - Provides a programming language neutral Service-Oriented Architecture (SOA)

• LexEVS 5.0 Grid Service - An interface which uses the caGRID infrastructure to provide access to the native LexEVS API via the caGRID Services

LexEVS – Lexical Enterprise Vocabulary Services

LexEVS Environment Architecture

Diagram showing architecture of LexEVS.

At the bottom, as Data Source, we have lexGrid MySQL database.

Core API layer on top of data source layer contains LexGrid Model Objects, Java APU and the Lucene Index Files.

Application Service layer on top of Core API, contain Distributed Java API and LexEVS CaGrid API.

And at the top, we have Client layer, with Client Application which could include Java QBE (Query By Example), Web/Grid Service (Soap/HTTP/Rest) and Java RMI.

Model Objects

LexEVScaCORESDK APIs

Java API

LexGridMySQL DB

LuceneIndex Files

Distributed Java

LexEVScaGrid

API

Java(QBE)

ApplicationService

ClientWeb/Grid Service(Soap/HTTP/Rest)

Java(RMI)

( Distributed)

Client Application

Core API

Data Source

RMI

LexGrid Model : Value Domain Definition

Diagram containing Value Domain part of LexGrid Logical Model.

Value Domain Definition :

URI should be unique

Versionable

DefinitionEntry

0 or more definitionEntry

Each definitionEntry can reference either code system or value domain or can contain entityReference

Multiple definitionEntries can be combined using local operators ‘OR’, ‘SUBTRACT’ or ‘AND’

entityReference

entityCode is the start concept and a required field.

Other fields are for rules.

• URI should be unique

• Versionable

• 0 or more definitionEntry

• Each definitionEntry can reference either code system or value domain or can contain entityReference

• entityCode is start concept

• referenceAssociation

• Applicable rules :

• leafOnly

• targetToSource

• transitiveClosure

• DefinitionEntries can be combined using logical operators ‘OR’, ‘SUBTRACT’, or ‘AND’

Value Domain Definitions Possible Forms

• Code system/concept code + relationship + additional rules (leaf only, immediate children, ...)

• Code system – all concept codes in the code system

• Other Value Domain – all resolved concept codes returned by referencing Value Domain

• Code system/concept code – individual code

• Combination of any of the above with or/and/difference operators

Value Domain Definition Resolution

• A value domain definition has to be made against a specific version of a code system.

• But it doesn’t have to be resolved against the same version.

• Even a simple list (a, b, c, d) needs to be resolved as, at some future date, “c” might be retired.

• Resolution does not create static artifact.

LexGrid Model : Pick List Definition

Diagram showing Pick List part of LexGrid Logical Model.

PickListId unique

Value Domain URI required

completeDomain – True/False

True – Dynamic, ignores pickListEntryNode

False – Static, default

0 or More pickListEntryNode

Each pickListEntryNode can contain either have pickListEntry or pickListEntryExclusion

PickListEntry :

pickText and entityCode are required fields.

PickListId unique

Value domain URI required

completeDomain – True/False

True – Dynamic, ignores pickListEntryNode.

False – Static, default

versionable

0 or more pickListEntryNode

Each pickListEntryNode can contain either pickListEntry or pickListEntryExclustion

versionable

pickText and entityCode are required fields.

Pick List Definitions Possible Forms

• Value Domain Definition – all concept codes in the value domain

• completeDomain = true

• Dynamic form

• Code system/concept code – individual code (inclusion and exclusion)

• completeDomain = false

• Subset of Value Domain

• Static form

Pick List Resolution

• A Pick List definition has to be made against a specific value domain.

• Even a simple list (a, b, c, d) needs to be resolved as, at some future date, “c” might be retired.

• Resolution does not create static artifact.

LexEVS Value Domain Services

Admin Services

• loadValueDomain(InputStream, boolean) - Loads value domain using inputStream.

• loadValueDomain(String, boolean) - Loads value domain by reading XML file.

• loadValueDomain(ValueDomainDefinition, String) - Loads supplied valueDomainDefinition object.

• removeAllValueDomains() - Removes all value domain definitions from the system.

• removeValueDomain(URI) - Removes supplied value domain definition from the system.

• validate(URI, int) - Perform validation of the value domain definition without loading data.

• dropValueDomainTables() - Drops value domain tables only if there are no value domain and pick list entries.

LexEVS Value Domain Services

Query Services

• getCodingSchemesInValueDomain(URI) - Returns list of coding scheme summary that is referenced by the supplied value domain.

• getValueDomainDefinition(URI) - Returns value domain definition for supplied value domain URI.

• getValueDomainEntitiesForTerm(String, URI, String) - Resolves the value domain supplied and restricts to the term and matchAlgorith supplied.

• listValueDomains(String) - Return the URI's for the value domain definition(s) for the supplied domain name. If the name is null, returns everything. If the name is not null, returns the value domain(s) that have the assigned name.

• resolveValueDomain(URI, AbsoluteCodingSchemeVersionReferenceList) - Resolve a value domain using the supplied set of coding scheme versions.

LexEVS Value Domain Services

Query Services (Cont…)

• isConceptInDomain(String, URI) - Determine if the supplied entity code is a valid result for the supplied value domain and, if it is, return the particular codingSchemeVersion that was used.

• isConceptInDomain(String, URI, CodingSchemeVersionOrTag, URI) - Determine if the supplied entity code is a valid result for the supplied domain.

• isDomain(String, String, CodingSchemeVersionOrTag) - Determine if the supplied entity code is of type valueDomain in supplied coding scheme and, if it is, return the true, otherwise return false.

• isSubDomain(URI, URI) - Check whether childValueDomainURI is a child of parentValueDomainURI.

LexEVS Pick List Services

Admin Services

• loadPickList(InputStream, boolean) - Loads pick list using inputStream.

• loadPickList(PickListDefinition, URI) - Loads supplied PickListDefinition object.

• loadPickList(String, boolean) - Loads pick list by reading XML file location supplied.

• removePickList(String) - Removes pick list definition from the system that matches supplied pickListId.

• validate(URI, int) - Perform validation of the picklist definition without loading data.

LexEVS Pick List Services

Query Services• getPickListDefinitionById(String) - Returns pickList definition for supplied pickListId

• getPickListDefinitionsForDomain(URI) - Returns all the pickList definitions that represents supplied valueDomain URI.

• getPickListValueDomain(String) - Returns an URI of the represented valueDomain of the pickList.

• listPickListIds() - Returns list of pickListIds that are available in the system.

• resolvePickList(String, boolean) - Resolves pickList definition for supplied pickListId.

• resolvePickListForTerm(String, String, String, String, String[], boolean) - Resolves pickList definition for supplied arguments - Term to restrict, match algorithm to use, language to restrict, list of context to restrict, sort flag (If True, will be sorted by text in ascending order.)

LexEVS 5.0 vs LexEVS 5.1

• LexEVS 5.0 Release

• Current Release

• Value Domain and Pick List services available as an extension

• LexEVS 5.1 Release

• November 2009

• Integrated within LexEVS API

Value Domain and Pick List Services in LexEVS

Both Value Domain and Pick List Services will be available in following components of LexEVS 5.1 :

• Java API - A Java interface based on the LexGrid 5.0 Object Model

• Distributed LexBIG (DLB) API - A Java interface based on the LexGrid 2009/01 data model and relies on a LexEVS Proxy and Distributed LexEVS Adapter to provide remote clients access to the native LexEVS API

Value Domain and Pick List Services in LexEVS

Value Domain and Pick List Services available in LexEVS EnvironmentDiagram showing architecture of LexEVS.

At the bottom, as Data Source, we have lexGrid MySQL database.

Core API layer on top of data source layer contains LexGrid Model Objects, Java APU and the Lucene Index Files.

Application Service layer on top of Core API, contain Distributed Java API and LexEVS CaGrid API.

And at the top, we have Client layer, with Client Application which could include Java QBE (Query By Example), Web/Grid Service (Soap/HTTP/Rest) and Java RMI.

Model Objects

LexEVScaCORESDK APIs

Java API

LexGridMySQL DB

LuceneIndex Files

Distributed Java

LexEVScaGrid

API

Java(QBE)

ApplicationService

ClientWeb/Grid Service(Soap/HTTP/Rest)

Java(RMI)

( Distributed)

Client Application

Core API

Data Source

RMI

Future Plans

Future plans for Value Domain and Pick List in LexEVS

• Persist resolved data.

• Include Value Domain and Pick List as part of following LexEVS components :• REST/HTTP Interface - Offers an HTTP based query mechanism. Results are

returned in either XML or HTML formats

• SOAP/Web Services Interface - Provides a programming language neutral Service-Oriented Architecture (SOA)

• LexEVS Grid Service - An interface which uses the caGRID infrastructure to provide access to the native LexEVS API via the caGRID Services

More Information

• caBIG® Vocabulary Knowledge Center :

• https://cabig-kc.nci.nih.gov/Vocab/KC/index.php/LexBig_and_LexEVS • Download Software• Documentation and Training• Presentations and Demos• LexGrid and LexBIG UML Diagram• LexGrid and LexBIG XML Schema• Sample Database

• caBIG LexEVS Developer Boot Camp :

• November 17th, 18th, and 19th, 2009

Questions ?

Questions ?