On Brain Atlases and GIS

Infrastructure for spatial data integration: across scales, species and projects

Ilya ZaslavskySDSC, BIRN-CC, Mouse-BIRN

Outline

• Atlas functionality• Spatial data integration

– Mouse brain atlas work– Spatial information infrastructure– Uses of ontology in spatial data

integration:• Ontology-based feature identification• Ontology-based location exchange

– Research portals for spatial data integration

• How BIRN atlas work influenced other cyberinfrastructure projects– STARE (ophthalmology)– GEON, CUAHSI, NIEHS Hurricane

Response portal

What is an atlas? a set of 2D images or a 3D volume in a specified

coordinate system, with anatomic feature delineations, and a set of spatially registered annotations

What we need is more than an atlas:• A gateway to large distributed databases of images,

volumes, segmentations, other spatially-registered data• Collection of spatial data sources• Collection of spatial data registries, services and

workflows: image registration, segmentation, spatial selection, spatial

analysis, integration of spatial data

• Collection of viewers and annotation tools with authenticated access

And APIs to make them all work together

SMaRT: Spatial Markup and Rendering Tool

Spatial Queries on Brain Structures, including 3D queries

The Smart Atlas: Spatial Integration over Distributed Data

WebStart from BIRN portal

Spatial integration of distributed multiscale data

The new version of the S.M.A.R.T. Brain Atlas is a MapObjects-Javabased Web applicationfor ontology-awarediscovery andintegration of distributed multiscale braindata registeredto the commonstereotaxic coordinate system.

GIS Challenges

Multiple collections of brain images/ atlases, but lack of interoperability across them• At the database and catalog levels• At the interface level

Queries such as: “which atlas sources provide images for user-selected area of the brain at

a given resolution”, or “display image segmentations available in one atlas over a newly obtained

image stored in a different location on the grid”, or “compare protein distributions in a given area of the brain available at

several atlas sources”

cannot be answered without a dedicated infrastructure that supports spatial integration of brain data

Heterogeneity in: image formats – conventions – spatial registration – access, retrieval and display mechanisms

The SMART Atlas is the prototype infrastructure to answer this challenge

Spatial sources and requests

Smart Atlas client

ArcIMS

MediatorSpatial Registry

SRB-controlled grid sources

image services feature service

ArcIMS

121

2

Smart Atlas client

ArcIMS

MediatorSpatial Registry

SRB-controlled grid sources

image services feature service

ArcIMS

121

2

1: getSources ({slice_plane_params, 2Dshape}, sp_rel)

2: getFragment(URI,plane_params,2Dshape,image_dims)

Interesting Issues

Use of Ontologies to Link Features

Structures on slices color coded by relationships contained in the UMLS

SMART Atlas usesUnified Medical Language System (UMLS) to query across multiple data sources and explore spatial relationships across brainslices indifferent coordinate systems (eg,across species)

Use of ontologies to delineate anatomic features

12

3

4

Color shows number of labels assigned to each polygon

Special Cases: 4 ns: nigrostriatal bundle al: ansa lenticularis,

C0152352 mfb: medial forebrain

bundle, C0025055 MCLH: magnocellular

nucleus of the lateral hypothalamus

LH: lateral hypothalamic area, C0020654 Capital letters designate nuclei (typically)

ANDdotted lines typically show nuclei

Fiber tracks: small letters and solid lines

mfb and LH are siblings in UMLS; a bundle of neurons (mfb) “runs through” a nucleus (LH) the polygon is assigned “LH”, with “mfb” as a secondary (fiber bundle) designation

Same picture with “al” vs “MCLH”

MBAT Programmers Meeting

From MBAT to WOMBAT

Web-based Ontology-based

Architecture

Basic data typeabstraction (orphysical databases)

Sources

Gene expression2D images2D vector segmentations3D volumes4+D volumes (FMRI)Time seriesPhenotype/behavioralsurfaces

Methods over the abstract representation –Shall have commonsignatures

getExpressed(gene, feature)translate (coords)convert (formats)

registries

Clients: MBAT etc.

Data integration tools

Interoperability desiderata

Read data from other atlases/databases

API for data retrieval and transformation

Find relevant data in other atlases/ databases

API for atlas catalogs

View the region of interest in another atlas

API for atlas state exchange

MBAT Drexel’s NTUCLA’s ShivaSmart Atlas

GensatAllen Brain AtlasCCDB, others

Spatial registry

Exchanging location information across scales and coordinate systems

Directly:• By space, using Stereotaxic coordinates

Formal coordinate system definitions Coordinate system translation services (incl. between image and

stareotaxic) Spatial registration of images and segmentations

• By anatomic feature names, via UMLS + Bonfire + Neuronames + … Ontological registration Semantic translation services

Indirectly• By verbal description of spatial locations relative to well-

defined features (a collection of location rules) • In relative coordinates; by matching derived indices:

histograms, shape indexes, etc.

Agreed-upon ROI description and state exchange protocol

STATE_INIT::

<atlas_state> <request name= "initialize_session">     <user>user</user>     <pwd>pwd</pwd>

 </request></atlas_state>

 <atlas_state>

 <response name= "initialize_session">    <sessionID>10009 </sessionID>

 </response></atlas_state>

  

STATE_SUBMIT

 <atlas_state> <request name="submit">    <sessionID>10009</sessionID>    <submitting_atlas>smart</submitting_atlas>

    <action name="highlight|...">       <locations>          <location coordsys="Stereotaxic">              <format type="pointlist3D">                     <ROI>                             <bbox>m1,n1,p1 m2,n2,p2</bbox>                             <data>x1,y1,z1 x2,y2,z2 ...</data>                     </ROI>                     <ROI>                              <bbox>m1,n1,p1 m2,n2,p2</bbox>                              <data>x1,y1,z1 x2,y2,z2 ...</data>                     </ROI>              </format>           </location>       </locations>     </action>   </request></atlas_state>

 

<atlas_state> <response name="submit">      <status>success</status> </response></atlas_state> 

STATE_RETRIEVE

 <atlas_state> <request name="retrieve">  <sessionID>10009</sessionID>  <targetatlas>shiva</targetatlas>  <targetterm ontology="UMLS|..."/>  <targetlocation coordsys="Stereotaxic" format="pointlist3D"/> </request></atlas_state>

<atlas_state > <response name="retrieve"> <submitting_atlas>smart</submitting_atlas> <action name="highlight"> <locations> <location coordsys="Stereotaxic">  <format type="pointlist3D">   <ROI>    <bbox>m1,n1,p1 m2,n2,p2</bbox>    <data>x1,y1,z1 x2,y2,z2 ...</data>   </ROI>   </ROI>   <ROI>    <bbox>m1,n1,p1 m2,n2,p2</bbox>    <data>x1,y1,z1 x2,y2,z2 ...</data>   </ROI>  </format> </location> </locations> </action> </response></atlas_state>

+ Java API

Integration across clients: ROI exchange API in action

INITIALIZESERVICE

SUBMITSTATE

RETRIEVESTATE

Position Synchronization

Accessing spatial registration sources from each viewer

Image registration workflow • Ability to plug in image registration/warping tools from

LONI, Drexel, UCSD, Ohio

Spatial registry at BIRN-CC:• Oracle Spatial database; contains:

Service URI (SRB:path, or ArcIMS:URL,service,layer) Coordinate system of the layer Plane to which the image or markup belongs: a,b,c,d wrt stereotaxic

(dorsal/ventral to bregma; anterior/posterior to bregma; lateral to midline) Positioning of image within the plane (6 elements of TFW) Image contour or feature extent as SDO Signal type, and link to additional signal metadata Additionally:

• to simplify search for non-oblique slices: slice types, associated numeric constant

• for visualization: thumbnail paths

Image Registration Portlet

Automatic generation of spatially-registered image services

Image Registration Portlet: Spatial Registry

Impact

ESRI Health Sciences Brochure

Presented at several GIS conferences: keynote at International Health GIS, 2004;Part of ESRI’s President’s address at GIS User Conference (2003), other…

The STARE (Structural

Analysis of the Retina)

project, UCSD

I

II

InferiorInferiorHemisphereHemisphere

PapMac180° 0°

I

II

InferiorInferiorHemisphereHemisphere

PapMac180° 0°

Foveal Coordinate System

Background

Coordinate System

Veins

Arteries

Optic nerve

Fovea

Cotton-Wool

Hemorrhages

SAN DIEGO SUPERCOMPUTER CENTER, UCSDSciR&D

BIRN Grid architecture in other projects

CI is about RE-USING data and research resources !!

Drinking water surface intakes and TRI facilities

Now at: http://www-apps.niehs.nih.gov/katrina/

TRI facilities within 3 miles of user-entered address

TRI facilities within a mile of drinking water intakes

Environmental justice applications: social and demographic variables overlaid with environmental effects

Data download and metadata are available for all spatial layers