INCF Infrastructure for Interoperable Atlases of Rodent Brain

Ilya ZaslavskySan Diego Supercomputer Center University of California San Diego

OUTLINE

1. What is INCF-DAI

2. A system of distributed atlas hubs and services

Anatomy of a service

Spatial transformations

3. INCF Central

4. The current status: introduction to the demo

5. How do you join the INCF atlasing network

6. Summary, current limitations, future work

Purpose of this INCF program: To enhance the interoperability, accessibility, and sharing of spatial data sets in neuroscience: INCF-sponsored standards

The Integration Problem: Whose standard?

What are we trying to do?

• We have many individual fragmented efforts, each using separate semantic and spatial frameworks

• Each is a research effort, i.e. constantly changing, imperfect, not necessarily designed for integration

• Our focus is not on perfecting tools, but integrating them into an interoperable system, by creating an INCF-DAI architecture and standardizing data exchange

• This is an iterative process: integration experiment -> finding “weakest links” -> prioritizing tool improvement -> new level of integration, with enhanced integration models…

1. What is INCF-DAI

2. A system of distributed atlas hubs and services

Anatomy of a service

Spatial transformations

3. INCF Central

4. The current status: introduction to the demo

5. How do you join the INCF atlasing network

6. Summary, current limitations, future work

INCF Central, and INCF Services and Hubs

INCF Central:

Service registry,Registry of

spaces,Metadata catalog

ABA Service

WHSService

UCSDService

EMAGEService

Other…

Other…

Atlas Services

ABA Service WHS Service UCSD Service EMAGE Service

Core Services

GetCapabilities, DescribeProcess

GetCapabilities, DescribeProcess

GetCapabilities, DescribeProcess

GetCapabilities, DescribeProcess

Known coordinate spaces

ABAvoxel, AGEA, ABAreference

WHS Paxinos EMAP

Lookup GetStructureNameByPOI

GetStructureNameByPOI

GetStructureNameByPOI

GetStructureNameByPOI

Coordinate translations

WHS AGEA,ABAreference ABAvoxel,AGEA ABAvoxel

Paxinos WHS EMAP WHS

Other, based on POI

Get2DImageAtPOI, GetCorrelationMap, GetGenesByPOI

Get2DImageAtPOI Get2DImageAtPOI, GetGenesByPOI

Based on structure

GetGenesByStructure GetGenesByStructure

Service = some functionality that you can call over the web in a standard manner

Internals of a service – demo implementation(with ABAServiceController as an Example)

• Service definition resides at the INCF server (as with all current services)

• Some methods are implemented locally: core methods, all space translations (with mandatory bridge to WHS), also structure lookup by POI

• Other methods wrap ABA API calls: GetCorrelationMap, Get2DImageatPOI, GetGenesAtPOI, GetGenesByStructure

• The services return either HTML (currently), or XML/txt

• GetCapabilities and DescribeProcess comply with OGC’s Web Processing Service (WPS) specification

API: Application Programming Interface

Some examples of service signatures

• Coordinate transformation:– http://132.239.131.188:8080/incf-services/service/ABAServiceController?request

=SpaceTransformation&FromSRSCode=WHS&ToSRSCode=AGEA&x=X&y=Y&z=Z&output=txt

• Structure name lookup:– http://132.239.131.188:8080/incf-services/service/ABAServiceController?request

=GetStructureNameByPOI&SRScode=ABAVoxel&x=X&y=Y&z=Z&filter=structures:anatomic&output=txt

• Find genes:– http://132.239.131.188:8080/incf-services/service/EmageServiceController?requ

est=GetGenesByStructure&vocabulary=ABAvoxel&term=term&filter=“structures:anatomic” &output=html

Services can be chained (i.e. output of one service becomes input of another) to implement different research workflows.

We attempted to make the service signatures as common across different atlas hubs as possible

The Central Role of Waxholm Space

Different types of transformations

• ABAvoxel to WHS– Large volumes for transforms in both directions, between 3D and 3D of different dimensions– Then simple lookup in the transformation volume

• ABAvoxel to ABAreference– Collection of conversion formulas for individual slices

• ABAvoxel to AGEA– Simple scaling

• WHS to Paxinos Mouse Atlas– Warping appropriate WHS cuts to match with Paxinos slices; translations per slice and brain

region

Transformation service issues (looking at WHS to Paxinos as example)

WHS is a voxel space, containing continuous information of a particular mouse’s brain, obtained elegantly from an MRI imaging. Paxinos is derived from a mouse brain sample, physically sliced into cross-sections and observed visually:

– Different animals, and different shape of the brain– There is no Paxinos information between two adjacent slices– Some Paxinos slides are “out of step” with the adjacent slides– Many Paxinos boundaries are estimated and not consistent with one another, particularly in the cerebellum.

Method:– Taking cross-sections of the MRI image, projecting them over Paxinos slides. – The human eye and brain is still a very powerful data processor. By looking at a WHS section over a

Paxinos slide, one can see what adjustments are needed: a nudge to the left, a twist clockwise, or a dilation in one direction. Not only can the shape of the brain itself be used, but the light-and-dark patches in the MRI match with Paxinos brain regions. Eventually, each Paxinos slide can match very closely with a WHS cross-section.

– Once these cross-sections are found, a clear mathematical correlation in the data gives a formula from one space to another.

Additional problems:– It is virtually impossible to make a perfect match, or ascertain accuracy– The anterior commisure region in WHS mouse happened to be located slightly higher than that in the

Paxinos mouse. The discrepancy distance is about one diameter. A query in the WHS ac would be likely to miss the Paxinos ac.

– The Olfactory Bulb is another difficult region; its cone shape is susceptible to deformation and has different relative dimensions not only in WHS and Paxinos, but in other spaces such as ABA as well.

– In some slides, while the internal regions match up very well, the outside extent of the cerebral cortex fails to match.

1. What is INCF-DAI

2. A system of distributed atlas hubs and services

Anatomy of a service

Spatial transformations

3. INCF Central

4. The current status: introduction to the demo

5. How do you join the INCF atlasing network

6. Summary, current limitations, future work

INCF hubs and central: extended

INCF Central: Registry of Atlas Spaces

• INCF role may play similar role to EPSG, the authority in coordinate systems – EPSG codes (e.g. EPSG:4326

for WGS84) are ubiquitous in GIS software. EPSG:900913 is the coordinate system used in Google Maps, Yahoo Maps, Virtual Earth.

• INCF may maintainits own registry ofatlas coordinatesystems (we have aprototype)

Other INCF Central Registries

• Service registry for atlas hubs (GetCapability requests)

• Spatial transformations registry, from all atlas hubs

• Ontology registry and repository (PONS)• Additional metadata: replicas of local

metadata catalogs and spatial registries, segmentations, etc.

Managing different data types in INCF-DAI

1. What is INCF-DAI2. A system of distributed atlas hubs and

servicesAnatomy of a serviceSpatial transformations

3. INCF Central4. The current status: introduction to the

prototype5. How do you join the INCF atlasing network6. Summary, current limitations, ongoing work

Demo ContributorsNeuroscience scenarios: Maryann Martone, Stephen Larson (UCSD), Jyl Boline

(Informed Minds), Lydia Ng and Mike Hawrylycz (Allen Institute), Al Johnson (Duke University)

WHS dataset: Al Johnson and Jeff Brandenburg (Duke University), Jonathan Nissanov and Pablo Burstein (Drexel University)

Other datasets and meshes: Maryann Martone and Stephen Larson (UCSD), Lydia Ng and Chris Lau (Allen Institute)

INCF Central and atlas service wrappers: Asif Memon (UCSD)

INCF hubs and local services: Lydia Ng (Allen Institute), Albert Burger (MRC, UK), Kenneth McLeod (Heriot-Watt Univ, UK), Asif Memon (UCSD)

Spatial transformations and anatomic structure lookup: Lydia Ng (Allen Institute), Steven Lamont and Alexander Young (UCSD)

Representation of coordinate spaces: Alexander Young (UCSD)

Client development: Stephen Larson and UCSD team; Seth Ruffins and UCLA team

Hardware and server setup: Larry Lui (UCSD)

Architecture: Albert, Fons, Jyl, Mike, Janis, Ilya

Demo overview: with WholeBrainCatalog

Testing transformation chains: from Paxinos Reference Plates to ABA Reference Plates via WHS and AGEA/ABA

Initial point: (1.0, 4.3, 1.78)

Anterior to Bregma at 1.78 mm, Fig. 16

Structure= AC (anterior commissure)

Testing, Step 1: Paxinos to WHS

The original

Transform to WHS using transformation service we built:http://incf-dev-mapserver.crbs.ucsd.edu:8080/incf-services/service/UCSDServiceController?request=SpaceTransformation&fromSRSCode=paxinos&toSRSCode=whs&x=1.0&y=4.3&z=1.78&output=html

Result = 308,642,224

WHS coronal cut WHS coronal cut fitted with Paxinos plate

Testing, Step 2: WHS to AGEA

The original

Transform to AGEA using Steve’s lookup over Lydia’s conversion matrix, wrapped in Asif’s service:http://incf-dev-mapserver.crbs.ucsd.edu:8080/incf-services/service/ABAServiceController?request=SpaceTransformation&fromSRSCode=whs&toSRSCode=AGEA&x=308&y=642&z=224&output=html

Result: 3825,5650,4650

http://tirebiter.ucsd.edu/cgi-bin/get_plane.cgi?atlas=whs&view=Y&x=308&y=642&z=224

http://tirebiter.ucsd.edu/cgi-bin/get_plane.cgi?atlas=whs&view=Z&x=308&y=642&z=224

WHS views

Looking at the result in AGEA:

http://mouse.brain-map.org/agea/all_coronal?correlation&seedPoint=3825,5650,4650

Testing, Step 2: WHS to AGEA;Results in AGEA

The original

http://mouse.brain-map.org/agea/all_coronal?correlation&seedPoint=3825,5650,4650

Wrapped in GetCorrelationMap service: http://incf-dev-mapserver.crbs.ucsd.edu:8080/incf-services/service/ABAServiceController?request=GetCorrelationMap&SRSCode=whs&x=308&y=642&z=224&filter=maptype:coronal&output=html

Testing, Steps 3 and 4: AGEA to ABA volume to ABA reference plates

The original

Transform to ABA (i.e. divide by 25):http://incf-dev-mapserver.crbs.ucsd.edu:8080/incf-services/service/ABAServiceController?request=SpaceTransformation&fromSRSCode=agea&toSRSCode=abavoxel&x=3825&y=5650&z=4650&output=html Result: 153,226,186

Transform to ABA reference plate coordinates, using Lydia’s formula implemented in Asif’s service:http://incf-dev-mapserver.crbs.ucsd.edu:8080/incf-services/service/ABAServiceController?request=SpaceTransformation&fromSRSCode=abavoxel&toSRSCode=abareference&x=153&y=226&z=186&output=html

Result: 1.194, 5.127,1.693

Check the result in ABA reference atlas at http://mouse.brain-map.org/atlas/ARA/Coronal/browser.html(see http://mouse.brain-map.org/viewImage.do?imageId=130973 ) – we are in Coronal level 38, as predicted!

The results, againThe original Paxinos

Placing a probe in the 3D atlas viewer

Wholebraincatalog.org

Lateral view, with the probe in hippocampus

Calling ABA services

Calling EMAGE services

Calling WHS services

Calling UCSD (Smart Atlas) services

Response from UCSD Get2DImageAtPOI

Lateral view, rotated

Responses from various ABA requests

Fine Structure Name: DGAnatomic Structure Name:HIP

WHC GetStructureNameByPOI: Hc

Responses from various ABA requests - 2

Response from EMAGE GetGenesByStructure

1. What is INCF-DAI

2. A system of distributed atlas hubs and servicesAnatomy of a service

Spatial transformations

3. INCF Central

4. The current status: introduction to the demo

5. How do you join the INCF atlasing network

6. Summary, current limitations, future work

Requirements• Flexibility

– Individual atlases may use different representations and data types– Atlases may support different functions– Atlases may adhere to known (already registered) or unique (but

defined) spatial and semantic frameworks– There may be different amount of funding available to bring the atlas in

compliance with INCF-DAI expectations

• A range of approaches:– Register a GetCapabilities service, then INCF Central will harvest the

rest of the metadata (implying that local coordinate system(s) and semantics are formally described and available via services): we need to provide sample software stack. This would be an ideal scenario, though need a monitoring infrastructure

– Work with INCF on a hybrid service, where some functions are hosted at the atlas hub, and other are created at INCF (e.g. space descriptions and WHS transformations)

– Have INCF host entire atlas

1. What is INCF-DAI

2. A system of distributed atlas hubs and services

Anatomy of a serviceSpatial transformations

3. INCF Central

4. The current status: introduction to the demo

5. How do you join the INCF atlasing network

6. Summary, current limitations, future work

Summary

• We developed concepts of INCF-DAI, atlas hubs and services, INCF Central server, and communications between them

• We built service-oriented architecture prototype for distributed digital atlases of mouse brain, that relies, where possible:– On standard service descriptions and exchange schemas– On standardized spaces, and a registry of spaces– On a collection of coordinate translation services, with WHS at the center– On standard terminology, with term cross-walks where needed

• The translation services have performed well, and several tools were built for testing their performance

• Data from several atlas hubs can be spatially integrated for the first time, via services and service chains

• The integration has been demonstrated with a novel3D Atlas Integration application

• The system is extensible

Limitations

• There are many other ways to link data to tell a neuroscience story– E.g. get genes from ABA, and check where these genes are expressed in EMAGE ( but not the

focus of the prototype)

• Spatial integration methods:– From coordinate-based and anatomic feature-based, to integration by spatial placement rules

• WHS meshes are “fitted” into ABA volume space, for now– Because otherwise need to adjust 2D images in the view, and perhaps have their warped copies

• Spatial selection– Now by POI; in the future: by ROIs, transects, trajectories (e.g. along neuronal paths)

• The client does not yet integrate data returned by spatial requests:– It is still Alpha; client integration is planned for the complete version (for 2D images, in particular)– XML schemas are not yet standardized for must queries

Future work• Additional atlasing hubs• Additional data types and associated standard

services (including simulation)• XML schemas for the services• Further coordinate space standardization• Additional reference spaces and transformations

(EMAP, in particular)• A registration and metadata harvesting system• A compelling neuroscience story• Working closely with the ontology and metadata

TFs on APIs and central registries• Versioning of INCF-DAI

NeuroHub Web Service

Atlas Web Service

Neuro Catalog Service

NIF Ontology Service

SRS and transformations Service

2D and 3D Map

Services

Authorization Service

Other services

Hub services

Hub

SRS

Fiducials and Transformations Page

List of hub services - 1

1. Core capability descriptions: GetCapabilities and DescribeProcess (WPS-based).

2. Basic information about the content of the services, harvested by INCF central server: ListGenes, ListStructures, etc.

3. Descriptions of available spatial reference systems: ListSRSs, DescribeSRS (also, possibly, ListStructures and DescribeStructure, ListFiducials and DescribeFiducial, etc.). These methods are expected to be supported by an atlas hub, even if the locally-used Spatial Reference System (SRS) is not unique for the hub. The ListSRSs and DescribeSRS should be also supported at the central server.

4. Spatial transformations: ListTransformations, DescribeTransformation, GetTransformationChain, TransformPOI. These methods should be supported by the INCF central server and by those hubs that rely on a local coordinate system

List of hub services - 2

5. Structure Lookup: GetStructureNamesByPOI. This method should support structure lookup for canonical set of segmentations of each atlas. If there are multiple versions of the segmentations by the maintainer of the canonical space, the latest version is returned (unless another one is requested explicitly).

6. Other POI-based methods. Most of these methods are likely to be implemented at just one or a handful of hubs. They include: Get2DImagesByPOI; GetCorrelationMapByPOI; GetGenesByPOI.

7. Other methods, including those based on structure names, e.g. GetGenesByStructure. May be the only way to connect to atlases for which spatial transformations have not been or cannot be developed, but ontology cross-walks exist.

8. Registration, monitoring and subscription/notification services.

Hub deployment model

General Service Request

http://<server-path>/HostServiceController?Service=WPS&version=1.0.0&request=<WPS_Request>&Identifier=<identifier_name>& ResponseForm ={format}&DataInputs={Encoded Inputs}

Web Processing Service 1.0.0; Waxholm Markup Language

BaseServiceUrl:: http://<server-path>/HostServiceController?Service=WPS

{BaseServiceUrl}&request=GetCapabilities

{BaseServiceUrl}&version=1.0.0&request=DescribeProcess&Identifier={ProcessToDescribe}

{BaseService}request=Execute&Identifier={ProcessToExecute}

Service Conventions

• References: PropertyId; PropertyCode; PropertyName, PropertyDescription; also: PropertyURL

• Method names:– List: list objects without a query. These requests should support paging

capabilities, and enable harvesting.– Describe: returns a single object using a persistent identifier– Get: returns one or more objects using a submitted query, or a pointer to a

service

• Prefixed by hub and SRS/vocabulary, e.g. ABA:ABAVoxel:Hip

• Support for paging; authentication/authorization (at which level?)

• Services for metadata and data?

Service descriptions (for atlas services)

• GetCapabilities• DescribeProcess• Execute identifiers:

– ListSRSs– DescribeSRS(SRSCode)…. Also, ListStructures, ListFiducials, etc.– ListTransformations– DescribeTransformation(transformationCode)– GetTransformationChain(fromSRS,toSRS,filter)– TransformPOI(POI,toSRS)– GetStructureNamesByPOI(POI, outputVocabulary, filter)

• (shall it be Feature or Structure?)– Get2DImagesByPOI (POI)– GetMap/GetCoverage (standard requests in WMS and WCS)– Also: GetCorrelationMapByPOI, GetGenesByPOI, etc.

Additional requests for generic INCF hubs and INCF Central

• ListHubs• DescribeHub (HubCode)• ListHubServices• DescribeHubService (HubServiceCode)• RegisterHubService, HarvestHubService,

GetHubServiceStatus, GetCatalogUpdates

Each HubService is expected to have its own capabilities method

Examples from spec{BaseService}request=Execute&Identifier=GetTransformationChain&DataInputs=inputSrsCode={Code};targetSrsCode={Code};filter={string}

Examples from spec{BaseService}request=Execute&identifier=GetStructureNamesByPOI&DataInputs=srsCode=ABAVoxel;x=295.0;y=113.0;z=146.0;vocabulary=ABAVoxel;filter=structureset:Fine

Examples from spec{BaseService}request=Execute&Identifier=TransformPOI&DataInputs=inputSRSCode=WHS;targetSRSCode=paxinos;x=280.0;y=112.0&z=162.0&filter=

Issues

• Scope and structure of WaxML, for short and long term

• WaxML specification for typical atlas responses (GetGenes? Get2dImageCollection? Others)

• How we separate data discovery from retrieval (what is metadata)

• Scope and structure of atlas service stack vs other services

• Ultimately – the issue of data models (for spaces, transformations, hubs, etc.); exposing them as API, and populating them via a community process this is how the meeting is structured

Next steps with the specification (optimistic)

• Initial development and editing, with contributions from DAI members and INCF research staff (to be completed within 1 month after the meeting - by May 1);

• Pilot implementations demonstrating the services (ongoing)

• The document is posted on the INCF site, with a two-month RFC time (May-June)

• Responses to the comments and discussion within DAI (July-August)

• The specification is presented for a vote, as recommended standard, by the III Neuroscience Congress in Kobe (end of August – September).