All Hands Meeting 2003

Technical Walkthroughs

Mark James

October 9, 2003

Objectives

Make the BIRN user community aware of the technical accomplishments across the BIRN scientific testbeds

Provide an overview of the tools and processes developed

Create a forum for the exchange of new ideas to be tested over the next several months

Each presenter will cover the following key items:• Overview of what was attempted• Review of technical issues• Results/Future

Smart Atlas

Ilya Zaslavsky• Brain Atlases: from images and sketches to interoperable

spatial databases• Haiyun He, Joshua Tran, Maryann Martone

SMaRT: Spatial Markup and Rendering Tool

Setting up atlas sources

Goal: to enable full set of spatial and attribute queries Converting drawings into GIS formats

• Planar enforcement and geometric corrections• Assigning labels to polygons (a multi-step procedure … an arduous

process…)• Creating coordinate system (in SVG: on the fly; in slice shapefiles:

registering to formally-defined stereotaxic coordinates)• Serving as ArcIMS feature services, or shapefiles

Result: spatial database, where most anatomic features have labels (which can be linked to UMLS Ids), and can be served and queried on the Web using a range of spatial and attribute queries

Converting images• Creating tfw files with image coordinates• Serving images via ArcIMS image services• Warping images as necessary

Automatic labeling results

12

3

4

Color shows number of labels assigned to each polygon

WebStart: http://animal.ucsd.edu:8030/load.html

Displaying CCDB cell locations

Overlaying images and vector markup from different sources (LONI, Paxinos)

Coordinated display: via UMLS bridge

Coordinated display: via coordinates bridge

UMLS-based interconnections

SMARTAtlas

QueryAtlasbonfire

Spatial or attribute selection UMLS Id

UMLS id

UMLS Id concept Id

concept id

Highlighting slices and areas with the concept Id (or its relatives),

doing literature search

QueryAtlas

SMARTAtlasbonfire

User-defined location concept Id

concept id

concept Id UMLS Id

UMLS id

Displaying slices and areas with the UMLS Id (or its relatives)

Towards an Atlas Environment: Mediator

Smart A Query A . . .

Let’s talk how we can connect !

Skull Stripping Project

Christine Fennema-Notestine • UCSD-fMRI• Other collaborators:

Massachusetts General Hospital Brigham & Women's Hospital UCLA – LONI FMRIB (UK)

Skull Stripping Performance

Manually

Stripped

Images

Outcomes from 3 Different Automated Methods

• Bias Correction: Un-corrected vs. N3 bias-corrected datasets (Sled et al., 1998)

• Different T1-weighted pulse sequences: Contemporary Morphometry BIRN protocol vs. Historical SPGR

• Different Diagnostic Groups:

Diagnostic Group

n Pulse

Sequence Mean Age

(sd)

4 PS#1 35.5 (13.5) Young Controls

4 PS#2 33.0 (15.1)

4 PS#1 75.0 (2.2) Elderly Controls

4 PS#2 74.5 (1.7)

4 PS#1 40.5 (13.3) Unipolar Depressed 4 PS#2 40.8 (10.8)

4 PS#1 76.0 (2.7) Alzheimer’s disease 4 PS#2 75.5 (1.7)

Study Conditions

• Manually stripped slices Two anatomists stripped six sections from each dataset

• Brain Extraction Tool (BET)Smith, 2002; Image Analysis Group, FMRIB, Oxford, UK

• 3dIntracranial (in AFNI)Ward, 1999; NIMH/MCW

• Hybrid Watershed algorithm (in FreeSurfer)Segonne et al., in prep; MGH NMR Center

• Brain Surface Extractor (BSE)Based on Shattuck et al., 2001; USC and LONI, UCLA

Methods Studied

• Significant effect of method: BSE & Hybrid Watershed both more similar to manual outcome, relative to 3dIntra and BET.

• Extent of neurodegeneration across subject groups significantly interacted with automated method. BSE & Hybrid Watershed both more robust across diagnostic group.

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

3dIntra BET BSE Hybrid Watershed

Method

Jacc

ard

Sim

ilar

ity

Ind

ex

YNC

DEPR

ENC

AD

Manual Jaccard Index (0.938)

Results to Date

• Interpretation of complex interactions within Jaccard analysis:

Bias Correction Contemporary BIRN protocol vs. Historical SPGR

• Further analyses with an Expectation/Maximization algorithm to better characterize the differences between methods sensitivity (non-brain removed) specificity (brain kept)

Analyses to Complete

Overall Project Goal

To develop a manuscript for publication that will:

• guide end-users towards a method appropriate to their datasets.

• improve efficiency of processing for large, multi-site neuroimaging studies, such as the BIRN.

• provide insight to the developers for the future.

Query Atlas

Greg Brown• BIRN CC, Brigham and Women’s, UCSD Clinical Imaging

Group

The Challenge of Interpretation

To interpret the results of an imaging study scientists must integrate brain location information with neuroscience information

The voluminous and complex neuroscience literature greatly complicates the process of interpretation.

Interpretation: Two crucial steps

• Relate specific brain locations to the relevant neuroscience literature

• Integrate brain location information and neuroscience information to create new knowledge.

Functions of the QueryAtlas

The QueryAtlas is a knowledge management tool that addresses both steps in the interpretive process

QueryAtlas’ primary methodology involves transforming spatial information (i.e. coordinate information) into anatomical labels that in turn are used to query bibliographic databases and specialized brain atlases.

The output is organized into a variety of knowledge structures

Structure of Query Atlas

QueryAtlas is structured to provide a variety of options for:

• Brain visualization

• Anatomic labeling

• Database and atlas querying

• Output Knowledge Representation

Brain Visualization Options

3D Slicer native viewing mode FreeSurfer Pial Surface FreeSurfer Inflated brain FreeSurfer Spherical brain FreeSurfer MNI Normalized Images AFNI original image brick (after transformation to .cor files)

AFNI Talairached image brick (after transformation to .cor files)

Overlaid functional maps

Example of the Challenge

Labeling Options

MGH Parcellation List

Talairach Daemon Anatomical Labels

Talairach Daemon Brodmann Labels

MGH Labels on FreeSurfer Inflated Brain

Databases and Atlases to Query

Click Search 

• Google• Direct PubMed• Arrowsmith augmented search of PubMed• Journal of Neuroscience

  Pop-up Window 

• Swanson Atlases• Bowden Atlas• PsycInfo

Query Page

Output Knowledge Representations

Output of PubMed (AND, OR) searches by date

Arrowsmith Mediated Outline

Arrowsmith Semantic Filter

A Arrowsmith search based on title words in Medline

Morphology of the frontal operculum:A volumetric magnetic resonance imaging study of the pars triangularis. J Neuroimaging 11:153-9, 2001

Initial magnetic resonance imagingvolumetric brain measurements and outcome in schizophrenia: a prospectiveLongitudinal study with 5-year follow-up.Biol Psychiatry 54:608-15, 2003

A

FrontalOperculum

C

SchizophreniaAn unexpected link

Sets of Medline records. A,C are defined to be disjoint. (Adapted from Swanson, 2003)

3 36

B Magnetic Resonance Imaging

e.g. epilepsy

Arrowsmith Output for Frontal Operculum and Schizophrenia Search

Exploring Large Data Sets

Example: KeyholeSeamless transitions across multiple scales – 7 TB DatabaseIntuitive User InterfaceDisplays where data exists and where notIntegration of Imaging and non-imaging dataMultiple ways to find the data of interest (by feature type, name, address, interactive exploration...)

How to Adapt/Expand this to Medical Images?

Query Atlas Interface Project

An interface to interact with the BIRN database as easily as Keyhole interacts with the earth dataDraw on skills of SDF (award-winning information design firm) in genomics/proteomics visualization

Short-term Goals

a. Complete automated access to Talairach Daemon

b. Explore further FreeSurfer’s capability to display functional statistical maps

c. Develop a pop-up window to access NeuroScholar

d. Use the QueryAtlas to query the Smart Atlas through UMLS IDs available to the Smart Atlas. Homologies between rodent and human brain would be demonstrated for a restricted set of brain regions (e.g. basal ganglia).

e. Move QueryAtlas to UCSD BIRN rack.

f. Write a paper.

Long-term Goals

a. Work with the Small Design Firm to improve the integration of Visual and Text information retrieved from QueryAtlas searches

b. Enhance the two-way interface between the Query and Smart Atlases

c. Add portal to BIRN query tools and databases

d. Explore additional collaborations with University of Southern California and Fox/Lancaster groups.

e. Write a manual.

Imaging Database Web Interface

Burak Ozyurt• UCSD-fMRI

Johnathan Sacks• Surgical Planning Laboratory @ Brigham & Women's

Hospital

Features

UCSD Human Imaging Database /UI current features• Generic assessment query building and result navigation

• Search result export for statistical analysis

• Scan and clinical visit view of a selected subject from the query results.

• Export of the MRI image series as AFNI BRIK (DICOM to AFNI conversion on the fly) from SRB.

• Uses Oracle label security for fine-grained database security

Future work• Derived Data/Data Provenance support

• Ontology / Mediator integration

• Data maintenance through Web UI capabilities

• Additional query wizards and report generators

Architecture

UCSD Human Imaging database Web UI architecture

Oracle Label Security

Assessment Selection

Score Selection

Query Criteria Selection

Query Result

Subject Visit Details

Function BIRN Developments

David Keator• UCI

Functional BIRN DevelopmentsFunctional BIRN Developments

Develop and extend BIRN human Develop and extend BIRN human imaging database schema for the FIRST imaging database schema for the FIRST BIRN.BIRN.

Develop a common data interface for Develop a common data interface for FIRST BIRN image formats using XML.FIRST BIRN image formats using XML.

Develop SRB storage hierarchies and Develop SRB storage hierarchies and upload scripts for FIRST BIRN non-upload scripts for FIRST BIRN non-human and human phantom datasets.human and human phantom datasets.

SubjectsExperiment

Visit

Series

MR DataAssociated Data

Scanner Protocol

ExperimentalProtocol

Database StructureDatabase Structure

Derived Data

AnalysisProtocol

Assessments

Protocols

SRB SRB / HID

XML Common Image FormatXML Common Image Format

Image metadata is preserved in translation using XML!

<bxh> <datarec type="image"> <!--AUTOGEN: Orientation is oblique axial --> <dimension type="x"> <units>mm</units> <size>64</size> </dimension> ... <byteorder>lsbfirst</byteorder> <elementtype>int16</elementtype> <filename>MR6C_20020430_03042_005_00_00001.dcm</filename> <fileoffset>9512</fileoffset> ... <magneticfield>1.5</magneticfield> <description>Head,Obl,2D,GRE</description> <fieldofview>240 240</fieldofview> <tr>1999.984000</tr> <te>40.000000</te></bxh>

Functional BIRN QA Script and XML Functional BIRN QA Script and XML – A Practical Example– A Practical Example

BIRN QA Analysis Pipeline

Analyze7.5

XML-awareBIRN QA

INPUTS

PROCESSINGSTEP

QA plots +image dataOUTPUTS

DICOM GE Pfile

A single BIRN QA script now supports all formats using XML interface!

Format X

add XML add XML add XML add XML

Human SRB TestbedHuman SRB Testbed

Research Project

Institution

Subject

Series

AFNISPMk-SpaceNative BRAINS2…....

XML

Visit

Study

Project ID

BIRN_ID, Gender, Age, AgeQualifier

Institution ID

VisitNumber, Age, SCID, AgeQualifier

Series Instance UID SeriesNumber, Sequence, SequenceVariant, AcqPlane, Matrix_X, Matrix_Y,Matrix_Z, PixelSpacing_X, PixelSpaciny_Y, SliceThickness, FlipAngle

StudyID, StationName, Modality, Weighting,Manufacturer, Model, MRAcquisitionTime

Human SRB Testbed and XMLHuman SRB Testbed and XML

Research Project

Institution

Subject

Series

AFNISPMk-SpaceNative BRAINS2…....

XML

Visit

Study

Future DirectionsFuture Directions

Include extensive front-side and back-side error Include extensive front-side and back-side error checking of both non-human and human data checking of both non-human and human data uploads. uploads.

Develop modularized XML infrastructure, storing Develop modularized XML infrastructure, storing important information at each level of the SRB important information at each level of the SRB hierarchy.hierarchy.

Extend software infrastructure to populate subject, Extend software infrastructure to populate subject, study, and visit metadata and modularized XML study, and visit metadata and modularized XML through clinical database queries.through clinical database queries.

Integrate and co-develop graphical user interfaces Integrate and co-develop graphical user interfaces with the Morphometry and BIRN portal groups for with the Morphometry and BIRN portal groups for database entry, SRB queries, SRB uploads, and database entry, SRB queries, SRB uploads, and SRB downloads.SRB downloads.

Integrate upload scripts into the BIRN portal.Integrate upload scripts into the BIRN portal.

LDMM Data in Morph BIRN SRB

Anthony Kolasny• Johns Hopkins University

Johnathan Sacks• Surgical Planning Laboratory @ Brigham & Women's

Hospital

Vt öt

JHU Tools

LDMM

VTK Conversion

Utility

Shape analysis Vt ötMGH Freesurfer

Images anonymized, segmented,

converted to analyze

MGH-JHU-BWH Pipeline 2003

BIRN SRB Database

BWH Slicer

Visualization tool

Goal: Create tools and procedures that will allow the sharing of data among sites.

Collections

Morphometric BIRNSRB collections and and minimal set of attributes (ldmm output)

Study ID

analyze_rawresultssmoothedraw

registered

Raw- Provided to CIS in May 2003

Analyze_raw cpu: 5 minuteshuman: 1 day real: 1 day

Smoothed cpu: minimalhuman: 45 man hours real: about 3 weeks

Processing Time

Registered cpu: minimalhuman: 20 man hours real: about 2 weeks

Results cpu: 48hrs human: minimal real: 48hrs

We received the data in May and the LDMM processing was complete at the end of July

Collections

Metadata

Morphometric BIRNSRB collections and and minimal set of attributes (ldmm output)

ldmm

hippocampusthalamus

deformation_maps inv_deformation_maps

deformed_target

deformed_template

velocity_fields

KiXmaps{1-n}

KiYmaps{1-n}

KiZmaps{1-n}

HXmaps{1-n}

HYmaps{1-n}

HZmaps{1-n}

velocityX{1-n}

velocityY{1-n}

velocityZ{1-n}

defTarget{1-n}

defTemplate{1-n}

...

Target1 Container Metadata TargetN

template

Atlas

...

AtlasN

raw vtk

deformed_target

deformed_template

velocity_fields

deformation_maps

inv_deformation_maps H-maps{1-n}

defTarget{1-n}

defTemplate{1-n}

Ki-maps{1-n}

velocities{1-n}

Template – will be segmented data set that we wish to compare (this data is local to Subject -> Visit -> Study).

filename{1-n} – represents timestep 1 to n. Each timestep has an file assosiated.

TargetN – will be a link to the nth compared target (data from another visit, study or an Atlas).

movies

Container Metadata

Software

Target

Template

Substructure

Alignment

Hippocampus target deformation animation created using Slicer

MIRIAD Project

MIRIAD: Multisite Imaging Research In the Analysis of Depression

Duke/Harvard (BWH)/UCLA mBIRN Collaboration

MIRIAD Group

Duke University Brigham and Women's UCLA

NIRL SPL LONI

Martha Payne Steve Pieper David Rex

James MacFall Kilian Pohl Arthur Toga

Brian Boyd Neil Weisenfeld

Ranga Krishnan Jonathan Sacks

Douglas McQuoid Simon Warfield

Elizabeth Flint Ron Kikinis

Overview/Rationale Duke Study of Major Depression in Late Life

• Hypotheses relate to effect of vascular lesions on brain/limbic structure and function

• Subjects (age ≥ 60 years) have MRIs at recruitment and every 2 years

• Assessment and care are given in the context of a naturalistic setting of standard-of-care

• Controls are recruited from Duke Center for Aging

Data already analyzed with semi-automated segmentation methods

• suitable for cross-sectional analysis

• difficulties in longitudinal analysis due to method variance

MIRIAD collaboration offers promise of BWH and UCLA tools that offer reduced variance and access to atlas-driven lobar and regional analysis

MIRIAD Plan/Data Flow Use Anonymization at Duke to avoid IRB delays

• select 50 depression subjects, baseline and year 2 MRI

• select 50 age-comparable normal subjects, baseline and year 2 MRI

• select metadata variables that will be needed for analysis

• anonymize data retaining a new BIRN number to link MRI and metadata that cannot be traced to original subject

UCLA Processing • atlas preparation and orientation/registration (rigid

body) of all subjects (also compute HO deformation parameters for segmentation)

• registration of BWH atlas to common data set• after BWH segmentation, perform atlas driven

lobar analysis

BWH Processing• EM Segmentation for gray, white, CSF• Atlas driven regional segmentation

The analysis is splendid demonstration of the promise of the BIRN infrastructure for collaborative studies

MIRIAD Plan/Data Flow

DukeArchives

UCLAAIR Registration

and Lobar Analysis

BWHIntensity Normalizationand EM Segmentation

DukeClinical Analysis

MIRIAD Data Flow1. Retrospective data

upload from Duke2. Lobar analysis and

Registration of Atlas to Subjects

3. Anatomical Segmentation

4. Comparison to Clinical History

1

2

3

4

BWH Probabilistic Atlas(one time transfer)

MIRIAD Status/Plans Status

• Scans uploaded to SRB rack at Duke (Project_0002)

• Metadata set created and anonymized (SAS format)

• UCLA and BWH have performed an analysis on two subjects for final verification of the process (10/6/03)

• Image processing expected to be complete by 10/31/03)

Plans • Each site will have lead author on one of the first four papers

"Senior Author" will be Human Brain Morphometry BIRN Methodology Lobar assessments Longitudinal grey and white matter (baseline vs. yr 2) Regions of Interest

• Improved analysis optimized intensity matching prior to segmentation identification/segmentation of vascular lesions

MIRIAD BIRN Suggestions

Just as there are many image formats, there are many metadata formats. BIRN should more intensively develop guidance/scripts for importing metadata

Sometimes the birn concept seems pretty rigid, e.g., the intermediate results we are generating now to tune the MIRIAD project need to be shared conveniently, but they aren't something we want to archive forever. So far the BIRN tools have been awkward to use in this context (because they are also under development), which is why we've relied on web sites and ftp for moving intermediate data.• Maybe we should have a mechanism so that someone from birn-cc would join

groups as they create their process. They could then help the birn-cc to build tools that facilitate specific projects.

SRB Enhancements

Roman Olschanowsky• BIRN-CC

New Features in SRB

Slsf script – reduces the time for issuing the Sls command Schmod – added a flag “-d” for improving the speed for the

assignment of permissions• You must be the person who originally placed the file in SRB

Error checking script – When using batch scripts for processing files with SRB and an error occurs, it will prevent further processing of the script

bulkUpload script - uses containers for fast data uploading and downloading, the containers are located in /container in a mirrored path of where your files are in /home

wipeallContainers script – used for fast and proper removal of files that were bulk uploaded

SRB Enhancements

Vicky Rowley• BIRN-CC

LUFS and SRB

Linux Userland File System extension for mounting SRB space as a local filesystem

Inserts a module in Linux kernel to handle mount command

Runs a daemon, “lufsd” to intercept commands on mounted directory

srbfs_c is an extension of this software

LUFS Usage

Use lufsmount script or the linux mount command to mount SRB space as part of the local filesystem

Can be configured with suid so each user can mount to their own mountpoint with their own permissions

Each mount operation is performed:• as a specific linux user• as a specific SRB user• with specific Unix permissions• to a specific SRB server/port

LUFS Schematic

Linux

SRB

Local file

system

mounted directory

/home/srb/

samba

W2kmac

Network drive

s:/home/srb/…

(exports only /home/srb)

Linux O/S

Mediator Schematic

Mediator Tools

Registration Tool • Displays all the tables and columns of your database that

are available to be exported. • You can optionally export only those table and columns

that you want BIRN users to query.• Training on the tool is provided by BIRN-CC

Vista • Creates integrated views across columns of data or

multiple sources • Training on the tool is provided by the BIRN-CC

Mediator Tools (continued)

Que• Creates customized queries accessible through the BIRN

Portal so that users can launch their queries against data sources.

• BIRN-CC will provide training on the tool.

Wrapper • The wrapper translates the query request from the

mediator into the correct syntax understood by the data source.

• Wrappers have been developed for Oracle and SRB.• Local configuration of the wrapper will be required for all

data sources. • Configuration support will be done by the BIRN-CC.

BIRN Portal - UMLS

Events Calendar

Message Board

Portal Applications Toolkit