7/12/2004 jhu/iacl jerry l. prince image analysis and communications laboratory dept. of electrical...

JHU/IACL 7/12/2004

Jerry L. Prince

Image Analysis and Communications Laboratory

Dept. of Electrical and Computer Engineering

Johns Hopkins University

Cortical Surface Segmentation and Topology

7/12/2004JHU/IACL

Acknowledgments

• Chenyang Xu• Dzung Pham• Xiao Han• Duygu Tosun• Bai Ying• Daphne Yu• Kirsten Behnke• Xiaodong Tao

• Susan Resnick• Mike Kraut• Maryam Rettmann• Christos Davatzikos• Nick Bryan• Aaron Carass• Ulisses Braga-Neto

Funding sources: NSF, NIH/NINDS, NIH/NIA

7/12/2004JHU/IACL

Outline

• Introduction

• Fuzzy Classification

• Nested Surface Segmentation

• Spherical Mapping and Partial Inflation

• Sulcal Segmentation

• Applications

7/12/2004JHU/IACL

Brain Cortex Reconstruction

Magnetic Resonance Images (MRI)

Cortical Surface

7/12/2004JHU/IACL

• Study geometry of cortex– relation to function

– changes in aging and disease

• Use in function mapping– EEG/MEG/PET signals

– localization on surface instead of volume

• Surgical planning– Automatic labels

– geometric plan

Why Cortex Reconstruction?

Extracranial Tissue

Cerebrospinal Fluid (CSF)Gray Matter (GM)

Outer Pial Surface

Central SurfaceInner WM/GM Surface

White Matter (WM)

7/12/2004JHU/IACL

Nested SurfacesInner

Central Outer

7/12/2004JHU/IACL

Some Difficulties

• Highly convoluted cortical folds Highly convoluted cortical folds • Image noiseImage noise • Image intensity inhomogeneity Image intensity inhomogeneity • Partial volume effect Partial volume effect

7/12/2004JHU/IACL

Some Requirements• Topology correctness • Valid 2D manifold

X

X

7/12/2004JHU/IACL

Four Steps

1. Fuzzy classification

2. Nested surface segmentation

3. Spherical mapping and partial inflation

4. Sulcal segmentation

7/12/2004JHU/IACL

Outline

• Introduction





• Applications

7/12/2004JHU/IACL

Preprocessing

7/12/2004JHU/IACL

Fuzzy Segmentation[Pham & Prince TMI 1999]

Gray matter

GM

White matter

WM

Cerebrospinal fluid

CSF

• Yields continuous-valued fuzzy membership functions, with values in the range of [0, 1]

7/12/2004JHU/IACL

Published Algorithms

• AFCM: Adaptive fuzzy c-means– smooth gain field; fuzzy clusters; yields pseudo

partial volume segmentation

• AGEM: Adaptive generalized Expectation Maximization– smooth gain field; MRF label smoothness;

posterior density is “fuzzy segmentation

• FANTASM– Fuzzy segmentation with smooth membership

functions and gain field

Pham and Prince

7/12/2004JHU/IACL

Membership Improvements

• White Matter– Modifications to fill interior, remove

extraneous surfaces, remove connectivity errors, and correct topology

• Gray Matter– Modification to provide evidence of CSF in

tight sulci

7/12/2004JHU/IACL

WM Isosurface

• Approximates WM/GM boundary

• Problems:– undesired surfaces– connectivity errors– handles

7/12/2004JHU/IACL

Autofill• WM isosurface should represent the

GM/WM interface of the cortex only

isosurface of WM segmentationbefore filling

isosurface of WM segmentationafter filling

7/12/2004JHU/IACL

Autofill WM Volume

7/12/2004JHU/IACL

WM Isosurface Principle

• 0.5 of WM membership approximates WM/GM interface

• 0.5 of WM+GM membership approximates GM/CSF interface

0.5WM GM CSF

7/12/2004JHU/IACL

Marching Cubes Isosurface

• Consider values on corners of voxel

• Label as– above isovalue– below isovalue

• Determine position of triangular mesh surface passing through voxel

• Linear interpolation

> 0.5< 0.5

Voxel values

7/12/2004JHU/IACL

Connectivity Errors

• Multiple meshes – select the largest mesh

• Touching vertices, edges, and faces– isovalue choice, or– adjust pixel values by epsilon

• Ambiguous faces and cubes– use saddle point methods, or– use connectivity consistent MC algorithm

Most isosurface algorithms use rules that lead to connectivity errors

7/12/2004JHU/IACL

Ambiguous Faces

Two possible tilings:

7/12/2004JHU/IACL

Ambiguous Cubes

Two possible tilings:

7/12/2004JHU/IACL

Digital Connectivity

• Consistent pairs: (foreground,background) → (6,18), (6,26), (18,6), (26,6)

6-connectivity

18-connectivity 26-connectivity

7/12/2004JHU/IACL

Connectivity Consistent MC Algorithm

• (black,white)• (18,6) choose b, f• (26,6) choose b, e

(a) (b) (c)

(d) (e) (f)

AmbiguousFace

AmbiguousCube

• (6,18) choose c, f• (6,26) choose c, f

7/12/2004JHU/IACL

Remaining Problem: Handles

multiple surfacesshared verticesshared edgesshared facesconnectivity errors

• handles

Taken from actual white matter

7/12/2004JHU/IACL

Removes Handles by Editing WM

Fill the backgroundCut the foreground OR

7/12/2004JHU/IACL

Euler Number

– Euler number of a triangular mesh:

– A simple closed surface is topologically equivalent to a sphere iff

– genus is handle

tunnel

A surface handle

Illustration

• Handles: easy to detect by computing the Euler number of the surface mesh

• Euler number provides no information about the location of the handles

2/1 g

7/12/2004JHU/IACL

GTCA Flow DiagramBODY

RESIDUE

SE

Opening

CTE

4

56

7

1

23

Component Labeling andConnection Analysis

Graph

Construction1

4

23

7

56Cycle

Breaking1

4

23

7

6

New Object

Original Object

Illustration of the basic ideas

(A) (B)

(C)(D)

Recycling

Illustration of our topology correction filter

7/12/2004JHU/IACL

1

23

4

56

78

Morphological Opening

structuring element

“body” “residue”

7/12/2004JHU/IACL

After Opening

• Divides object into two components:– “body”– “residue”

• Build graph? Throw out residue pieces? NO!– residue are often very large, but thin sheets– opening may create holes that did not exist

before

7/12/2004JHU/IACL

Conditional Topological Expansion• Grow body by adding “nice” points from

residue: prohibits creation of handles; allows filling of holes

7/12/2004JHU/IACL

Build a Graph

1

23

4

56

7

1

23

4

5

6

7

connected components

connectivity

7/12/2004JHU/IACL

Detect and Remove Cycles

• Find a cycle using depth-first search

• Find the smallest residue connected component in the cycle and remove it

• Repeat until no more cycles remain

1

23

4

5

6

7X

X

7/12/2004JHU/IACL

Restore Residue

• Add remaining residue connected components back to body

• Run conditional topological expansion again.– restores some points

that were discarded prior to graph construction.

7/12/2004JHU/IACL

Success?

• Compute isosurface of binary volume

• Compute Euler number– If less than 2; repeat on background

• Compute Euler number again– If less than 2; repeat with larger structuring

element, and so on…

• Is isosurface algorithm consistent with digital topology?– wrong algorithm connectivity paradoxes

7/12/2004JHU/IACL

Topology Correction: Result

Before Topology Correction After Topology Correction

¹WM ¹WM

^

7/12/2004JHU/IACL

Results: Quantitative

Ratio of voxels changed to original genus is around 2

Genus of resulting volume.

Brain S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15

Init. Genus 724 955 1376 744 1031 776 562 886 688 825 986 597 1944 1280 801

b1 46 31 31 39 31 24 16 33 26 23 20 17 57 36 20f1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0b2 : : : : 1 : : 0 : : : : : : :f2 : : : : 0 : : : : : : : : : :

Changes 1371 1915 2526 1434 1984 1352 1049 1576 1257 1493 1717 1051 3812 2477 1498

ANVCPH 1.89 2 1.84 1.93 1.92 1.74 1.87 1.78 1.83 1.81 1.74 1.76 1.96 1.93 1.87

Number of voxels changed in volume.

7/12/2004JHU/IACL

GM/WM Interface• Topologically correct• No self intersections• Sub-voxel resolution• Close to

– WM/GM surface– GM central surface– pial surface

• Represented by – triangle mesh, or– level set function

7/12/2004JHU/IACL

Gray Matter Isosurface

• Misses tight sulci

7/12/2004JHU/IACL

Partial Volume Effect

Imaging

GMCSF

partial volumeaveraging

WM

GM CSF

WM

Gyri

Sulci

7/12/2004JHU/IACL

Weighted Distance Skeleton

Distance functionfrom the GM/WM

interface in

Compute its Laplacian and normalize to [ , ]0 1

L( )in

7/12/2004JHU/IACL

Anatomically Consistent Enhancement (ACE)

GM GMold

in ( ( ))1 L

if in 0 CSF CSFold

GMold

in L( )

if in 0

Outside

^

^

7/12/2004JHU/IACL

ACE Result

Original GM ACE GM

7/12/2004JHU/IACL

Outline

• Introduction





• Applications

7/12/2004JHU/IACL

Deformable Surface Model

• Want to move the initial WM/GM mesh

7/12/2004JHU/IACL

Nested Deformable Surfaces

Pial Surface

Inner Surface

Central Surface

TGDM-3

Initial WM Isosurface

TGDM-2TGDM-1

7/12/2004JHU/IACL

• Parametric deformable models (PDMs)

─ Represent curves or surfaces through explicit parameterization

─ e.g. curves tessellated with nodes,

surfaces tessellated with triangles

• Geometric deformable models (GDMs)

– Implicit implementation – uses level set numerical

method

Deformable Models

7/12/2004JHU/IACL

Parametric Deformable Models

p = location on contour

[Kass, Witkin, & Terzopolous, 1987]

• Curves/surfaces that deform with a speed law derived Curves/surfaces that deform with a speed law derived from image information and prior knowledge about object from image information and prior knowledge about object shape (e.g. boundary smoothness and continuity)shape (e.g. boundary smoothness and continuity)

7/12/2004JHU/IACL

x

y

One Extra Dimension

C p t( , )

z 0

z

xy

z x y t( , , )

Level Set Method

C p t x x t x R R( , ) { | ( , ) }, ) 0 2 3(or

[Osher and Sethian 1988]

7/12/2004JHU/IACL

Advantages of GDMs

• Produce closed, non-self-intersecting contours

• Independent of contour parameterization

• Easy to implement: numerical solution of PDEs on regular computational grid

• Stable computation

7/12/2004JHU/IACL

Parametric to Geometric[Osher & Sethian 1988]

0||||

Ft

Level Set PDE:

Contour Deformation:

0

t

C

t

0)),,(( ttpC

||||

FF

7/12/2004JHU/IACL

Topology Behavior of Deformable Contour Models

• Parametric self intersection problem

• Geometric cannot control topology

• TGDM (ours) preserves topology

Parametric Geometric TGDM

7/12/2004JHU/IACL

Digital Embedding of Contour Topology

White Points:

0)( x

Black Points:

• Contour topology is determined by signs of the level set function at pixel locations

• Topology of the implicit contour is the same as the topology of the digital object

7/12/2004JHU/IACL

Connectivity Rule of Contour

• Topology of digital contour determined by connectivity rule

n n 4 8, n n 8 4,

Same digital object, different topologies

7/12/2004JHU/IACL

Topology Preservation Principle

• Preserving contour topology is equivalent to maintaining the topology of the digital object

• The digital object can only change topology when the level set function changes sign at a grid point

• Which sign changes can be allowed, and which cannot?

• To prevent the digital object from changing topology, the level set function should only be allowed to change sign at simple points

7/12/2004JHU/IACL

Simple Point• Definition: a point is simple if adding or removing the point

from a binary object will not change the object topology • Determination: can be characterized locally by the

configuration of its neighborhood (8- in 2D, 26- in 3D) [Bertrand & Malandain 1994]

SimpleNon-

Simple

7/12/2004JHU/IACL

x is a Simple Point

0)( x

x

0)( x

xx

7/12/2004JHU/IACL

x is Not a Simple Point

n n 4 8,

0)( x 0)( xX

X

7/12/2004JHU/IACL

Topology Preserving Geometric Deformable Model (TGDM)

• Evolve level set function according to GDM• If level set function is going to change sign,

check whether the point is a simple point– If simple, permit the sign-change– If not simple, prohibit the sign-change

(replace the grid value by epsilon with same sign)– (Roughly, this step adds 7% computation time.)

• Extract the final contour using a connectivity consistent isocontour algorithm

7/12/2004JHU/IACL

SGDM TGDM

A 2D Demonstration

7/12/2004JHU/IACL

PDM Result TGDM Result

No Self-intersections

7/12/2004JHU/IACL

A 3D TGDM DemonstrationOriginalObject

SGDMInit #1

#1

#2

SGDMInit #2

TDGMInit #1

TDGMInit #2

7/12/2004JHU/IACL

TGDM for Inner Surface

Initial WM Isosurface Final GM/WM Interface

7/12/2004JHU/IACL

TGDM for Inner Surface

• Evolution Equationt R x x ( ( ) ( )) 1 2

( ) ( )x

Mean Curvature:

1 2and are weighting factors

R x x( ) ( ) 2 1WMRegion Force:

7/12/2004JHU/IACL

TGDM for Central Surface

Initialize with GM/WM surface Final Central Surface

7/12/2004JHU/IACL


• Gradient Vector Flow [Xu & Prince TIP98]

7/12/2004JHU/IACL


( ) ( )x

Mean Curvature:

Gradient Vector Flow Force:

F xGVF GMGVF( ) ( )

1 3, and are weighting factors2

Region Force:

R xx

x x( )

,

( ) ( ),

if ( ) 0.5

otherwiseGM

WM CSF

0

• Evolution Equation

t R x x F x ( ( ) ( )) ( ) 1 2 3 GVF

7/12/2004JHU/IACL

Nesting Constraint

• Nested surfaces:– Central is outside GM/WM– Pial is outside central

• If level set function wants to go negative to positive – allow if inner level set function is positive – otherwise set to small positive epsilon

7/12/2004JHU/IACL

TGDM for Outer Surface

Final Pial SurfaceStart from Central Surface

7/12/2004JHU/IACL

TGDM for Outer Surface

• Evolution Equationt R x x F x ( ( ) ( )) ( ) 1 2 3 GVF

R x x x( ) ( ) ( ) GM CSFRegion Force:

( ) ( )x

Mean Curvature:

Gradient Vector Flow Force:

F xGVF GM WMGVF( ) ( ( ) ) 1 3, and are weighting factors2

7/12/2004JHU/IACL

Coronal

Results Visual Inspection

Sagittal

• Slice views of three surfaces overlaid on cross-sections of the original image

Axial

7/12/2004JHU/IACL

Repeatability Analysis

• 3 subjects, each scanned twice

• Surface pairs rigidly registered

• Average errors:– signed distance– absolute distance

7/12/2004JHU/IACL

Repeatability Results (mm)

7/12/2004JHU/IACL

Landmark Validation Study

7/12/2004JHU/IACL

Landmark Validation Analysis

• Raters: 12• Brains: 2 • Landmarks: 10 per

region• Sulci: 33 / brain• Geometry: 11 fundi, 11

gyri, 11 banks• Surface: Inner & Pial• Statistical software: “R”

version 1.8.1

• CRUISE surfaces are reference surfaces: yield “landmark offset”– signed and absolute

• Membership values– white matter– gray matter

• Statistical factors:– Brain– Geometry– Sulci

7/12/2004JHU/IACL

Landmark Validation: Results

• MANOVA revealed significant factors: – geometry & sulci, but

not brain

• Landmark offset– mean = - 0.35 mm– std = 0.65 mm– 16% farther than 1

mm from reference

• ACE regions show smaller offsets

• Signed distance consistently negative

• outward bias of CRUISE– differs for geometry

(largest for fundi)– differs for surface

• Note: we are optimizing parameters

7/12/2004JHU/IACL

Nested Surface Segmentation• Nearly fully automated

– skull-stripping is semi-automated (10 minutes)– AC & PC need to be picked manually (5 minutes) – The rest is fully automated

• Less than 25 minutes for each brain – (Previous PDM version takes 2-3 hours)

• More than 200 brain datasets processed so far – average error is about 1/3 voxel– highly repeatable scanner errors dominate

Han et al, 2004

7/12/2004JHU/IACL

Outline

• Introduction





• Applications

7/12/2004JHU/IACL

Spherical and Partial Flattening[Tosun et al, 2003]

7/12/2004JHU/IACL

Surface Inflation

• Coarsen shape• More regular mesh

structure• Use relaxation

operator:

• Check norm of mean curvature:

7/12/2004JHU/IACL

Atlas Registration

• Simpler surface registered using modified ICP

• Atlas labels transfer easily

Atlas Subject

(a)

(b)

(c)

(d)

7/12/2004JHU/IACL

Spherical Mapping

• Single conformal map from atlas

• Inverse stereographic projection

7/12/2004JHU/IACL

Automatic Labelling

• Brains mapped to sphere• Segmented sulci compared to labelled atlas• Simple voting scheme leads to >90%

accuracy

7/12/2004JHU/IACL

Outline

• Introduction





• Applications

7/12/2004JHU/IACL

Sulcal SegmentationGoals: • Automatically segment sulci • carry out cortical parcellation

Applications:• Localizing activation sites in functional images• Brain registration• Understanding morphological changes in normal aging and disease

Principle:• Based on depth from “outer” surface

7/12/2004JHU/IACL

Sulcal RegionsDefined as buried cortical regions that

surround sulcal spaces

7/12/2004JHU/IACL

Classifying Gyral and Sulcal Regions

• Generate a shrink-wrap surface• Sulcal regions distinguished

from gyral regions based on distance to shrink-wrap surface

7/12/2004JHU/IACL

Sulcal/Gyral Classification

sulcal regions (red)andgyral regions (blue)

Euclidean distance to outer surface

sulci > 2 mmfrom outer surface

7/12/2004JHU/IACL

Watershed Segmentation

• Classification does not separate sulci

• Further segmentation is required

• Watershed by immersion is intuitive idea:

7/12/2004JHU/IACL

Geodesic Distance Computation• use Fast Marching (Kimmel and Sethian, ’98)

• initial contour at time zero is gyral/sulcal boundary

• Propagation at unit speed in normal direction on mesh

• geodesic distance is arrival time of evolving contour

7/12/2004JHU/IACL

Watershed Computation

Each local minimumproduces acatchment basin (CB).

Critique:• true sulci are separated • single sulci are over-segmented.

7/12/2004JHU/IACL

Merging Algorithm

• Addresses over-segmentation problem

• Small ridges in sulcal regions result in formation of separate CBs

• Criterion for merging CBs:

1) height of ridge

2) size of CB

• Provides different “levels” of merging

7/12/2004JHU/IACL

Sulcal Segmentation Results

Height threshold = 1 cmSize threshold = 3 cm2

Rettmann et al. MMBIA 2000

7/12/2004JHU/IACL

Sulcal Segmentation Results

7/12/2004JHU/IACL

Cross-Sections

7/12/2004JHU/IACL

7/12/2004JHU/IACL

Outline

• Introduction





• Applications

7/12/2004JHU/IACL

Repeat Scan Validation

Superiorfrontal sulcus

scan 1 scan 2 scan 3

7/12/2004JHU/IACL

Shape Analysis

Left

Right

Cingulate

Subject 1 Subject 2

7/12/2004JHU/IACL

Geometric Features

mean curvature

geodesic depth

7/12/2004JHU/IACL

Cortical Thickness[Yezzi et al, 2003]

7/12/2004JHU/IACL

Baltimore Longitudinal Study of Aging

• PI: Susan Resnick (NIA)

• 1994-2003

• Ages 55-85, 158 participants

• >1000 separate scans, 1 per year per subject

• volumetric SPGR brain scans

• 0.9375x0.9375x1.5mm voxel size

7/12/2004JHU/IACL

Thickness Map from CRUISETypical Thickness Map

7/12/2004JHU/IACL

Cross-sectional Study of Cortical Thickness

• Preliminary study on 35 subjects

7/12/2004JHU/IACL

The END

7/12/2004 jhu/iacl jerry l. prince image analysis and communications laboratory dept. of electrical...

Documents