department of radiology and bric, unc-chapel hill links: learning-based multi-source integration...

IDEA Image Display, Enhancement, and Analysis

Department of Radiology and BRIC, UNC-Chapel Hill

LINKS: Learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images

Li Wang, Yaozong Gao, Feng Shi, Gang Li, Dinggang ShenPresented by Li Wang

09-18-2014


Content

Motivation Proposed method Experimental results Conclusion


Motivation

Limitations of multi-atlas label fusion1. nonlinear registrations2. simple intensity patch3. equal weight for different modality

Fractional anisotropy (FA) was calculated from Diffusion MRIs.

Our proposed work will 1. linear registrations2. appearance features and context features3. adaptive weights for different modality

2-weeks

6-months

12-months

T1 T2 FA Manual segmentation


Flowchart of our proposed work

Context features

Appearance features

Classifier 2

Ground truth

T1 T2 FAAppearance

features

Probability mapsSequence classifier

Feature vectors

Context features

Appearance features

Classifier τ

Haar-like featuresClassifier 1

Random forests


Result of an unseen target subject

T1 T2 FA

Original images

Iteration 1

Iteration 2

Iteration 10

Ground truth


Probabilities of training image by the random forest

Post-processing: Anatomical constraint

To deal with the possible artifacts due to independent voxel-wise classification, we use patch-based sparse representation to impose an anatomical constraint [1] into the segmentation.

1. Wang, L., Shi, F., Gao, Y., Li, G., Gilmore, J.H., Lin, W., Shen, D., 2014. Integration of sparse multi-modality representation and anatomical constraint for isointense infant brain MR image segmentation. NeuroImage 89, 152-164.

Ground truth of training images

Probabilities of target image by the random forest

𝛼1 𝛼 𝑖

Without anatomical With anatomical Ground truth


Dataset

Dataset 1: UNC 119 infants consisting of 26, 22, 22, 23, and 26 subjects at 0-, 3-, 6-, 9- and 12-months of age, respectively.

Dataset 2: NeoBrainS12 MICCAI2012 Challenge. Dataset 3: SATA MICCAI2013 Challenge.


Importance of the context features

Iterations Iterations Iterations


Importance of the multi-source


Dataset 1: UNC 119 infants

(a) Majority voting (MV)(b) Nonlocal label fusion [1](c) Atlas forest [2](d) Patch-based sparse labeling [3](e) Proposed1 (Random forest)(f) Proposed2 (Random forest + Anatomical constraint)

1. Coupé, P., Manjón, J., Fonov, V., Pruessner, J., Robles, M., Collins, D.L., 2011. Patch-based segmentation using expert priors: Application to hippocampus and ventricle segmentation. NeuroImage 54, 940-954.

2. Zikic, D., Glocker, B., Criminisi, A., 2013. Atlas Encoding by Randomized Forests for Efficient Label Propagation. MICCAI 2013, pp. 66-73.3. Wang, L., Shi, F., Gao, Y., Li, G., Gilmore, J.H., Lin, W., Shen, D., 2014. Integration of sparse multi-modality representation and anatomical

constraint for isointense infant brain MR image segmentation. NeuroImage 89, 152-164.


T1 T2 FA

(a) Majority voting(b) Nonlocal label fusion

(c) Atlas forest(d) Patch-based sparse labeling

Ground truth

(e) Proposed1 (f) Proposed2

Slice comparisons

Segmentation

Difference maps with the ground truth


Inner surface comparisons

(a) Majority voting(b) Nonlocal label fusion

(c) Atlas forest(d) Patch-based sparse labeling

(e) Proposed1 (f) Proposed2 (g) Ground truth


Quantitative measurement

Methods MV Nonlocal

label fusion Atlas Forest

Patch-based Sparse labeling

Proposed1 Proposed2

Time cost 1h 1.2h 12m 2h 5m 1.8h

WM

0 81.6±0.28 89.0±0.74 88.9±0.60 89.7±0.59 91.7±0.64 92.1±0.62 3 76.6±1.48 85.0±1.21 85.1±1.33 85.3±1.71 88.8±1.09 89.1±0.95 6 80.1±0.83 83.6±0.80 82.1±0.91 84.2±0.78 86.4±0.79 87.9±0.68 9 79.2±0.98 86.1±2.00 84.2±1.34 87.1±1.89 89.0±0.78 89.4±0.56 12 82.5±1.05 88.6±1.22 87.2±1.29 90.3±1.42 90.7±0.74 91.8±0.65

GM

0 78.6±1.02 85.1±0.78 87.1±0.76 86.7±0.81 89.6±0.66 90.8±0.42 3 77.3±1.42 83.4±0.78 85.5±1.12 85.3±0.51 88.1±1.00 88.3±0.90 6 79.9±1.04 83.9±0.83 83.1±0.93 84.8±0.77 88.2±0.77 89.7±0.59 9 83.6±0.69 88.1±0.75 87.4±0.66 87.4±0.54 90.0±0.49 90.3±0.54 12 84.9±1.01 89.3±0.90 88.8±1.02 88.9±0.57 90.3±0.74 90.4±0.68

CSF

0 76.6±1.57 80.2±1.87 77.7±4.52 76.1±2.59 83.9±2.20 84.2±2.02 3 80.6±1.55 84.1±1.88 82.4±2.17 80.1±1.10 83.7±1.52 85.4±1.49 6 71.2±0.71 79.2±1.69 86.7±1.16 83.0±0.77 92.7±0.63 93.1±0.55 9 68.7±1.27 80.6±2.40 84.1±1.57 81.0±2.27 85.8±1.53 86.7±1.09 12 65.2±3.69 81.5±1.66 83.6±1.83 81.7±2.59 84.1±1.90 85.2±1.69


Dataset 2: NeobrainS12 MICCAI Challenge

2 training images with the manual segmentations. 3 target images for testing.


Our results of 3 target images



Table 1. Dice ratios (DC) and modified Hausdorff distance (MHD) of different methods on NeoBrainS12 MICCAI Challenge data. (Bold indicates the best performance)

WM CGM BGT BS CB CSF

Placed Team Name DC MHD DC MHD DC MHD DC MHD DC MHD DC MHD UNC-IDEA 0.92 0.35 0.86 0.47 0.92 0.47 0.83 0.9 0.92 0.5 0.79 1.18 1

Imperial 0.89 0.70 0.84 0.73 0.91 0.8 0.84 1.04 0.91 0.7 0.77 1.55 2 Oxford 0.88 0.76 0.83 0.61 0.87 1.32 0.8 1.24 0.92 0.63 0.74 1.82 3 UCL 0.87 1.03 0.83 0.73 0.89 1.29 0.82 1.3 0.9 0.92 0.73 2.06 4

UPenn 0.84 1.79 0.80 1.01 0.8 4.18 0.74 1.96 0.91 0.85 0.64 2.46 5

http://neobrains12.isi.uu.nl/mainResults_Set1.php


Dataset 3: SATA MICCAI2013 Challenge

35 training images with the 14 ROIs in subcortical regions. 12 target images for testing.


Our results on one target image



Team Name Submission Date/Time

Mean (Median) DSC

Mean (Median) Hausdorff Distance (mm)

UPENN_SBIA_MAM 12-Jul-2013 0.8686 (0.8772) 3.3043 (3.1006)

PICSL 02-Jul-2013 0.8663 (0.8786) 3.5381 (3.2369)

LINKS 04-May-2014 0.8613 (0.8722) 3.6453 (3.3637)

deedsMIND 12-Jul-2013 0.8402 (0.8573) 4.1027 (3.8983)

MSRC_AF_NEW 18-Feb-2014 0.8247 (0.8392) 3.8437 (3.6799)

MSRC_AF_NEW_STAPLE 18-Feb-2014 0.8063 (0.8169) 4.6494 (4.3760)

deedsMIND no marginals 15-Jul-2013 0.7216 (0.7539) 6.1614 (5.5120)

Table 2. Dice ratios (DC) and Hausdorff distance (HD) of different methods on SATA MICCAI Challenge data.

http://masi.vuse.vanderbilt.edu/submission/leaderboard.html


Conclusion

We have presented a learning-based method (LINKS) to effectively integrate multi-source images and the tentatively estimated tissue probability maps for infant brain image segmentation.

Experimental results on 119 infant subjects and MICCAI grand challenge show that the proposed method achieves better performance than other state-of-the-art automated segmentation methods.


Thanks for your attention!

http://www.unc.edu/~liwa/

Google scholar



http://scholar.google.com/citations?user=xjzMf0cAAAAJ&hl=en

department of radiology and bric, unc-chapel hill links: learning-based multi-source integration...

Documents

uncchapel hillimportance

uncchapel hillresult

uncchapel hillit

uncchapel hilldatasetdataset

infant image

month image

idea image display

gang li