deformable organ contour transfer with deep inverse shape...
TRANSCRIPT
![Page 1: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/1.jpg)
Deformable Organ Contour Transfer with
Deep Inverse Shape Encoding (DISE) Networks
for Auto-segmentation in Low Contrast Regions
Tiancheng Liu1 Xiaobai Sun1 Fang-fang Yin2 Lei Ren2
1Department of Computer Science, Duke University, USA2Department of Radiation Oncology, Duke University School of Medicine, USA
60th AAPM Annual Meeting, Nashville, TN
Aug 2nd, 2018
1 / 15
![Page 2: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/2.jpg)
Outline
� Purpose: Robust automatic segmentation of thoracic and
abdominal CT images with low CNR
� Method: Deep Inverse Shape Encoding (DISE) networks
◦ Inverse Shapes for coarse regional partition
◦ Sparse registration via shape matching
◦ Reference guided labeling
� Results and Evaluation
2 / 15
![Page 3: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/3.jpg)
Introduction: Automatic segmentation
Existing automatic methods: based on
dense voxel-wise registration using
� patch texture1
� gradients2
Shortcomings:
� Time consuming
� Not robust to low CNR images
Reference Image Target Image
Provided labels Generated labels3
1 Korfiatis et al. IEEE Trans Inf Technol Biomed, 20102 Sotiras et al. IEEE TMI, 20133 Liu et al. SU-K-201-14 (Snap Oral), AAPM 2017
3 / 15
![Page 4: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/4.jpg)
Purpose
Axial Coronal Sagittal
� Robustly, automatically segment CT (CBCT) images of thoracic,
abdominal regions with low CNR
� To replace manual delineation and segmentation based on dense
registration4 / 15
![Page 5: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/5.jpg)
Method: deep inverse shape encoding (DISE) network
5 / 15
![Page 6: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/6.jpg)
Inverse shape: basic concept with 2D illustration
Gradient:
∇I (x0) ≈ I (x0+∆x)∆x
local in spatial support
non-robust to noisy change in intensity
Inverse Shape:
invShape(I0,∆I ) = {x |I (x) ∈ [I0, I0 + ∆I ]}non-local in spatial support
robust in shape to noisy change in intensity
6 / 15
![Page 7: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/7.jpg)
Illustration: inverse shape on XCAT phantom
Five inverse shapes1 are shown by their boundaries, each in a unique color
1 XCAT Phantom data from Konik et al. Phys. Med. Biol. 2014 7 / 15
![Page 8: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/8.jpg)
Illustration: sparse samples on a pair of inverse shapes
50
100
150
200
250
300
250200 300250200150 150100
Reference
50
100
150
200
250
300
250200 300250200150 150100
Target
Sparse samples on inverse shape containing liver, spleen, diaphragm and aorta
5064 samples (from 1.34M voxels) on reference, 5059 samples (from 1.33M voxels) on target
8 / 15
![Page 9: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/9.jpg)
Shape encoding via shape context descriptor1 (2D)
2 4
Radii( ) bin index
2
4
6
8
10
12
An
gle
()
bin
in
de
x
2 4
Radii( ) bin index
2
4
6
8
10
12
An
gle
()
bin
in
de
x
Reference Target
◦ Log-polar histogram of samples on the shape
(equally-spaced bins along circumference, larger radii bins at coarser scale)
◦ Capture topology at multiple scales
1 Belongie et al. IEEE PAMI 2002 9 / 15
![Page 10: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/10.jpg)
Inverse shape encoding via shape context descriptors (3D)
Geometric depiction data structure for shape descriptor
3D Shape context descriptor in log-spherical histograms10 / 15
![Page 11: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/11.jpg)
DISE network for inverse shape matching
11 / 15
![Page 12: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/12.jpg)
Illustration: sparse registration
Reference Target
Correspondence between shape descriptors =⇒ matching between inverse shapes12 / 15
![Page 13: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/13.jpg)
Results and Evaluation: liver contour transferring
� XCAT Phantom 1
� Fast, robust segmentation by DISE
network
� DSC on Liver: 0.98
DSC =2|Vgen ∩ Vgt ||Vgen|+ |Vgt |
� Robust to noise
Ground Truth liver contour on target image
Generated liver contour on target image
1 Konik et al. Phys. Med. Biol. 2014 13 / 15
![Page 14: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/14.jpg)
Results and Evaluation: liver contour transferring
� XCAT Phantom 1
� Fast, robust segmentation by DISE
network
� DSC on Liver: 0.98
DSC =2|Vgen ∩ Vgt ||Vgen|+ |Vgt |
� Robust to noise
Liver contour on reference image
Generated liver contour on target image
1 Konik et al. Phys. Med. Biol. 2014 13 / 15
![Page 15: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/15.jpg)
Summary of DISE network
� Achieve both robustness and efficiency for anatomical segmentation
◦ Coarse partition of image domain into inverse shapes
induced by intensity bins
◦ Sparse representation of the inverse shape
via sparse sampling and shape context descriptor
◦ Contour transfer
Shape matching via DISE network
◦ Can faciliate finer registration and other analysis tasks
14 / 15
![Page 17: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/17.jpg)
References i
X. Bai, S. Bai, Z. Zhu, and L. J. Latecki.
3D shape matching via two layer coding.
IEEE transactions on pattern analysis and machine intelligence,
37(12):2361–2373, 2015.
S. Belongie, J. Malik, and J. Puzicha.
Shape matching and object recognition using shape contexts.
IEEE transactions on pattern analysis and machine intelligence,
24(4):509–522, 2002.
![Page 18: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/18.jpg)
References ii
A. M. Bronstein, M. M. Bronstein, and R. Kimmel.
Rock, paper, and scissors: extrinsic vs. intrinsic similarity of
non-rigid shapes.
In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference
on, pages 1–6. IEEE, 2007.
D. Ji, J. Kwon, M. McFarland, and S. Savarese.
Deep view morphing.
In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pages 2155–2163, 2017.
![Page 19: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/19.jpg)
References iii
A. Konik, C. M. Connolly, K. L. Johnson, P. Dasari, P. W. Segars, P. H.
Pretorius, C. Lindsay, J. Dey, and M. A. King.
Digital anthropomorphic phantoms of non-rigid human respiratory
and voluntary body motion for investigating motion correction in
emission imaging.
Physics in Medicine & Biology, 59(14):3669, 2014.
T. Liu, D. Floros, N. Pitsianis, X. Sun, F.-f. Yin, and L. Ren.
Robust automatic co-segmentation of multiple medical images.
SU-K-201-14, presented at AAPM 2017.
![Page 20: Deformable Organ Contour Transfer with Deep Inverse Shape ...tcliu/publication/AAPM-2018/AAPM2018-… · 3D shape matching via two layer coding. IEEE transactions on pattern analysis](https://reader035.vdocument.in/reader035/viewer/2022071017/5fd082d4c75fe210bf19f60c/html5/thumbnails/20.jpg)
References iv
O. Nomir and M. Abdel-Mottaleb.
Hierarchical contour matching for dental X-ray radiographs.
Pattern Recognition, 41(1):130–138, 2008.
H. Tabia, H. Laga, D. Picard, and P.-H. Gosselin.
Covariance descriptors for 3D shape matching and retrieval.
In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pages 4185–4192, 2014.
Y. Zhang, F.-F. Yin, W. P. Segars, and L. Ren.
A technique for estimating 4D-CBCT using prior knowledge and
limited-angle projections.
Medical physics, 40(12), 2013.