dtu medical visionday may 27, 2009 generative models for automated brain mri segmentation koen van...
TRANSCRIPT
![Page 1: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/1.jpg)
DTU Medical Visionday May 27, 2009
Generative models for automated brain MRI segmentation
Koen Van Leemput
Athinoula A. Martinos Center for Biomedical Imaging
Department of Radiology, MGH
Harvard Medical School, USA
Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology, USA
![Page 2: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/2.jpg)
MRI of the brain
Magnetic resonance imaging:– Harmless– Three dimensional (3-D)– High soft tissue contrast– High spatial resolution– Extremely versatile– Possibly multi-spectral
Ideal for studying the living human brain
“voxel”
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 3: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/3.jpg)
Segmentation of brain MRI
– Delineating structures of interest in the images
– Segmentation is important: Basic neuroscience Uncovering disease mechanisms Diagnosis, treatment planning,
and follow-up Clinical drug trials …
– Automated computational methods are needed
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 4: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/4.jpg)
Overview
Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 5: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/5.jpg)
Overview
Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 6: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/6.jpg)
MRI image
The problem to be solved
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 7: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/7.jpg)
MRI image
The problem to be solved
Koen Van Leemput DTU Medical Visionday May 27, 2009
Label image
![Page 8: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/8.jpg)
One solution: generative modeling
– Formulate a statistical model of how an MRI image is formed
– The model depends on some parameters
“labeling model”
“imaging model”
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI image Label image
![Page 9: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/9.jpg)
MRI image
Segmentation = inverse problem
Koen Van Leemput DTU Medical Visionday May 27, 2009
Label image
![Page 10: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/10.jpg)
MRI image
Segmentation = inverse problem
Koen Van Leemput DTU Medical Visionday May 27, 2009
Label image
Bayesian inference– Start from our statistical model of image formation– Play with the mathematical rules of probability
![Page 11: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/11.jpg)
Practical approximation
Involves two optimizations:– First estimate the optimal model parameters– Then find the optimal segmentation based on those parameter
estimates
Bayesian inference
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 12: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/12.jpg)
Example: Gaussian mixture model
– The label in each voxel is drawn independently with a probability for tissue type k
– Assume a uniform prior for the labeling model parameters
“labeling model”
“imaging model”
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI image Label image
![Page 13: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/13.jpg)
Example: Gaussian mixture model
– The intensity in each voxel is drawn independently from a Gaussian distribution associated with its label
– The imaging model parameters are the mean and variance of each Gaussian:
– Assume a uniform prior
“labeling model”
“imaging model”
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI image Label image
![Page 14: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/14.jpg)
Model parameters are unknown
Example: Gaussian mixture model
Koen Van Leemput DTU Medical Visionday May 27, 2009
Mean and variance of each Gaussian
Relative weight of each Gaussian
three labels
![Page 15: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/15.jpg)
Optimization 1: parameter estimation
– Given an MRI image to be segmented, what is the MAP parameter estimate ?
– Parameter optimization with an Expectation Maximization (EM) algorithm
current estimate
– Repeatedly maximize a lower bound to the objective function
– Iterative parameter optimizer using only closed-form parameter updates!
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 16: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/16.jpg)
Koen Van Leemput DTU Medical Visionday May 27, 2009
Optimization 1: parameter estimation
![Page 17: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/17.jpg)
Koen Van Leemput DTU Medical Visionday May 27, 2009
Optimization 1: parameter estimation
![Page 18: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/18.jpg)
Optimization 2: segmentation
white matter
gray matterCSF
Koen Van Leemput DTU Medical Visionday May 27, 2009
Upon completion of the parameter estimation algorithm, assign each voxel to the MAP label
![Page 19: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/19.jpg)
Overview
Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 20: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/20.jpg)
MRI bias field artifact
Intensity inhomogeneities across the image area
Imaging artifact in MRIequipment limitations
patient-induced electrodynamic interactions
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI data after intensity windowing…
![Page 21: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/21.jpg)
MRI bias field artifact
Causes segmentation errors with our segmentation procedure so far…
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 22: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/22.jpg)
MRI bias field artifact
Koen Van Leemput DTU Medical Visionday May 27, 2009
Causes segmentation errors with our segmentation procedure so far…
![Page 23: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/23.jpg)
Improved imaging model
“labeling model”
“imaging model”
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI image Label image
![Page 24: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/24.jpg)
Improved imaging model
“labeling model”
“imaging model”
old model
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI image Label image
![Page 25: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/25.jpg)
Improved imaging model
“labeling model”
“imaging model”
+
old model polynomial bias field model
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI image Label image
![Page 26: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/26.jpg)
Model parameter estimation
– Polynomial coefficients are part of the model parameters
– Parameter optimization with a Generalized Expectation Maximization (GEM) algorithm
current estimate
– Repeatedly improve a lower bound to the objective function
– Iterative parameter optimizer using only closed-form parameter updates! [Van Leemput et al., IEEE TMI 1999]
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 27: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/27.jpg)
Example
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI data Estimated bias field
Bias-corrected MRI data
![Page 28: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/28.jpg)
Example
MRI data
White matter without bias field model
White matter with bias field model
Estimated bias field
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 29: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/29.jpg)
Example
MRI data
White matter without bias field model
White matter with bias field model
Estimated bias field
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 30: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/30.jpg)
Overview
Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 31: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/31.jpg)
Improving the labeling model
“labeling model”
“imaging model”
Koen Van Leemput DTU Medical Visionday May 27, 2009
– So far our labeling model just expresses the relative frequency of occurrence of different labels
– Too simplistic for segmenting the brain into 30+ subregions
A more realistic labeling model is needed!
MRI image Label image
![Page 32: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/32.jpg)
Improving the labeling model
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 33: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/33.jpg)
Improving the labeling model
Try to find the underlying probability distribution
Manual segmentations in N individuals (training data)
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 34: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/34.jpg)
Modeling the training data (2-D)
Triangular mesh representation
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 35: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/35.jpg)
Modeling the training data (2-D)
Assign label probabilities to each mesh node• Flat prior• Label probabilities are linearly interpolated over triangle areas
“atlas”
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 36: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/36.jpg)
Modeling the training data (2-D)
Mesh node positions are sampled from a topology-preserving Markov
random field prior
“atlas”
warped atlases
Koen Van Leemput DTU Medical Visionday May 27, 2009
“knob” that controls the flexibility of the atlas warp
![Page 37: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/37.jpg)
Modeling the training data (2-D)
Example segmentations are sampled according to the
deformed atlases
atlas
warped atlases
example segmentations
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 38: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/38.jpg)
Bayesian inference [Van Leemput, IEEE TMI 2009]
Given a collection of manual segmentations– what is the most probable atlas?– what is the most likely value of the parameter controlling the
flexibility of the deformations?– what is the most likely mesh
representation?
Good models explain regularities in the manual segmentations– Automatically yields sparse representations that explicitly avoid
overfitting to the training data– cf. Minimum Description Length
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 39: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/39.jpg)
Example atlas
Koen Van Leemput DTU Medical Visionday May 27, 2009
Derived from manual segmentations of 36
brain substructures in 4 individuals
Has average “shape”
![Page 40: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/40.jpg)
Overview
Segmentation basics: modeling and inference Modeling MRI bias fields Mesh-based brain atlases Whole-brain segmentation
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 41: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/41.jpg)
Whole-brain segmentation
– Tetrahedral mesh-based atlas– The labeling model parameters are the location of the
mesh nodes– The prior is the topology-preserving MRF model
(penalizes deformations)
“labeling model”
“imaging model”
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI image Label image
![Page 42: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/42.jpg)
Whole-brain segmentation
“labeling model”
“imaging model”
+
Gaussian mixture model polynomial bias field model
Koen Van Leemput DTU Medical Visionday May 27, 2009
MRI image Label image
![Page 43: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/43.jpg)
– Model parameter estimation:
– Fully automated segmentation procedure• No need for pre-processing (skull stripping, bias field corr., …)• Automatically adapts to different scanners and acquisition sequences!• Fast!
Whole-brain segmentation
Improve the imaging model parameters (Generalized Expectation-Maximization;
closed-form expressions)
Improve the atlas warp (registration; gradient in analytical form)
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 44: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/44.jpg)
Examples (validation under way)
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 45: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/45.jpg)
Examples (validation under way)
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 46: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/46.jpg)
Examples (validation under way)
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 47: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/47.jpg)
Examples (validation under way)
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 48: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/48.jpg)
Examples (validation under way)
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 49: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/49.jpg)
Examples (validation under way)
Koen Van Leemput DTU Medical Visionday May 27, 2009
![Page 50: DTU Medical Visionday May 27, 2009 Generative models for automated brain MRI segmentation Koen Van Leemput Athinoula A. Martinos Center for Biomedical](https://reader036.vdocument.in/reader036/viewer/2022062423/56649e855503460f94b86db2/html5/thumbnails/50.jpg)
Thanks!
Koen Van Leemput DTU Medical Visionday May 27, 2009