medical image registration yujun guo dept.of cs kent state university
Post on 21-Dec-2015
218 views
TRANSCRIPT
![Page 1: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/1.jpg)
Medical Image RegistrationMedical Image Registration
Yujun GuoYujun Guo
Dept.of CSDept.of CS
Kent State UniversityKent State University
![Page 2: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/2.jpg)
OutlineOutline
Why registrationWhy registration
Registration basicsRegistration basics
Rigid registrationRigid registration
Non-rigid registrationNon-rigid registration
ApplicationsApplications
![Page 3: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/3.jpg)
Modalities in Medical ImageModalities in Medical Image
Computed Tomography (CT), Magnetic Resonance (MR) imaging, Ultrasound, and X-ray give anatomic information.
Positron Emission Tomography (PET) and Single Photon Emission CT (SPECT) give functional information.
![Page 4: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/4.jpg)
RegistrationRegistration
• Monomodality:Monomodality: A series of same modality A series of same modality images (CT/CT, MR/MR, images (CT/CT, MR/MR,
Mammogram pairs,…). Mammogram pairs,…). Images may be acquired weeks or months apart; Images may be acquired weeks or months apart;
taken from different viewpoints.taken from different viewpoints. Aligning images in order to detect subtle changes in Aligning images in order to detect subtle changes in
intensity or shapeintensity or shape
• Multimodality:Multimodality: Complementary anatomic and functional information Complementary anatomic and functional information
from multiple modalities can be obtained for the from multiple modalities can be obtained for the precise diagnosis and treatment.precise diagnosis and treatment.
Examples:PET and SPECT (low resolution, functional Examples:PET and SPECT (low resolution, functional information) need MR or CT (high resolution, information) need MR or CT (high resolution, anatomical information) to get structure anatomical information) to get structure information.information.
![Page 5: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/5.jpg)
Registration Problem DefinitionRegistration Problem Definition
p = (825,856)
q = (912,632)
q = T(p;a)
Pixel location in first image Homologous pixel location in second image
Pixel location mapping function
![Page 6: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/6.jpg)
Example Mapping FunctionExample Mapping Function
p = (825,856)
q = (912,632)
Pixel scaling and translation
![Page 7: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/7.jpg)
Image RegistrationImage Registration
Define a transform T that will map Define a transform T that will map one image onto another image of the one image onto another image of the same object such that some image same object such that some image quality criterion is maximized.quality criterion is maximized.
A mapping between two images both A mapping between two images both spatially and with respect to intensityspatially and with respect to intensity
II22 = g (T(I = g (T(I11))))
![Page 8: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/8.jpg)
Registration SchemeRegistration Scheme
![Page 9: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/9.jpg)
ComponentsComponents
Feature SpaceFeature Space
Search Space or transformationSearch Space or transformation
Similarity MetricSimilarity Metric
Search StrategySearch Strategy
![Page 10: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/10.jpg)
Feature SpaceFeature SpaceGeometric landmarks:Geometric landmarks:Points Points Edges Edges Contours Contours Surfaces, etc.Surfaces, etc.Intensities:Intensities:Raw pixel valuesRaw pixel values
23 35
24 56
Feature-based Feature-based Intensity-basedIntensity-based
![Page 11: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/11.jpg)
Image transformationsTransformation
Inputimage
Output
image
w
y
x
w
y
x
876
543
210
'
'
'
mmm
mmm
mmm
100
543
210
mmm
mmm
affineM
Affine transformation
100
cossin
sincos
y
x
rigid t
t
M
Rigid transformationOriginal
shape
Rigid
Non-rigid
![Page 12: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/12.jpg)
Similarity MetricSimilarity Metric
Absolute differenceAbsolute difference
SSD (Sum of Squared Difference)SSD (Sum of Squared Difference)
Correlation CoefficientCorrelation Coefficient
Mutual Information / Normalized Mutual Information / Normalized Mutual InformationMutual Information
![Page 13: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/13.jpg)
Search StrategySearch Strategy
Powell’s direction set methodPowell’s direction set method
Downhill simplex methodDownhill simplex method
Dynamic programmingDynamic programming
Relaxation matchingRelaxation matching
Hierarchical techniquesHierarchical techniques
![Page 14: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/14.jpg)
Multi-modality Brain image Multi-modality Brain image registrationregistration
Intensity-basedIntensity-based3D/3D Rigid transformation, DOF=6 3D/3D Rigid transformation, DOF=6 (3 translations, 3 rotations)(3 translations, 3 rotations)Maximization of Normalized Mutual Maximization of Normalized Mutual InformationInformationSimplex DownhillSimplex DownhillMulti-resolutionMulti-resolutionDataset: Vanderbilt UniversityDataset: Vanderbilt University
http://www.vuse.vanderbilt.edu/~image/registration/http://www.vuse.vanderbilt.edu/~image/registration/results.htmlresults.html
![Page 15: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/15.jpg)
Mutual Information as Similarity Mutual Information as Similarity MeasureMeasure
Mutual informationMutual information is applied to measure the is applied to measure the statistic dependence between the image intensities of statistic dependence between the image intensities of corresponding voxels in both images, which is assumed to corresponding voxels in both images, which is assumed to be maximal if the images are geometrically aligned.be maximal if the images are geometrically aligned.
a b BA
ABAB bPaP
baPbaPBAI
)()(
),(log),(),(
)|()(
)|()(
),()()(
ABHBH
BAHAH
BAHBHAH
![Page 16: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/16.jpg)
Normalized Mutual InformationNormalized Mutual Information
Extension of Mutual InformationExtension of Mutual Information
Maes et. al.:Maes et. al.:
Studholme et. Al.:Studholme et. Al.:
Compensate for the sensitivity of MI to Compensate for the sensitivity of MI to changes in image overlapchanges in image overlap
)()(
)(2),(
),(),(),(
BHAH
AMIBANMI
BAMIBAHBANMI
),(
)()(),(
BAH
BHAHBANMI
![Page 17: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/17.jpg)
Geometry TransformationGeometry Transformation Image Coordinate transform:
The features (dimension, voxel size, slice spacing, gantry tilt, orientation) of images, which are acquired from different modalities, are not the same.
From voxel units (column, row, slice spacing) to millimeter units with its origin in the center of the image volume.
11000
0
0
0
1
'
'
'
),,(222120
121110
020100
z
y
x
aaa
aaa
aaa
z
y
x
zyxT
![Page 18: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/18.jpg)
Target Image & Template ImageTarget Image & Template Image
Target Image Grid
j
i
y
x
Target ImagePhysical Coordinates
y’
x’
Template ImagePhysical Coordinates
Template Image Grid
j
i
Space Transform
![Page 19: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/19.jpg)
Images from the same patientImages from the same patient
Images provided as part of the project: “Retrospective Image Registration Evaluation”, NIH, Project No. 8R01EB002124-03, Principal Investigator, J. Michael Fitzpatrick, Vanderbilt University, Nashville,
TN.
256 x 256 pixels
MRI-T2
128 x 128 pixels
PET
Target Image ?
Template Image ?
![Page 20: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/20.jpg)
InterpolationInterpolation
Nearest NeighborNearest Neighbor
Tri-linear InterpolationTri-linear InterpolationPartial-Volume Interpolation Partial-Volume Interpolation
Higher order partial-volume Higher order partial-volume interpolationinterpolation
![Page 21: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/21.jpg)
Evaluating similarity measure for Evaluating similarity measure for each transformationeach transformation
y
Template Image
Transform
x
y
Target Image
x
![Page 22: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/22.jpg)
OptimizationOptimization
Powell’s Direction Set methodPowell’s Direction Set method
Downhill Simplex methodDownhill Simplex method
![Page 23: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/23.jpg)
Multi-resolutionMulti-resolution
Why Multi-resolutionWhy Multi-resolution Methods for detecting optimality can not guarantee that Methods for detecting optimality can not guarantee that a global optimal value will be found.a global optimal value will be found.
Time to evaluate the registration criterion is proportional Time to evaluate the registration criterion is proportional to the number of voxels.to the number of voxels.
The result at coarser level is used as the starting The result at coarser level is used as the starting point for the finer level.point for the finer level.
Currently multi-resolution approaches:Currently multi-resolution approaches:Sub-sampling Sub-sampling
AveragingAveraging
WaveletWavelet
![Page 24: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/24.jpg)
Registration Result (I)Registration Result (I)
A typical superposition of CT-MR images.
Left : before registration Right: after registration.
![Page 25: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/25.jpg)
Rigid transformation (II)Rigid transformation (II)
A typical superposition of MR-PET images.
Left : before registration Right: after registration.
![Page 26: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/26.jpg)
MammographyMammographyBreast cancer is the second leading cause Breast cancer is the second leading cause of death among women in USA.of death among women in USA.Detected in its early stage, breast cancer Detected in its early stage, breast cancer is most treatable.is most treatable.MammographyMammography is the main tool for is the main tool for detection and diagnosis of breast detection and diagnosis of breast malignances.malignances.It reduces breast cancer mortality by 25% It reduces breast cancer mortality by 25% to 30% for women in the 50 to 70 age to 30% for women in the 50 to 70 age group group
![Page 27: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/27.jpg)
Mammogram RegistrationMammogram Registration
Temporal/bilateral mammograms Temporal/bilateral mammograms varyvary– Breast compressionBreast compression– Breast positionBreast position– Imaging TechniqueImaging Technique– Change in BreastChange in Breast
![Page 28: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/28.jpg)
Mammogram registration Mammogram registration techniquestechniques
Whole breast area vs. regionalWhole breast area vs. regional
Nipple locationNipple location
Control-point locationControl-point location
Rigid & non-rigid registrationRigid & non-rigid registration
![Page 29: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/29.jpg)
Non-rigid Mammogram RegistrationNon-rigid Mammogram Registration
Intensity-basedIntensity-based
Elastic transformationElastic transformation
Multi-resolutionMulti-resolution
Demons algorithm (Thirion, 1996)Demons algorithm (Thirion, 1996)
![Page 30: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/30.jpg)
DemonsDemons
Scene (Target)
Model (Template)
Transform
![Page 31: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/31.jpg)
Demons (Cont.)Demons (Cont.)
Scene
Model
Transform
Forces
![Page 32: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/32.jpg)
Demons (Cont.)Demons (Cont.)
Scene
Gradient
Intensity
Space
Desired Displacement
CurrentEstimation
![Page 33: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/33.jpg)
DemonsDemons From Optical Flow From Optical Flow
Scene: f, Model: gScene: f, Model: g
Assumption: The intensity of a Assumption: The intensity of a moving object is constant with time moving object is constant with time
(1)
(2)
![Page 34: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/34.jpg)
Description of the ApproachDescription of the Approach1.1. Select demon points.Select demon points.2.2. Compute the force Compute the force uu on the model on the model
at each of the selected demonsat each of the selected demons3.3. Determine a global transformation Determine a global transformation
based on the computed based on the computed uu and apply and apply it to the modelit to the model
4.4. If the model images is now If the model images is now registered to the scene image, stop. registered to the scene image, stop. Else, go to Step 2.Else, go to Step 2.
![Page 35: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/35.jpg)
Registration ComponentsRegistration Components
Image IntensitiesImage Intensities
Non-rigid transformation, one Non-rigid transformation, one displacement vector for each pixeldisplacement vector for each pixel
Bilinear interpolationBilinear interpolation
Absolute difference as similarity Absolute difference as similarity metricmetric
Multi-resolutionMulti-resolution
Dataset: MIAS,DDSMDataset: MIAS,DDSM
![Page 36: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/36.jpg)
Demons Results (I) Synthetic Images
Level=2
Level=3
Level=4
Level=5
![Page 37: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/37.jpg)
Demons Result (II) MIASDemons Result (II) MIAS
Before registration
After rigid registration
Original images
After non-rigid registration
![Page 38: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/38.jpg)
Ongoing registration topicsOngoing registration topics
Trade-off of computation and Trade-off of computation and accuracyaccuracy
Evaluation of registration resultsEvaluation of registration results
Visualization of registrationVisualization of registration
![Page 39: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/39.jpg)
Applications: Change DetectionApplications: Change Detection
Images taken at different timesImages taken at different times
Following registration, the Following registration, the differences between the images may differences between the images may be indicative of changebe indicative of change
Deciding if the change is really there Deciding if the change is really there may be quite difficult may be quite difficult
![Page 40: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/40.jpg)
Other ApplicationsOther ApplicationsMulti-subject registration to develop Multi-subject registration to develop organ variation atlases.organ variation atlases.– Used as the basis for detecting Used as the basis for detecting
abnormal variationsabnormal variations
Object recognition - alignment of Object recognition - alignment of object model instance and image of object model instance and image of unknown object (segmentation)unknown object (segmentation)
![Page 41: Medical Image Registration Yujun Guo Dept.of CS Kent State University](https://reader035.vdocument.in/reader035/viewer/2022062516/56649d605503460f94a41fb0/html5/thumbnails/41.jpg)
ReferencesReferencesMaes F,Collignon A, et al. “Multimodality image Maes F,Collignon A, et al. “Multimodality image registration by maximization of mutual registration by maximization of mutual information.” information.” IEEE Trans. Med. ImagingIEEE Trans. Med. Imaging. 1997, . 1997, V16,pp187-198V16,pp187-198
L.G.Brown, “A survey of image registration L.G.Brown, “A survey of image registration techniques,” techniques,” ACM Computing SurveysACM Computing Surveys, vol. 24, , vol. 24, no. 4, pp. 325–376, 1992.no. 4, pp. 325–376, 1992.
Jean-Philippe Thirion, “Non-Rigid Matching Using Jean-Philippe Thirion, “Non-Rigid Matching Using Demons,” IEEE Conference on Computer Vision Demons,” IEEE Conference on Computer Vision and Pattern Recognition,1996and Pattern Recognition,1996