multimodal registration of medical data prof. leo joskowicz school of computer science and...
Post on 19-Dec-2015
215 views
TRANSCRIPT
![Page 1: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/1.jpg)
Multimodal Registration of Medical Data
Prof. Leo Joskowicz
School of Computer Science and Engineering
The Hebrew University of Jerusalem
![Page 2: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/2.jpg)
Intensity-based rigid registration (1)
• Use intensity information to define the measure of similarity between two data sets
• Rationale: the closer the data sets are, the more similar their intensity values are.
• No segmentation is necessary! The entire data set is used. Slow, especially for 3D data sets.
• The parametric space of transformations is searched incrementally from an initial configuration. The search space is six-dimensional (3 rotations and 3 translations)
![Page 3: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/3.jpg)
Intensity-based registration (2)• Similarity measures
– cross correlation– histogram correlation– mutual information– intensity values
• Uses: brain CT/MRI, Xray/CT• Example: fluoroscopic Xray to CT
![Page 4: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/4.jpg)
Xray/CT registration
• Problem definition: given– preoperative CT data set of rigid structure– intraoperative Xray images from a calibrated
camera at relatively known spatial configurations
• Find a rigid transformation that matches the CT data set to the intraoperative Xrays so that if Xrays of the CT were taken from the transformed camera positions, the resulting Xray images would be identical to the intraoperative ones.
![Page 5: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/5.jpg)
Xray/CT registration setup
2D/3D registration problem!
Ref C-arm
Ref patientCt slices
Ref ct volume
DRR
Fluoroscopic image
![Page 6: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/6.jpg)
Xray to CT registration algorithm
Input: preop CT, intraop Xray Ifluoro , intrinsic Xray camera parameters, initial guess p0 for camera pose
1. generate simulated Xray IDRR (called digitally reconstructed radiograph, or DRR) at camera pose pi
2. Compute dissimilarity between IDRR and Ifluoro by comparing their intensities
3. Compute a new camera pose pi+1 = pi + d that best reduces the dissimilarity between IDRR ,and Ifluoro)
repeat until no progress can be made
![Page 7: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/7.jpg)
Digitally reconstructed radiographs
![Page 8: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/8.jpg)
Generating DRRs
For each pixel in the DRR plane, construct the ray emanating from the camera focal point. Sum up the intensities of the CT voxel values according to the Xray attenuation formula to obtain the gray level value of the DRR pixel.
DRR
CT
Camera
![Page 9: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/9.jpg)
Generated DRRs
![Page 10: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/10.jpg)
Real X-ray vs DRR
![Page 11: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/11.jpg)
Similarity measure Pairwise comparison of normalized pixel
intensity values:• IDRR(i,j) and Ifluoro (i,j) are the pixel values
• IDRR and Ifluoro are the average image values
• T is the region of interest
![Page 12: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/12.jpg)
Examples of initial poses registrations (DRRs only)
![Page 13: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/13.jpg)
Actual use: radiation therapy with the Cyberknife (radiation therapy)
![Page 14: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/14.jpg)
Frameless radiation therapyStereotactic setup
Track head with Xraysbefore each dose application
![Page 15: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/15.jpg)
Matching skull X-ray and DRR
Match only regions
![Page 16: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/16.jpg)
Intensity-based registration
• Advantages – no segmentation, automatic– selective regions – potentially accurate
• Disadvantages– large seach space, many local minima– slow
![Page 17: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/17.jpg)
Deformable registration: scope
• Necessary for soft tissue organs and for cross-patient comparisons– brain images before and during surgery– anatomical structures at different times or from
patients: tumor growth, heart beating, compare– matching to atlases
• Much more difficult than rigid registration!– problem is ill-posed; solution is not unique– error measurements and comparisons are difficult– local vs. global deformations?
![Page 18: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/18.jpg)
Deformable registration: properties• Mapping transformation can be
– global, e.g., a bi- or tri-variate polynomial– local, e.g.a fine grid with displacement vectors
• Define an energy function that should be minimized to make the data sets match.
• Usually comes after rigid registration to get an approximate position estimate.
• Both geometry based and intensity-based techniques exist.
![Page 19: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/19.jpg)
rigid affine triliear quadratic
quadratictransform
Mathematics of deformationsGlobal transformations
Local spline deformation
![Page 20: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/20.jpg)
Square shift
Global deformation transformations
Scale
![Page 21: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/21.jpg)
Local grid-based deformable registration
image 1 image 2image 1+2 deformation map
![Page 22: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/22.jpg)
Example: MRI slice matching image 1 image 2 after registration
difference image without deformation
difference image with deformation
![Page 23: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/23.jpg)
Brain tumor matching - 2D map
![Page 24: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/24.jpg)
Brain tumor matching - 3D map
source target match
![Page 25: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/25.jpg)
Initial configuration
After rigid registration
After deformableregistration with local splines
Example: spine matching
![Page 26: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/26.jpg)
Deformable registration techniques
• Too many to list here!– Optical flow model– Physics-based: elastic and fluid models
• Use an elastic or deformable model
• Validation is difficult
![Page 27: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/27.jpg)
Commercial products• Medical image processing software packages
include some registration capabilities (manual or semi-automatic feature selection)
• Contact-based rigid registration of CT and optical tracker in orthopaedics and neurosurgery (half a dozen companies)
• Intraoperative Open MR to tracker rigid registration
![Page 28: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/28.jpg)
The future: research directions• In many areas, the problem is far from solved:
similar to image segmentation!• Much clinical validation is needed. More
coverage of other anatomy (60% focus on brains!)
• Interleave segmentation and registration
• Difficult data sets: 2D and 3D ultrasound images, video sequences, portal images
• Model-based techniques are the most likely to be sufficiently robust for clinical use
• Integration requirements are very important.
![Page 29: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/29.jpg)
Bibliography (1)• Two chapters on registration in Computer-Integrated
Surgery, Taylor et al, MIT Press, 1995. • Medical Image Registration, Hajnal et al, CRC Press 2001• “A survey of medical image registration”, Maintz and
Viergever, Medical Image Analysis Journal, 2(1), Oxford University Press 1998 (over 150 references!)
• “A method for registration of 3D shapes”, Besl and McKay, IEEE Trans. on Pattern Analysis, 14(2), 1992.
• Special issue on Biomedical Image Registration, Image and Vision Computing, Vol 19(1-2), 2001.
• “Deformable models in medical image analysis: a survey”, McInerney and Terzopolous, Medical Image Analysis 1(2), 1996.
![Page 30: Multimodal Registration of Medical Data Prof. Leo Joskowicz School of Computer Science and Engineering The Hebrew University of Jerusalem](https://reader038.vdocument.in/reader038/viewer/2022110207/56649d405503460f94a19926/html5/thumbnails/30.jpg)
Bibliography (2)• “Retrospective registration of tomographic brain
images”, J. Mainz, PhD Thesis, Utrecht U., 1996 www.cs.ruu.nl/people/twan/personal/list.html
• “Localy affine registration of free-form surfaces”, J. Feldmar and N. Ayache, Proc. IEEE CVPR , 1994.
• “Matching 3D anatomical surfaces with non-rigid deformations using octree splines”, R. Szeliski and S. Lavallee, Int. Journal of Computer Vision 18(2), 1996.
• “Fast intensity-based non-rigid matching”, P. Thirion, Proc. Conf. Medical Robotics and CAS, 1995.
• “Multimodal volume registration by maximization of mutual information, W.Wells, P.Viola, R.Kikinis. (idem)