![Page 1: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/1.jpg)
Unified Joint Feature Registration for Brain Anatomical Alignment
Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan
Image Processing and Analysis GroupDepartments of Electrical Engineering
Yale University
![Page 2: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/2.jpg)
Brain Anatomical Alignment• Brains are different:
– Shape.– Structure.
• Direct comparison of brains between different subject is not very accurate.
• Statistically and quantitatively more accurate study requires the brain image data to be put in a common “normalized” space through alignment.
• Examples of areas that need brain registration:– Studying structure-function connection.– Tracking temporal changes.– Generating probabilistic atlases.– Creating deformable atlases.
![Page 3: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/3.jpg)
Studying Function-Structure Connection
Brain Function
Image
Alignment of Subjects
Comparison of Subjects After Alignment
Direct Comparison of Subjects Distribution Before Alignment
Distribution After Alignment
![Page 4: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/4.jpg)
Inter-Subject Brain Registration
• Inter-subject brain registration: – Alignment of brain MRI images from different
subjects to remove some of the shape variability.
• Difficulties:– Complexity of the brain structure.– Variability between brains.
• Brain feature registration: – Choose a few salient structural features as a
concise representation of the brain for matching.
– Overcome complexity: only model important structural features.
– Overcome variability: only model consistent features.
![Page 5: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/5.jpg)
Previous Work: 3D Sulcal Point Matching
Feature Extraction Extracted Point Features
![Page 6: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/6.jpg)
Previous Work: 3D Sulcal Point Matching
Overlay of 5 subjects before TPS alignment:
After TPS alignment:
![Page 7: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/7.jpg)
A Unified Feature Registration Method
Outer Cortex Surface
Major Sulcal Ribbons
All FeaturesPoint Feature
Representation
Point Feature Representation
Feature Extraction Feature Fusion
Feature
Matching
Subject I
Subject II
![Page 8: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/8.jpg)
Non-rigid Feature Point Registration
![Page 9: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/9.jpg)
Unification of Different Features
• Ability to incorporate different types of geometrical features.– Points.
– Curves.
– Open surface ribbons.
– Closed surfaces.
• Simultaneously register all features --- utilize the spatial inter-relationship between different features to improve registration.
![Page 10: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/10.jpg)
Joint Clustering-Matching Algorithm (JCM)
![Page 11: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/11.jpg)
Overcome Sub-sampling Problem
• Sub-sampling (e.g. clustering) reduces computational cost for matching.
• In-consistency problem with sub-sampling:
• The in-consistency can be overcome by sub-sampling (clustering) and matching simultaneously.
![Page 12: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/12.jpg)
Joint Clustering-Matching Algorithm (JCM)
• JCM:
• Reduce computational cost using sub-sampled cluster centers.
• Accomplish optimal cluster placement through joint cluster-matching.
• Symmetric: two way matching.
MatchingClusters Center Set V
Clustering
Cluster Center Set U
Clustering
Point Set X Point Set YOriginal RPM
• Diagram:
![Page 13: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/13.jpg)
JCM Energy Function
MatchingClusters Center Set V
Clustering
Cluster Center Set U
Clustering
Point Set X Point Set Y
K
a iai
xai
xN
vxm1 1
2||||
K
a jaj
yaj
yN
uym1 1
2||||
K
axaxa Lfvfu
1
22 ||||||),(||
K
ayaya Lfufv
1
22 ||||||),(||
Annealing: .loglog1 11 1
K
a
N
i
yai
yai
K
a
N
i
xai
xai
yx
mmTmmT
![Page 14: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/14.jpg)
JCM Energy Function
• Fuzzy assignment + least squares energy function:
• Row and column summation constraints.
),( ME
K
a iai
xai
xN
vxm1 1
2||||
K
a jaj
yaj
yN
uym1 1
2||||
K
axaxa Lfvfu
1
22 ||||||),(||
K
ayaya Lfufv
1
22 ||||||),(||
.loglog1 11 1
K
a
N
i
yai
yai
K
a
N
i
xai
xai
yx
mmTmmT
![Page 15: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/15.jpg)
JCM Example
• Matching 2 face patterns with JCM (click to play movie).
![Page 16: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/16.jpg)
Experiments
![Page 17: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/17.jpg)
Comparison of Different Features
• Different features can be used in our approach.
• Two types of features investigated:– Outer cortex surface.
– Major sulcal ribbons.
• Comparison of different methods:
Method I Method II Method III
![Page 18: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/18.jpg)
Synthetic Study Setup
Template True Deformation (GRBF)
Target
Template RecoveryEstimated Deformation
(TPS)
Error Evaluation
Feature Matching
Change the choice of features to
compare method I, II and III
![Page 19: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/19.jpg)
Results: Method I vs. Method III
• Outer cortical surface alone can not provide adequate information for sub-cortical structures.
• Combination of two features works better.
![Page 20: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/20.jpg)
Results: Method II vs. Method III
• Major sulcal ribbons alone are too sparse --- the brain structures that are relatively far away from the ribbons got poorly aligned.
• Combination of two features works better.
![Page 21: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/21.jpg)
Conclusion
• Combination of different features improves registration.
• Unified brain feature registration approach:– Capable of estimating non-rigid transformations without the
correspondence information.
– General + unified framework.
– Symmetric.
– Efficient.
![Page 22: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/22.jpg)
Acknowledgements
• Members of the Image Processing and Analysis Group at Yale University: – Hemant Tagare.– Lawrence Staib. – Xiaolan Zeng. – Xenios Papademetris. – Oskar Skrinjar. – Yongmei Wang.
• Colleagues in the brain registration project:– Joseph Walline.
• Financial support is provided by the grants from the Whitaker Foundation, NSF, NIH.
![Page 23: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/23.jpg)
Future Work
![Page 24: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/24.jpg)
Estimating An Average Shape
• Given multiple sample shape (sample point sets), compute the average shape for which the joint distance between the samples and the average is the shortest.
Average ?
• Difficult if the correspondences between the sample points are unknown.
![Page 25: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/25.jpg)
“Super” Clustering-Matching Algorithm (SCM)
• Diagram:
MatchingMatchable
ClustersOutlier Cluster
Clusters Center Set V
Clustering
Matchable Clusters
Outlier Cluster
Clusters Center Set U
Clustering
Point Set X Point Set Y
Average Point Set Z
Matching and
Estimating
![Page 26: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/26.jpg)
![Page 27: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/27.jpg)
End
• Further Information:– Web site: http://noodle.med.yale.edu/~chui/
![Page 28: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/28.jpg)
End
![Page 29: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/29.jpg)
2D Examples of RPM
![Page 30: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/30.jpg)
![Page 31: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/31.jpg)
![Page 32: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/32.jpg)
Point Matching
Example Application: Face Matching
![Page 33: Unified Joint Feature Registration for Brain Anatomical Alignment Haili Chui, Robert Schultz, Lawrence Win, James Duncan and Anand Rangarajan Image Processing](https://reader035.vdocument.in/reader035/viewer/2022062407/56649d365503460f94a0eef6/html5/thumbnails/33.jpg)
Example Application: Face Matching