Quantitative analysis of radiologic images:
Image segmentation and registration, statistical atlases
Christos Davatzikos, Ph.D.
Professor of Radiology
Section of Biomedical Image Analysis
http://www.rad.upenn.edu/sbia
You can control a quantity if you can measure or weigh it
Lord Kelvin, 1824-1907
Need to develop tools that obtain accurate and precise measurement from image data
Expert 1: Total Lesion volume: 15,635 mm^3 Expert 2: Total Lesion volume: 7,560 mm^3
Human limitations in measuring: inter-rater differences
Major limitation for:
1) Diagnosis of disease stage
2) Monitoring the effect of treatments
Quantification/measurement:
- ~3% longitudinal atrophy of the hippocampus in early AD patients
- Contraction pattern of the cardiac muscle
- a 5% change in radiologic signal could be indicative of evolving pathology
More human limitations
Visually detecting morphological abnormalities
Scan 1 Scan 2
Visually detecting morphological abnormalities
Scan 1 Scan 2
30% atrophy!
Manual Drawing of anatomical structures
Visual evaluation of a 3% atrophy is practically impossible Laborious and not well-reproducible manual outlining is required
•Evaluating complex spatio-temporal patterns of radiologic signal change, especially if the magnitude of the signal change is small and anatomical variability is large
Kahneman and Tversky in their Nobel prize winninng careers studied human reasoning under uncertainty and demonstrated the limitations of human reasoning in evaluating conjunctions, i.e. A and B and C …
Even more fundamental limitations of human evaluation
Detecting spatially complex very subtle anatomical abnormalities
Normal Schizophrenia patient
?
Healthy Mildly Cognitively impaired:Prodromal stage to Alzheimer’s
Detecting spatially complex very subtle anatomical abnormalities
?
Functional activity during truth telling and lying
Lies
Truths
Brain and criminal behavior
-2
-1.8
-1.6-1.4
-1.2
-1
-0.8
-0.6-0.4
-0.2
0
TotalFrontal
Left
TotalFrontalRight
TotalParietal
Left
TotalParietalRight
TotalTemporal
Left
TotalTemporal
Right
TotalOccipital
Left
TotalOccipital
Right
lateralventricle
left
lateralventricle
right
Computers can complement and assist humans in many ways
Statistical anatomical atlases: from single-individual anatomical examples, to atlases
capturing variability in a population
Analogous to training of a human reader
• Disease identification (learn variation of normal anatomy identify abnormality as a deviation from normal variation)
• Integration of data from multiple individuals in order to discover systematic relationships among radiologic and clinical measurements
-Does a lesion in a particular part of the brain correlate with a certain neurological deficit?
-Does prostate cancer appear uniformly throughout the prostate or does it tend to appear in certain regions more frequently what is the optimal way of biopsying/treating a patient in order to maximize probability of cancer detection/elimination?)
-What is the normal variation of hippocampal size for a given age?
- What is the normal variation of cardiac shape and deformation?
Image Registration: Integration and Comparative Analysis of Images from different individuals / modalities / times /conditions
BeforeSpatialNormalization
AfterSpatialNormalization
--Image integration and co-registration helps generalize from the individual to the group, and to construct normative data abnormalities can be distinguished from normal statistical variation
Underlying biological process that results in abnormal signal, or simply normal tissue whose normal variability, in terms of image properties, needs to be measured
Overlay/Comparison of such images?
Registration and Measurement of Biological Shape D’Arcy Thompson, 1917:
• The deformation function measures the local deformation of the template:
Deformation 1 Deformation 2
Local structural measurements can be measured by analyzing the deformation functions with standard statistical methodologies
Template Shape 1 Shape 2
Red: ContractionGreen: Expansion
≈
High-Dimensional Shape Transformations
Template MR image Warped template
Significant 4-year GM changes in 107 older adults
From the cover page of
the Lancet, Neurology
RIGHT LEFT
Voxel-based analysis of tissue density maps
Effect Size Maps
NC > FTD
NC > AD
FTD > AD
Tissue atrophy map of an AD patient, relative to cognitively normal controls
Template Space
Patient’s scan
Regions of differences between schizophrenics and normal controls
Average of 148 brain images, after deformable registration to the atlas
Atlas with optimal needle positions
Apex
Base
Left Right
6
7
4
3
1 2
5
Apex
Base
Left Right
Targeted Prostate Biopsy Using Mathematical Optimization
100 Samples Template
…
Segmented 3D Prostate
Warped Prostate Atlas
US prostate image
MRI prostate image
Deformable Segmentation of Prostate Images
20 subjects, average age 64.70 20 subjects, average age 83.05
Quantitative analysis meets
visual image interpretation
40174 mm320564 mm3
“Younger Old Adult” Average Model
“Older Old Adult” Average Model
Average age 64.7 Average age 83
Using a statistical atlas to guide WM lesion segmentation
Spatial distribution of WM abnormalities in 50 older adults
(BLSA)
HAMMER: Hierarchical Attribute Matching Mechanism for Elastic
Registration
Pattern Matching: Finding Anatomical Correspondences
Attribute vector based on wavelet analysis of the anatomical context around each voxel morphological signature of each voxel
Template
A brain MRI before warping and after warping
Model
Measuring volumes of anatomical structures :An atlas with anatomical definitions is registered to the patient’s images
Subject
HAMMER HAMMER
To summarize:
• Anatomical definitions are used to create an atlas analogous to the knowledge of anatomy by humans
• Pattern matching performed hierachically at various scales is used to match the atlas to the individual
Can we use these quantitative image analysis tools as diagnostic tools?
-Combine all morphological, physiological, and clinical measurements into a broader phenotypic profile
-Use high-dimensional pattern classification and machine learning techniques
Problem: Potentially high statistical overlap for any single anatomical structure, if disease is not focal
Where is the problem?
0.001
0.0012
0.0014
0.0016
0.0018
0.002
0.0022
0.0024
0.0026
0.0028
0.0017 0.0027 0.0037 0.0047
Hippocampus Volume
En
torh
inal
Co
rtex
Vo
lum
e
Normal Controls
MCI
Data from Baltimore Longitudinal Study of Aging, Davatzikos et.al. Neurobiology of aging, in press
Pattern
Classification
Abnormality
score
A pattern is sampled by measuring brain volumes and blood flow in a number of brain regions
• Local tissue volumes and PET O15 are combined
• 15-20 brain regions (clusters) build a multi-parametric imaging profile
Abnormality Score
Measurement and Integration of Structural and Functional Patterns
0.001
0.0012
0.0014
0.0016
0.0018
0.002
0.0022
0.0024
0.0026
0.0028
0.0017 0.0027 0.0037 0.0047
Hippocampus Volume
En
torh
ina
l Co
rte
x V
olu
me
Normal Controls
MCI
PE
T-po
st c
ingu
late
Individual Diagnosis
• High-dimensional Pattern Classification (Machine learning)
• Evaluate spatial patterns of GM, WM, CSF, PET signal distribution
• Use these pattern to construct an image-based classifier, using support vector machines
L-E
RC
w
Anterior L-hipp
Brain regions that collectively contributed to classification
All GMEffect size
WM Effect size
PETEffect size
Images in radiology convention
Classification Rate vs. Number of Regions
Change of abnormality scores over time
* Clinically normal, has now gone through autopsy with Braak 4 and
moderate plaques meets AD pathology criteria
*
After removing this one participant
Normals: -0.3
MCI at latest scan: 0.26
MCI at year of conversion: 0.15
Already significant structural abnormality on year of conversion to MCI
Abnormality scores when converting from normal to MCI
Data from ADNI
AD vs CN classifier applied to MCI: most MCI’s have AD-like MRI profiles
MMSE decline
fMRI for Lie Detection: A Card Concealment Experiment
• Experiments performed by the Brain and Behavior Laboratory (Psychiatry)
• Particiapnts were asked to lie about the possession of a card of their choice
• 22 participants, both true/lie responses
• Parameter images were created using the GLM with double gamma HRF
Most discriminative brain region: 63.1%
Region1 /Structure 1
Region 2/Structure 2
Focal effects
Non-focal effects
H
P
HP
The power of true multi-variate analysis vs. mass-univariate
Training Results
-6
-4
-2
0
2
4
6
De
cis
ion
Va
lue
s
Lie - 99.26%Truth - 99.27%
Testing Results
-6
-4
-2
0
2
4
6
1 21 41 61 81 101
De
cis
ion
Va
lue
s
Lie - 90.00%Truth - 85.83%
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
1 3 5 7 9 11 13 15 17 19 21
Decis
ion
Valu
es
Lie - 95.5%Truth - 95.5%
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
1 3 5 7 9 11 13 15 17 19 21
Dec
isio
n V
alu
es
Lie - 90.9%
Truth - 86.36%
Pattern classification results
Individual images
Average images
Lie
Truth
Set of regions with predictive power
Statistical maps of group differences
Multi-variate analysis continued……..
…..combining different types of images
Image1
Image2
No single image says it all!
Computer result by combining 4 different MR acquisition protocols
ConclusionComputers can complement humans in:• Quantification
• Increased reproducibility
• Analysis of non-focal disease
• Evaluating complex spatio-temporal patterns
-patterns of longitudinal change of structure and function
- patterns of tissue motion and deformation
In the heart of computational image analysis is the notion of statistical atlases, which represent normal variation and help identify disease as a deviation from this normal range
http://www.rad.upenn.edu/sbia