statistics of anatomic geometry: information theory and automatic model building

Combining the strengths of UMIST andThe Victoria University of Manchester MICCAI 2005: Statistics of Anatomic Geometry

Statistics of Anatomic Geometry:

Information Theory and Automatic Model Building

Carole Twining

Imaging Science and Biomedical Engineering (ISBE)

University of Manchester, UK

Contributions from:

Rhodri Davies, Stephen Marsland, Tim Cootes, Vlad Petrovic,

Roy Schestowitz, & Chris Taylor


Slide 2

Overview Recap of Point Distribution/Statistical Shape Models PDMs/SSMs

● Correspondence Problem: Shape Representation & Correspondence Correspondence & Statistics Methods for establishing correspondence

● Automatic Methods for Groupwise Shape Correspondence Manipulating Correspondence not Shape Minimum Description Length objective function Optimisation

● Extension to Images:

MDL Groupwise Registration

• automatic models from unannotated image sets

● Model Evaluation Criteria


Slide 3

Point Distribution Models (PDMs)Statistical Shape Models (SSMs)

Set of Shapes& Corresponding

PointsShape Space

PCA

ModelPDF


Slide 4

Adding Image Information

Shape Space Shape & Appearance Space


Slide 5

● Include image information from

whole region

● Correlation between shape & texture

Adding Image Information

Shape Model Shape & Texture Model


Slide 6

Active Shape & Appearance Models

ASM Search

AAMSearch


The Correspondence Problem


Slide 8

Shape Representation & Correspondence

● Non-Local Representations

Fourier descriptors (e.g., SPHARM)

Medial descriptors (e.g., MREPS)

● Local Representations

Point based (e.g., PDMs/SSMs)

● Common Representation of training set => Correspondence

Non-local tends to give implicit correspondence

Point based gives explicit correspondence

● Why does the correspondence matter?


Slide 9

Correspondence & Statistics

Shape Space Shape Space

Varying correspondence varies the shape statistics


Slide 10

Establishing Correspondence

● Manual landmarking

● Arbitrary parameterisations

Kelemen, Hill, Baumberg & Hogg

● Shape features

Wang, Brett

● Image registration

models from deformation field

Christensen, Joshi, Lavalle, Reuckert, Twining


Slide 11

Manual Methods for Correspondence

● Manual Landmarks

Interpolate for dense

correspondence

May need to adjust

● Problems:

Time-consuming

Subjective

Requires expert anatomical knowledge

Very difficult in 3D


Slide 12

Arc-Length Parameterisation● Equally-space landmarks around each shape

(Baumberg & Hogg)


Slide 13

Shape Features● e.g. Curvature-based methods

● Intuitive

● But:

What about regions without such features?

Not really groupwise, since depends on local properties of each shape

Is it really the best correspondence?


Automatic Groupwise Correspondence


Slide 15


Desirable features:

● Groupwise:

Depends on whole set of shapes

● Automatic – little or no user intervention

● 2D & 3D

● Runs in reasonable time!


Slide 16


Optimisation Problem Framework:

● Method of manipulating correspondence:

2D & 3D

● Objective function:

quantifies the ‘quality’ of the correspondence

● Optimization Scheme


Manipulating Correspondence


Slide 18

Manipulating Correspondence● Point-to-Point:

Shape 1 Shape 2

Shape Points

Correspondence Points

Varying correspondence varies shape!

Vary correspondence but not shape!


Slide 19

Manipulating Correspondence● Continuous parameterisation of shape

● Re-parameterising varies correspondence


Slide 20

● Generalises to 3D

● Map surface to parameter sphere - no folds or tears

● Varying parameterisation on sphere

Manipulating Correspondence

ShapeSphere & Spherical Polar coordinates


Objective Function


Slide 22

Objective Function● Varying Correspondence = Varying Statistics

● Objective function based on model probability density function

number of model modes

compactness

quality of fit to training data

number of model parameters

Shape Space Shape Space


Slide 23

Shape Space

MDL Objective Function

● Transmit training set as encoded binary message

● Shannon:

Set of possible events {i} with probabilities {pi}

Optimal codeword length for event i: -log pi

● Encode whole training set of shapes:

Encoded Model: mean shape, model modes etc

• Reconstruct shape space and model pdf

Each training shape: pi from model pdf

• Reconstruct all training shapes

● MDL Objective function = total length of message


Slide 24

MDL Objective Function

● Fit between model pdf and training data:

Probabilities for training points => better the fit, shorter the message

● Too complex a model:

model parameter term large

● Too few modes:

Bad fit to data & large residual

● Badly chosen modes:

Bad fit to data


Slide 25

Optimisation● Genetic algorithm search (Davies et al, 2002)

All parameters optimised simultaneously

Slow, scales badly with no of examples

● More recent, multi-scale, multi-resolution approaches:

better convergence

fast enough for routine use

scales approximately linearly with no of examples

(Davies et al, IPMI 2003)


Slide 26

Results● Quantitatively better results compared to SPHARM

● Differences tend to be subtle

● Comparing techniques, have to go beyond visual inspection

(see section on Model Evaluation Criteria)


MDL Groupwise Image Registration


Slide 28

Image & Shape Correspondence● Groups of Shapes:

groupwise dense correspondence

statistical models of shape variability

• analysis of variation across & between populations

• assist in analysing unseen examples (ASM & AAM)

● Groups of Images:

groupwise dense correspondence = groupwise registration

statistical models of shape & appearance

• as above

● MDL technique for correspondence can be applied to both

(Twining et al 2005)


Slide 29

● Spatial Correspondence between images Shape variation

● Warp one to another Difference is texture variation

● Repeat across group => Appearance model of image set

Image Registration


Slide 30

Groupwise Image Registration● MDL Objective Function

Combined shape & texture model

● Define dense correspondence triangulated points on each image & interpolate

● Manipulate Correspondence

● Increase resolution of mesh & repeat


Slide 31

Results● 104 2D brain slices

● Appearance

Model


Model Evaluation Criteria


Slide 33

Model Evaluation Criteria● Need to go beyond visual inspection, subtle differences

● Generalisability:

the ability to represent unseen shapes/images which belong to the same class as those in the training set

● Specificity:

the ability to only represent images similar to those seen in the training set

● Quantitative comparison of models


Slide 34

General but not Specific

Specificity and Generalization

Specific but not General

Training Set:

Sample Set from model pdf:

Space of Shapes/Images


Slide 35

Specificity

Training Set

Sample Set

:distance on image/shape space


Slide 36

Generalisation Ability

Sample Set

Training Set


Slide 37

Validation

● Annotated/Registered Data

● Perturb Registration

GeneralisationSpecificity

Size of Perturbation

Objective function


Slide 38

Evaluating Brain Appearance Models


Slide 39

Summary● Manipulating Correspondence

Shown to produce quantitatively better models

Large-scale Optimisation problem - so far, only linear models

Extension to other shape representation methods (e.g. MREPS)

Topology – manipulate parameter space:

• simple, fixed topology

Multi-part objects

Differences tend to be subtle - go beyond visual inspection of results

• Model evaluation criteria

Extension to groupwise image registration


Questions?


Slide 41

Resources & ReferencesAAMs, ASMs

● [1] T. F. Cootes, G. J. Edwards, and C. J. Taylor,

Active appearance models,

IEEE Trans. Pattern Anal. Machine Intell., vol. 23, no. 6, pp. 681-685, 2001.

● [2] T. F. Cootes, C. J. Taylor, D. H. Cooper and J. Graham,

Active shape models – their training and application,

Computer Vision and Image Understanding, 61(1), 38-59, 1995

● [3] T. F. Cootes, A. Hill, C. J. Taylor, and J. Haslam,

The use of active shape models for locating structures in medical images,

Image and Vision Computing, vol. 12, no. 6, pp. 276-285, July 1994.

● [4] B. van Ginneken, A.F.Frangi, J.J.Stall, and B. ter Haar Romeny,

Active shape model segmentation with optimal features,

IEEE Trans. Med. Imag., vol. 21, pp. 924-933, 2002.

● [5] P. Smyth, C. Taylor, and J. Adams,

Vertebral shape: Automatic measurement with active shape models,

Radiology, vol. 211, no. 2, pp. 571-578, 1999.

● [6] N. Duta and M. Sonka,

Segmentation and interpretation of MR brain images: An improved active shape model,

IEEE Trans. Med. Imag., vol. 17, pp. 1049-1067, 1998.

Further references, as well as notes on the historical meanderings in the development of these techniques

can be found on Tim Cootes’ website:

http://www.isbe.man.ac.uk/~bim/


Slide 42

Resources & References MREPS● [7] S. M. Pizer, D. Eberly, D. S. Fritsch, and B. S. Morse,

Zoom-invariant vision of figural shape: The mathematics of cores,

Computer Vision and Image Understanding, vol. 69, no. 1, pp. 055-071, 1998.

Fourier descriptors, spherical harmonics & SPHARM

● [8] C. Brechb¨uhler, G. Gerig, and O. Kubler,

Parameterisation of closed surfaces for 3D shape description,

Computer Vision, Graphics and Image Processing, vol. 61, pp. 154-170, 1995.

● [9] A. Kelemen, G. Szekely, and G. Gerig,

Elastic model-based segmentation of 3D neurological data sets,

IEEE Trans. Med. Imag., vol. 18, no. 10, pp. 828-839, Oct. 1999.

● [10] C. Brechb¨uhler, G. Gerig, and O. K uhler,

Parametrization of closed surfaces for 3D shape description,

Computer Vision and Image Understanding, vol. 61, no. 2, pp. 154-170, 1995.

● [11] G. Szekely, A. Kelemen, C. Brechbuhler, and G. Gerig,

Segmentation of 2-D and 3-D objects from MRI volume data using constrained elastic deformations

of flexible fourier contour and surface models,

Medical Image Analysis, vol. 1, pp. 19-34, 1996.


Slide 43

Resources & ReferencesFourier descriptors, spherical harmonics & SPHARM

● [12] D. Meier and E. Fisher,

Parameter space warping: Shape-based correspondence between morphologically different objects,

IEEE Trans. Med. Imag., vol. 21, no. 1, pp. 31-47, Jan. 2002.

● [13] M. Styner, J. Liberman, and G. Gerig,

Boundary and medial shape analysis of the hippocampus in schizophrenia,

in Proc. International Conference on Medical Image Computing and Computer Aided Intervention

(MICCAI), 2003, pp. 464-471.

Feature-Based Shape correspondence● [14] A. D. Brett, A. Hill, and C. J. Taylor,

A method of automatic landmark generation for automated 3D PDM construction,

Image and Vision Computing, vol. 18, pp. 739-748, 2000.

● [15] Y. Wang, B. S. Peterson, and L. H. Staib,

Shape-based 3D surface correspondence using geodesics and local geometry,

in Proc. IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2000, pp. 644-651.

● [16] G. Subsol, J. Thirion, and N. Ayache,

A scheme for automatically building three-dimensional morphometric anatomical atlases: application

to a skull atlas,

Medical Image Analysis, vol. 2, no. 1, pp. 37-60, 1998.


Slide 44

Resources & ReferencesElastic and Distortion based methods of shape correspondence● [17] M. Kaus, V. Pekar, C. Lorenz, R. Truyen, S. Lobregt, and J. Weese,

Automated 3-D PDM construction from segmented images using deformable models,

IEEE Trans. Med. Imag., vol. 22, no. 8, pp. 1005-1013, Aug. 2003.

● [18] C. Shelton,

Morphable surface models,

International Journal of Computer Vision, vol. 38, pp. 75-91, 2000.

● [19] S. Sclaroff and A. P. Pentland,

Modal matching for correspondence and recognition,

IEEE Trans. Pattern Anal. Machine Intell., vol. 17, no. 6, pp. 545-561, 1995.

● [20] F. L. Bookstein,

Landmark methods for forms without landmarks: morphometrics of group differences in outline shape,

Medical Image Analysis, vol. 1, no. 3, pp. 225-244, 1997.

Minimum Description LengthThis is the information theory stuff behind MDL.

● [21] J. Rissanen, Lectures on Statistical Modeling Theory,

http:\\www.cs.tut.fi\~rissanen\papers\lectures.pdf

● [22] J. Rissanen,

Stochastic Complexity in Statistical Inquiry,

World Scientific Press, 1989.


Slide 45

Resources & ReferencesMDL for Shape CorrespondenceApproximate MDLNote that the freely available code distributed by Thodberg is only approximate MDL, not full state-ofthe-

art MDL as used by other groups. In fact, the objective function used in these papers is equivalent

to what is used to initialise other algorithms. This fact has caused a little confusion in the literature.

● [23] H. Thodberg,

MDL shape and appearance models,

in Proc. 18th Conference on Information Processing in Medical Imaging (IPMI), 2003, pp. 51-62.

● [24] H. Thodberg and H. Olafsdottir,

Adding curvature to MDL shape models,

in Proc. 14th British Machine Vision Conference (BMVC), vol. 2, 2003, pp. 251-260.

● [25] T. Heimann, I. Wolf, T. G. Williams, and H.-P. Meinzer,

3D Active Shape Models Using Gradient Descent Optimization of Description Length ,

IPMI 2005.

MDL for 2D ShapeThis uses the initial genetic algorithm search, which was later improved upon.

● [26] R. H. Davies, C. J. Twining, T. F. Cootes, J. C. Waterton, and C. J. Taylor,

A minimum description length approach to statistical shape modelling,

IEEE Trans. Med. Imag., vol. 21, no. 5, pp. 525-537, May 2002.

● [27] R. H. Davies, C. J. Twining, P. D. Allen, T. F. Cootes, and C. J. Taylor,

Building optimal 2D statistical shape models,

Image and Vision Computing, vol. 21, pp. 1171-1182, 2003.


Slide 46

Resources & ReferencesMDL for 3D Shape

● [28] R. H. Davies, C. J. Twining, T. F. Cootes, J. C. Waterton, and C. J. Taylor,

3D statistical shape models using direct optimisation of description length,

in Proc. 7th European Conference on Computer Vision (ECCV), 2002, pp. 3-21.

MDL for Image Registration● [29] C. J. Twining, T. Cootes, S. Marsland, V. Petrovic, R. Schestowitz, and C. J. Taylor,

A Unified Information-Theoretic Approach to Groupwise Non-Rigid Registration and Model

Building, Presented at IPMI 2005

● [30] C. J. Twining, S. Marsland, and C. J. Taylor,

Groupwise Non-Rigid Registration: The Minimum Description Length Approach,

In Proceedings of BMVC 2004.

● [31] C.J. Twining and S. Marsland,

A Unified Information-Theoretic Approach to the Correspondence Problem in Image Registration,

International Conference on Pattern Recognition (ICPR), Cambridge, U.K. 2004.

statistics of anatomic geometry: information theory and automatic model building

Documents