landmark methods for forms without landmarks

Medical Image Analysis (1996/7) volume 1, number 3, pp 225–243c© Oxford University Press

Landmark methods for forms without landmarks: morphometrics ofgroup differences in outline shape

Fred L. Bookstein∗

Institute of Gerontology, University of Michigan, 300 North Ingalls Building, Ann Arbor, MI48109-2007, USA

AbstractMorphometrics, a new branch of statistics, combines tools from geometry, computer graphicsand biometrics in techniques for the multivariate analysis of biological shape variation. Althoughmedical image analysts typically prefer to represent scenes by way of curving outlines or surfaces,the most recent developments in this associated statistical methodology have emphasized thedomain of landmark data: size and shape of configurations of discrete, named points in twoor three dimensions. This paper introduces a combination of Procrustes analysis and thin-platesplines, the two most powerful tools of landmark-based morphometrics, for multivariate analysisof curving outlines in samples of biomedical images. The thin-plate spline is used to assign point-to-point correspondences, called semi-landmarks, between curves of similar but variable shape,while the standard algorithm for Procrustes shape averages and shape coordinates is altered toaccord with the ways in which semi-landmarks formally differ from more traditional landmarkloci. Subsequent multivariate statistics and visualization proceed mainly as in the landmark-basedmethods. The combination provides a range of complementary filters, from high pass to low pass,for effects on outline shape in grouped studies. The low-pass version is based on the spectrum ofthe spline, the high pass, on a familiar special case of Procrustes analysis. This hybrid method isdemonstrated in a comparison of the shape of the corpus callosum from mid-sagittal sections ofMRI of 25 human brains, 12 normal and 13 with schizophrenia.

Keywords: morphometrics, multivariate analysis, outlines, Procrustes analysis, thin-plate spline

Received January 15, 1996; revised December 12, 1996; accepted January 3, 1997

1. INTRODUCTION

Over the last decade, previously scattered or fragmentary tac-tics and techniques from medical image analysis, multivari-ate statistics and computational geometry have been interwo-ven very effectively in a newly standardized methodology forlandmark data. This morphometric synthesis binds togetherthe Riemannian structure of David Kendall’s shape space,multivariate statistical maneuvers in the tangent space at theProcrustes average form and graphical approaches for visual-izing a wide variety of signals in the resulting data sets. Thesynthesis brings to the analysis of medical images a biomet-rical spirit—a concern for optimal description of causes andeffects—that was hitherto limited to the more esoteric reaches

∗Corresponding author(e-mail: [email protected])

of evolutionary biology. Recent reviews include Bookstein(1996a, 1997) and the papers in Marcus et al. (1996).

The power of the synthesis for scientific description, how-ever, has thus far typically been bound to the very demandingabstraction of landmark point data. This is inconvenient fora number of reasons. Such data cannot at present be sup-plied automatically with any reliability, but instead require thescientist either to locate these loci herself or continually tocorrect the erroneous selections produced by automatic algo-rithms. Landmark data seem unavailable for large extents ofscientifically important images, and for others, such as render-ings of the human cerebral cortex, landmarks cannot be de-clared with assurance to correspond across reasonable rangesof normal adult variation (to say nothing of disease states).Most seriously, the discrete structure of landmark data doesnot particularly suit the semantics of scientific explanations

226 F. L. Bookstein et al.

of form, which are typically concerned with causes or conse-quences of variation in surface area or volume or with ‘forces’(biomechanics, ultrastructure) associated with image contentover extended regions.

The two techniques at the core of contemporary morpho-metrics are the Procrustes-projection construction of shapecoordinates and the visualization of localized shape phenom-ena by thin-plate splines. (Both will be reviewed in the nextsection.) There have been several previous attempts to extendthese techniques from the somewhat constricted domain oflandmarks to the curving form more generally. The thin-platespline, for instance, has been extended to incorporate arbitraryinformation about affine derivatives (Bookstein and Green,1993) and curvature (Little and Mardia, 1996; Mardia et al.,1996) and to treat whole extended curves spanning landmarks(Cutting et al., 1995). Procrustes analysis is likewise beingextended to whole curves (see, for instance, Sampson et al.,1996). But in these extensions there has not hitherto been anyformal coherence analogous to the statistical geometry thatbinds together the two approaches for landmarks.

This paper introduces such a combination of these tools fora problem that apparently has not previously been formalized:the task of effectively describing group differences in datafrom curving forms that, while not featureless, neverthelessneed not have any reliably point-like landmarks anywherealong the arcs. We will proceed by construing each compo-nent of the synthesis, the spline and the Procrustes fit, as a gen-tly nonlinear filter for regional differences of outline shape.We will find the filters to be related by a directional comple-mentarity of band-pass characteristics, so that the power ofthe Procrustes-based filter is greatly heightened after a spline-based preprocessing.

The data set used here to exemplify the localization tech-niques of this paper has been exploited previously to demon-strate landmark-based techniques (Bookstein, 1995a, 1996b).The original images are single thick parasagittal slices, ofnot particularly high quality, selected from clinical MR brainscans of 12 doctors and 13 patients from the Adult PsychiatryUnit, University of Michigan Hospitals. Previous analyses ofthe group difference began with a data set of 13 landmarks lo-cated by John DeQuardo, MD; the discrimination was sharp-ened after I included four additional intersects placed aroundthe inner boundary of the corpus callosum. But of the original13 landmarks, three were already positioned rather arbitrar-ily along that arc and two others lay with equal arbitrarinessalong the falx cerebri. My colleague, Bill Green, in preparingto develop the algorithms for averaging here (Green, 1995,1996), deleted those five landmarks in favor of augmentingthe data set by tracings of the two arcs in full. For the completefeature set, the average of all 25 forms, against which eachcase has been relaxed, is shown in Figure 1. From this scheme

we will use only the callosal outline itself, a polygon of 26semi-landmarks in each of the forms. These 25 polygons areshown in Figure 2. The bulb at the far right (toward the back

Figure 1. Scheme for relaxing information about the curving formof a parasagittal brain section: eight landmark points, a callosal arcof 26 points and a calvarial/dural arc of 12 points. Only the callosalarc is used in this paper, but assignments of homology are based ona spline relaxation that involves the other structures as well. Herethe geometry corresponds to the Procrustes mean shape. The head isfacing to the left.

Figure 2. Our data set: 25 26-point polygons around the callosalborder as traced by W. D. K. Green. Medical staff, first 12 forms;schizophrenics, last 13. Sample and planes selected by John De-Quardo, Adult Psychiatry Unit, University of Michigan. The bul-bosity at the rightmost (posterior) end is named ‘splenium’; the oneat the left (anterior), ‘genu’.

Landmark methods for outlines 227

of the head) is called splenium; that at the left (anterior), genu(‘knee’). The narrowing of the arch near splenium is the isth-mus. Owing to limitations in the original images, we chose tointentionally blunt the rostrum of the corpus callosum, whichotherwise would appear as a right-facing cusp at their lowerleft. The first 12 images in this series pertain to the doctorsand the last 13 to their patients.

2. PRINCIPAL TECHNIQUES OF THEMORPHOMETRIC SYNTHESIS

2.1. Procrustes shape distanceIn ordinary language the shape of an object is described bywords or quantities that do not vary when the object is moved,rotated, enlarged or reduced. The translations, rotations andchanges of scale we thereby ignore constitute the similaritygroup of transformations of the plane. When the ‘objects’are finite ordered point sets (either landmarks or the semi-landmarks to be introduced presently), it turns out to be usefulto say that their shape is simply the set of all point sets that‘have the same shape’. That is, we have formally definedthe shape of a set of points as the equivalence class of thatpoint set, within the collection of all point sets of the samecardinality, under the operation of the similarity group.

Almost all of the multivariate statistics to follow will relyon one simple construction: a distance measure for landmarkshapes. It proves most natural to define the squared Procrustesdistance between two shapes (that is, two equivalence classes)as the minimum of summed squared distances between cor-responding points over the similarities that studies of shapeare to ignore. As Kendall (1984) explains, this is the only ap-proach consistent with how we want a shape distance to relateto Euclidean distance in the original plane. Once the scale ofa landmark set A is fixed, the squared shape distance betweenA and another landmark set B is the minimum of summedsquared Euclidean distances between the landmarks of A andthe corresponding landmarks in point set C as C ranges overthe whole set of shapes equivalent to B. Under this metric,the set of equivalence classes actually becomes a Riemannianmanifold, Kendall shape space, which is a submersion of theoriginal R2k or R3k.

The Procrustes metric is most easily understood graphi-cally. The top row of Figure 3 shows two quadrilaterals ofpoints presumed to be landmarks. Compute the centroid ofeach set of four, then rescale so that the sum of squares of thedistances shown is fixed at unity (second row). The originalroot sum of squares, the scaling factor here, is usually called‘centroid size’ in the statistical literature; it is the same asthe square root of the net moment of the original set of land-marks around their centroid. Superimpose one of the scaledforms over the other at their centroids and spin it (third row).

In general, there will be a unique rotation (fourth row, left)that minimizes the sum of squares of the distances betweencorresponding landmarks. The squared Procrustes distancebetween the forms is the sum of squares of those residualdistances at its minimum. It is proportional to the total areaof the circles shown at the lower right.

Figure 3. Procrustes shape distance for two quadrilaterals of land-marks (top row). Each form is scaled separately (second row) to sumof squares 1 around its own centroid. After the centroids are super-imposed (third row left), either form is rotated about the other (right)until the sum of squared distances between matched landmarks isminimized. Fourth row, the squared Procrustes distance betweenthe original quadrilaterals is the minimum sum of squared distancesbetween corresponding points; it is proportional to the sum of theareas of the circles drawn here.


Formulas for this procedure have been familiar to the mul-tivariate statistical community since Gower’s original matrixformulation (see, for instance, Rohlf and Slice, 1990). If X1

and X2 are k× p matrices (p = 2 or 3) for the coordinates ofk landmarks after the centering and rescaling steps indicatedin the figure, and if the singular-value decomposition of Xt

1 X2

is U DV t with all elements of D positive, then the rotation re-quired to superimpose X2 upon X1 optimally is just the matrixVUt, 2× 2 or 3× 3. In a different notation, for two complexvectors z j = (z1 j , . . . , zkj )

t, j = 1, 2, with∑

i zi j = 0 and∑i zi j zi j = 1, the superposition of the second form upon the

first is approximately

z2 →(∑

i

zi 1zi 2

)z2 (1a)

and the Procrustes distance between the two shapes is approx-imately

PD2(z1, z2) = 1−∣∣∣∣∑

i

zi 1zi 2

∣∣∣∣2. (1b)

This version takes the form of an ‘error variance’ 1 − R2

in a univariate regression (Bookstein, 1991). A formulationaccording better with the global geometry of shape space, butless explicitly indicating its origin in a least-squares problem,is PD(z1, z2) = cos−1 |∑i zi 1zi 2|. The rotation in the fourthrow of the figure is by an angle arg

∑i zi 1zi 2.

2.2. Averaging shapes and the Procrustes tangent spaceThe average of an ordinary list of numbers—their sum dividedby their count—has a least-squares property: it is the quantityabout which they have the least sum of squared differences.This notion goes over directly to the study of shapes (equiv-alence classes) now that we have a distance measure. (Ingeneral, this indirect approach to averaging complex struc-tures in a metric space is called a Frechet mean.) The neces-sary minimization is not too difficult (Kent, 1995). Indeed,for two-dimensional data, it can be expressed as a closed-form eigenanalysis. For a sample of complex vectors zt

j =(z1 j , . . . , zkj ), j = 1, . . . N with

∑i zi j = 0 and

∑i zi j zi j =

1, all j , the Procrustes average is the shape z that is the firsteigenvector of the matrix

∑j z j zt

j , k× k.In practice, for reasons I will explain in the next paragraph,

this average is more often computed by the iterative approachsketched in Figure 4—an alternation between fitting to a tenta-tive average and averaging of the fitted locations landmark bylandmark. As in this toy example, beginning from any shapein a sample, typically the algorithm converges to sufficientaccuracy by the second iteration. After we have computedthe average, we can put each individual shape down over the

average using the similarity transformation that made the sumof squares from the average a minimum for that particularcase, resulting in the diagram at the lower center in Figure 4.

This tactic, far from being a mere graphical aid, is in factthe most crucial step in the morphometric toolkit. Kendallshape space is of dimension 2k − 4, 2k original Cartesiancoordinates decremented by the four degrees of freedom (twofor translation, one for rotation, one for scale) of the similaritygroup of the plane. The tangent space to this manifold is like-wise of dimension 2k−4. The construction at the lower centerin Figure 4 actually shows the projection of each shape of oursample onto the tangent space to that manifold at the sampleaverage shape. That is to say, the Riemannian metric (in thesmall, in the vicinity of the sample average) is preserved—foreach pair of specimens, the summed squared distance betweenthe representatives of the cases in Figure 4 is the same as theirProcrustes shape distance, while the representation is flattenedto a linear space of the correct dimension. The k coordinatepairs of that figure actually serve as a set of 2k redundantcoordinates for this (2k− 4)-dimensional tangent space.

A tangent space can be imagined as a hyperplane touchinga hypersurface, but the modern approach construes it insteadas a linear space in its own right, the space of all linear germsof scalar functions along curves through the point at which it‘touches’ the manifold. To the statistician, these functions arejust what we mean by shape variables—algebraic functions ofthe coordinates that are invariant under changes of position,orientation or scale (Bookstein, 1991). That is, the tangentspace construction of Figure 4 provides the setting for allpossible linear multivariate analyses of the information in theshapes of the landmark configurations. Furthermore, the usualmultivariate sum-of-squares metric of that representation isthe underlying Procrustes metric of the manifold, and so themultivariate statistics will be commensurate with the originalProcrustes geometry.

For shape variations generated by circular Gaussian varia-tions of landmark location, whatever the means and howeverlarge that circular variance, the complete statistical theory ofthese shape representations is known exactly. Small (1996) isa good introduction to this distribution, the ‘Mardia–Drydendistribution’, which was first announced in 1989, and Goodall(1991) evaluates the associated multivariate normal approxi-mations. For shapes that are concentrated in a small regionof the full shape space (as any within-species sample of theshape of an organ is likely to be), one can carry out most ofthe ordinary maneuvers of multivariate statistical analysis—tests of group differences, correlations of shape with causesor effects, principal-components analysis with respect to Pro-crustes distance—directly in terms of this redundant basis forthe tangent space. The suitability of these linear approxima-tions is very high (Bookstein, 1991; Kent, 1995). We will


make use of this when we average shapes over groups inFigure 8.

In three dimensions the Kendall manifold lacks the sym-metries of the two-dimensional setting and the mathematicalstatistics of all this is much less elegant; nevertheless three-dimensional versions of all these data-analytic maneuvers areknown. For Procrustes distance see Rohlf and Slice (1990);for the tangent spaces and approximations see Goodall (1991).

2.3. The thin-plate spline for landmark points and itseigenspaces

The other foundation for the landmark methodology that thispaper extends to curves is a very simple and useful visualiza-tion of shape difference as deformation: the thin-plate splineinterpolant between two sets of landmarks. Let U be thefunction U (r) = r 2 log r, and consider a reference shape(in practice, a sample Procrustes average) with landmarks

original form 1 original form 2 original form 3 original form 4

form 1 fitted to original form 1 form 2 fitted to original form 1 form 3 fitted to original form 1 form 4 fitted to original form

fits above and their average scatter of fits, updated average Procrustes average of four shapes

Figure 4. Procrustes averaging and Procrustes shape coordinates. Top row, four forms of four landmarks. Middle row, Procrustes fit of each (×s)to an arbitrary starting guess (dots: the first form). Bottom left, the next estimate of the average (dots) is the average of the fitted locations fromthe previous step. A second round of fits and averages changes it hardly at all—the algorithm seems to have converged already. Bottom right,the Procrustes shape coordinates (×s) are the locations of the landmarks after the fitting step upon the average shape (dots) at the convergence ofthe algorithm. These actually represent the orthogonal projection of the sample onto the tangent space to Kendall’s shape manifold at the shapeon the right (see text).


Pi = (xi , yi ), i = 1, . . . , k. Writing Ui j = U (Pi−Pj ), buildup matrices

K =

0 U12 . . . U1k

U21 0 . . . U2k...

.... . .

...

Uk1 Uk2 . . . ψ 0

, ψ Q =

1 x1 y1

1 x2 y2...

......

1 xk yk

,(2)

and

L =(

K QQt O

), (k+ 3)× (k+ 3),

where O is a 3 × 3 matrix of zeros. The thin-plate splinef (P) having heights (values) hi at points Pi = (xi , yi ), i =1, . . . , k, is the function

f (P) =k∑

i=1

wi U (P − Pi )+ a0 + axx + ayy (3)

where

W = (w1, . . . , wk,a0,ax,ay)t = L−1 H (4)

with H = (h1, h2, . . . , hk, 0, 0, 0

)t. Then we have f (Pi ) =

hi , all i : f interpolates the heights hi at the landmarks Pi .Moreover, the function f has the minimum bending energyof all functions that interpolate the heights hi in that way: theminimum of∫∫

R2

((∂2 f

∂x2

)2

+ 2

(∂2 f

∂x∂y

)2

+(∂2 f

∂y2

)2)

(5)

where the integral is taken over the entire picture plane. Thevalue of this bending energy is

1

8πWt H = 1

8πH t

kL−1k Hk,ψ (6)

where L−1k , the bending energy matrix, is the k × k upper-

left submatrix of L−1 and Hk is the corresponding k-vector of‘heights’

(h1, h2, . . . , hk

).

A plane-to-plane interpolation is couched as a Cartesianpair ( fx, fy) of these functions based in the same matrix Land for which fx uses a vector Hx of x-coordinates of atarget form and fy uses a vector Hy of y-coordinates. Thebending energy being minimized is now the quadratic formH t

x L−1k Hx + H t

yL−1k Hy. Examples of these grids, applied

to semi-landmarks instead of landmarks, can be seen in Fig-ures 5, 9 and 10.

The spline helps visualize statistical summaries based onthese Procrustes coordinates in a remarkably effective man-ner. Any vector computed in the course of a multivariateanalysis—a mean difference, for example—can be visualized

as a deformation in this way. That the integral in Equation (5)is minimized means that the spline fits shape changes with thesmallest possible variation of affine derivative (shapes of thelittle grid cells in those figures). If a given change of affinederivative can be managed over a larger interval, its contribu-tion to the integral of squares is lower; hence the spline tries torepresent deformations ‘as globally as possible’. In this way itleads the trained eye to a remarkably reproducible assessmentof both global and local features of change. Eigenvectors ofthe bending-energy matrix L−1

k of Equation (6) can be shown(Bookstein, 1995b; Mardia, 1995) to be orthonormal in theProcrustes geometry of the tangent space, not just the Carte-sian geometry of the original digitizing space. The eigen-vectors emerge paired by any orthogonal pair of Cartesiandirections. In general, they emerge in order of localization(Bookstein, 1991), but we will not need to exploit that facthere.

template target, bending energy 0.241

relaxed, bending energy 0.065 unrelaxed and relaxed

-0.3 -0.1 0.1

-0.1

0.1

0.3

Figure 5. The geometry of relaxation using a thin-plate spline.Upper left, a ‘template’ or starting form of 19 points. Upper right,arbitrary set of pre-assigned homologues, with the ordinary thin-plate spline interpolating these point-pairs. Lower left, landmarksafter a relaxation along escribed chords (of which short segmentsare shown) through starting positions. Lower right, the principaleffect of the relaxation is the counter-rotation of the opposing curvesegments at the center, as viewed in the regularization of the grid atits center between the upper right and lower left panels. That thebending energy at small scales is so sensitive to digitizing noise is onereason it is not often used as a metric in the morphometric literature(see Bookstein, 1996a, 1997).


Uniform transformations (affine transformations, shears)enter into this series as a ‘zeroth pair’, corresponding to the theeigenspace of eigenvalue zero. Augmented thereby, the set ofeigenvectors of bending energy rotates the redundant basis of2k Procrustes shape coordinates into a complete orthonormalset of 2k − 4 values. The associated vector decomposition,the components of which are called partial warps, decomposesany observed change uniquely into a sum of fundamentalmodes of variation based only on the average landmark shape,without any assumptions about material properties in betweenthe landmarks. These and other helpful aspects of the splineare reviewed in Bookstein (1996a, 1997).

The three-dimensional version of this same thin-platespline has been known for as long as the 2-D version. Thekernel r 2 log r is replaced by |r |, so that the spline is no longersmooth at the landmarks. The matrix Q of Equation (2) gainsa fourth column of z-coordinates, the vectors H and W gain(k+4)th terms 0 and az, respectively, and the integral (5) nowhas six terms instead of three. The bending energy minimizedby the spline is now −(H t

x L−1k Hx + H t

yL−1k Hy+ H t

zL−1k Hz

).

3. SPLINE RELAXATION ALONG CURVES

In the application to landmark points, although the bendingenergy (6) for the spline is a global minimum of the integral(5), this property does not drive the calculus of minimization;it is a formal identity. Under other circumstances, however,the characterization can become an algorithm for optimiza-tion. In the present application it allows the spline method tobe extended so that some of the target landmarks are freed toslide along lines (Bookstein, 1991). Let there be a ‘nominalset’ of right-hand landmarks Y0

1 , . . . ,Y0k collected coordinate-

wise as the vector Y0 = (Y1x, . . . ,Ykx,Y1y, . . . ,Yky), theconcatenation of the two vectors H of the preceding treat-ment. We seek the spline of one set of landmarks X1 . . . Xk

onto another set of landmarks Y1 . . .Yk of which a sublistYi1 . . .Yim are free to slide away from their nominal positionsY0

i jalong directions u j = (u jx , u jy). It seems entirely natural

to proceed by minimizing the corresponding bending energy.That energy is the quantity Yt

x L−1k Yx + Yt

yL−1k Yy where

the landmarks Yi j of the sublist range over lines Y0i j+ t j u j .

To minimize, collect the parameters t1, . . . , tm in a vectorT of length m. T is the parameter vector over which wewill minimize the energy of the corresponding spline. Theminimization is easiest to notate if we collect the directionalconstraints u1, . . . ,um in a matrix of 2k rows and m columnsin which the (i j , j )th entry is u jx and the (k + i j , j )th entryis u jy, otherwise zeros. The task is now to minimize the form

Yt

(L−1

k 00 L−1

k

)Y ≡ YtL−1

k Y (7)

over the hyperplane Y = Y0+UT. The solution to this famil-iar generalized or weighted least-squares problem is achievedfor the parameter vector

T = −(U tL−1k U

)−1U tL−1

k Y0. (8)

Figure 5 demonstrates this computation for a little schemeof 19 ‘landmarks’ slightly simpler than the real average formwe will actually be using. The thin-plate spline from thetemplate to the ‘raw data’ shows considerable local shear inthe middle of the arch and at its tips. The relaxation allowspoints along the upper and lower arcs of this arch to slidein opposite directions from their starting positions (large ×sat lower left) to the positions that jointly minimize bendingenergy and allow the tips to relax to the left or right similarly.In this example, the directions (u j 1, u j 2) of relaxation are setto escribed chords u j = Yj+1 − Yj−1. (An escribed chordto a polygon is a line through a vertex parallel to the join ofits two nearest neighbors. At convex vertices it lies outsidethe polygon.) This quick-and-dirty method works adequatelyexcept where the polygon is turning rapidly, for which moresophisticated procedures are required. Deviations of the targetpolygon in a direction normal to the template boundary arenot relaxed, as one can see under the ‘shoulders’ of this form,where most of the remaining bending is to be found. In ourexample, all the landmarks were allowed to slide. But therepresentation of Equation (8) does not require m= k, so anysubset of landmarks could be fixed in advance. We will notneed that extension here.

The minimization in Equation (8) accords with the integralformulation (5): the relaxation will try to bring the variationof all first-order derivatives toward zero. Sometimes it canachieve that minimum. For instance, if the method is appliedto a hexagon of landmarks, it will always arrive at a com-pletely affine (linear) transformation, as in Figure 6. (Thenumber of parameters that must be fitted is now reduced tosix, which is the dimension of the complete space of affinetransformations, and the machinery of Equation (8) is linear.)For more complicated data, the minimization will tend to castchanges of affine derivative at the largest spatial scales. Itis therefore suited to multivariate statistical procedures thatscan down a hierarchy of spatial scales, such as the nested T2

convincing us of the significance of the group difference here(see below).

In three dimensions, the vector Y0 is of length 3k andL−1

k = diag(L−1k , L−1

k , L−1k ). The matrix U has three non-

contiguous rows per landmark and one or two columns persliding landmark, as each is constrained to a curve (one degreeof freedom) or to a surface patch (two degrees of freedom).In the former case, the entries Ui j , j , Uk+i j , j , U2k+i j , j are thedirection cosines of the tangent to the curve; for the constraint


to surfaces, the corresponding two columns of U are anytwo perpendicular vectors in the (approximate) tangent plane.These features are built into the Green’s program packageEdgewarp3D, release of which is expected early in 1997.

The minimization in Equation (8) will generate a relaxationno matter what the geometry of the constraint vectors Uj ,as long as the matrix U tL−1

k U is nonsingular. In Figures 5–7 these vectors were taken as finite differences Yi+1 − Yi−1

corresponding to escribed chords of the polygon. These canlead to relaxations some distance from the starting outlineswhen forms are sampled at a spacing too sparse to suit theactual curvature. In Green (1995), they are instead taken asYi − Yi±1, i.e. actual edge segments of the nominal startingpolygon. It would be simple to improve this preliminaryalgorithm in several other ways, for instance, incorporatingB-spline representations of the starting polygons instead ofthis crude C1 construction, then iterating (Green, 1996) toensure that points are clamped to the original outline represen-tations. However, there is an additional step in the procedurewhich moots all these second-order optimizations of the singleoutline—the Procrustes averaging we have already introducedfor landmarks, which aggregates over a sample of first-ordervariations.

3.1. Procrustes averaging of curvesFor the application to curves, the Procrustes averaging algo-rithm of Figure 4 is altered by inserting an additional stepcorresponding to Figure 5 for every form in every cycle.The resulting iterative sequence is as in Figure 7. One muststart with all shapes in some approximate registration, suchas by centering and rotating to common principal moments.Upon a starting estimate of the average, perhaps the first formin the data set, one carefully assigns a reasonable numberof points spaced roughly inverse to the outline curvature,then samples every other outline by a purely local geometriccorrespondence (such as by closest-edge projection). Thisinitialization could be done automatically (W. D. K. Green,

personal communication). The point count is maintainedthrough all subsequent computations. Each outline of thedata set is relaxed along itself to that starting set of pointsand the loci that result from the relaxation are Procrustes-averaged by the usual algorithm of Figure 4. The average thatresults is the starting form for the next round of relaxations.In practice this algorithm always converges. At convergence,the loci that represent each outline form of the data set by aselection of points from its edges correspond between everypair of individual forms of the data set by virtue of havingarisen by spline relaxation from the same point of the finalProcrustes average. They thus share some of the criteria thatmake landmarks useful for scientific analysis; here we callthem semi-landmarks. They fail to be true landmarks in that incontrast to the requirements of Bookstein (1991) they cannotbe defined on a single image—they exist only in the contextof a group average. Of the two coordinates of each semi-landmark, one (the coordinate along the curve) is the vari-ation of the ‘definition’—this coordinate is set for all casesat the same time, early in the algorithm of Figure 7. Onlythe coordinate normal to the outline carries information aboutdifferences between specimens or groups.

Green (1996) shows how to extend the algorithm used hereto any combination of open or closed curves and landmarkpoints. The data of Figure 2 have already been preprocessedby this algorithm using the landmark-and-arc scheme of Fig-ure 1. When there are reliable ‘corners’ of the curves, theyare matched by the spline regardless of whether they appearin the slip list i1, . . . , im. (In effect, it costs less energy tobend one corner into another corner than to bend a flat spotinto a corner while simultaneously flattening another cor-ner nearby.) In other words, features that behave like goodlandmark points (Bookstein, 1991) are treated like pointswhether or not the investigator labels them as such in advance(cf. McEachen et al., 1994). This landmark–outline compos-ite may already subsume a great many approaches suggestedelsewhere for various special cases, including open curves

starting form target form spline relaxations on six landmarks are affine

Figure 6. Spline relaxation of a hexagon of landmarks will almost always result in an exactly affine transformation: the quadratic form inEquation (7) can be zeroed.


between two landmarks (Dean et al. 1996a, Sampson et al.1996) and landmarks with directions through them (Booksteinand Green, 1993). Green conjectures that this extended algo-rithm can be expected to converge to useful representations ofsample average shapes and variation around those averages inall of these extended settings.

4. PROCRUSTES DESCRIPTION OF GROUPSHAPE DIFFERENCES

The final step in the Procrustes approach to landmark statis-tics, Figure 4 lower right, represents shape variation bythe fitted locations of each landmark when superposed by

least-squares upon the sample average. This algorithmicstep persists in the hybrid with the spline relaxation (seeFigure 7). For this little data set of 26 semi-landmarks for 25callosal shapes, there results the scatter of Procrustes shapecoordinates at the upper left in Figure 8. (Remember thatalthough this is a Procrustes scatter, it is taken around thespline-relaxed average.) The component variations ‘at’ theseparate semi-landmarks are mostly well behaved and onlymodestly noncircular. The outlying points at the lower rightcenter derive from one callosum of extremely recurved sple-nium, the form at the right in the third row of Figure 2.

As for landmarks, most multivariate analyses of the shapesof these outlines can be carried out directly on these Procrustes

Figure 7. New algorithm for combining Procrustes averaging with spline relaxation. Top row, four ‘original forms’, generated by randomvariation around the template of Figure 5. Middle, each original outline is relaxed against an initial estimate of the average, in this case, the firstoriginal outline. Relaxation is along escribed chords as in the text. Bottom left, the Procrustes average of the relaxed forms from the precedingstep becomes the next estimate of the average outline. Bottom right, the algorithm has converged by the end of the second iteration.


shape coordinates. For this paper the simplest such analysiswill suffice: the computation and comparison of two groupaverage shapes. These averages (simple centroids by groupof the points scattered in Figure 8, upper left) are shown inthe upper right-hand panel. The largest displacement is atsemi-landmark 15. The lower left-hand panel shows scattersby group at semi-landmarks 14, 15 and 16, the subarc thatappears most interesting. In this registration, the difference atsemi-landmark 15 (×s versus +s) is significant separately (byHotelling’s T2 test on its pair of coordinates) at p ∼ 0.02.

This is not evidence enough for a claim of statistical sig-nificance, of course, since we selected this point as the mostinteresting out of 26. The morphometric synthesis offers anassortment of global significance tests that do not require anyselection of points or features (Bookstein, 1996c). One thatis appropriate in this context is a nested series of Hotelling’sT2 tests on the partial warps derived from the bending-energymatrix L−1

k above. (The original data would not be amenable

to ordinary T2 testing at all: there are at least 26 degrees offreedom for shape, but only 25 specimens in the sample.) Thegroup difference is orthogonally projected (in the Procrustesgeometry) onto a series of subspaces of dimensions of thetangent space that do not exceed a certain degree of ‘bend-ing’. One first tests the 2-subspace (plane) of uniform (affine)changes, then the span of the uniform and the largest-scalepartial warp, and so on until degrees of freedom are exhausted.The most significant of these tests is for the span of the uni-form part and the first five warps, corresponding to vectormultiples of the eigendeformations in Figure 9 along with uni-form shears. Projected onto these six two-dimensional modesof deformation, the group difference is statistically significantby Hotelling’s T2 test at p ∼ 0.002. As we had 11 ofthese tests to choose from (half the sample size, minus 1), thecorrected probability is a little above 2%, which is, indeed,satisfactorily significant. The projected (filtered) mean out-lines with a significant difference are shown at the lower rightin the figure; compare with the upper right panel of Figure 8.

The same spline visualization that we used to relax individ-ual outlines to the emerging mean outline shape can be usedto represent the difference between the two group averageshapes of Figure 8. The relation between them is shown inFigure 10 magnified by a factor of three (since the spline isa matrix manipulation, no iteration is required). This versionof the finding is easier to report to a scientific audience thanthe statistical display of the same information in Figure 8. Itsmain features are quite clear: the average callosum of theseschizophrenics differs from that of the normal group mainlyin a substantial upward-rightward deviation of the isthmustogether with a small diminution of genu size. There seems tobe no alteration in size or shape of the main arc of the arch.The assumption that these averaged semi-landmarks can betreated as landmarks, without further slipping, is sustained bythe plot of the ‘forces’ (coefficients Wx, Wy) of the mapping,shown at the right in the figure. Almost all of these lie normalto the average outline curve. This means that the comparisonof averages is relaxed already, as it should be.

5. LOCALIZING CHANGES OF OUTLINE SHAPE

Figure 8 shows a displacement of semi-landmark 15 betweenthe groups in the direction normal to the average boundaryby 136% of the averaged within-group variance—a signal-to-noise ratio (S/N) of 1.36. But some of this noise can becircumvented. Its denominator (net within-group variation)incorporates displacements from the Procrustes optimizationalong other parts of the outline. For instance, variations insize of genu, the structure at the far left, will displace thecentroid of the entire form in ways that have nothing to dowith the shape difference in the vicinity of point 15. But

Figure 8. Procrustes analysis of the whole callosal form. Upper left,conventional scatter of Procrustes-fit coordinates (least-squares su-perposition to an average) at convergence, where it is the fixed-pointof the slipping algorithm as well. Upper right, group mean contoursin this registration. Solid line, normals; dashed line, schizophren-ics. Lower left, enlargement of scatters at points 14, 15, and 16.×, schizophrenics; +, normals. The difference at point 15 is signif-icant at about the 2% level by ordinary T2 test. Lower right, key tosemi-landmark numbers referred to in the text.


because the Procrustes step recenters the whole form, and be-cause the vector toward genu is oblique to the average outlinetangent at point 15, this variation at genu will contribute tothe denominator of the S/N ratio we have just inspected. TheProcrustes analysis can also attenuate the numerator of theS/N ratio. In Figure 8, the systematic difference at genu hasshifted the dashed (schizophrenic) form leftward, diminishingthe shift at semi-landmark 15 that we are trying to assess.Since there are such group differences at the other end, it isbest to exclude them from our judgment of what is happeningat point 15.

We experiment with a different starting Procrustes fit: justhalf the form, the ‘back half’, as in Figure 11. (To divide into‘halves’ this way is not an arbitrary choice. The susceptibilityof Procrustes fits to large-scale changes is largest along thelonger principal moment of the mean shape, which is obvi-ously this horizontal direction. Empirically, the largest twoprincipal components of shape are aligned with this contrast as

well, so that to whiten the large-scale noise one would subsetleft from right in preference to any other statistical surgery.)This change does not involve any further spline relaxations.But the individual shifts and rotations to the average and theindividual scalings by centroid size have been recomputed denovo. The net effect is to alter the superposition of the meancontours and the geometry of variation around them, whileleaving their separate shapes pretty much unchanged. Forthis posterior part of the callosum, the variance of Procrustesshape coordinates is distinctly less than before. The groupdifference at semi-landmark 15 now has a S/N of 1.74, andthe finding has a significance level (by Hotelling’s T2) of0.0015 (which still needs correction by a factor of 26). A plotshowing the original polygonal outlines, lower right, confirmsthis tendency of separation for the locus at which the outlinestransect their approximate shared ‘normal’ near point 15.

Instead of trying harder and harder to find subsets of fea-tures onto which to project, we might try another strategy, the

partial warp 1 group difference ( -0.0079 , -0.0039 ) partial warp 2 group difference ( -0.014 , -0.0099 ) partial warp 3 group difference ( 0.0031 , 0.0044 )

partial warp 4 group difference ( 0.0026 , -0.0072 ) partial warp 5 group difference ( -0.0012 , -0.0067 ) group means projected onto first six eigenspaces-0.3 -0.2 -0.1 0.0 0.1 0.2

-0.1

0.0

0.1

0.2

0.3

0.4

. . . . . ....

....

..

....

.

.

..

.

.

.. . . . . .

...

....

.....

..

.

..

.

.

.

Figure 9. First five partial warps of the averaged outline (Figure 8) along with their components in the group difference there and (lower right)the low-pass-filtered difference analysis of this order.


omnibus test, that accepts features at all scales simultaneously,but requires that they be combined according to an a prioriformula (in this case, the usual Procrustes sum of squares). Ifour data came in the form of the circular isotropic Gaussianvariances of the mathematical development, we could useGoodall’s (1991) equivalent F-distribution. But for data ashighly nonlinearly processed as these slipped spline residuals,there is no ‘distribution’ available. We exploit, instead, a per-mutation test (Good, 1994; Bookstein, 1997). In a permuta-tion test, one chooses a statistic (in our case, summed squaredProcrustes distance normal to the average curve, weighted in-versely by the pooled variance of that distance) and computesits distribution when the true group assignment is replaced bymany hundreds or thousands of random permutations of casenumber over group. The permutation process makes no as-sumptions whatever about the structure of variation across thelandmarks within the full pool of 25 cases; it is concerned onlywith the relevance of the group label to the average shape. Foreach such permutation, there is a squared Procrustes distancebetween group means. The appropriate tail probability forthe hypothesis of a group mean difference in shape, in themetric of this Procrustes sum of squares, is exactly the fractionof times a random permutation results in a group mean Pro-crustes distance larger than the one afforded when the grouplabels were taken at their true values.

For this posterior half of the form, this test reports a sig-nificance level of 1.4%. The p-value must be corrected forthe number of degrees of freedom it represents from the shapespace as a whole. Here, the fully-slipped data set bears 26degrees of freedom (one per landmark) and the ‘splenium

half’ of the form is tested between landmarks 3 and 15, whichis 13 of those 26; so the correction is by a factor of 2. (Theremaining degrees of freedom are 11 for the other ‘half’ ofthe form and 2 for the relation between the two halves.) Thisresults in a significance level of 2.8%, which is in adequateagreement with the implications of that single T2 at semi-landmark 15. The same test applied to the outline as a wholeis too badly confounded by large-scale variation to be usablein this omnibus form.

5.1. An adaptive high-pass filter for shape differencesWhat this development hints at can be restated explicitly ina suggestive way. The Procrustes-based T2 at one landmarkis working as a signal detection filter at the smallest scale,the scale of one landmark at a time. It thus complements theanalysis in Figure 9, the low-pass analysis exploring the spec-trum of the group difference from its global end. The signalbeing sought, a shift of one semi-landmark on a backgroundof a subset of the others presumed unchanging in mean posi-tion, is a special case of the ‘resistant residual’ familiar to themorphometric community (Rohlf and Slice, 1990). Any meanshift signal will be passed through the ‘Procrustes projection’(Kent, 1995) like any other quasilinear shape signal. Thisprojection attenuates more or less as the landmark suspectedof shift lies near to or far from the centroid of the mean Pro-crustes form. Thus we should be searching in neighborhoodscentered over the target landmark. The preliminary spline-relaxation step has augmented the power of this filter in avery suggestive way. Because of the strong anisotropy of itsspectrum, one can assume any local changes along the curve

Figure 10. Thin-plate spline for the comparison of group mean semi-landmark shapes. Left, grid for the threefold magnification of the changefrom normal mean to schizophrenic mean. Right, coefficients of the spline, showing perpendicularity to the mean curve at points of lowcurvature.


to be negligible by comparison with those across the curve.Hence the Procrustes filter needs to be read only in one singledirection, the direction normal to the mean curve.

The complementarity of these two filters can be usefullyexplored in the one-dimensional equivalent shown in Fig-ure 12. Instead of outlines, consider two samples of functionsover a wide interval of integers centered at x = 0. In thissimulation, the function has an expected value of 0 at everyvalue except x = 0; there, one group has an expected valueof 0, the other, some nonzero value α (here 0.6). The obser-vations vary around this expected value by the sum of twoprocesses of quite different spatial scales. One is a completelyuncorrelated process of Gaussian noise i.i.d. N(0, σ 2), at eachpoint of the domain. The other is a process of very long-rangeorder indeed—for each case an independent normal deviate εi ,distributed as N(0, τ 2), multiplying an exact parabola y = x2

applying throughout the domain. (Here σ 2 = 0.01, τ 2 =0.001.) The figure displays one realization of this compos-ite process for two groups of 25. The small-scale process

is intended as the analogue of ordinary edge-location noise,while the long-range order corresponds to large-scale shapedifferences or signals from other parts of the tissue understudy.

By analogy with our Procrustes-based search for a normaldeviation, in this set-up we apply a filter that contrasts thevalue of the function at 0 to its average value across someneighborhood of zero. This is the ‘δ-filter’ with coefficients

(2m)−1( m︷︸︸︷−1, . . . ,−1, 2m,

m︷︸︸︷−1, . . . ,−1

)centered at 0. As m varies, the filter will supply an estimateα of the mean shift α at 0 whose error variance is a func-tion of m having a proper minimum somewhere. In fact,the variance of the filter output for the single case is σ 2 +(2m)−2

(2mσ 2+m(m+1)(2m+1)τ 2/3

), which is minimized

for m =√

32σ 2

τ 2 + 12 . For the values of σ 2 and τ 2 here, this

Procrustes fit coordinates, points 2 through 17 only

-0.4 -0.2 0.0 0.2

-0.2

0.0

0.2

0.4

0.6

........ ... ... ....... ..

...................... ....

................. ....... .

.... ... .......... ....... . . ... ........... .. .

...... . .. .. ........... .. ....

... ... .... ................ .. .

........ .................... ..................

....

... .. ............. ..... ..... ....

.. ......... ..... ........... ......... ......

.......... ...........................

............................................... ... ... ....

.......

enlargement, points 14, 15, 16

-0.20 -0.10 0.0

-0.0

50.

050.

15

......... .

.......

.

...

.

..

............

....

....

......

........ ... .. .....

.....

..

outlines, posterior half of form

-0.4 -0.2 0.0 0.2

-0.2

0.0

0.2

0.4

0.6

........ ... ... ....... ..

....................... ....

................. ....... .

.... ... .......... ....... . . ... ........... .. .

...... . .. .. ........... .. ....

... ... .... ................ .. .

........ .................... ..................

....

..... ............. ..... ..... ....

.. ......... ..... ........... ......... ......

.......... ...........................

............................................... ... ... ....

.......

Figure 11. Procrustes analysis for the posterior half of the form,rotated to its own Procrustes horizontal. Upper left, means andProcrustes-fit coordinates. Lower left, enlargement for points 14,15, and 16. As a multiple of its own standard error, the group meandifference at point 15 in the direction normal to the outline is 30%larger than it was in Figure 8. Lower right, superposition of originaloutline polygons indicates the extent of separation at point 15 normalto the shared tangent direction.

one-dimensional analogue for the delta-filter

vert

ical

dis

plac

emen

t

-10 -5 0 5 10

-4-2

02

4

Figure 12. One-dimensional analogue for the claim that the δ-filterwill be complementary to the large-scale analysis of shape differenceprovided by the spectrum of the thin-plate spline. The functionshere are a superposition of uncorrelated noise of variance 0.01 andmultiples of x2 by Gaussians of variance 0.001 over a mean shift of0.6 at the single argument 0 (obvious sharp peak at the center). Thetwo groups are shown as two different types of line (dotted and solid).The text derives the optimal neighborhood width for detecting thepeak at 0. This optimal interval is (−4, 4), smaller than one mightexpect, owing to the pernicious effect of the long-range order in thequadratic term.


yields m∼ 4: an interval less than half the width of the plotaround the target point 0.

Back in the domain of morphometrics, this modelof quadratic growth (the function x2) for long-range orlarge-scale effects is qualitatively quite plausible—growthgradients, the largest partial warps, and other realisticcandidates have always been modeled by such supralinearscaling (Bookstein, 1991, Sections 7.3 and 7.4). If localshape differences are also present, they might be expectedto be found as optima of the two-dimensional version ofthe δ-filter at some intermediate window size. Furthermore,because the spline relaxation of every specimen curve againstthe average preferentially filtered out tangential differencesof group mean shape within ‘small’ neighborhoods, it is fairto characterize candidate findings by the statistical signalnormal to the average curve, which is to say, by exactly theS/N ratio of the two experiments above.

We are thus led to the analysis shown in Figure 13: a searchover k(k−2) potential neighborhoods in search of local max-ima of S/N ratios for the group difference signal normal to themean curve. We hope for a ‘corrugated’ surface, with maximathat are sharply a function of position on the outline (semi-landmark number) and only very mildly a function of neigh-borhood size. By analogy with Figure 12, the neighborhoodsare centered at the landmarks at which S/N is computed. Thelist of candidate neighborhoods is then just the union of all thelists of j nearest neighbors of each semi-landmark in turn, j =2, 3, . . . , k−1. (The hierarchy of neighborhoods is computedfrom the Procrustes average shape.) For legibility, the figurebegins at j = 5. The values plotted at the right ends of thesetraces correspond to the neighborhoods of size 26 (which areall the same ‘neighborhood’, the whole set of points—thenumerators of these ratios are the displacements displayed inFigure 8). The figure suggests another global significance testfor the net shape difference: the univariate t-test of groupdifference in the average shift of semi-landmarks 14 and 15in the original Procrustes registration. That t-ratio is 4.0,significant at p ∼ 0.0005. Multiplied by 26 (the number ofsuch segments), that is a probability of 1.3%, comparable withthe figure of 2% which we arrived at by the complementaryfilter using the hierarchy of eigenspaces of bending energy. Itis not necessary to fit all of these windows, as the dependenceon scale (Figure 13) is quite smooth, but there seems to be noharm in doing so.

This novel diagram is remarkably informative about as-pects of the data omitted from Figures 8 or 10. Clearly thecomparison of callosal form between the groups shows justone region of localized shape change, the arc 14–15 we havealready noticed in the global splines. The single small regionof best S/N is the neighborhood of size eight around semi-landmark 15, for which the S/N is 1.76 and the directional

t-ratio a satisfactory 4.4, p ∼ 0.0002. Multiplied by 26(points) and further by a factor of∼3, the effective dimension-ality of the curves in Figure 13, this is about ∼2% again, thesame range at which we have arrived by two earlier methods.

The Procrustes analysis that produced this S/N, the corre-sponding superposition of mean outlines and the classifica-tion afforded thereby are all collected in Figure 14. Our twocomplementary filters arrived at the same conclusion aboutthe shape change. It is mainly a rearrangement of the isthmusbetween splenium and arch. At large scale (Figure 10) thishas the appearance of a displacement; here at small scale, theappearance is instead of a sculpting or bending. At largerscales, according to Figure 13, semi-landmark 14 is a bit moreinformative separately, but the corresponding visualization isalready apparent in Figure 10. Although two other semi-landmarks, 2 and 11, seem to embody some small-scale infor-mation, clearly the S/N surface is ‘corrugated’. Just as we hadhoped, it expresses widely spaced local effects. The output ofthis center-versus-periphery Procrustes filter is very robust tovariation of neighborhood size but highly sensitive to positionalong the curve.

At the lower left in Figure 14 is the classification affordedby this optimal local filter. Classification is clearly improvedover that in Figure 8, reproduced at lower right, by attendingto the ‘resculpting’ of the isthmus. This is in spite of thedifficulty in digitizing this arc by hand. (In many originalimages the fornix springs from this arc to the left, requiring asubjective decision as to where the underlying outline shouldbe located. It was estimated, of course, without any knowl-edge of diagnosis.)

The model underlying the filter here contrasts strongly withthe basic distributional model for Procrustes-type analyses ofdiscrete landmarks. Our standard landmark methods (Book-stein, 1995–97) cast a 26-‘landmark’ data set like this one intoa 48-dimensional space of shape phenomena within which astrenuous symmetry is enforced a priori. Variation tangentialto and normal to outlines must be weighted equally and groupdifferences are modeled as equally plausible regardless of thedirection along which they lie in this space. For instance,every set of three landmarks, neighboring or not, is consideredan equally likely locus of some potentially crucial aspect ofshape discrimination. In this context, the best single overallmultivariate test statistic is Goodall’s (1991) omnibus F-ratio(see also Bookstein, 1996a, 1997), which is a multiple of theratio of Procrustes distance between the group means to theProcrustes variance within the groups separately. For the datahere its value is 1.32 on 48 and 1104 d.f., significant at p ∼0.07 only. That was, in essence, a single test of the displace-ment between mean curves in Figure 8 with respect to the netdegree of variation around those means aggregated over all thesemi-landmarks separately. Because the F could not take into


account the concentration of the finding in both direction andlocation, it was blocked from finding anything particularlyimplausible about the configuration of group differences as awhole.

An earlier attempt to localize the shape difference betweenthese same callosa (Bookstein, 1995a, 1996b) exploited ahighly irregular landmark analysis to arrive at the same find-ing. From a preliminary landmark-based image averaging ofthe region of callosum, a shift was found between the aver-age lower borders. A new landmark was constructed on theindividual border as the intersection of a straight line from‘splenium’ to ‘genu’, where these two notions are now con-strued as points rather than regions. Analysis of the pro-portion into which this new landmark divided the diameterof the callosum resulted in a finding about as strong as thatin Figure 8. In the alternate method, the equivalent of theδ-filter is a consideration of the shapes formed by this newlandmark with respect to the two or three landmarks nearby.Addition of just one more point, on colliculus, resulted in a

finding with about the same power as that in Figure 14 here.However, as several colleagues have stubbornly pointed outquite sharply, this hybrid technique corresponds to no formaltheory of morphometrics and affords no significance tests.

6. RELATED APPROACHES

The adaptive filter optimization here is partly analogous totwo otherwise disparate techniques that have been the sub-ject of previous discussions in the literature of medical imageanalysis. Techniques of anisotropic diffusion (Perona et al.1994; Whitaker and Gerig, 1994) modify images toward theequilibration of picture content within regions without allow-ing information to diffuse across sharp boundaries. They thusachieve part of the purpose of the processing needed here—an enhancement of the signal normal to the boundary, by flat-tening gradients elsewhere—without the corresponding relax-ation along the boundary that seems the key to the successof our filter. That lacuna is understandable, of course, inthat these diffusion techniques deal with analysis of pictures

size of neighborhood (count of landmarks)

S/N

rat

io in

nor

mal

dire

ctio

n

5 10 15 20 25

0.0

0.5

1.0

1.5

2.0

1

1

2

2

3

34

45

5

6

6

7

7

8 8

9

9

1010

11

11

12

12

1313

1414

15

15

16

16

171718

1819

19

20

20

2121

22 22

23

23

24

24

25

25

26

26

Figure 13. Performance of the δ-filter at all semi-landmarks and all neighborhood sizes from 6 through 26. The S/N plotted is in the directionnormal to the mean outline. Semi-landmarks are numbered at both ends of the trace of their S/N ratios over the range of neighborhood sizes.The chart argues for a signal at small scale restricted to the vicinity of semi-landmarks 14 and 15 (see text).


one at a time, whereas what is relaxed by the spline is not aproperty of any single image, but of a pair or a whole data setof instances in relation to a template or an average.

The δ-filter method can be considered as a limiting case ofa contrast-of-annuli filter. In light of the strongly anisotropicnature of the preliminary spline processing, these annuli orGaussians are better taken not as circular but as severely elon-gated in the direction of the edge under study. The combinedmethod here can be thought of as an even larger-scale adap-tation of such contrasts to the vicissitudes of large-scale cur-vature at some distance—it is as if the edge were straightenedbefore these parametrically simple filters were applied. Thelarge-scale approach of Figure 9 is thus complementary to thisentire family of filters; cf Coggins and Huang (1993). Had wechosen a great many more points on these outlines, perhaps260 rather than 26, we would experiment with alterations ofthe inner radius of the Procrustes filter, to allow the consider-ation of more than one ‘center’ at a time. For the equivalentin the one-dimensional analogue of Figure 12, see Bookstein(1996d).

But the δ-filter is not at all closely related to other extantmethods of multivariate statistical analysis of curving form,which share most of their spectral characteristics with thecomplementary, large-scale landmark technique introducedin connection with Figure 9. In the hands of other multi-variate statisticians, samples of outlines have almost beenanalyzed by one or another decomposition into orthogonalseries [Nastar and Ayache, 1994; Cootes and Taylor, 1996;Sampson et al., 1996; see Rohlf (1990) for a review of theclassical work]. Sometimes these are geometric orthogo-nal functions, eigenfunctions of energetics of the mean form(Fourier decompositions, ‘normal modes of bending’) andsometimes they are empirical orthogonal functions (princi-pal components of covariations of position). In the formerapproach, features at small scale are required to be orthogo-nal to those at large scale, so that the last few terms of anydecomposition look like high-frequency oscillations aroundthe average outline and thus cannot localize. In the latterapproach, small effects on large regions of outline domi-nate large effects on small regions, as was the case for the

Figure 14. Analysis for the optimal neighborhood around point 15. Upper left, group mean outlines and Procrustes scatter. Upper right,individual outlines. Lower left, output of the δ-filter at point 15. Lower right, comparable output of the full Procrustes fit (Figure 8) with whichwe began. N, normals; S, schizophrenics. All panels are in the orientation of the entire form, Figure 8.


callosal outlines here, and so a local signal that was de-tected at all would already have been low-pass filtered—itwould be visualized at too large a spatial scale to be infor-mative. The finding at semi-landmarks 14–15 here suits noneof these function-space approaches. If it is developed fur-ther, a wavelet decomposition of curving form such as thatsuggested by Ackroyd and Mardia (1996) may prove com-plementary to the large-scale approaches via orthogonal de-composition. But because wavelet decompositions are, ingeneral, not invariant against the Euclidean similarity group,considerable modifications will be required before there canbe a wavelet description of any scientific phenomenon likethe local boundary shift that has proved so robust a findinghere.

Tagare et al. (1995) have suggested a curve-to-curvematching protocol optimizing a functional that curiously com-plements this slipping spline. In effect, the correspondencethey compute minimizes the integral along the curve of a sum-of-squares that combines acceleration or deceleration alongthe curve together with differences of bending in the normaldirection. When the weight of this second term (which servesas a regularization) is set suitably, the combination is nearlythe same as the first and third terms in the integrand for bend-ing energy, Equation (5). But in the spline method the integralis taken over the entire picture plane and so allows aspectsof the global differential geometry of the curves to affect thecorrespondence. It is the linearization of that global aspect,not the local, that is most conducive to subsequent statisticalanalyses.

Davatzikos et al. (1996) studied callosal outlines similarto ours that were acquired by an automatic active contourmethod. Outlines of a sample are related to an a priori normby an elastic relaxation. The relation of each individual to thenorm is then described by an areal distortion function, in effectthe area of the little squares in our spline figures. Groups(in their example, eight male and eight female elderly fromBaltimore) are compared by averaging this derived quantityat corresponding points of the normative form or over regions,and thresholding those differences at various effect sizes.

Davatzikos was kind enough to send me his data setfor reanalysis by the method of the present manuscript.The permutation test of spline-slipped coordinates for theposterior ‘half’, this time comprising 50 semi-landmarks (for16 cases!), finds a shape difference between the sexes that issignificant at p ∼ 0.015 before the correction factor of 2is applied. The difference is primarily a vertical extensionof the splenium. Davatzikos et al. (1996) found a change ofarea in this region, but did not report any directionality andapparently could not test for statistical significance in a sampleso small. Indeed, they offered no graphical representation ofsample variation at all.

What differences between the methods might account forthis difference in power? Both algorithms have a matchingstep in which points of a candidate callosal outline are as-signed to points of a standard outline by minimizing a formu-lation in terms of energy and both ‘energies’ are abstract, non-physical. But in general, description by the areal-ratio scalar,like any other scalar description, uses one parameter (here thedeterminant) from the three that together describe the tensorfield of affine derivatives, and so discards more than half of theinformation available for discussions of variability. Restoringit in the form of those tensors, furthermore, is not a verygood idea (Bookstein, 1991). Statistical analysis works muchbetter using the more linear spaces of shape features reviewedhere. If transformations are uniform, specialized Procrustesmethods are optimized for their estimation, comparison andtesting (see Bookstein, 1996a, 1997). Visualization by arealratio following elastic matching is tuned to one particularform of shape change: relatively homogeneous increase inarea over compact regions that the elastic model treats in asomewhat homogeneous fashion. In the present data set, forinstance, changes of area might partially describe a reshapingof splenium, but would not detect the shift of isthmus stronglyto the upper right, which is our principal finding.

Of course, the initial steps in the Davatzikos procedure,active contour analysis and elastic relaxation, are familiaraspects of the analysis of individual images. In particular,the active contour method is obviously better than manualdigitizing of the outline of the callosum where it blurs intoneighboring structures (such as the fornix mentioned above).In this context, standard imaging tools can adequately simu-late the performance of a skilled human. They do not extendto the further task of biometric analysis, however, at whichunaided humans typically do not do very well either. OurProcrustes tactics and the thin-plate spline, while simpler, al-gebraically totally explicit, and exactly suited to the specifictask of quantitative group comparison, are familiar mainly tothe evolutionary biologist. They are not to be found in gradu-ate curricula for imaging or in textbooks and their sources inthe refereed literature were, until recently, quite sparse. Butthey can find scientific signals that are out of reach of familiarmethods tuned to other purposes, such as object recognition orimage segmentation. There is no particular reason to expectmethods suited for one task to be suited for another, any morethan the biometric techniques of this paper would bring anyparticular efficacy to the context of segmentation.

7. CONCLUDING REMARK

The finding here is obviously tentative. The callosum is acomplex three-dimensional structure not adequately sampledby its midsagittal section, certainly not the crudely positioned


approximations here, and our 13 patients were not of anyhomogeneous clinical class. Instead this presentation wasintended as a prototype for new and potentially powerfulmethodological possibilities.

To be able to treat curving form by landmark methods with-out requiring their acquisition in landmark-anchored form isa significant step forward in the morphometrics of biomedicalimages. For many three-dimensional data sets, for instance,representation of high-contrast surfaces works well by themethod of ridge curves or crest lines (Thirion, 1994; Deanet al., 1996a), which extracts reliable three-dimensional lociat which the surface is locally most like a sharp edge. Sta-tistical methods for analysis of such data have hitherto eitherbeen limited to very simple parametric models—rigid motion,polynomial warping—or have required the extraction of spe-cific landmark points specimen by specimen (typically, cur-vature maxima along these curves). Such landmarks are typi-cally much noisier than the arcs on which they lie. The com-bination of the spline relaxation and the associated multiscalemultivariate statistics makes it possible to analyze such curvesas a whole, by assigning semi-landmarks that describe the re-lation to a template without requiring any geometric semanticsof characterization upon the individual form. In this aspectthese new methods are converging with other approaches toform-comparison from McGill, Washington University andelsewhere, in which parametrization of the individual form issubordinated to parametrization of the relation to a template.For landmark data sensu stricto, these parametrizations areidentical a priori: the points of shape space serve equally wellas vector specifications of the corresponding splines (Book-stein, 1996a).

The spline relaxation and mixed multivariate methods usedhere will sustain valid findings to the extent that the multi-scale model driving them is an adequate preliminary repre-sentation of the information discriminating the image groupsunder study: discrete group differences of either large orsmall scale. In this aspect it joins the mainstream of mor-phometrics to the most active current in analysis of medi-cal images separately. Morphometrics can thus contributeto medical image analysis as a method for multiscale im-age comparison, a theme absent from many other current ap-plications of the multiscale theme. [It does, however, ap-pear in the McGill approach to atlas matching: see Evanset al. (1996).] This approach may, for instance, be helpfulin studies of myocardial wall motion, the effect of diseaseon which has often been modeled as local defect of ‘motion’with respect to a globally twisting shape change at quite largescale (e.g., McEachen et al., 1994). It may also be helpfulin more detailed consideration of the shapes of fluid-filledcavities such as the cerebral ventricles (Dean et al., 1996)or other complex shapes having subtle but crucial functional

implications, shapes that have hitherto proved difficult foranalyses that followed conventional image-by-image process-ing.

ACKNOWLEDGEMENTS

Preparation of this contribution was supported by NIH grantsDA-09009 and GM-37251 to Fred L. Bookstein. The formergrant is jointly supported by the National Institute on DrugAbuse, the National Institute of Mental Health and the Na-tional Institute on Aging as part of the Human Brain Project.All the statistical graphics were produced in the Splus packageavailable from MathSoft, Inc., Seattle. The callosal outlinedata were acquired by Bill Green using an experimental ver-sion of his program package edgewarp that will eventu-ally be available by FTP from the directory pub/edgewarpon brainmap.med.umich.edu. I thank Christos Da-vatzikos for access to the callosal outlines from his 1996publication.

REFERENCES

Ackroyd, R. G. and Mardia, K. V. (1996) A MCMC approach towavelet warping. In Mardia, K. V., Gill, C. A. and Dryden, I.L. (eds), Proceedings in Image Fusion and Shape VariabilityTechniques, pp. 129–140. Leeds University Press, Leeds.

Bookstein, F. L. (1991) Morphometric Tools for Landmark Data:Geometry and Biology. Cambridge University Press, Cam-bridge.

Bookstein, F. L. (1995a) How to produce a landmark point: the sta-tistical geometry of incompletely registered images. In Melter,R., Wu, A., Bookstein, F. and Green, W. D. K. (eds), VisionGeometry IV. SPIE Proc., Vol. 2573, pp. 266–277.

Bookstein, F. L. (1995b) Metrics and symmetries of the morphomet-ric synthesis. In Mardia, K. V. and Gill, C. A. (eds), Proceedingsin Current Issues in Statistical Shape Analysis, pp. 139–153.Leeds University Press, Leeds.

Bookstein, F. L. (1996a) Biometrics, biomathematics, and the mor-phometric synthesis. Bull. Math. Biol., 58, 313–365.

Bookstein, F. L. (1996b) Biometrics and brain maps: the promise ofthe morphometric synthesis. In Koslow, S. and Huerta, M. (eds),Neuroinformatics: an Overview of the Human Brain Project.Progress in Neuroinformatics, Vol. 1, pp. 203–254. LawrenceErlbaum, Hillsdale, NJ.

Bookstein, F. L. (1996c) Combining the tools of geometric morpho-metrics. In Marcus, L. F., Corti, M., Loy, A., Naylor, G. J. P.and Slice, D. E. (eds), Advances in Morphometrics. NATO ASISeries A: Life Sciences, Vol. 284, pp. 131–151. Plenum, NewYork.

Bookstein, F. L. (1996d) Landmark methods for forms withoutlandmarks: localizing group differences in outline shape. InAmini, A., Bookstein, F. and Wilson, D. (eds), Proc. Work-shop on Mathematical Methods in Biomedical Image Analysis,


San Francisco, pp. 279–289. IEEE Computer Society Press, LosAlamitos, CA.

Bookstein, F. L. (1997) Shape and the information in medical im-ages: a decade of the morphometric synthesis. Comp. VisionImage Understanding, to appear.

Bookstein, F. L. and Green, W. D. K. (1993) A feature space foredgels in images with landmarks. J. Math. Imag. Vision, 3, 231–261.

Coggins, J. M. and Huang, C. (1993) Defining optimal feature setsfor segmentation by statistical pattern recognition. In Wilson,D. C. and Wilson, J. N. (eds), Mathematical Methods in MedicalImaging II, SPIE Proc., Vol. 2035. pp. 80–88.

Cootes, T. F. and Taylor, C. J. (1996) Active shape models: a reviewof recent work. In Mardia, K. V. and Gill, C. A. (eds), Proceed-ings in Current Issues in Statistical Shape Analysis, pp. 108–114. Leeds University Press, Leeds.

Cutting, C., Dean, D., Bookstein, F., Haddad, B., Khorramabadi,D., Zonnefeld, F. and McCarthy, J. (1995) A three-dimensionalsmooth surface analysis of untreated Crouzon’s disease in theadult. J. Craniofacial Surgery, 6, 444–453.

Davatzikos, C., Vaillant, M., Resnick, S. M., Prince, J. L., Letovsky,S. and Bryan, R. N. (1996) A computerized approach for mor-phological analysis of the corpus callosum. J. Comp. Assis. To-mogr., 20, 88–97.

Dean, D., Marcus, L. and Bookstein, F. (1996a) Chi-square test ofbiological space curve affinities. In Marcus, L. F., Corti, M.,Loy, A., Naylor, G. J. P. and Slice, D. E. (eds), Advances inMorphometrics, NATO ASI Series A: Life Sciences, Vol. 284,pp. 211–233. Plenum, New York.

Dean, D., Buckley, P., Bookstein, F., Kamath, J., Kwon, D., Fried-man, L. and Lys, C. (1996b) Three-dimensional MR-based mor-phometric comparison of schizophrenic and normal cerebralventricles. In Hohne, H. and Kikinis R. (eds), Visualization inBiomedical Computing, Lecture Notes in Computer Science,Vol. 1131, pp. 363–372. Springer-Verlag, New York.

Evans, A. L., Collins, D. L. and Holmes, C. J. (1996) Computationalapproaches to quantifying human neuroanatomical variability.In Toga, A. W. and Mazziotta, J. C. (eds), Brain Mapping: theMethods, pp. 343–361. Academic Press, San Diego, CA.

Good, P. (1994) Permutation Tests. Springer-Verlag, New York.Goodall, C. R. (1991) Procrustes methods in the statistical analysis

of shape. J. R. Stat. Soc. B, 53, 285–339.Green, W. D. K. (1995) Spline-based deformable models. In Melter,

R., Wu, A., Bookstein, F. and Green, W. D. K. (eds), VisionGeometry IV, SPIE Proc., Vol. 2573, pp. 290–301.

Green, W. D. K. (1996) The thin-plate spline and images with curv-ing features. In Mardia, K. V., Gill, C. A. and Dryden, I. L. (eds),Proceedings in Image Fusion and Shape Variability Techniques,pp. 79–87. Leeds University Press, Leeds.

Kendall, D. G. (1984) Shape-manifolds, Procrustean metrics, andcomplex projective spaces. Bull. London Math. Soc., 16, 81–121.

Kent, J. T. (1995) Current issues for statistical inference in shapeanalysis. In Mardia, K. V. and Gill, C. A. (eds), Proceedings

in Current Issues in Statistical Shape Analysis, pp. 167–175.Leeds University Press, Leeds.

Little, J. A. and Mardia, K. (1996) Edgels and tangent planes inimage warping. In Marcus, L. F., Corti, M., Loy, A., Naylor, G.J. P. and Slice, D. E. (eds), Advances in Morphometrics, NATOASI Series A: Life Sciences, Vol. 284, pp. 263–270. Plenum,New York.

Marcus, L. F., Corti, M., Loy, A., Naylor, G. J. P. and Slice, D. E.(eds) (1996) Advances in Morphometrics, NATO ASI Series A:Life Sciences, Vol. 284. Plenum, New York.

Mardia, K. V. (1995) Shape advances and future perspectives. InMardia, K. V. and Gill, C. A. (eds), Proceedings in Current Is-sues in Statistical Shape Analysis, pp. 57–75. Leeds UniversityPress, Leeds.

Mardia, K. V., Kent J., Goodall, C. and Little, J. (1996) Kriging andsplines with derivative information. Biometrika, 81, 207–221.

McEachen, J. C., II, Nehorai, A. and Duncan, J. (1994) A recursivefilter for temporal analysis of cardiac motion. In Proc. IEEEWorkshop on Biomedical Image Analysis, pp. 124–133. IEEEComputer Society Press, Los Alamitos, CA.

Nastar, C. and Ayache, N. (1994) Classification of nonrigid mo-tion in 3D images using physics-based vibration analysis. InProc. IEEE Workshop on Biomedical Image Analysis, pp. 61–69. IEEE Computer Society Press, Los Alamitos, CA.

Perona, P., Shiota, T. and Malik, J. (1994) Anisotropic diffusion. Inter Haar Romeny, B. M. (ed.), Geometry-Driven Diffusion inComputer Vision, pp. 73–92. Kluwer, Dordrecht.

Rohlf, F. J. (1990) Fitting curves to outlines. In Rohlf, F. J. andBookstein, F. (eds), Proc. Michigan Morphometrics Workshop,pp. 167–177. University of Michigan Museums, Michigan, MI.

Rohlf, F. J. and Slice, D. (1990) Extensions of the Procrustes methodfor the optimal superposition of landmarks. Systematic Zoology,39, 40–59.

Sampson, P. D., Bookstein, F., Sheehan, F. and Bolson, E. (1996)Eigenshape analysis of left ventricular outlines from contrastventriculograms. In Marcus, L. F., Corti, M., Loy, A., Nay-lor, G. J. P. and Slice, D. E. (eds), Advances in Morphomet-rics, NATO ASI Series A: Life Sciences, Vol. 284, pp. 211–233.Plenum, New York.

Small, C. G. (1996) The Statistical Theory of Shape. Springer-Verlag,New York.

Tagare, H. D., O’Shea, D. and Rangarajan, A. (1995) A geometriccriterion for shape-based non-rigid correspondence. Int. Conf.on Computer Vision, Boston, MA.

Thirion, J.-P. (1994) The extremal mesh and the understanding of 3Dsurfaces. In Proc. IEEE Workshop on Biomedical Image Anal-ysis., pp. 3–12. IEEE Computer Society Press, Los Alamitos,CA.

Whitaker, R. and Gerig, G. (1994) Vector-valued diffusion. In terHaar Romeny, B. M. (ed.), Geometry-Driven Diffusion in Com-puter Vision, pp. 93–134. Kluwer, Dordrecht.

landmark methods for forms without landmarks

Documents