jointregistrationand shapeanalysisof curves...

14
Joint Registration and Shape Analysis of Curves and Surfaces Jingyong Su, Sebastian Kurtek, and Anuj Srivastava Florida State University Abstract The shapes of boundaries of objects in images can be conveniently represented as parameterized curves and surfaces. Their analysis, however, is made complicated by the fact that a re-parameterization is shape preserv- ing, and one needs metrics and other analysis tools that are invariant to re-parameterizations. We summarize recent progress in developing Rieman- nian methods that remove the parameterization variability using equivalence relations and quotient spaces. They exploit the parameterization variability as a tool for registration, i.e. matching of points across objects, and provide elastic Riemannian metrics that allow for both simultaneous registration and analysis of shapes. The resulting deformations and shape statistics improve performance over the existing methods in terms of feature preservation and model efficiency. While the original metrics are complicated, we discuss novel mathematical representations, of both curves and surfaces, that reduce shape spaces to standard Hilbert spaces and many of the existing algorithmic tools can be applied. We will illustrate these ideas using examples from shape analysis of curves and surfaces. 1 Introduction There are several meanings of the word shape. Although the use of words shape or shape analysis is very common in computer vision, its definition is sel- dom made precise in a mathematical sense. According to the Oxford English Dictionary, it means “the external form or appearance of someone or some- thing as produced by their outline”. Kendall [8] described shape as a mathe- matical property that remains unchanged under certain transformations such as rotation, translation, and global scaling. Shape analysis seeks to represent shapes as mathematical quantities, such as vectors or functions, that can be manipulated using appropriate rules and metrics. Statistical shape analysis is 1

Upload: others

Post on 20-May-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

Joint Registration and Shape Analysis of

Curves and Surfaces

Jingyong Su, Sebastian Kurtek, and Anuj SrivastavaFlorida State University

Abstract The shapes of boundaries of objects in images can be convenientlyrepresented as parameterized curves and surfaces. Their analysis, however,is made complicated by the fact that a re-parameterization is shape preserv-ing, and one needs metrics and other analysis tools that are invariant tore-parameterizations. We summarize recent progress in developing Rieman-nian methods that remove the parameterization variability using equivalencerelations and quotient spaces. They exploit the parameterization variabilityas a tool for registration, i.e. matching of points across objects, and provideelastic Riemannian metrics that allow for both simultaneous registration andanalysis of shapes. The resulting deformations and shape statistics improveperformance over the existing methods in terms of feature preservation andmodel efficiency. While the original metrics are complicated, we discuss novelmathematical representations, of both curves and surfaces, that reduce shapespaces to standard Hilbert spaces and many of the existing algorithmic toolscan be applied. We will illustrate these ideas using examples from shapeanalysis of curves and surfaces.

1 Introduction

There are several meanings of the word shape. Although the use of wordsshape or shape analysis is very common in computer vision, its definition is sel-dom made precise in a mathematical sense. According to the Oxford EnglishDictionary, it means “the external form or appearance of someone or some-thing as produced by their outline”. Kendall [8] described shape as a mathe-matical property that remains unchanged under certain transformations suchas rotation, translation, and global scaling. Shape analysis seeks to representshapes as mathematical quantities, such as vectors or functions, that can bemanipulated using appropriate rules and metrics. Statistical shape analysis is

1

Page 2: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

2 Authors Suppressed Due to Excessive Length

concerned with quantifying shape as a random quantity and developing toolsfor generating shape registrations, comparisons, averages, probability mod-els, hypothesis tests, Bayesian estimates, and other statistical procedures onshape spaces.

Shape is an important physical property of objects that characterizes theirappearances, and can play an important role in their detection, tracking, andrecognition in images and videos. A significant part has been restricted tolandmark-based analysis, where shapes are represented by a coarse, discretesampling of the object contours [1, 16]. This approach is limited in that auto-matic detection of landmarks is not straightforward, and the ensuing shapeanalysis depends heavily on the choice of landmarks. One usually restricts tothe boundaries of objects, rather than the whole objects, for shape analysisand that leads to a shape analysis of curves (for 2D images) and surfaces (for3D images). Fig. 1 suggests that shapes of boundaries can help characterizeobjects present in images.

Fig. 1 Shapes of boundary curves are useful in object characterizations.

To understand the issues and challenges in shape analysis, one has to lookat the imaging process since that is a major source of shape data. A picturecan be taken from an arbitrary pose (arbitrary distance and orientation ofthe camera relative to the imaged object), and this introduces a randomrotation, translation, and scaling of boundaries in the image plane. Therefore,any proper metric for shape analysis should be independent of the pose andscale of the boundaries. A visual inspection also confirms that any rotation,translation, or scaling of a boundary, while changing its coordinates, does notchange its shape.

In case of parameterized curves and surfaces, an additional challenge ariseswhen it comes to invariance. Let β : [0, 1] → R

2 represent a parameterizedcurve and let γ : [0, 1] → [0, 1] be a smooth, invertible function such thatγ(0) = 0 and γ(1) = 1. Then, the composition β̃(t) ≡ (β ◦ γ)(t) represents acurve with coordinate functions that are different from those of β(t) but havethe same shape. β̃ is called a re-parameterization of β. Fig. 2 illustrates thisissue with a simple example. It shows that the coordinate functions of the

Page 3: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

Joint Registration and Shape Analysis of Curves and Surfaces 3

re-parameterized curve, β̃x(t) and β̃y(t), as functions of t, are different from

the original coordinate functions βx(t) and βy(t). But when β̃x(t) is plotted

versus β̃y(t), it traces out the same sequence of points, i.e. the same shape,as that traced by βx(t) versus βy(t). This results in an additional invariancerequirement in shape analysis of parameterized curves (and similarly for sur-faces). That is, the shape metrics should be invariant to how the curves areparameterized. Similarly, for parameterized surfaces, a change of parameter-

00.5

1

−1−0.50

−1

−0.5

0

0.5

1

tx

y

0 0.2 0.4 0.6 0.8 1−1.4

−1.2

−1

−0.8

−0.6

−0.4

−0.2

0

t

x, x

t

0 0.2 0.4 0.6 0.8 1−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

t

y, y

t

−1.5 −1 −0.5 0

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

x, xt

y, y

t

0 0.2 0.4 0.6 0.8 1

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

t

γ(t)

(a) (b) (c) (d) (e)

Fig. 2 Re-parameterized curve has different coordinate functions but same shapeas the original curve. (a): curves (t, βx(t), βy(t)) and (t, β̃x(t), β̃y(t)); (b): βx(t) and

β̃x(t); (c): βy(t) and β̃y(t); (d): curves (βx(t), βy(t)) and (β̃x(t), β̃y(t)); (e): γ(t).

ization does not change the shape of the object. Fig. 3 shows three differentparameterizations of a surface but the overall shape of the objects remainsthe same. We will treat a closed surface as an embedding of unit sphere inR

3, i.e. f : S2 → R3. If γ is an arbitrary diffeomorphism of S2, then f ◦ γ is

nothing but a re-parameterization of the surface. We seek shape metrics andtechniques for analysis which will be invariant to the introduction of arbi-trary γ in shape representations. For example, a parameterization-invariantmetric between the three surfaces shown in Fig. 3 should be zero.

f ◦ γ

γ

Fig. 3 The same surface with different parameterizations.

Page 4: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

4 Authors Suppressed Due to Excessive Length

Another important issue in comparing shapes is registration, i.e. matchingof points across objects. Any shape metric requires a registration componentto help decide which point on one shape is compared to which point on theother. A registration is good when it matches points across shapes that havesimilar geometric features. A majority of current techniques in shape analysisneed registration as an input or they perform registration-shape analysis ina sequential fashion. These two steps are often performed under two differ-ent criteria. Even if one can achieve optimal solutions under the individualsteps, the overall system will be suboptimal due to these differing criteria. Abetter strategy is to solve the two problems simultaneously under the sameobjective function. This is accomplished in a Riemannian framework by form-ing quotient spaces under the re-parameterization group. To understand thisimportant idea, let us take two curves β1, β2 : [0, 1] → R

n. By default, thepoint β1(t) is matched with the point β2(t), for any t ∈ [0, 1]. However, if were-parameterize one of the curves, say β2 using β2 ◦γ, then the registration ofpoints depends on γ. Thus, optimal registrations between curves and surfacescan be controlled using re-parameterizations. The key point is to find metricsthat serve two purposes: (1) form an objective function for finding optimal re-parameterizations and (2) lead to proper parameterization-invariant metricsfor comparing shapes.

There are several papers in the literature that consider the problem of reg-istering curves [4, 3, 2, 15]. While this literature represented a major progressin that the authors recognized the importance of curve registration, the pro-posed criteria for registration had certain major limitations. We elaboratethe main limitation next. Once again consider any two curves β1 and β2. It iseasy to show that an identical re-parameterization of β1 and β2 (β1 ◦γ, β2◦γ)does not change the correspondence. Thus, any criterion (a cost function or ametric) used for determining optimal correspondences between curves shouldsatisfy the isometry property: d(β1, β2) = d(β1 ◦ γ, β2 ◦ γ). Note that the un-der the standard L

2 metric, we have ‖β1−β2‖ 6= ‖β1 ◦γ−β2 ◦γ‖, in general.Most criteria used in the current literature (including [4, 3, 2, 15]) do not sat-isfy this fundamental property. In other words, the quantity for registrationof curves is not invariant to the re-parameterization action. Many criteria,such as minimum description length, mutual information, relative entropy,etc, are not even proper metrics on representation spaces. One exception isthe framework used in [19], but it applies to only to curves in R

2. In addi-tion, isometry is needed for subsequent statistical analysis of shapes such asdefining geodesic paths, distances, and summary statistics such as means andcovariances.

In this paper, we summarize recent progress in using Riemannian methodsfor shape analysis of curves and surfaces. The advantages of these methodsare as follows: 1) they allow for a joint solution to the problem of shape reg-istration and analysis, 2) the criterion used for registration and analysis isbased on an elastic Riemannian metric, 3) the analysis is fully invariant totranslation, scale, rotation and re-parameterization, and 4) they allow one

Page 5: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

Joint Registration and Shape Analysis of Curves and Surfaces 5

to define geodesic paths between shapes, sample means and covariances andother statistics on manifolds containing curves and surfaces. (These samplestatistics can be further used for deriving probability models.) Furthermore,because of these desirable properties, the use of these methods in patternrecognition tasks such as shape classification and tracking significantly im-proves performance over existing methods. We will consider shape analysisof curves and surfaces separately in the next two sections.

2 Shape Analysis of Curves

Although the framework described here is valid for curves in any Euclideanspace Rn, we will focus primarily on planar closed curves. These curves orig-inate as boundaries of imaged objects in 2D images and their shapes forman important feature in object classification and recognition. While manycurrent techniques, such as the active shape model and Kendall’s shape anal-ysis (KSA) [8, 1] use discrete points (or landmarks) sampled from the curvesfor analyzing their shapes, we will work with full parameterized curves. Asmentioned earlier, an important aspect of this framework is that shape dis-tances, geodesics, and statistics should be invariant to how the curves areparameterized. For details please refer to the papers [6, 18, 17].

Mathematical Representation

We start by describing the mathematical framework for the elastic shapeanalysis of curves. Let a parameterized closed curve be denoted as β : S1 →R

2. (The domain of parameterization for closed curves is naturally chosen tobe S1 rather than an interval.) In order to analyze its shape, β is represented

by its square-root velocity function (SRVF): q(t) = β̇(t)√‖β̇(t)‖

∈ R2. The SRVF

q includes both the instantaneous speed and the direction of the curve βat time t. The use of the time derivative makes the SRVF invariant to anytranslation of curve β. In order for the shape analysis to be invariant to scale,one can rescale each curve to length one. The set of all unit length, closedcurves in R

2, represented by their SRVFs, is called the preshape space C. If qis the SRVF of a curve β, then the SRVF of β ◦ γ is (q, γ) = (q ◦ γ)√γ̇. Animportant property of SRVFs is that for any two curves, the correspondingSRVFs satisfy ‖q1 − q2‖ = ‖(q1, γ) − (q2, γ)‖ for any re-parameterizationfunction γ.

There are four shape-preserving transformations for curves: translation,scale, rotation, and re-parameterization. Of these, the first two have alreadybeen eliminated from the representations, but the other two remain. Curvesthat are within a rotation and/or a re-parameterization of each other result

Page 6: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

6 Authors Suppressed Due to Excessive Length

in different elements of C despite having the same shape. Let SO(2) be thegroup of 2×2 rotation matrices and Γ be the group of all re-parameterizations(they are actually orientation-preserving diffeomorphisms of the unit circleS1). In order to unify all elements in C that denote the same shape, one can

define equivalence classes of the type: [q] = {O(q ◦γ)√γ̇|O ∈ SO(2), γ ∈ Γ} .Each such equivalence class [q] is associated with a shape uniquely and viceversa. The set of all these equivalence classes is called the shape space S;mathematically, it is a quotient space of the preshape space: S = C/(SO(2)×Γ ).

An important advantage of the SRVF representation is that the elastic Rie-mannian metric defined by Mio et al. [14], turns into the standard L

2 metric,as shown by Joshi et al. [6], Srivastava et al. [18]. That is, one can alterna-tively compute the path lengths, or the sizes of deformations between curves,using the cumulative norms of the differences between successive curves alongthe paths in the SRVF space. This turns out to be much simpler and a veryeffective strategy for comparing shapes of curves, by finding the paths withleast amounts of deformations between them, where the amount of defor-mation is measured by an elastic metric. This gives a proper distance dcfor comparing elements of C. Another distinct advantage of using SRVFs isthat the distance between any two curves remains same if they are rotatedand re-parameterized in the same way, i.e. dc(q1, q2) = dc((q1, γ), (q2, γ)) anddc(q1, q2) = dc(Oq1, Oq2) for all O ∈ SO(2) and γ ∈ Γ . Consequently, a shapedistance between any two curves is given by:

ds([q1], [q2]) = infγ∈Γ,O∈SO(2)

dc(q1, O(q2 ◦ γ)√

γ̇) . (1)

Shape Matching and Geodesics

According to Eqn. 1, the distance between any two shapes is given by thelength of the shortest path, called a geodesic, connecting them in that man-ifold. An interesting feature of this framework is that it not only providesa distance between shapes of two curves, but also a geodesic path betweenthem in S. The geodesics are actually computed using the differential ge-ometry of the underlying space S. One technique for finding geodesics iscalled path straightening [9]. It is an iterative technique that initializes anarbitrary path and then iteratively “straightens” it by updating it along thenegative gradient of the cost function. This gives a geodesic and a geodesicdistance between SRVFs in C but the goal is to compute geodesic paths in S.In other words, geodesic paths between the equivalence classes [q1] and [q2]are needed, not just between q1 and q2. This desired geodesic is obtained byfinding the shortest geodesic amongst all pairs (q̃1, q̃2) ∈ ([q1] × [q2]). Thissearch is further simplified by fixing an arbitrary element of [q1], say q1, andsearching over all rotations and re-parameterizations of q2 to minimize thegeodesic length, as stated in Eqn. 1. The minimization over SO(2) is in con-

Page 7: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

Joint Registration and Shape Analysis of Curves and Surfaces 7

ventional way and the optimization over Γ is accomplished using the dynamicprogramming algorithm or a gradient-type approach [18].

We present two examples of optimization over Γ in Fig. 4. The parametriza-tion of a curve is displayed using colors, i.e. same color implies the same valueof t. It can be seen that the matching after the optimization over Γ is betterin matching similar geometric features. Several examples of geodesic paths inthe shape space S are shown in Fig. 5; these geodesics are compared with thegeodesics obtained by KSA, for the same shapes. KSA is a method that onlyconsiders rigid transformation and scaling. It is easy to see that the geodesicsresulting from our elastic shape analysis (ESA) appear to have more naturaldeformations as they are better in matching features across shapes.

β1 β2 β2 ◦ γ∗ β1 β2 β2 ◦ γ∗

Fig. 4 Comparison of initial matching and matching after optimization over Γ .

KSA

ESA

KSA

ESA

KSA

ESA

Fig. 5 Examples of geodesic paths between shapes using KSA and ESA.

Page 8: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

8 Authors Suppressed Due to Excessive Length

Shape Statistics of Curves

The richness of this framework comes from its ability to provide shape statis-tics, such as sample mean or sample PCA, under proper shape metrics. Thenotion of a mean on a nonlinear manifold is typically established using theKarcher mean [7]. For a given set of curves β1, β2, . . . , βn, represented by theirSRVFs q1, q2, . . . , qn, their Karcher mean is defined as the quantity that sat-isfies: [µ] = argmin[q]∈S

∑ni=1 ds([q], [qi])

2. There is a gradient-based iterativealgorithm for finding the minimizer of this cost function that can be foundin [7, 13, 17]. Shown in Fig. 6 are some examples of mean shapes. The topthree rows show a set of given curves and bottom rows display their meanscomputed using the two methods discussed here: KSA and ESA.

Sample Shapes

1

2

3

KSA ESA KSA ESA KSA ESA

Fig. 6 Examples of mean shapes under two different methods.

For computing and analyzing the second and higher moments of a shapesample, the tangent space to the shape manifold S at the point µ is used. Thisspace, denoted by Tµ(S), is convenient because it is a vector space and one canapply more traditional methods here. The details are omitted for brevity. Fig.7 shows the principal geodesic paths along two different dominant directions,respectively. The middle points in each row are the mean shapes.

One important use of means and covariances of shape families is in devis-ing “Gaussian”-type probability densities on the shape space S. In order totackle the nonlinearity of the shape space, a common approach is to impose aGaussian distribution on the tangent space Tµ(S) since that is a vector space.In case of ESA this space is infinite-dimensional, so the Gaussian model is ac-

Page 9: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

Joint Registration and Shape Analysis of Curves and Surfaces 9

PC1 PC2

KSA

ESA

KSA

ESA

KSA

ESA

Fig. 7 Two principal directions of variability in shapes shown in Fig. 6.

tually imposed on a finite-dimensional subspace, e.g. a principal subspace, ofTµ(S). Shown in Fig. 8 are examples of random samples from S using meansand covariances estimated from data shown in Fig. 6. For comparison, thisfigure also shows random samples from similar Gaussian models but usingKSA.

It is easy to observe the superiority of the results obtained using ESA.Consider the example of hand, the mean using ESA is much more represen-tative. In the principle geodesic paths, using ESA, the last finger shrinks andthe fourth one grows in the first principle direction and both fingers shrinkin the second principle direction. The shapes get KSA are distorted, includ-ing the mean, the principle geodesic paths and random samples. The samedistortion can be observed in another two cases using KSA.

Method “Gaussian” Random Samples

KSA

ESA

Fig. 8 Random samples from “Gaussian”-type distributions under the two methods:KSA, and ESA, for parameters estimated from the given shapes shown in Fig. 6.

Page 10: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

10 Authors Suppressed Due to Excessive Length

3 Shape Analysis of Parameterized Surfaces

In this section we describe a similar framework for studying shapes of pa-rameterized surfaces using novel mathematical representations. For detailsof this approach, please refer to the papers by Kurtek et al. [11, 10, 12].We assume that the surfaces are closed and have genus zero, so they canbe represented as embeddings of S2 in R

3, i.e. f : S2 → R3. The function

f also denotes a parameterization of S. Let the set of parameterized sur-faces be F = {f : S2 7→ R

3|∫

S2‖f(s)‖2ds < ∞ and f is smooth}, where

ds is the standard Lebesgue measure on S2. A re-parameterization γ of a

surface is given by a diffeomorphism of S2 to itself; let Γ be the set of allre-parameterizations. The re-parameterization of a surface f is then given bythe composition f ◦ γ.

Mathematical Representation

To endow F with a Riemannian metric, we begin by defining a new represen-tation of surfaces, called q-maps, defined as q(s) =

κ(s)f(s), where κ(s) isthe area-multiplication factor at the point s ∈ S

2. (An alternative mathemat-ical representation using the normal vector field on a surface is described ina recent paper [5] .) If a surface f is re-parameterized as f ◦γ, then its q-mapis given by (q, γ) ≡ (q ◦ γ)

Jγ , where Jγ is the determinant of Jacobianof γ. For comparing shapes, we choose the natural L2 metric on the spaceof q-maps. Similar to SRVFs for curves, an important advantage of usingthese q-maps is that a simultaneous re-parameterization of any two surfacesdoes not change the distance between them. That is, for any two surfacesf1 and f2, represented by their q-maps, q1 and q2 respectively, we have that‖q1 − q2‖ = ‖(q1, γ)− (q2, γ)‖. Actually, the Riemannian metric that we willuse on F is the pullback of the L

2 metric from the space of q-maps. Withthis induced metric, F becomes a Riemannian manifold.

Shape analysis of surfaces can be made invariant to translation and scal-ing by normalizing. With a slight abuse of notation, we define the space ofnormalized surfaces as F . F forms the pre-shape space in our analysis. Theremaining groups – rotation and re-parameterization – are dealt with dif-ferently, by removing them algebraically from the representation space. Theequivalence class of a surface f is given by [f ] = {O(f ◦γ)|O ∈ SO(3), γ ∈ Γ}and the set of all such equivalence classes is defined to be S.

Shape Matching and Geodesics

The next step is to define geodesic paths in S. Similar to the curve case, apath-straightening approach is used to find geodesics in F [12]. Once we havean algorithm for finding geodesics in F , we can obtain geodesics and geodesic

Page 11: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

Joint Registration and Shape Analysis of Curves and Surfaces 11

lengths in S by solving an additional minimization problem over SO(3)×Γ .Let f1 and f2 denote two surfaces and let 〈〈·, ·〉〉 be the inherited Riemannianmetric on F . Then, the geodesic distance between shapes of f1 and f2 willbe given by quantities of the following type:

minγ,O

minF : [0, 1] → F

F (0) = f1, F (1) = O(f2 ◦ γ)

(∫ 1

0

〈〈Ft(t), Ft(t)〉〉(1/2) dt)

. (2)

Here F (t) is a path in F indexed by t, and the quantity∫ 1

0〈〈Ft(t), Ft(t)〉〉(1/2)dt

denotes the length of F where Ft is used for dFdt . The minimization inside the

brackets, thus, denotes the problem of finding a geodesic path between thesurfaces f1 and O(f2 ◦ γ), where O and γ stand for an arbitrary rotation andre-parameterization of f2, respectively. The minimization outside the bracketseeks the optimal rotation and re-parameterization of the second surface so asto best match it with the first surface. In simple words, the outside optimiza-tion solves the registration or matching problem while the inside optimizationsolves for both an optimal deformation (geodesic, F ∗) and a formal distance(geodesic distance) between shapes.

We demonstrate these ideas using some examples in Fig. 9. These examplesalso highlight improvements in registration of surfaces during the optimiza-tion over SO(3)×Γ , by comparing corresponding geodesic paths between thesame pairs of surfaces in F and S. In all of these experiments, we notice thatthe geodesic distances in S are much smaller than the geodesic distances inF .

Shape Statistics of Surfaces

Here we briefly present some examples for computing the Karcher mean fora set of surfaces using the method similar to planar closed curves.

We present some examples of Karcher mean shapes using toy objects. Forcomparison, we also display f̃ = (1/n)

∑ni=1 fi, i.e. without any rotational or

re-parameterizational alignment. For each example we show the decrease inthe gradient of the cost function during the computation of the Karcher mean.In the top part of Fig. 10, we present means for ten unimodal surfaces withrandom peak placements on a sphere. The f̃ surface has ten very small peaksat the locations of the peaks in the sample. On the other hand, the mean inS has one peak, which is of the same size as all of the peaks in the sample. Inthis simple example one can clearly see the effect of feature preservation dueto rotational and re-parameterizational alignment. In the bottom part of Fig.10, we present mean shapes of nine surfaces with dual peaks. We note thatthe mean in F has one peak aligned (at the location of the common peak in

Page 12: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

12 Authors Suppressed Due to Excessive Length

Pre-Shape Space Shape Space

E(F∗) = 0.228 E(F∗) = 0.161

E(F∗) = 0.283 E(F∗) = 0.171

E(F∗) = 0.288 E(F∗) = 0.098

E(F∗) = 0.203 E(F∗) = 0.133

E(F∗) = 0.192 E(F∗) = 0.127

Fig. 9 Comparison of geodesics in F and S, and their geodesic distances.

the sample) and one very wide and small peak, which could be considered asa failure mode. The very wide peak happens due to averaging out of features.The mean in S has two peaks due to a crisp alignment and thus is a muchbetter representative of the sample.

4 Conclusion

In this paper we have described recent progress in using Riemannian meth-ods in shape analysis of curves and surfaces. An important attribute of thisframework is that it performs shape comparison and registration jointly underthe same metric. The choice of elastic metric and novel mathematical repre-

Page 13: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

Joint Registration and Shape Analysis of Curves and Surfaces 13

Sample

1 2 3 4 5 6 7

3

4

5

6

7

8

9

x 10−6

Pre-Shape Mean Shape Mean Energy

Sample

1 2 3 4 5 6 7 8

1

2

3

4

5

x 10−5

Pre-Shape Mean Shape Mean Energy

Fig. 10 Mean computation for a sample of surfaces with one peak (top) and dualpeaks (bottom). The figures compare mean computations in F and S.

sentations (SRVFs for curves and q-maps for surfaces) enable us to use L2

norms and standard optimization tools. This framework provides geodesics –optimal deformations – between shapes, and tools for computing statisticalsummaries of sets of shapes.

References

1. Dryden, I.L., Mardia, K.: Statistical shape analysis. John Wiley & Son (1998)2. Frenkel, M., Basri, R.: Curve matching using the fast marching method. In:

Proceedings of Energy Minimization Methods in CVPR, pp. 35–51 (2003)3. Gdalyahu, Y., Weinshall, D.: Flexible syntactic matching of curves and its appli-

cation to automatic hierarchical classification of silhouettes. IEEE Trans. PatternAnalysis and Machine Intelligence 21(12), 1312–1328 (1999)

4. Geiger, D., Gupta, A., Costa, L., Vlontzos, J.: Dynamic programming for detect-ing, tracking, and matching deformable contours. IEEE Trans. Pattern Analysisand Machine Intelligence 17(3), 294 –302 (1995)

5. Jermyn, I.H., Kurtek, S., Klassen, E., Srivastava, A.: Elastic shape matching ofparameterized surfaces using square root normal fields. In: Proceedings of ECCV(2012)

6. Joshi, S.H., Klassen, E., Srivastava, A., Jermyn, I.H.: A novel representation forRiemannian analysis of elastic curves in Rn. In: Proceedings of CVPR, pp. 1–7(2007)

7. Karcher, H.: Riemannian center of mass and mollifier smoothing. Communica-tions on Pure and Applied Mathematics 30(5), 509–541 (1977)

Page 14: JointRegistrationand ShapeAnalysisof Curves andSurfacesjinsu/papers/JointRegistration-Special.pdf · performance over the existing methods in terms of feature preservation and model

14 Authors Suppressed Due to Excessive Length

8. Kendall, D.G.: Shape manifolds, Procrustean metrics and complex projectivespaces. Bulletin of London Mathematical Society 16, 81–121 (1984)

9. Klassen, E., Srivastava, A.: Geodesics between 3D closed curves using path-straightening. In: Proceedings of ECCV, Lecture Notes in Computer Science,pp. I: 95–106 (2006)

10. Kurtek, S., Klassen, E., Ding, Z., Jacobson, S., Jacobson, J., Avison, M., Srivas-tava, A.: Parameterization-invariant shape comparisons of anatomical surfaces.IEEE Trans. Medical Imaging 30(3), 849–858 (2011)

11. Kurtek, S., Klassen, E., Ding, Z., Srivastava, A.: A novel Riemannian frameworkfor shape analysis of 3D objects. In: Proceedings of CVPR, pp. 1625–1632 (2010)

12. Kurtek, S., Klassen, E., Gore, J.C., Ding, Z., Srivastava, A.: Elastic geodesicpaths in shape space of parameterized surfaces. IEEE Trans. Pattern Analysisand Machine Recognition 34(9), 1717– 1730 (2012)

13. Le, H.L., Kendall, D.G.: The Riemannian structure of Euclidean shape spaces: anovel environment for statistics. Annals of Statistics 21(3), 1225–1271 (1993)

14. Mio, W., Srivastava, A., Joshi, S.H.: On shape of plane elastic curves. Intl.Journal of Computer Vision 73(3), 307–324 (2007)

15. Sebastian, T., Klein, P., Kimia, B.: On aligning curves. IEEE Trans. PatternAnalysis and Machine Recognition 25(1), 116 –125 (2003)

16. Small, C.: The statistical theory of shape. Springer (1996)17. Srivastava, A., Joshi, S.H., Mio, W., Liu, X.: Statistical shape analysis: Cluster-

ing, learning and testing. IEEE Trans. Pattern Analysis and Machine Intelligence27(4), 590–602 (2005)

18. Srivastava, A., Klassen, E., Joshi, S.H., Jermyn, I.H.: Shape analysis of elasticcurves in Euclidean spaces. IEEE Transactions on Pattern Analysis and MachineIntelligence 33, 1415–1428 (2011)

19. Younes, L., Michor, P.W., Shah, J., Mumford, D., Lincei, R.: A metric on shapespaces with explicit geodesics. Matematica E Applicazioni 19(1), 25 – 57 (2008)