1Numerical geometry of non-rigid shapes Non-rigid correspondence

Numerical geometry of non-rigid shapes

Non-rigid correspondence

Alexander Bronstein, Michael Bronstein, Ron Kimmel© 2007 All rights reserved

2Numerical geometry of non-rigid shapes Non-rigid correspondence

Correspondence problems

Given two objects and , find a mapping

copying features to corresponding similar features

Not always well-defined semantically

Aesthetic rather than geometric considerations often apply

Yet, if objects are sufficiently similar (nearly isometric),

correspondence

is likely to have a geometric meaning

3Numerical geometry of non-rigid shapes Non-rigid correspondence

One-dimensional intuition

A closed curve has an arc-length parametrization

Arc-length parametrization is unique up to choice of starting point

and direction

Correspondence of curves: bring and to arc-length

parametrization

Find the isometry aligning features

4Numerical geometry of non-rigid shapes Non-rigid correspondence

One-dimensional intuition

5Numerical geometry of non-rigid shapes Non-rigid correspondence

Intrinsic parametrization

Bad news: no equivalent of the canonical arc-length parametrization for

surfaces

We can still find an intrinsic parametrization

Given and its bending find a method to compute a

parametrization such that

Intrinsic parametrization gives a correspondence between shapes

6Numerical geometry of non-rigid shapes Non-rigid correspondence

Euclidean embedding

Embedding defined up to congruence in

Requires alignment

Inaccuracies due to embedding error

7Numerical geometry of non-rigid shapes Non-rigid correspondence

Minimum distortion correspondence

Minimum distortion correspondence: a map

Correspondence relates “similar parts to similar parts”

The correspondence is defined up to self-isometries of and

Isometry groups are trivial, unless the shapes have symmetries

BBK, IEEE TVCG, 2007

8Numerical geometry of non-rigid shapes Non-rigid correspondence

Minimum distortion correspondence

If shapes are symmetric, minimum distortion correspondence is not

unique

Intrinsic information is insufficient to select any of them

Adding extrinsic information (e.g. orientation) can resolve ambiguity

Photometric information can be added as well

9Numerical geometry of non-rigid shapes Non-rigid correspondence

Texture transfer

GMDS provides a natural correspondence between and

Define new texture on

BBK, IEEE TVCG, 2006

Problem: transfer texture from to

10Numerical geometry of non-rigid shapes Non-rigid correspondence

Reference Transferred texture

11Numerical geometry of non-rigid shapes Non-rigid correspondence

Virtual makeup

BBK, IEEE TVCG, 2006

A “virtual mask” following the facial deformations

12Numerical geometry of non-rigid shapes Non-rigid correspondence

Reference Transferred texture

12Numerical geometry of non-rigid shapes Non-rigid correspondence

13Numerical geometry of non-rigid shapes Non-rigid correspondence 13

Reference Transferred texture

Numerical geometry of non-rigid shapes Non-rigid correspondence

14Numerical geometry of non-rigid shapes Non-rigid correspondence

Calculus of non-rigid shapes

Extrinsic geometry can also be manipulated

Correspondence makes affine combination of shapes well-defined

Establishes a calculus of shapes

BBK, IEEE TVCG, 2006

15Numerical geometry of non-rigid shapes Non-rigid correspondence

Extrapolation

Abstract manifold of shape deformations

Shape animation: trajectory

Minimum-distortion correspondence allows creating a (locally) linear

space, in which shapes are represented as vectors

Calculus of non-rigid shapes

BBK, IEEE TVCG, 2006

Interpolation

16Numerical geometry of non-rigid shapes Non-rigid correspondence

Calculus of non-rigid shapes

Extrinsic coordinates and texture interpolation

CORRESPONDENCE

Extrinsic geometry

Texture

BBK, IEEE TVCG, 2006

17Numerical geometry of non-rigid shapes Non-rigid correspondence

Interpolation

0 10.50.25 0.75

I N T E R P O L A T E D F R A M E S

Temporal super-resolution: increase frame rate of 3D video by

adding interpolated frames

Interpolation of geometry and texture

BBK, IEEE TVCG, 2006

18Numerical geometry of non-rigid shapes Non-rigid correspondence

Extrapolation

Expression exaggeration: synthesize new expressions using a

non-convex combination

Interpolation of geometry and texture

0 1.51

NEUTRAL EXPRESSION EXAGGERATEDEXPRESSION

BBK, IEEE TVCG, 2006

19Numerical geometry of non-rigid shapes Non-rigid correspondence

Non-rigid correspondence

ISOMETRIC

NEARLY-ISOMETRIC

NON-ISOMETRIC

20Numerical geometry of non-rigid shapes Non-rigid correspondence

Texture substitution

BBK, IEEE TVCG, 2006

ALICE BOB ALICE’S TEXTUREBOB’S GEOMETRY

21Numerical geometry of non-rigid shapes Non-rigid correspondence

Metamorphing

0 10.50.25 0.75

Convex combination between two different objects

Morphing of geometry and texture

SOURCE TARGET

BBK, IEEE TVCG, 2006

22Numerical geometry of non-rigid shapes Non-rigid correspondence

GMDS can be used to find non-rigid correspondence

Correspondence allows to establish calculus of shapes

Conclusions so far…