mesh processing course : mesh parameterization

Parameterization and Flattening

Gabriel PeyréCEREMADE, Université Paris-Dauphine

www.numerical-tours.com

http://www.ceremade.dauphine.fr/~peyre/cours/manifold/

http://www.ceremade.dauphine.fr/~peyre/cours/manifold/

parameterization

Mesh Parameterization - Overview

2

texturemappingM � R3

parameterization

D � R2

Mesh Parameterization - Overview

2

��1

re-sampling

zoom

texturemappingM � R3

Overview•Dirichlet Energy on Meshes

• Harmonic Parameterization

• Spectral Flattening

• Barycentric Coordinates for Warping

• Geodesic Flattening

• High Dimensional Data Analysis3

Local Averaging

4

Local operator: W = (wij)i,j�V where wij =

�> 0 if j � Vi,

0 otherwise.(Wf)i =

�

(i,j)�E

wijfj .

Examples: for i � j,

wij = 1combinatorial

wij = 1||xj�xi||2

distancewij = cot(�ij) + cot(⇥ij)

conformal

Local averaging operator W = (wij)i,j�V : ⇥ (i, j) � E, wij =wij�

(i,j)�E wij.

W = D�1W with D = diagi(di) where di =�

(i,j)⇥E

wij .

Averaging: W1 = 1.

(explanations later)

Voronoi and Dual Mesh

5

Definition for a planar triangulation M of a meshM � R2.

Voronoi for vertices: ⇧ i ⇤ V, Ei = {x ⇤M \ ⇧ j ⌅= i, ||x� xi|| ⇥ ||x� xj ||}

Voronoi for edges: ⌅ e = (i, j) ⇥ E, Ee = {x ⇥M \ ⌅ e� ⇤= e, d(x, e) � d(x, e�)}

Partition of the mesh: M =�

i�V

Ei =�

e�E

Ee.

i

j

Ai

cf

i

j

A(ij)cf

Dual mesh 1:3 subdivided mesh

Approximating Integrals on MeshesApproximation of integrals on vertices and edges:

⇥

Mf(x)dx �

�

i�V

Ai f(xi) ��

e=(i,j)�E

Ae f([xi, xj ]).

Theorem : ⇥ e = (i, j) � E,

Ae = Area(Ee) =12

||xi � xj ||2 (cot(�ij) + cot(⇥ij))

i

j

A(ij)cf

Approximating Integrals on MeshesApproximation of integrals on vertices and edges:

⇥

Mf(x)dx �

�

i�V

Ai f(xi) ��

e=(i,j)�E

Ae f([xi, xj ]).

Theorem : ⇥ e = (i, j) � E,

Ae = Area(Ee) =12

||xi � xj ||2 (cot(�ij) + cot(⇥ij))

i

j

A(ij)cf

A

B

C

O

�

�

�

��

�

h

� + � + � =�

2

A(ABO) = ||AB||� h = ||AB||� ||AB||2

tan(�)

A(ABO) =||AB||2

2tan

�⇤

2� (� + ⇥)

�

Proof:

Cotangent WeightsSobolev norm (Dirichlet energy): J(f) =

�

M||�f(x)||2dx

1.2 Linear Mesh Processing

The smoothing property corresponds to W1 = 1 which means that the unit vector is an eigen-vector of W with eigenvalue 1.Example 3. In practice, we use three popular kinds of averaging operators.

Combinatorial weights: they depends only on the topology (V,E) of the vertex graph

⇧ (i, j) ⇤ E, wij = 1.

Distance weights: they depends both on the geometry and the topology of the mesh, but do notrequire faces information,

⇧ (i, j) ⇤ E, wij =1

||xj � xi||2.

Conformal weights: they depends on the full geometrical realization of the 3D mesh since theyrequire the face information

⇧ (i, j) ⇤ E, wij = cot(�ij) + cot(⇥ij). (1.3)

Figure 1.2 shows the geometrical meaning of the angles �ij and ⇥ij

�ij = ⇥(xi, xj , xk1) and ⇥ij = ⇥(xi, xj , xk2),

where (i, j, k1) ⇤ F and (i, j, k2) ⇤ F are the two faces adjacent to edge (i, j) ⇤ E. We will seein the next section the explanation of these celebrated cotangent weights.

xi

xj

xk1

xk2

�ij

�ij

Figure 1.2: One ring around a vertex i, together with the geometrical angles �ij and ⇥ij used tocompute the conformal weights.

One can use iteratively a smoothing in order to further filter a function on a mesh. The resultingvectors Wf, W 2, . . . , W kf are increasingly smoothed version of f . Figure 1.3 shows an example ofsuch iterations applied to the three coordinates of mesh. The sharp features of the mesh tend todisappear during iterations. We will make this statement more precise in the following, by studyingthe convergence of these iterations.

1.2.3 Approximating Integrals on a Mesh

Before investigating algebraically the properties of smoothing operators, one should be carefulabout what are these discrete operators really approximating. In order for the derivation to besimple, we make computation for a planar triangulation M of a mesh M ⇥ R2.

In the continuous domain, filtering is defined through integration of functions over the mesh. Inorder to descretize integrals, one needs to define a partition of the plane into small cells centeredaround a vertex or an edge.

Definition 10 (Vertices Voronoi). The Voronoi diagram associated to the vertices is

⇧ i ⇤ V, Ei = {x ⇤M \ ⇧ j ⌅= i, ||x� xi|| � ||x� xj ||}

5

Cotangent Weights

7

Approximation of Dirichelet energy:

where wij = cot(�ij) + cot(⇥ij).

⇥

M||⇤xf ||2dx ⇥

�

e�E

Ae|(Gf)e|2 =�

(i,j)�E

Ae|f(xj)� f(xi)|2

||xj � xi||2

=�

(i,j)�E

wij |f(xj)� f(xi)|2

Sobolev norm (Dirichlet energy): J(f) =�

M||�f(x)||2dx

1.2 Linear Mesh Processing

The smoothing property corresponds to W1 = 1 which means that the unit vector is an eigen-vector of W with eigenvalue 1.Example 3. In practice, we use three popular kinds of averaging operators.

Combinatorial weights: they depends only on the topology (V,E) of the vertex graph

⇧ (i, j) ⇤ E, wij = 1.

Distance weights: they depends both on the geometry and the topology of the mesh, but do notrequire faces information,

⇧ (i, j) ⇤ E, wij =1

||xj � xi||2.

Conformal weights: they depends on the full geometrical realization of the 3D mesh since theyrequire the face information

⇧ (i, j) ⇤ E, wij = cot(�ij) + cot(⇥ij). (1.3)

Figure 1.2 shows the geometrical meaning of the angles �ij and ⇥ij

�ij = ⇥(xi, xj , xk1) and ⇥ij = ⇥(xi, xj , xk2),

where (i, j, k1) ⇤ F and (i, j, k2) ⇤ F are the two faces adjacent to edge (i, j) ⇤ E. We will seein the next section the explanation of these celebrated cotangent weights.

xi

xj

xk1

xk2

�ij

�ij

Figure 1.2: One ring around a vertex i, together with the geometrical angles �ij and ⇥ij used tocompute the conformal weights.

One can use iteratively a smoothing in order to further filter a function on a mesh. The resultingvectors Wf, W 2, . . . , W kf are increasingly smoothed version of f . Figure 1.3 shows an example ofsuch iterations applied to the three coordinates of mesh. The sharp features of the mesh tend todisappear during iterations. We will make this statement more precise in the following, by studyingthe convergence of these iterations.

1.2.3 Approximating Integrals on a Mesh

Before investigating algebraically the properties of smoothing operators, one should be carefulabout what are these discrete operators really approximating. In order for the derivation to besimple, we make computation for a planar triangulation M of a mesh M ⇥ R2.

In the continuous domain, filtering is defined through integration of functions over the mesh. Inorder to descretize integrals, one needs to define a partition of the plane into small cells centeredaround a vertex or an edge.

Definition 10 (Vertices Voronoi). The Voronoi diagram associated to the vertices is

⇧ i ⇤ V, Ei = {x ⇤M \ ⇧ j ⌅= i, ||x� xi|| � ||x� xj ||}

5

Cotangent Weights

7

Approximation of Dirichelet energy:

Theorem : wij > 0�⇥ �ij + ⇥ij < ⇤

where wij = cot(�ij) + cot(⇥ij).

⇥

M||⇤xf ||2dx ⇥

�

e�E

Ae|(Gf)e|2 =�

(i,j)�E

Ae|f(xj)� f(xi)|2

||xj � xi||2

=�

(i,j)�E

wij |f(xj)� f(xi)|2

Sobolev norm (Dirichlet energy): J(f) =�

M||�f(x)||2dx

Overview• Dirichlet Energy on Meshes

•Harmonic Parameterization





Mesh Parameterization

3D space (x,y,z)

2D parameter domain (u,v)

boundaryboundary 9

Parameterization: bijection � :M �⇤ D ⇥ R2.

�

Hypothesis: � = (�1,�2) is smooth, minimizes

⇥xi � ⇥M, �(i) = �0(i) � ⇥D.

min��0

�

(i,j)�E

wi,j(|�1(i)� �1(j)|2 + |�2(i)� �2(j)|2)

With boundary conditions �0:


3D space (x,y,z)


boundaryboundary 9


�⇥ sparse linear system to solve.

�


⇥xi � ⇥M, �(i) = �0(i) � ⇥D.

min��0

�

(i,j)�E

wi,j(|�1(i)� �1(j)|2 + |�2(i)� �2(j)|2)

�⇥ i /� ⇥M, (L�1)(i) = (L�2)(i) = 0⇥ i � ⇥M, �(i) = �0(i) � ⇥D.

Optimality conditions:



3D space (x,y,z)


boundaryboundary 9


�⇥ sparse linear system to solve.

Theorem: (Tutte) if � i, j, wij > 0,

then � is a bijection.

�

⇥ i /� ⇥M, �(i) =�

(i,j)�E

wi,j�(j).


⇥xi � ⇥M, �(i) = �0(i) � ⇥D.

min��0

�

(i,j)�E

wi,j(|�1(i)� �1(j)|2 + |�2(i)� �2(j)|2)

�⇥ i /� ⇥M, (L�1)(i) = (L�2)(i) = 0⇥ i � ⇥M, �(i) = �0(i) � ⇥D.

Optimality conditions:

Remark: each point is theaverage of its neighbors:


Examples of Parameterization

Com

bina

tori

alC

onfo

rmal

Mesh

10

Examples of Parameterization

Com

bina

tori

alC

onfo

rmal

Mesh

11

Application to Remeshing

12

parameterization �

��1

zoom

re-sampling

P. Alliez et al., Isotropic Surface Remeshing, 2003.

Application to Texture Mapping

13parameterization

�

texture g(u)

color g(�(x))

Mesh Parameterization #1

14

�

S

S0

�

� : S0 �⇥ S�S0 �⇥ �S

⇥x � S0\⇥S0, �� = 0


14

W = make_sparse(n,n);for i=1:3 i1 = mod(i-1,3)+1; i2 = mod(i ,3)+1; i3 = mod(i+1,3)+1; pp = vertex(:,faces(i2,:)) - vertex(:,faces(i1,:)); qq = vertex(:,faces(i3,:)) - vertex(:,faces(i1,:)); % normalize the vectors pp = pp ./ repmat( sqrt(sum(pp.^2,1)), [3 1] ); qq = qq ./ repmat( sqrt(sum(qq.^2,1)), [3 1] ); % compute angles a = 1 ./ tan( acos(sum(pp.*qq,1)) ); a = max(a, 1e-2); % avoid degeneracy W = W + make_sparse(faces(i2,:),faces(i3,:), a, n, n ); W = W + make_sparse(faces(i3,:),faces(i2,:), a, n, n );end

�

S

S0

�

� : S0 �⇥ S�S0 �⇥ �S

⇥x � S0\⇥S0, �� = 0

Discretization of �:


14


D = spdiags(full( sum(W,1) ), 0, n,n);L = D - W; L1 = L; L1(boundary,:) = 0; L1(boundary + (boundary-1)*n) = 1;

�

S

S0

�

� : S0 �⇥ S�S0 �⇥ �S

⇥x � S0\⇥S0, �� = 0

Formation of the linear system:



14


D = spdiags(full( sum(W,1) ), 0, n,n);L = D - W; L1 = L; L1(boundary,:) = 0; L1(boundary + (boundary-1)*n) = 1;

Rx = zeros(n,1); Rx(boundary) = x0;Ry = zeros(n,1); Ry(boundary) = y0;x = L1 \ Rx; y = L1 \ Ry;

�

S

S0

�

� : S0 �⇥ S�S0 �⇥ �S

⇥x � S0\⇥S0, �� = 0

Formation of the linear system:

Formation of the RHS and resolution:



15

Exercise: perform the linear interpolation of the parameterization.

Exercise: display the geometry image using a checkboard texture.

Geometry image: re-sample X/Y/Z coordinates of � on a grid.�� store the surface as a color (R/G/B) image.


16

Exercise: Locate the position of the eyes / the mouth

Exercise: Compute an a�ne transformation to re-align the texture.

in the texture and on the mesh.

Mesh Deformations

17

Initial position: xi � R3.

Displacement of anchors:� i � I, xi �� x�

i = xi + �i � R3

Linear deformation:x�

ixiI

� i � I, �(i) = �i

� i /� I, ��(i) = 0

x�i = xi + �(i)

Mesh Deformations

17



i = xi + �i � R3


ixiI

% modify LaplacianL1 = L; L1(I,:) = 0;L1(I + (I-1)*n) = 1;% displace verticesvertex = vertex + ( L1 \ Delta0' )';

� i � I, �(i) = �i

� i /� I, ��(i) = 0

x�i = xi + �(i)

Mesh Deformations

17



i = xi + �i � R3


ixi

x�iNon-linear deformation:

xi = xi + �i x�i

coarsescale

details

xi �� x�iLinear deformation:

x�i = xi + ��i, ni�n�

iExtrusion along normals:

n�i

I

% modify LaplacianL1 = L; L1(I,:) = 0;L1(I + (I-1)*n) = 1;% displace verticesvertex = vertex + ( L1 \ Delta0' )';

� i � I, �(i) = �i

� i /� I, ��(i) = 0

x�i = xi + �(i)



•Spectral Flattening




Mesh Flattening

19

min�1⇥Rn

||G�1||2 =⇥

i�j

wi,j |�1(i) � �1(j)|2 with

�||�1|| = 1,

⇥�1, 1⇤ = 0.

min�2⇥Rn

||G�2||2 =⇧

i�j

wi,j |�2(i) � �2(j)|2 with

�⌅⇤

⌅⇥

||�2|| = 1,

⇥�2, �1⇤ = 0,

⇥�2, 1⇤ = 0.

No boundary condition, minimize:

Mesh Flattening

19

min�1⇥Rn

||G�1||2 =⇥

i�j

wi,j |�1(i) � �1(j)|2 with

�||�1|| = 1,

⇥�1, 1⇤ = 0.

min�2⇥Rn

||G�2||2 =⇧

i�j

wi,j |�2(i) � �2(j)|2 with

�⌅⇤

⌅⇥

||�2|| = 1,

⇥�2, �1⇤ = 0,

⇥�2, 1⇤ = 0.

(�1(i),�2(i)) � R2

Theorem: ⇥i = �i+1L⇥i,where �0 = 0 � �1 � �2 . . .�n�1 are eigenvalues of L = G⇥G.

No boundary condition, minimize:

conformalcombinatorial

Proof

20

L = G�G = D �W = U�U�Spectral decomposition:

� = diag(�i) where 0 = �1 < �2 � . . . � �n

U = (ui)ni=1 orthonormal basis of Rn. u1 = 1

Proof

20


� = diag(�i) where 0 = �1 < �2 � . . . � �n

U = (ui)ni=1 orthonormal basis of Rn.

If ��, 1� = 0, then

u1 = 1

E(⇥) = ||G⇥||2 =n�

i=1

�i|�⇥, ui⇥|2

E(⇥) =n�

i=2

�iai where ai = |�⇥, ui⇥|2

Proof

20


� = diag(�i) where 0 = �1 < �2 � . . . � �n

U = (ui)ni=1 orthonormal basis of Rn.

If ��, 1� = 0, then

u1 = 1

E(⇥) = ||G⇥||2 =n�

i=1

�i|�⇥, ui⇥|2

Constrained minimization:

� linear program: minimum reached at a = �i.

minPni=2 ai=1

n�

i=2

�iai

±u2 = argmin��, 1⇥=0,||�||

E(�)

E(⇥) =n�

i=2

�iai where ai = |�⇥, ui⇥|2

Flattening Examples

21

Main issue: No guarantee of being valid (bijective).

conformalcombinatorial




•Barycentric Coordinates for Warping



Barycentric Coordinates

23

x1

x2

x3x

�i(x) =A(x, xi+1, xi+2)

A(x1, x2, x3)


23

x1

x2

x3

x1

x3

x2x x

Barycentric coordinates: {�i(x)}i�I

Positivity: �i(x) � 0.

Interpolation:

Reproduction of a�ne functions:

�i(x) =A(x, xi+1, xi+2)

A(x1, x2, x3)

Normalized: �i(x) =�i(x)�j �j(x)

�

i�I

�i(x)xi = x

⇥i(xj) = �i,j


23

x1

x2

x3

x1

x3

x2x x



Interpolation:


Application: interpolation of data {fi}i�I

�i(x) =A(x, xi+1, xi+2)

A(x1, x2, x3)


�

i�I

�i(x)xi = x

f(x) =�

i�I

�i(x)fi

⇥i(xj) = �i,j


23

x1

x2

x3

x1

x3

x2x x



Interpolation:


Application: interpolation of data {fi}i�I

Application: mesh parameterization:

xi

xj�i(x) =

A(x, xi+1, xi+2)A(x1, x2, x3)


�

i�I

�i(x)xi = x

f(x) =�

i�I

�i(x)fi

wi,j = �i(xj)

⇥i(xj) = �i,j

Mean Value Coordinates

24

x

xi+1

xi

xi

�i

�i�i

�i

Conformal Laplacian weights:

⇥i(x) = cotan(�i(x)) + cotan(�i(x))

Mean-value coordinates:

�� not necessarily positive.

�� valid coordinates.�� extend to non-convex coordinates (oriented angles).

⇥i(x) =tan(�i(x)/2) + tan(�i(x)/2)

||x� xi||

�1(x) �2(x)

Barycentric Coordinates for Warping

25

Example: textured grid, 3D model, etc.Data points: {yj}j�J � C.

Cage C: polygon with vertices {xi}i�I .

Initialization: data anchoring, compute� j � J, � i � I, �i,j = �i(yj).

Data warping: yj � y�j =�

i�I

�i,jx�i

Satisfies: yi =�

i�I

�i,jxi

Cage warping: xi � x�i

xix�iyi

y�i

Harmonic Coordinates

26

Mean value Harmonic

Mean value coordinates:

“non-physical” behavior,

passes “through” the cage.

⇥x � C, ��i(x) = 0.

⇥x � ⇥C, �i(x) = �0i (x).

Harmonic mapping:

Boundary conditions:

Warping Comparison

27

Mean value HarmonicInitial shape





•Geodesic Flattening


Geodesic Distances

29

Length of a curve �(t) �M: L(�) def.=� 1

0W (�(t))||��(t)||dt.

Geodesic distance: dM(x, y) = min�(0)=x,�(1)=y

L(�)

Euclidean Shape Isotropic W �= 1 Surface

dM(x, y) = L(��)Geodesic curve ��:

Computation of Geodesic Distances

30

Non-linear PDE:

Distance map to a point: Ux0(x) = dM(x0, x).Ux0

x0

�||�Ux0(x)|| = W (x)Ux0(x0) = 0,(viscosity)

Upwind finite di�erences approximation.Fast Marching: front propagation in O(N log(N)) operations.

Manifold Flattening

31

Input manifoldM, dM geodesic distance onM.

Input geodesic distance matrix: D = (dM(xi, xj)2)i,j , for xi �M.

x1x1

x2x2

M R3

M R2

x1

x2

x1

x2

Flattening: find X = (xi)pi=1 ⇤ Rn�p such that ||xi � xj || ⇥ dM(xi, xj).

Surface parameterization Bending invariant

Stress Minimization

32

Geodesic stress: di,j = dM(xi, xj)S(X) =�

i,j

|||xi � xj ||� di,j |2,

SMACOF algorithm: X(�+1) =1N

X(�)B(X(�))

where B(X)i,j =di,j

||xi � xj ||

�

Non-convex functional : X(�) � X� local minimizer of S.

Projection on Distance Matrices

33

D(X)i,j = ||xi � xj ||2Di,j = d2

i,j

||xi � xj ||2 = ||xi||2 + ||xj ||2 � 2⇥xi, xj⇤

minX1=0

�

i,j

|||xi � xj ||2 � d2i,j |2 = ||D(X)�D||2,

=⇥ D(X) = d1T + 1d� 2XTX where d = (||xi||2)i ⇤ Rn


33

D(X)i,j = ||xi � xj ||2Di,j = d2

i,j

||xi � xj ||2 = ||xi||2 + ||xj ||2 � 2⇥xi, xj⇤

minX1=0

�

i,j

|||xi � xj ||2 � d2i,j |2 = ||D(X)�D||2,


For centered points:

Centering matrix: J = Idn � 11T /N

�12JD(X)J = XTX

JX = X

J1 = 0


33

D(X)i,j = ||xi � xj ||2Di,j = d2

i,j

||xi � xj ||2 = ||xi||2 + ||xj ||2 � 2⇥xi, xj⇤

minX1=0

�

i,j

|||xi � xj ||2 � d2i,j |2 = ||D(X)�D||2,


For centered points:

Centering matrix: J = Idn � 11T /N

�12JD(X)J = XTX

JX = X

J1 = 0Replace ||D(X)�D|| by

Explicit solution: diagonalize � 12JDJ = U�UT �i � �i�1

�k = diag(�0, . . . ,�k�1),Uk = (u0, . . . , uk�1)

T,X� =

��kUk

minX

||� J(D(X)�D)J/2|| = ||XTX + JDJ/2||

Isomap vs. Laplacian

34

Flattening: f = (f1, f2)� R2.

Isomap: global constraints: ||f(x)� f(y)|| ⇥ dM(x, y).�⇥ (f1, f2) eigenvectors (#2,#3) of L = GTG.

Laplacian: local smoothness: fi = argmin ||Gf || subj. to ||f || = 1.

Mesh Lapl. combin. Lapl. conformal Isomap

Bije

ctiv

eN

otbi

ject

ive

�⇥ (f1, f2) eigenvectors (#1,#2) of �J(dM(xi, xj)2)ijJ .

Bending Invariants of Surfaces

35

Bending invariants: [Elad, Kimmel, 2003].

SurfaceM, Isomap dimension reduction: x ⇥M ⇤� IM(x) ⇥ R3.

[Elad, Kimmel, 2003].

M

IM


35



Geodesic isometry � :M�M�: dM(x, y) = dM�(�(x),�(y)).

Theorem: up to rigid motion, IM is invariant to geodesic isometries:IM(x) = v + UIM�(�(x)) where U � O(3) and v � R3.

[Elad, Kimmel, 2003].

M

IM


35



Geodesic isometry � :M�M�: dM(x, y) = dM�(�(x),�(y)).

Theorem: up to rigid motion, IM is invariant to geodesic isometries:IM(x) = v + UIM�(�(x)) where U � O(3) and v � R3.

[Elad, Kimmel, 2003]. [Bronstein et al., 2005].

M

IM

Bending Invariants

36IM

M

Face Recognition

37

Rigid similarity Non-rigid similarity Alex

A. M. Bronstein et al., IJCV, 2005






•High Dimensional Data Analysis38

High Dimensional Data Sets

39

Graph and Geodesics

40

Isomap Dimension Reduction

41

Isomap vs PCA Flattening

42

Laplacian Spectral Dimension Reduction

43

Parameterization of Image Datasets

44

Library of Images

45

When Does it Works?

46

Local patches in images

47

mesh processing course : mesh parameterization

Education

j ewhere i

cotij j

f x dx i

j e wherewij

j e wijw

cotangent weights i

j etheorem

j eexamples