a deterministic approach to the least squares mean on ...constantin.vernicos.org › articles ›...

A deterministic approach to the least squaresmean on Hadamard spaces

Yongdo Lim

Kyungpook National University

Overview

There has recently been considerably interest in defining “means”(averaging, barycenters, centroids) on manifolds/ metric spaces.A natural and attractive candidate of an averaging procedure on aCartan-Hadamard Riemannian manifold is the least squares mean.This mean has appeared under a variety of other designations:center of gravity, Frechet mean, Cartan mean, Riemannian centerof mass, Riemannian geometric mean, or frequently, Karcher mean,the terminology we adopt. It plays a central role in imageprocessing (subdivision schemes), medical imaging (DT-MRI),radar systems, and biology (DNA/Genomes), to cite a few.

Our purposes in this talk include the following for Hadamardspaces:

(1) A deterministic (“no dice”) approach to the Karcher mean

(2) A limit theorem for contractive means

(3) A no dice conjecture and some geometry of contractive means

Contractive means

Let (M, δ) be a complete metric space and let G : Mn → M.

(1) G is a mean if G(a, . . . ,a) = a.

(2) G is symmetric if G(aσ(1), . . . ,aσ(n)) = G(a1, . . . ,an).

(3) G is contractive ifδ(G(a1, . . . ,an),G(b1, . . . ,bn)) 6 1

∑ni=1 δ(ai,bi).

For contractive n-means G and H, define

δ(G,H) = supa∈Mn

∆(a)6=0

δ(G(a),H(a))

∆(a),

where ∆(a1, . . . ,an) = max16i,j6n δ(ai,aj).

• The set of all contractive n-means is a metric space withDiam 6 1. Question: Which metric spaces support contractiven-means (for some or all n > 2) ?

Contractive means

Let (M, δ) be a complete metric space and let G : Mn → M.

(1) G is a mean if G(a, . . . ,a) = a.

(2) G is symmetric if G(aσ(1), . . . ,aσ(n)) = G(a1, . . . ,an).

(3) G is contractive ifδ(G(a1, . . . ,an),G(b1, . . . ,bn)) 6 1

∑ni=1 δ(ai,bi).

For contractive n-means G and H, define

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a),

where ∆(a1, . . . ,an) = max16i,j6n δ(ai,aj).

• The set of all contractive n-means is a metric space withDiam 6 1. Question: Which metric spaces support contractiven-means (for some or all n > 2) ?

Least Squares Mean

The Karcher mean of a1, . . . ,an ∈ M, is defined as the uniqueminimizer (provided it exists) of the optimization problem

Λ(a1, . . . ,an) = argminx∈M

n∑i=1

δ2(x,ai).

This idea had been anticipated by Elie Cartan, who showed amongother things that such a unique minimizer exists if the points all liein a convex ball in a Riemannian manifold. Alternatively, it arisesas the unique solution of the Karcher equation:

n∑i=1

logx ai = 0.

• Recent research includes: Numerical methods for itsapproximation, e.g., Newton and gradient descent methods(exhibit local convergence, heavy dependence on initial points andstep length); detailed investigation for the case of positive definitematrices. Question: What about its contractiveness?

Means and S.L.L.N.

For {1, 2, . . . ,n} equipped with a probability measure,Pn := {1, 2, . . . ,n}N is a probability space w.r.t. the productmeasure, e.g.,

w∗ : N → {1, 2, 3}, ω∗ : 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, . . . , .

Let µ = {Gk}∞k=1 be a sequence of means Gk : Mk → M. TheStrong Law of Large Numbers for µ can be stated as follows: forany n-tuple a = (a1, . . . ,an) ∈ Mn,

∃ limk→∞ Gk(aω(1), . . . ,aω(k)), a.e. ω ∈ Pn.

• Define µ∗(a1, . . . ,an) to be the “common” limit.• Find such a µ = {Gk}∞k=1 and describe the mean µ∗.• “No Dice” Conjecture:

limk→∞ Gk(aω∗(1), . . . ,aω∗(k)) = µ∗(a1, . . . ,an).

Means and S.L.L.N.

For {1, 2, . . . ,n} equipped with a probability measure,Pn := {1, 2, . . . ,n}N is a probability space w.r.t. the productmeasure, e.g.,

w∗ : N → {1, 2, 3}, ω∗ : 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, . . . , .

Let µ = {Gk}∞k=1 be a sequence of means Gk : Mk → M. TheStrong Law of Large Numbers for µ can be stated as follows: forany n-tuple a = (a1, . . . ,an) ∈ Mn,

∃ limk→∞ Gk(aω(1), . . . ,aω(k)), a.e. ω ∈ Pn.

• Define µ∗(a1, . . . ,an) to be the “common” limit.• Find such a µ = {Gk}∞k=1 and describe the mean µ∗.• “No Dice” Conjecture:

limk→∞ Gk(aω∗(1), . . . ,aω∗(k)) = µ∗(a1, . . . ,an).

Contractive means on Busemann spaces

There are several types of contractive means on Busemann NPCspaces. A metric space with midpoint selector is a triple (M, δ,#)

such that (M, δ) is a metric space and (x,y) 7→ x#y is a binaryoperation that assigns to each point a metric midpoint in such away that x#y = y#x. Menger proved that a complete metricspace with midpoint selector is a geodesic metric space; for alldistinct x,y ∈ M, there exists a minimal geodesic γx,y; [0, 1] → M

from x to y. Denote

x#ty := γx,y(t), (x#y = x# 12y)

the t-weighted geometric mean of a and b.

DefinitionA complete metric space with midpoint selector (M, δ,#) is saidto satisfy the Busemann NPC-inequality, if for all a,b, c,d

δ(a#b, c#d) 61

2[δ(a, c) + δ(b,d)].

LemmaLet (M, δ,#) be a complete metric space with midpoint selectorsatisfying the Busemann NPC-inequality. Then for 0 6 t 6 1,

δ(a#tb, c#td) 6 (1 − t)δ(a, c) + tδ(b,d).

• Hadamard spaces. • Symmetric cones equipped with Thompson

metric: Domains of positivity, Jordan-Banach algebras (e.g. theconvex cone of positive definite operators on a Hilbert space,forward light cones)

• Finite dim. symmetric cones: self-dual homogeneous cones,Euclidean Jordan algebras, having rich (Finsler) geometricstructures inherited from symmetric gauge norms (equivalently,unitarily invariant norms; Schatten p-norms and spectral norm).

Inductive mean and Birkhoff shortening

Let (M, δ,#) be a complete metric space with midpoint selectorsatisfying the Busemann NPC-inequality.The inductive mean (convex combin.) of a1, . . . ,an is defined as

S(a1) = a1, S(a1, . . . ,an) = S(a1, . . . ,an−1)# 1nan.

The Birkhoff shortening of a1, . . . ,an is the common limit of thethe following sequences

γ(a1, . . . ,an) = (a1#a2,a2#a3, . . . ,an#a1)

limk→∞ γk(a1, . . . ,an) = (x∗, x∗, . . . , x∗)

γ(a1, . . . ,an) = (a1#a2,a2#a3, . . . ,an#a1)

limk→∞ γk(a1, . . . ,an) = (x∗, x∗, . . . , x∗)

γ(a1, . . . ,an) = (a1#a2,a2#a3, . . . ,an#a1)

limk→∞ γk(a1, . . . ,an) = (x∗, x∗, . . . , x∗)

S(a1) = a1, S(a1, . . . ,an) = S(a1, . . . ,an−1)# 1nan.

The Birkhoff shortening of a1, . . . ,an is the common limit of thefollowing sequences

γ(a1, . . . ,an) = (a1#a2,a2#a3, . . . ,an#a1)

limk→∞ γk(a1, . . . ,an) = (x∗, x∗, . . . , x∗)

• Indeed, the common limit exists.

• Inductive mean and Birkhoff shortening are contractive, butnon-symmetric (can vary with permutations).

ALM and BMP Means

There are alternative symmetric and contractive means, namelyALM = {Almn}∞n=2 and BMP = {Bmpn}∞n=2 means, on a completemetric space with midpoint selector satisfying the BusemannNPC-inequality via “symmetrization procedures” and “induction”.

• See Ando-Li-Mathias for positive definite matrices, Es-Sahib andHeinich on locally compact Hadamard spaces, Bini-Meini-Polonifor positive definite matrices.

• Alm3 = the Birkhoff shortening.

• There are infinitely many symmetric and contractive means onon a complete metric space with midpoint selector satisfying theBusemann NPC-inequality.

ALM and BMP Means

Hadamard Space

A complete metric space (M, δ) is called a Hadamard space if itsatisfies the semiparallelogram law; for each x,y ∈ M, there existsan m ∈ M satisfying

δ2(m, z) 61

2δ2(x, z) +

2δ2(y, z) −

4δ2(x,y),∀z ∈ M.

• m is the unique metric midpoint between x and y.

• unique minimal geodesic γa,b; a#tb := γa,b(t).

• Hadamard space =⇒ Busemann NPC space

• Inductive, Birkhoff shortening, ALM and BMP means exist.

• The Karcher mean exists and unique:Λ(a1, . . . ,an) = argmin

∑ni=1 δ2(x,ai)

Examples of Hadamard Spaces

• Cartan-Hadamard Riemannian manifolds; e.g., the convexcone of positive definite matrices with trace metric

δ(A,B) = || log A−1/2BA−1/2||2.

In this case A#tB = A1/2(A−1/2BA−1/2)tA1/2, thet-weighted matrix geometric mean of A and B.

• (Infinite dim.) Lorentz cones

• Phylogenetic Trees (DNA sequences and Genomes)

• Booklets, metric trees,

• Spiders equipped with Taxicab metric

• Subsets, images, products and Gromov-Hausdorff limits ofHadamard spaces

The 3-spider

The 3-spider is defined as the set of 3-distinct half-rays in theplane with a common point at the origin.

Problem: Compute the Karcher, ALM and BMP ofAi = L(ti), i = 1, 2, 3.

Formulae

Let Ai = Li(ti), i = 1, 2, 3. We may assume by the “permutationinvariancy” that t1 > t2 > t3 > 0.

• [T. Sturm] Λ =

{0, if t1 6 t2 + t3,L1(

t1−t2−t33 ), if t1 > t2 + t3.

• Alm3 = L1

(t1−t2

• Bmp3 =

{0, if t1 = t2,

((2n+2−1)(t1−t2)−t3

), if t1 6= t2,

⌊log2

(t1 − t2 + t3

t1 − t2

The least squares mean, ALM and BMP means are indeed distinct.The ALM mean is computationally cumbersome and BMP meanexhibits more rapid convergence properties for positive definitematrices. Not true on the 3-spider.

Least Squares Mean

Let (M, δ) be a Hadamard space. The Karcher mean isΛ(a1, . . . ,an) = argmin

∑ni=1 δ2(x,ai).

• [T. Sturm] S.L.L.N holds for the inductive mean;

limk→∞ S(aω(1), . . . ,aω(k)) = Λ(a1, . . . ,an), a.e. ω ∈ Pn.

• [J. Holbrook] For positive definite matrices

limk→∞ S(Aω∗(1), . . . ,Aω∗(k)) = Λ(A1, . . . ,An).

The proof depends heavily on matrix analysis and the Riemanniantrace metric. Very slow in convergence. The “No Dice” conjectureremains open for general Hadamard spaces (no differentialstructures).[cf. Es-Sahib and Heinich’s S.L.L.N and no dice via ALM mean]

Least Squares Mean

Let (M, δ) be a Hadamard space. The Karcher mean isΛ(a1, . . . ,an) = argmin

∑ni=1 δ2(x,ai).

• [T. Sturm] S.L.L.N holds for the inductive mean;

limk→∞ S(aω(1), . . . ,aω(k)) = Λ(a1, . . . ,an), a.e. ω ∈ Pn.

• [J. Holbrook] For positive definite matrices

limk→∞ S(Aω∗(1), . . . ,Aω∗(k)) = Λ(A1, . . . ,An).

The proof depends heavily on matrix analysis and the Riemanniantrace metric. Very slow in convergence. The “No Dice” conjectureremains open for general Hadamard spaces (no differentialstructures).[cf. Es-Sahib and Heinich’s S.L.L.N and no dice via ALM mean]

Geometric power means

Let (M, δ) be a Hadamard space.

TheoremLet G : Mn → M be a contractive mean and let t ∈ (0, 1]. Then foreach (a1, . . . ,an) ∈ Mn, the following equation has a uniquesolution in M :

x = G(x#ta1, . . . , x#tan). (1)

Define by Gt(a1, . . . ,an) the unique solution of (1).

• G1 = G and each Gt is a contractive mean.

• The map x → G(x#ta1, . . . , x#tan) is a strict contraction withthe least contraction coefficient less than or equal to 1 − t.

• The preceding hold on any complete metric space with midpointselector satisfying the Busemann NPC-inequality.

DefinitionA map G : Mn → M is said to satisfy the extended metricinequality, EMI for short, if

δ2(x,G(a)) 61

n∑i=1

δ2(x,ai)

for all x ∈ M, a = (a1, . . . ,an) ∈ Mn. We denote En by the set ofall contractive n-means satisfying EMI.

• E2 = {#}.

• S, Almn, Bmpn and Birkhoff shortening belong to En.

• Gt ∈ En for all t ∈ (0, 1] if G ∈ En.

• G#tH ∈ En if G,H ∈ En, where (G#tH)(a) = G(a)#tH(a).

• [Gt]s = Gst, that is, {Gt}t∈(0,1] forms a one-parametersemigroup.

δ2(x,G(a)) 61

n∑i=1

δ2(x,ai)

• E2 = {#}.

δ2(x,G(a)) 61

n∑i=1

δ2(x,ai)

• E2 = {#}.

δ2(x,G(a)) 61

n∑i=1

δ2(x,ai)

• E2 = {#}.

δ2(x,G(a)) 61

n∑i=1

δ2(x,ai)

• E2 = {#}.

δ2(x,G(a)) 61

n∑i=1

δ2(x,ai)

• E2 = {#}.

A limit theorem

TheoremLet G ∈ En. Then

δ(Gt(a),Gs(a)) 6

√s + t

2∆(a), ∀a ∈ Mn.

Furthermore, limt→0+ Gt(a) = Λ(a).

Key idea: δ2(x,G(a)) 6 1n

∑ni=1 δ2(x,ai) implies that

δ2(x,Gt(a)) 61

n∑i=1

δ2(x,ai) −1 − t

n∑i=1

δ2(Gt(a),ai).

As t → 0+,

n∑i=1

δ2(G0(a),ai) 6n∑

δ2(x,ai) − nδ2(x,G0(a)).

A limit theorem

δ(Gt(a),Gs(a)) 6

√s + t

2∆(a), ∀a ∈ Mn.

δ2(x,Gt(a)) 61

n∑i=1

δ2(x,ai) −1 − t

n∑i=1

δ2(Gt(a),ai).

As t → 0+,

n∑i=1

δ2(G0(a),ai) 6n∑

δ2(x,ai) − nδ2(x,G0(a)).

A limit theorem

δ(Gt(a),Gs(a)) 6

√s + t

2∆(a), ∀a ∈ Mn.

δ2(x,Gt(a)) 61

n∑i=1

δ2(x,ai) −1 − t

n∑i=1

δ2(Gt(a),ai).

As t → 0+,

n∑i=1

δ2(G0(a),ai) 6n∑

δ2(x,ai) − nδ2(x,G0(a)).

A limit theorem

δ(Gt(a),Gs(a)) 6

√s + t

2∆(a), ∀a ∈ Mn.

• A deterministic approach; limt→0+ St(a) = Λ(a). Almost allproperties of the “inductive mean” are preserved in the limit.

• Λ ∈ En.

• Every closed ball (convex set) is stable for the Karcher mean.

• Λt = Λ for all t ∈ (0, 1].

• Gt = G for some t ∈ (0, 1) if and and only if G = Λ.

• More effective computationally than the “no dice”/ approach as∆(a) → 0.

A limit theorem

δ(Gt(a),Gs(a)) 6

√s + t

2∆(a), ∀a ∈ Mn.

• Λ ∈ En.

• Λt = Λ for all t ∈ (0, 1].

A limit theorem

δ(Gt(a),Gs(a)) 6

√s + t

2∆(a), ∀a ∈ Mn.

• Λ ∈ En.

• Λt = Λ for all t ∈ (0, 1].

A limit theorem

δ(Gt(a),Gs(a)) 6

√s + t

2∆(a), ∀a ∈ Mn.

• Λ ∈ En.

• Λt = Λ for all t ∈ (0, 1].

A limit theorem

δ(Gt(a),Gs(a)) 6

√s + t

2∆(a), ∀a ∈ Mn.

• Λ ∈ En.

• Λt = Λ for all t ∈ (0, 1].

A limit theorem

δ(Gt(a),Gs(a)) 6

√s + t

2∆(a), ∀a ∈ Mn.

• Λ ∈ En.

• Λt = Λ for all t ∈ (0, 1].

Metric structures of En

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

• (En, δ) is a metric space with Diam(En) 6 1.For any G,H,G ′,H ′ ∈ En,

• δ(Gt,Ht) 6√

• δ(Gt,Λ) 6√

t2 . In particular, δ(G,Λ) 6 1√

• Upper bounds are independent on n.

• δ(G#sH,G#tH) = |s − t|δ(G,H).

• δ(G#tH,G ′#tH′) 6 (1 − t)δ(G,G ′) + tδ(H,H ′).

• Completeness of (En, δ)? True for any bounded Hadamardspaces.

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

Define ln : En → [0, 1√2], ln(G) = δ(G,Λ).

• Find an explicit formula for ln(G); e.g., ln(S).

• On the 3-spider, l3(St) = t2+t , l3(S) = l3(Alm) = 1

For A = L1(a),B = L2(b),C = L3(c) with a > b > c,

St(A,B,C) =

(a−b−c

)= Λ(A,B,C), if a > b + c,

(b+c−a

2+t t), if a < b + c.

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

St(A,B,C) =

(a−b−c

)= Λ(A,B,C), if a > b + c,

(b+c−a

2+t t), if a < b + c.

δ(G,H) = supa∈Mn

∆(a) 6=0

δ(G(a),H(a))

∆(a).

St(A,B,C) =

(a−b−c

)= Λ(A,B,C), if a > b + c,

(b+c−a

2+t t), if a < b + c.

Monotonicity

The monotonicity of the Karcher mean of positive definite matricesequipped with the Lowner partial order A 6 B if B − A is positivesemidefinite,

Λ(A1, . . . ,An) 6 Λ(B1, . . . ,Bn) if Ai 6 Bi,∀i,was conjectured by Bhatia and Holbrook and is one of keyaxiomatic properties of matrix geometric means. It was recentlyestablished by Lawson and L. via S.L.L.N and by Bhatia andKarandikar via some probabilistic counting arguments, botharguments depending heavily on basic inequalities for theRiemannian metric. Also by Holbrook via a “No dice” approachand by L. and Palfia by arithmetic power mean approach.Our approach to the Karcher mean yields a simple, structured, anddeterministic proof of the monotonicity extending to all finite dim.symmetric cones and infinite dim. Lorentz cones; the inductivemean S and its geometric power mean St are monotonic and hencethe Karcher mean, the limit of St, also is.

Arithmetic power means

Let A1, . . . ,An be m×m positive definite matrices. For t ∈ (0, 1],the following equation has a unique positive definite solution:

n∑i=1

(X#tAi). (2)

Define by Pt(A1, . . . ,An) the unique solution of (2). Then

limt→0+

Pt(A1, . . . ,An) = Λ(A1, . . . ,An).

• X → 1n

∑ni=1 X#tAi is a strict contraction for the “Thompson

metric” with least contraction coefficient 6 1− t. For positive reals,

Pt(a1, . . . ,an) =[

∑ni=1 at

] 1t , the usual power mean.

• [The Karcher equation]

n∑i=1

log(X1/2A−1i X1/2) = 0.

Further work

Extend the previous work to the case of Infinite dimensionalsymmetric cones equipped with Thompson metric (Finsler metric).There are infinitely many midpoints between points, but anysymmetric cone is a complete metric space with midpoint selector(the geometric mean defined via the Riccati Lemma) satisfying theBusemann NPC-inequality with respect to the Thompson metric.Lorentz cones (second order cones) are Cartan-HadamardRiemannian manifolds.

• Karcher means and Karcher equations on symmetric cones (cf.Lawson and L., for the special case of positive definite operators ona Hilbert space).

• Establish limit theorems for geometric and arithmetic powermeans on symmetric cones.

• Extend S.L.L.N and “no dice” conjecture for contractive means(e.g., inductive, Birkhoff shortening, ALM and BMP) to symmetriccones.

References-matrices

[1] T. Ando, C.-K. Li and R. Mathias, Geometric means, LAA(2004).

[2] M. Moakher, A differential geometric approach to thegeometric mean of symmetric positive-definite matrices, SIAM J.Matrix Anal. Appl. (2005).

[3] R. Bhatia and J. Holbrook, Riemannian geometry and matrixgeometric means, LAA (2006).

[4] D. Bini, B. Meini and F. Poloni, An effective matrix geometricmean satisfying the Ando-Li-Mathias properties, Math. Comp.(2010).

[5] D. Bini and B. Iannazzo, Computing the Karcher mean ofsymmetric positive definite matrices, LAA (2012).

References-monotonicity

[6] J. Lawson and Y. Lim, Monotonic properties of the leastsquares mean, Math. Ann. (2011).

[7] R. Bhatia and R. Karandikar, Monotonicity of the matrixgeometric mean, Math. Ann. (2012).

[8] J. Holbrook, No dice: a determinic approach to the Cartancentroid, J. Ramanujan Math. Soc. (2012)

[9] Y. Lim and M. Palfia, The matrix power means and theKarcher mean, J. Functional Analysis (2012).

References-metric spaces

[10] H. Karcher, Riemannian center of mass and mollifiersmoothing, Comm. Pure Appl. Math. (1977).

[11] A. Es-Sahib and H. Heinich, Barycentre canonique pour unespace metrique a courbure negative, Seminaire de probabilites(Strasbourg) (1999).

[12] K.-T. Sturm, Probability measures on metric spaces ofnonpositive curvature, Contemp. Math. (2003).

[13] J. Lawson and Y. Lim, A general framework for extendingmeans to higher orders, Colloq. Math. (2008).

[14] J. Lawson, H. Lee and Y. Lim, Weighted geometric means,Forum Math. (2012).

[15] Y. Lim and M. Palfia, A deterministic approach for theKarcher mean on Hadamard spaces, submitted.

[16] Y. Lim, Y. Cho et al, Means on the 3-spider, in preparation.

a deterministic approach to the least squares mean on ...constantin.vernicos.org › articles ›...

Documents