geometry and expansion: a survey of recent results
DESCRIPTION
Geometry and Expansion: A survey of recent results. Sanjeev Arora Princeton. ( touches upon: S. A., Satish Rao, Umesh Vazirani, STOC’04; S. A., Elad Hazan, and Satyen Kale, FOCS’04; S. A., James Lee, and Assaf Naor, STOC’05 + papers that are not mine). | E(S, S c )|. - PowerPoint PPT PresentationTRANSCRIPT
Geometry and Expansion: A survey of recent results
Sanjeev Arora Princeton
( touches upon:
S. A., Satish Rao, Umesh Vazirani, STOC’04; S. A., Elad Hazan, and Satyen Kale, FOCS’04; S. A., James Lee, and Assaf Naor, STOC’05
+ papers that are not mine)
Sparsest Cut / Edge Expansion
SS
G = (V, E)
c- balanced separator
Both NP-hard
(G) = minS µ V
| E(S, Sc)|
|S||S| < |V|/2
c(G) = minS µ V
| E(S, Sc)|
|S|c |V| < |S| < |V|/2
Why these problems are important
• Analysis of random walks, PRAM simulation, packet routing, clustering, VLSI layout etc.
• Underlie many divide-and-conquer graph algorithms (surveyed by Shmoys’95)
• Discrete analog of isoperimetry; useful in Riemannian geometry (via 2nd eigenvalue of Laplacian (Cheeger’70)
• Graph-theoretic parameters of inherent interest (cf. Lipton-Tarjan planar separator theorem)
The three main characters
Expansion Isoperimetry
(continuous analog of expansion)
Geometry
(and geometric embeddings of finite metric spaces)
Outcome: New plog n –approximations for various NP-hard problems; Derived using geometric insights, & which led to new geometry thms.
Previous approximation algorithms for expansion problems
1) Eigenvalue approaches (Cheeger’70, Alon’85, Alon-Milman’85)
Only yield factor n approximation. 2c(G) ¸ (G) ¸ c(G)2 /2
2) O(log n) -approximation via LP (multicommodity flows) (Leighton-Rao’88) • Approximate max-flow mincut theorems
• Region-growing argument
(Linial, London, Rabinovich’94, AR’94)
3) Embeddings of finite metric spaces into l1
• Geometric approach; more general result (but still O(log n) approximation)
New results of [ARV’04]
1. O( ) -approximation to sparsest cut and conductance
2. O( )-pseudoapproximation to c-balanced separator (algorithm outputs a c’-balanced separator, c’ < c)
3. Existence of expander flows in every graph (approximate certificates of expansion)
log n
log n
Disparate approaches from previous slide get “unified”
Outline:• Graph partitioning problems: intro and history
• New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani]
• Uses of “S. T.” in geometric embeddings
• Introduction to expander flows and O(n2) time algorithms
• Outline of proof of “S. T.”
• Open problems
Next: Semidefinite relaxations for c-balanced separator
(and how to round the solution)
c-balanced separator
c(G) = minS µ V
| E(S, Sc)|
|S|c |V| < |S| < |V|/2
SS
Assign {+1, -1} to v1, v2, …, vn to minimize
(i, j) 2 E |vi –vj|2/4
Subject to i < j |vi –vj|2/4 ¸ c(1-c)n2
+1
-1
|vi –vj|2/4 =1
Semidefinite relaxation for
Find unit vectors in <n
|vi –vj|2 + |vj –vk|2 ¸ |vi –vk|2 8 i, j, k
Triangle inequality
“cut semimetri
c”
|vi –vj|2 =0
Unit l22 space
Unit vectors v1, v2,… vn 2 <d
|vi –vj|2 + |vj –vk|2 ¸ |vi –vk|2 8 i, j, k
Vi
Vk
Vj
non obtuse !
Example: Hypercube {-1, 1}k
|u – v|2 = i |ui – vi|2 = 2 i |ui – vi| = 2 |u – v|1
In fact, l2 and l1 are subcases of l22
Structure Theorem for l22 spaces [ARV’04]
Subsets S and T are -separated if for every vi 2 S, vj 2 T |vi –vj|2 ¸
¸
Thm: If i< j |vi –vj|2 = (n2) then 9 S, T of size (n) that are -separated for = ( 1 )
<d
log n
G = Graph in which (i,j) is an edge iff |vi –vj|2 ·
Equiv: G is an “expander” iff · 1
p log n
Main thm ) O( )-approximationlog n
v1, v2,…, vn 2 <d is optimum SDP soln; SDPopt = (i, j) 2 E |vi –vj|2
S, T : –separated sets of size (n)
Do BFS from S until you hit T. Take the level of the BFS tree with the fewest edges and output the cut (R, Rc) defined by this level
(i, j) 2 E |vi –vj|2 ¸ |E(R, Rc)| £
) |E(R, Rc)| · SDPopt /
· O( SDPopt) log n
Other new -approximation algorithms
• MIN-2-CNF deletion and several graph deletion problems. [Agarwal, Charikar, Makarychev, Makarychev’05]
• MIN-LINEAR ARRANGEMENT [Charikar, Karloff, Rao’05]
• General SPARSEST CUT [A., Lee, Naor ’05]
• Min-ratio VERTEX SEPARATORS and Balanced VERTEX SEPARATORS [ Feige, Hajiaghayi, Lee, ’05]
log n
All use the Structure Theorem (+ other ideas)
Example: Structure Theorem (Agarwal, Charikar, Makarychev2 ‘05)
d : directed version of l22 metric;w: weight function on the nodesG = (V, E): any graph on the nodes.
S
There exists a subset S that contains 1/10 of the total weight and such that e leaves S d(e) is at Most p log n £ e 2 E d(e).
(Useful in rounding SDP for MIN-2CNF-DELETION.)
Outline:
• Graph partitioning problems: intro and history
• New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani]
• Geometric embeddings of metric spaces
• Introduction to expander flows and O(n2) time algorithms
• Outline of proof of “S. T.”
• Open problems
Finite metric space (X, d)
x
y
d(x,y)
<k (with l2 norm)
f
distortion of f is minimum C>1 such that
d( x, y) · |f(x ) – f( y)|2 · C d( x, y) 8 x, y
Thm (Bourgain’85): For every n-point metric space, a map exists with distortion O(log n)
[LLR’94]: Efficient algorithm to find the map; Proof that O(log n) cannot be improved in general
Qs: Improve O(log n) for X = l22 (say) or l1 ?
f(x)
f(y)
Embeddings and Cuts (LLR’94, AR’94)
Recall: Cut semi-metric
1
0
Fact: Metric (X, d) embeds isometricallyin l1 iff it can be written as a positive combination of cut semimetrics
Embedding l22 into l1 gives a way to produce cuts from SDP solution
Status report of this area
l1 into l2log0.5 n[Enflo’69]
l22 into l1 1.16 [Zatloukal’04]
Superconstant[Khot, Vishnoi’04]
l22 into l2log0.5 n[Enflo’69]
Best lowerbound Best upperbound
Exactly the integrality gap of SDP for general SPARSEST CUT[LLR’94, AR’94]
log n[Bourgain’85]
log0.75 n
[Chawla,Gupta,Racke ’04]
log0.5 n log log n[A., Lee, Naor’04]
Uses fourier techniques developed for PCPs!
Disproves Goemans-Linial
conjecture
Note: l2 µ l1 µ l22
Embedding Upperbounds:Frechet’s recipe to embed metric space (X, d) into Rk
Pick k suitable subsets A1, A2, …, Ak of X
Map x 2 X to (d(x, A1), d(x, A2), … , d(x, Ak))Aix
In recent embeddings, Ai’s are chosen using S.T.and “Measured descent” idea of [Krauthgamer, Lee, Naor, and Mendel’04]
Note: d(x, A1) – d(y, A1) · d(x, y)
Why S.T. useful: If S obtained from S.T., then in the mapping x ! d(x, S), “many” x’s (namely, all those in T) map far from 0.
Embedding lowerbounds (Khot-Vishnoi’05)
Explicit unit- l22 space (X, d) that requires distortion log log log n into l1
Main observation: Need good handle on cut structure of X
Use hypercube as building block !
Cut ´ Boolean Function
Number of cut edges = average sensitivity
(Fourier analysis a la KKL, Friedgut, Hastad, Bourgain etc. ) isoperimetric theorems)
[Khot-Naor]: Lowerbounds for embedding earth-mover & edit metrics into l1
Outline:
• Graph partitioning problems: intro and history
• New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani]
• Outline of proof of “S. T.”
• Uses of “S. T.” in geometric embeddings
• Introduction to expander flows and O(n2) time algorithms
• Open problems
Expander flows: Motivation
G = (V, E)
SS
Idea: Embed a D-regular (weighted) graph such that
8 S w(S, Sc) = (D |S|)
Cf. Jerrum-Sinclair, Leighton-Rao(embed a complete graph)
“Expander”
Weighted Graph w satisfies (*) iff L(w) = (1) [Cheeger]
(*)
Our Thm: If G has expansion , then a D-regular expander flow exists in it where D=
log n
(certifies expansion = (D) )
Can be found in O(n2) time (A., Hazan, Kale ’04)
Example of expander flow
n-cycle
Take any 3-regular expander on n nodesPut a weight of 1/3n on each edge
Embed this into the n-cycle
Routing of edges does not exceed any capacity ) expansion =(1/n)
Outline:
• Graph partitioning problems: intro and history
• New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani]
• Uses of “S. T.” in geometric embeddings
• Introduction to expander flows and O(n2) time algorithms
•Outline of proof of “S. T.”
• Open problemsS
T (Algorithm to produce -separatedsets S, T, of size (n) )
Outline of proof of S. T.
Algorithm to produce two –separated sets
<d
u
Su
Tu
0.01
d
Easy: Su and Tu likely to have size (n)
If Su, Tu still have size (n), output them
Main difficulty: Show that whp only o(n) points get deleted
d
“Stretched pair”: vi, vj such that |vi –vj|2 · and | h vi –vj, u i | ¸ 0.01
Obs: Deleted pairs are stretched and they form a matching.
Delete any vi 2 Su, vj 2 Tu s.t. |vi –vj|2 < . (till no such pair remains)
Naïve analysis of random projection fails
<dv
u
<u, v> ??1
d
1
d
e-t
2/2
Stretched pair: |vi –vj|2 < ; |<vi –vj, u>| > 0.01d
standard deviations
E[# of stretched pairs] = n2 exp(-) À n
= O( 1 )
Vi
Ball (vi , )u
Vj
0.01
d
Proof by contradiction: Suppose matching of (n) size exists with probability (1)…
….stretched pairs are almost everywhere you look!
Idea: Put stretched pairs together; derive very improbable event
Walks in unit l22 space
Unit vectors v1, v2,… vn 2 <d
|vi –vj|2 + |vj –vk|2 ¸ |vi –vk|2 8 i, j, k
Vi
Vk
Vj
Angles are non obtuse
Taking r steps of length s
only takes you squared distance rs2
(i.e. distance r s)
ss
ss
Proof by contradiction (contd.)
ss
ss r steps of length s )
squared distance rs2 (distance r s)
Stretched pair: |vi –vj|2 < ; |<vi –vj, u> ¸ 0.01
Claim: 9walk on stretched edges
u
d
0.01
d
0.01
d
….
<vfinal –v0, u> ¸ r 0.01
d|vfinal –v0| · r
Projection = r
£ standard deviation
VERY UNLIKELY IF r large enough) Walk impossible (CONTRADICTION)
How to produce walk: delicate argument; measure concentration
OPEN PROBLEMS
nlog• Better approximation factor than O( )? (For general SPARSEST CUT, log log n “lowerbound” )
• Better distortion bound for embedding l22 into l1? ( upperbound v/s loglog n lowerbound.)
• Remove need for solving SDPs (i.e., design combinatorial algorithms) (similar to one for SPARSEST CUT from [A., Hazan, Kale] )
• O(m) time algorithm for SPARSEST CUT instead of O(n2)? (not known even for Leighton-Rao’88 O(log n) approximation)
• Other applications of expander flows? (Useful in some geometric results [Naor, Rabani, Sinclair’04])
nlog
Looking forward to more progress…
Thanks !
New Result (A., Hazan, Kale;FOCS’04)
O(n2) time algorithm that given any graph G finds for some D >0
• a D-regular expander flow • a cut of expansion O( D )log n
Ingredients: Approximate eigenvalue computations; Approximate flow computations (Garg-Konemann; Fleischer) Random sampling (Benczur-Karger + some more)
Idea: Define a zero-sum game whose optimum solution is an expander flow; solve approximately using Freund-Schapire approximate solver.
)D) · (G) ·O(D )log n
Expander flows: LP view
LP feasible ) ¸ (D)LP feasible ) ¸ (D)
GG
· D· D
· 1· 1
Thm [ARV]: 9 0 s.t. the LP is
feasible with D = /√log n
Thm [ARV]: 9 0 s.t. the LP is
feasible with D = /√log n
Open problems (circa April’04)
• Better running time/combinatorial algorithm?
• Improve approximation ratio to O(1); better rounding??(our conjectures may be useful…)
• Extend result to other expansion-like problems (multicut, general sparsest cut; MIN-2CNF deletion)
• Resolve conjecture about embeddability of l22 into l1; of l1 into l2
• Any applications of expander flows?
O(n2) time; [A., Hazan, Kale]
log3/4 n distortion; [Chawla,Gupta, Racke]
Integrality gap is (log n) [Charikar]
Yes [Naor,Sinclair,Rabani]
Better embeddings of lp into lq [Lee]
Various new results
O(n2) time combinatorial algorithm for sparsest cut (does not use semidefinite programs)
[A., Hazan, Kale’04]
New results about embeddings: (i) lp into lq [J. Lee’04]
(ii) l22 and l1 into l2 [CGR’04]
(approx for general sparsest cut)
Clearer explanation of expander flows and their connection to embeddings [NRS’04]
Formal statement : 9 0 >0 s.t. foll. LP is feasible for d = (G)
log n
fp ¸ 0 8 paths p in G
8i j p 2 Pij fp = d (degree)
Pij = paths whose endpoints are i, j
8S µ V i 2 S j 2 Sc p 2 Pij fp ¸ 0 d |S| (demand graph is
an expander)
8e 2 E p 3 e fp · 1 (capacity)
A concrete conjecture (prove or refute)
G = (V, E); = (G)
For every distribution on n/3 –balanced cuts {zS} (i.e., S zS =1)
there exist (n) disjoint pairs (i1, j1), (i2, j2), ….. such that for each k,
• distance between ik, jk in G is O(1/ )
• ik, jk are across (1) fraction of cuts in {zS}
(i.e., S: i 2 S, j 2 Sc zS = (1) )
Conjecture ) existence of d-regular expander flows for d =
log n
log n
nlog
Example of l22 space: hypercube {-1, 1}k
|u – v|2 = i |ui – vi|2 = 2 i |ui – vi| = 2 |u – v|1
In fact, every l1 space is also l22
Conjecture (Goemans, Linial): Every l22 space is l1 up to distortion O(1)
LP Relaxations for c-balanced separator
Motivation: Every cut (S, Sc) defines a (semi) metric
1
1
1
0 0
Xij 2 {0,1}
i< j Xij ¸ c(1-c)n2
Xij + Xj k ¸ Xik
0 · Xij · 1
Semidefinite
There exist unit vectors v1, v2, …, vn 2 <n such that Xij = |vi - vj|2 /4
Min (i, j) 2 E Xij
Semidefinite relaxation (contd)
Min (i, j) 2 E |vi –vj|2/4
|vi|2 = 1
|vi –vj|2 + |vj –vk|2 ¸ |vi –vk|2 8 i, j, k
i < j |vi –vj|2 ¸ 4c(1-c)n2
Unit l22 space
Many other NP-hard problems have similar relaxations.
Algorithm to produce two –separated sets
<d
u
Su
Tu
0.01
d
Check if Su and Tu have size (n)
If any vi 2 Su and vj 2 Tu satisfy
|vi –vj|2 ·
repeat until no such vi, vj remain
delete them and
If Su, Tu still have size (n), output them
Main difficulty: Show that whp only o(n) points get deleted
d
“Stretched pair”: vi, vj such that |vi –vj|2 · and | h vi –vj, u i | ¸ 0.01
Obs: Deleted pairs are stretched and they form a matching.
Next 10-12 min: Proof-sketch of Structure Thm
( algorithm to produce -separated S, T of size (n); = 1/ )nlog
S
T
“Matching is of size o(n) whp” : naive argument fails
d
“Stretched pair”: vi, vj such that |vi –vj|2 · and | h vi –vj, u i | ¸ 0.01
O( 1 ) £ standard deviation
) PrU [ vi, vj get stretched] = exp( - 1 )
= exp( - )log n
E[# of stretched pairs] = O( n2 ) £ exp(- ) log n
Generating a contradiction: the walk on stretched pairs
u
Vi
Vj
0.01
d
0.01
d
r steps
0.01
d
r
|vfinal - vi| < r
| <vfinal – vi, u>| ¸ 0.01rd
= O( r ) x standard dev.
vfinal
Contradiction if r is large enough!
Measure concentration (P. Levy, Gromov etc.)
<d
A
A : measurable set with (A) ¸ 1/4
A : points with distance · to A
A
A) ¸ 1 – exp(-2 d)
Reason: Isoperimetric inequality for spheres
Expander flows
(approximate certificates of expansion)