generic rounding schemes for sdp relaxations
DESCRIPTION
Generic Rounding Schemes for SDP Relaxations. Prasad Raghavendra Georgia Institute of Technology, Atlanta. ``Squish and Solve” Rounding Schemes [ R,Steurer 2009]. Rounding Schemes via Dictatorship Tests [R,2008]. Rounding SDP Hierarchies via Correlation - PowerPoint PPT PresentationTRANSCRIPT
Generic Rounding Schemesfor
SDP Relaxations
Prasad RaghavendraGeorgia Institute of Technology,
Atlanta
``Squish and Solve” Rounding Schemes
[R,Steurer 2009]
Rounding Schemes via Dictatorship Tests
[R,2008]
Rounding SDP Hierarchies via Correlation
[Barak,R,Steurer 2011] [R,Tan 2011]
``Squish and Solve” Rounding Schemes
[R,Steurer 2009]
Max Cut
10
15
3
7
11
Max CUTInput: A weighted graph G
Find:A Cut with maximum number/weight of crossing edges
Fraction of crossing edges
Semidefinite Program:[Goemans-Williamson 94]
Embedd the graph on the N - dimensional unit ball, Maximizing
¼ (Average Squared Length
of the edges)
Eji
jiij vvw),(
2||41
Semidefinite Program[Goemans-Williamson 94]
Variables : v1 , v2 … vn |vi|2 = 1
Maximize
MaxCut
10
15
3
7
11
1
1
1
-1
-1
-1
-1-1
-11 -1
Max Cut ProblemGiven a graph G,Find a cut that maximizes the number of crossing edges
v1
v2
v3
v4
v5
MaxCut Rounding
v1
v2
v3
v4
v5
Cut the sphere by a random hyperplane, and output the induced graph cut.
-A 0.878 approximation for the problem.
[Goemans-Williamson]
SQUISH AND SOLVE ROUNDING
Approximation using Finite Models
¦-CSP Instance =
¦-CSP Instance =finite
variablefolding
(identifyingvariables)
optimal solution for =finite
approximate solution for =
unfolding ofthe assignment
constant time
Challenge: ensure = finite has a good solution
10
15
3
7
11
1
1
-1
-1
-1
-1-1
-1
-1
1
1
11
Approximation using Finite Models
[Frieze-Kannan]For a dense instance =, it is possible to construct finite model =finite
OPT(=finite) ≥ (1-ε) OPT(=)
General Method for CSPs
What we will do :
SDP value (=finite) > (1-ε)SDP value (=)
PTAS for dense instances
Analysis of Rounding Scheme¦-CSP Instance
=¦-CSP Instance
=finite
SDP value ®
SDP value > ® - ²
OPT value¯
rounded value¯
010001001010001001
Hence: rounding-ratio for = < (1+²) integrality-ratio for = finite
unfolding
CONSTRUCTING FINITE MODELS (MAXCUT)
v1
v2
v3
v4
v5
STEP 1 : Dimension Reduction
• Pick d = 1/ Є4 random Gaussian vectors {G1 , G2 , .. Gd} • Project the SDP solution along these directions.Map vector V
V → V’ = (V G∙ 1 , V G∙ 2 , … V G∙ d)v
1
v3v
4 v5
Constant dimensions
STEP 2 : SurgeryScale every vector V’ to unit length
STEP 3 : Discretization•Pick an Є –net for the
d dimensional sphere• Move every vertex to the nearest point in the Є –net
v2v
2
FINITE MODEL Graph on Є –net points
To Show:
SDP value (=finite) > (1-ε)SDP value (=)
Johnson Lindenstrauss Lemma : “Distances are almost preserved under random
projections”
If V’,U’ are random projections of unit vectors U, V on 1/ ε4 directions,
Pr [ |V U – V’ U’| > ∙ ∙ ε] < ε2
STEP 1 : Dimension Reduction•Project the SDP solution along 1/ Є4 random directions.
STEP 2 : SurgeryScale every vector V’ to unit length
STEP 3 : Discretization•Pick an Є –net for the
d dimensional sphere• Move every vertex to the nearest point in the Є –net
For SDP value (=)Contribution of an edge e = (U,V)
|U-V|2 = 2-2 V U ∙
To Show:
SDP value (=finite) > (1-ε)SDP value (=)SDP Vectors for =finite = Corresponding vectors in Є –net
STEP 1With probability > 1- Є2 ,
| |U-V|2 - |U’-V’|2 | < 2Є
STEP 2With probability > 1- 2Є2 , 1+ Є < |V’|2 ,|U’|2 < 1- Є, Normalization changes distance by at most 2Є
STEP 3Changes edge length by at most 2Є
For SDP value (=)Contribution of an edge e = (U,V)
|U-V|2 = 2-2 V U ∙
To Show:
SDP value (=finite) > (1-ε)SDP value (=)SDP Vectors for =finite = Corresponding vectors in Є –net
STEP 1With probability > 1- Є2 ,
| |U-V|2 - |U’-V’|2 | < 2Є
STEP 2With probability > 1- 2Є2 , 1+ Є < |V’|2 ,|U’|2 < 1- Є, Normalization changes distance by at most 2Є
STEP 3Changes edge length by at most 2Є
ANALYSISWith probability 1-3Є2,The contribution of edge e changes by < 6Є
In expectation,For (1-3Є2) edges, the contribution of edge e changes by < 6Є
SDP value (=finite) > SDP value (=) - 6Є – 3Є2
Drawbacks•Running Time(A) On CSP over alphabet size q, arity k
•No explicit approximation ratio
Generic Rounding For CSPs
rounding – ratioA ( ¦ )(approximation ratio) ≥
(1-²) integrality gap of a natural SDP ( ¦ )(SDP is optimal under UGC)
=
[Raghavendra Steurer08]
For any CSP ¦ and any ²>0, there exists an efficient algorithm A,
Unifies a large number of existing rounding schemes, and the resulting algorithm A as good as all known algorithms for CSPs (without dependence on n)
)(2)/1,,(2 npoly
qkpoly
Computing Integrality Gaps
Theorem:
For any CSP ¦ and any ²>0, there exists an algorithm A to compute integrality gap (¦) within an accuracy ²
Running Time(A) On CSP over alphabet size q, arity k
)/1,,(22qkpoly
Run through all instances of size exp(poly(k,q,1/²)
Rounding Schemes via Dictatorship Tests
[R,2008]
Dictatorship TestGiven a function F : {-1,1}R {-1,1}•Toss random coins•Make a few queries to F •Output either ACCEPT or REJECT
F is a dictator functionF(x1 ,… xR) = xi
F is far from every dictator function
(No influential coordinate)
Pr[ACCEPT ] = Completeness
Pr[ACCEPT ] =Soundness
UG Hardness
Rule of Thumb: [Khot-Kindler-Mossel-O’Donnell]A dictatorship test where • Completeness = c and Soundness = αc•the verifier’s tests are predicates from a CSP Λ
It is UG-hard to approximate CSP Λ to a factor better than α
A Dictatorship Test for Maxcut
CompletenessValue of Dictator Cuts
F(x) = xi
SoundnessThe maximum value attained by a cut far from a dictator
A dictatorship test is a graph G on the hypercube.A cut gives a function F on the hypercube
Hypercube = {-1,1}100
Overviewv1
v2
v3
v4
v5
10
15
3
71
1
100 dimensional hypercube
Graph G SDP Solution
CompletenessValue of Dictator Cuts =
SDP Value (G)
SoundnessGiven a cut far from every dictator :It gives a cut on graph G with the same value.
Rounding Scheme:
• Construct the dictatorship test gadget from graph G• Try all possible cuts far from dictator, and obtain a cut back in the graph G.
Guarantee:Algorithm’s Output Value ≥ Soundness of the Dictatorship Test Gadget
UG Hardness
Dictatorship Test
Completeness CSoundness S [KKMO]
UG Hardness“On instances, with
value C, it is NP-hard to output a solution of
value S, assuming UGC”
In our case,
Completeness = SDP Value (G)Soundness < Algorithm’s Output
Cant get better approximation assuming UGC!
The Goalv1
v2
v3
v4
v5
10
15
3
71
1
100 dimensional hypercube
Graph G SDP Solution
CompletenessValue of Dictator Cuts =
SDP Value (G)
SoundnessGiven a cut far from every dictator :It gives a cut on graph G with the same value.
Influences
Definition: Influence of the ith co-ordinate on a function F:{0,1}R [-1,1] under a product distribution μR is defined as:
Infiμ (F) = E [ Variance [F] ]
Random Fixing of All Other Coordinates from μR-1
over changing the ith coordinate as per μ
Definition: A function is τ-quasirandom if for all i, Infi
μ (F) ≤ τ
(For the ith dictator function : Infiμ (F) is as large as variance of F)
v1
v2
v3
v4
v5
Max Cut SDP:
Embed the graph on the N - dimensional unit ball,
Maximizing
¼ (Average Squared Length
of the edges)
v1
v2
v3v
4 v5
Constant dimensional hyperplaneProject to random 1/ Є2 dimensional space.New SDP Value = Old SDP Value + or - Є
100
Dimension Reduction
Making the Instance Harder
v 1
v 2
v 3
v 4
v 5
v1
v2
v3
v4
v5
SDP Value = Average Squared Length of an Edge
Transformations• Rotation does not change the SDP value.• Union of two rotations has the same SDP value
Sphere Graph H :Union of all possible rotations of G.
v 1
v 2
v 3
v 4
v 5v1
v2
v3
v4
v5v 1
v 2v 3
v 4
v 5
SDP Value (Graph G) = SDP Value ( Sphere Graph H)
Making the Instance Harder
v 1
v 2
v 3
v 4
v 5
v1
v2
v3
v4
v5
MaxCut (H) = S
MaxCut (G) ≥ S
Pick a random rotation of G and read the cut induced on it.Thus,
v 1
v 2
v 3
v 4
v 5v1
v2
v3
v4
v5v 1
v 2v 3
v 4
v 5
v1
v2v3
v4
v5
MaxCut (H) ≤ MaxCut(G)SDP Value (G) = SDP Value (H)
Hypercube Graph v1
v2
v3
v4
v5
SDP Solution
100 dimensional hypercube : {-1,1}100
For each edge e, connect every pair of vertices in hypercube separated by the length of e
Generate Edges of Expected Squared Length = d
1) Starting with a random x Є {-1,1}100 ,1) Generate y by flipping each bit of x with probability d/4
Output (x,y)
Dichotomy of Cuts
Dictator CutsF(x) = xi
Cuts Far From Dictators(influence of each coordinate on function F is small)
A cut gives a function F on the hypercube
F : {-1,1}100 -> {-1,1}
Hypercube = {-1,1}100
1
1 1
1
-1 -1
-1
Dictator Cuts
100 dimensional hypercube
v1
v2
v
uv5
For each edge e = (u,v), connect every pair of vertices in hypercube separated by the length of e
Value of Dictator Cuts = SDP Value (G)
Pick an edge e = (u,v), consider all edges in hypercube corresponding to e
Fraction of red edges cut by horizontal dictator .
Fraction of dictators that cut one such edge (X,Y)
Number of bits in which X,Y differ
=|u-v|2/4
=
X
Y
=
Fraction of edges cut by dictator = ¼ Average Squared Distance
Cuts far from Dictatorsv1
v2
v3
v4
v5
100 dimensional hypercube
-1
-1-1
1
1
1
Intuition:
Sphere graph : Uniform on all directions
Hypercube graph : Axis are special directions
If a cut does not respect the axis, then it should not distinguish between Sphere and Hypercube graphs.
v1
v2
v3
v4
v5v 1
v 2
v 3
v 4
v 5
v 1
v 2v 3
v 4
v 5
The Invariance PrincipleCentral Limit Theorem
``Sum of large number of {-1,1} random variableshas similar distribution as
Sum of large number of Gaussian random variables.”
Invariance Principle for Low Degree Polynomials[Rotar] [Mossel-O’Donnell-Oleszkiewich], [Mossel 2008]
“If a low degree polynomial F has no influential coordinate, then F({-1,1}n) and F(Gaussian) have similar distribution.”
Hypercube vs Sphere
H
F:{-1,1}100 -> {-1,1} is a cut far from every dictator.
P : sphere -> Nearly {-1,1} is the multilinear extension of F
By Invariance Principle, MaxCut value of F on hypercube ≈ Maxcut value of P on
Sphere graph H
Rounding SDP Hierarchies via Correlation
[Barak,R,Steurer 2011] [R,Tan 2011]
The Unique Games Barrier
It is Unique Games-Hard to approximate to a factor better than that given by Simple SDP Relaxation for
• Constraint Satisfaction Problems [R08]• Metric Labelling Problems [Manokaran-Naor-R.-Schwartz 08]• Ordering Constraint Satisfaction Problems
[Guruswami-Hastad-Manokaran-R. ]• Kernel Clustering Problems [Khot Naor 09]
• Grothendieck Problem [R.-Steurer 09]
• Monotone-Hard-Constraint CSPs [Kumar-Manokaran-Tulsiani-Vishnoi]
[R-Steurer 09]Unconditionally, Adding all valid constraints on at most 2^O((loglogn)1/4) variables to the simple SDP does not improve the approximation ratio for
Constraint Satisfaction Problems
Metric Labelling Problems Ordering Constraint Satisfaction
Problems
Kernel Clustering Problems
Grothendieck Problem
For the non-believers
Stronger SDP Relaxations
Possibility:``Certain Strong SDP Relaxations yield better
approximations and disprove the Unique Games Conjecture”
(five rounds of Lasserre hierarchy)
Even Otherwise:For what problems do these relaxations help?
How does one use these stronger SDP relaxations?
Difficulty
.
Successes of Stronger SDP Relaxations:• [Arora-Rao-Vazirani] used an SDP with triangle inequalities to improve approximation for Sparsest Cut from log n to sqrt(log n).
• Stronger SDPs for better approximations for graph and hypergraph independent set in [Chlamtac] [Arora-Charikar-Chlamtac] [Chlamtac-Singh]
Very few general techniques to extract the power of stronger SDP relaxations.
Eji
jiij vvw),(
2||41
Semidefinite Program
Variables : v1 , v2 … vn | vi |2 = 1
Maximize
SDP for MaxCutQuadratic Program
Variables : x1 , x2 … xn xi = 1 or -1
Maximize
Eji
jiij xxw),(
2)(41
Relax all the xi to be unit vectors instead of {1,-1}. All products are replaced by inner products of vectors
10
15
3
7
11
1
11
-1
-1
-1
-1-1
-11 -1
Ideally, these vectors are convex combination of integral solutions.
-- the SDP can be thought of as a distribution over cuts
Instead, we force vectors to look like integral solutions locally (on every k vertices)
k-round Lasserre-SDP for MaxCut
1015
3
7111
11
-1-1
-1
-1-1
-11 -1
1 -1 1 -1 …………….. 1 1 1 -1 1 1 - 1
1 -1 -1 -1 …………….. 1 1 1 -1 1 1 1
1 -1 -1 -1 …………….. 1 1 1 -1 1 1 - 1 1 -1 1 -1 …………….. 1 1 1 -1 1 1 - 1
1 1 1 -1 …………….. 1 1 1 -1 1 1 - 1
1 1 1 -1 …………….. 1 1 1 -1 1 1 - 1
-……………………………………………………………………
X1 X2 X3 X4 …………….. X15 …………………. Xn
Local distribution μS
For any subset S of k vertices,
A local distribution μS over {+1,-1} assignments to the set S
Conditioned SDP Vectors {vi|Sα}
For any subset S of k vertices, and an assignment α in {-1,1}k ,
An SDP solution {vi|Sα} corresponding to the SDP solution conditioned on S being assigned α
CorrelationsCorrelation:`` Two random variables are correlated, if the fixing the
value of one changes the distribution of the other’’Measuring Correlation:Mutual information between the two random variables.
]|)([)|( aYXHEYXHa
]Pr[1log]Pr[)(aX
aXXH
Entropy of X Conditional Entropy of X|Y
Mutual Information I(X,Y) = H(X) - H(X|Y)
Global CorrelationGlobal Correlation is the average correlation between random
pairs of vertices in the instance.
GC = E {a,b}[ I(Xa , Xb) ]
Crucial ObservationConditioning the SDP solution on the value of a random vertex Xa reduces average entropy by GC
Proof:average entropy = E{b} H(Xb) average entropy after conditioning Xa = E{a} [E{b} H(Xb | Xa)]
Hence the decrease isE{b} H(Xb) - E{a} [E{b} H(Xb | Xa)] = E{a,b} [H(Xb)- H(Xb | Xa)] = E{a,b} [I(Xb , Xa)]
Progress By Global CorrelationsSuppose an SDP solution has global correlation > ε,Then we sample and condition on the value of a random vertex,Average entropy drops by ε
If global correlation always remains > ε, then after 1/ ε conditionings, the average entropy ≈ 0
The variables are almost frozen, and the conditioned SDP solution is nearly integral.
CorollaryWithin O(1/ ε) conditionings, the global correlation of the SDP solution becomes < ε
Application: Max Bisection
10
15
3
7
11
Max BisectionInput: A weighted graph G
Find:A Cut with maximum number/weight of crossing edges
with exactly ½ of the vertices on each side of the cut.
Halfspace Rounding?
v1
v2
v3
v4
v5
Cut the sphere by a random hyperplane, and output the induced graph cut.
The expected fraction of vertices on each side of the cut is half.
However, the actual number of vertices might always be away from half
-- no concentration
Independence among random variables Concentration (Ex: Chernoff bounds)
Lack of concentration lack of independence
Bounding Variance Let Z1 , Z2 , .. Zn denote the random projections,Suppose the rounding function is F : R [0,1]Fraction of vertices on one side of the cut = E{a} [F(Za)]
Variance of this random variable= EZ [ E{a,b} [F(Za)F(Zb)] - E{a} [F(Za)]E{b} [F(Zb)] ]
= E{a,b} [ Covariance(F(Za), F(Zb)) ] Low global correlation E{a,b} [I(Za ,Zb)] is small
the above variance is small.
CSPs with Global Cardinality Constraint
[R, Tan 2011]Given an instance of Max Bisection/Min Bisection with
value 1-ε, there is an algorithm running in time npoly(1/ε) that finds a solution of value 1-O(ε1/2)
[R, Tan 2011]For every CSP with global cardinality constraint, there is a
corresponding dictatorship test whose Soundness/Completeness = Integrality gap of poly(1/ ε) - round
Lasserre SDP.
Another Application: 2-CSPs on ``expanding instances”
Locally, the constraints of the CSP introduce correlations among the variables.
If the graph is a sufficiently good expander, these local correlations must translate in to global correlations.
10
15
3
7
11
1
1
1
-1
-1
-1
-1-1
-1
1 -1
Low-Rank Graphs
If the adjacency matrix of the graph is “low rank” – approximated by few eigen vectors.
10
15
3
7
11
1
1
1
-1
-1
-1
-1-1
-1
1 -1
Lemma: If the number of eigen values > δ is less than d, then an SDP solution with local correlation > δ has global correlation O(1/d2)
2-CSP on random constraint graphs[Barak-Raghavendra-Steurer]
Given an instance of 2-CSP whose constraint graph is a degree d random graph,
poly(1/ε, k, d) round Lasserre SDP hierarchy has value <
Optimum + od(1)
Another Application
Subexponential Time Algorithm for Unique Games[Arora-Barak-Steurer]
Given an instance of Unique Games with value 1-ε,in time exp(nε), the algorithm finds a solution of value 1-εc
Used a combination of brute force and spectral decomposition, but no SDPs
Subexponential Time Algorithm for Unique Games via SDPs[Barak-Raghavendra-Steurer] [Guruswami-Sinop]
Given an instance of Unique Games such thatnO(ε) – rounds of SDP hierarchy has value with value 1-ε,
there exists an assignment of value 1-εc
Future Work
Can one use local-global correlations to prove [Arora-Rao-Vazirani] or something weaker?
subexponential time algorithms beating the current best for MaxCut, Sparsest Cut?
Thank You
Rounding
1015
3
7111
11
-1-1
-1
-1-1
-11 -1
1 -1 1 -1 …………….. 1 1 1 -1 1 1 - 1
-1 -1 -1 -1 …………….. 1 1 1 -1 1 1 1
1 -1 -1 -1 …………….. 1 1 1 -1 1 1 - 1 1 -1 1 -1 …………….. 1 1 1 -1 1 1 - 1
1 1 1 -1 …………….. 1 1 1 -1 1 1 - 1
1 1 1 -1 …………….. 1 1 1 -1 1 1 - 1
-……………………………………………………………………
X1 X2 X3 X4 …………….. X15 …………………. Xn
Case 1: average entropy < ε,
The SDP solution is nearly integral (it can be rounded to integral solution with value c – O(ε) )
Case 2: average entropy > ε,
if we condition on a random vertex, the average entropy drops by δ
Main Theorem (Informal):
If an instance I of a problem satisfies (c,ε,δ)-global correlation property,
Then, (1/δ)-round SDP solution on the instance I is within O(ε) of the integral value.
Do instances have this global correlation property arise?