limits of local algorithms for randomly generated constraint satisfaction problems
TRANSCRIPT
Limits of local algorithms for random graphsLimits of local algorithms for random graphs
David Gamarnik
Joint work with Madhu Sudan (Microsoft Research, New England)
Yandex workshop on extremal combinatorics
June 24, 2014
Local Algorithms
Computer Science. Applications: hardware, fault-tolerant models of computation, sensor networks.
Lynch (book) [1996]Nguyen & Onak [2008]Rubinfeld, Tamir, Vardi, Xie [2011]Suomela (survey) [2011]
Wireless communicationsApplications: communication protocols with low overhead
Shah [2008] Shin & Shah [2012]
Who is interested in local algorithms?
Network Science. Economic team modelingApplications: models of social interaction, structure of social networks
Rusmevichientong, Van Roy [2003]Judd, Kearns & Vorobeychik [2010] Borgs, Brautbar, Chayes, Khanna & Lucier [2012]
Signal Processing. Machine learning.Applications: image processing, coding theory, wireless communication, gossip algorithms.
Wainwright & Jordan [2008], Mezard & Montanari [2009], Shah [2008] Shin & Shah [2012]
Physics.Applications: Spin glass theoryMezard & Montanari [2009]
Who is interested in local algorithms?
Mathematics/Combinatorics.Applications: graph limits.
Lovasz (book) [2012] Hatami, Lovasz & Szegedy [2012] Elek & Lippner [2010] Lyons & Nazarov [2011] Czoka & Lippner [2012] Aldous [2012].
Who is interested in local algorithms?
Random n-node d-regular graph
I. Local algorithms for random graphs : i.i.d. factors
locally regular tree like
Local algorithms: i.i.d. factors
Hatami-Lovasz-Szegedy [2012] framework
Generate i.i.d. U[0,1] weights
Apply some local rule f: decorated tree ! {0,1} for every node
f
0 or 1
Conjecture. Hatami, Lovasz & Szegedy [2012] There exists a local rule f which produces a nearly largest independent set in a random d-regular graph
Example. f =1 iff the weight of the node is larger than the weights of all of its neighbors
IN OUT
far from optimal !
Local algorithms: i.i.d. factors
Algorithms for independent sets in random graphs
Frieze & Luczak [1992]
Best known algorithm: Greedy
Hatami-Lovasz-Szegedy Conjecture: there exists f such that
Theorem. [G & Sudan ] Hatami-Lovasz-Szegedy conjecture is not valid: No
local rule f can produce an independent set larger than factor of the optimal, for large enough d.
Theorem. [Rahman & Virag 2014] No local rule f can produce an independent set larger than factor of the optimal, for large enough d.
Notes:
1. Rahman & Virag’s result is the best possible: factor ½ can be achieved by local algorithms, Lauer & Wormald [2007], G & Goldberg [2010]
2. Proof technique. Spin Glass theory: geometry of large independent sets
Main result
Geometry of solutions: clustering phenomena
Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’ independent sets either have a very ‘’small’’ or a very ‘’large’’ intersection:
Geometry of solutions: clustering phenomena
Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’ independent sets either have a very ‘’small’’ or a very ‘’large’’ intersection:
Proof sketch: argue by contradiction. Suppose rule f exists
?
(a) Generate two i.i.d. sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Intersection size belongs to the ‘’non-existent interval’’
Theory is great not when it is correct but when it is interesting… Murray Davis, 1 9 7 1
II. Sequential local algorithms for random NAE-K-SAT problem
Not-All-Equal-K-Satisfiability formula on n boolean variables and m clauses.Each clause contains K (i.e. K=3)
Assignment is satisfiable if every clause is satisfied by at least one variable, and is not satisfied by at least some other variable
Formula is satisfiable if there exists at least one satisfying assignment.
NP-Complete.
Sequential local algorithms for random NAE-K-SAT problem
Random NAE-K-SAT problem
Generate m clauses independently uniformly at random from the space of all possible clauses (with replacement). d=m/n – clause density.
Theorem. [Coja-Oglan and Panagiotou 2014] The threshold for probability that the formula is satisfiable
Best algorithm Unit Clause succeeds only when
Achlioptas et al. 2001
K gap!
Sequential local algorithms for the random NAE-K-SAT problem
Sequential r-local algorithms
1.Fix a “local rule” which maps formula © with variable x to probability value.
2.For i=1,2,…,n apply rule ¿ to variables xi and depth r sub-formulas ©i rooted at xi : fix xi to be 1 with probability ¿(©i, xi), and 0 with probability 1- ¿(©i, xi)
Notes:
Belief Propagation (BP) guided decimation and Survey Propagation (SP) guideddecimation algorithms are sequential local algorithms when the number of iterations is bounded by a constant independent of the number of variables
Unit Clause algorithm is a sequential local algorithm
Belief Propagation and Survey Propagation algorithms – message passing type algorithms proposed by statistical physicists. Work remarkably well for low values of K (K=3,4,5).
Mezard, Parisi & Zecchina [2003]Krzakala, Montanari, Ricci-Tersenghi, Semerjian, Zdeborova [2007]Mezard & Montanari (book) [2009]
Theorem. G & Sudan [2013] Every sequential local algorithm fails to find a satisfying assignment when
simple algorithm (Unit Clause) exist
no sequential local algorithms exist
no solutions exist
Sequential local algorithms cannot bridge the 1/K gap
Proof Technique
(a) Clustering property for random NAE-K-SAT(b) Interpolation between m randomly generated solutions(c) Decisions for variables are localized
UNSAT
Proof Technique
(a) Clustering property for random NAE-K-SAT
Theorem. Let
With probability approaching unity as n increases, there does not exist m satisfying assignments such that all pairwise Hamming distances are
Proof: first moment method.
Some further thoughts
• The approach should apply to other problems with “symmetry”, for example coloring of graphs. But applying the approach to random K-SAT is problematic.
• In practice Survey Propagation algorithm is run for many iterations. Challenge: establish similar result when there is no bound on the number of iterations. Coja-Oghlan [2010] – true for Belief Propagation for K-SAT.
• Establish limits for the performance of local algorithms for random constraint satisfaction problems using the most general (Computer Science) definition of local algorithms.
Mick can’t get satisfaction above (from Sequential Local Algorithms)
Chvatal & Reed [1998] ’’Mick gets some (the odds are on his side) [satisfiability]’’
Parting thoughts …
Thank you