600.325/425 declarative methods - j. eisner 1 random 3-sat sample uniformly from space of all...

600.325/425 Declarative Methods - J. Eisner

1

Random 3-SAT

Random 3-SAT sample uniformly from

space of all possible 3-clauses

n variables, l clauses

Which are the hard instances? around l/n = 4.3

slide thanks to Henry Kautz


2

Random 3-SAT

Varying problem size, n

Complexity peak appears to be largely invariant of algorithm backtracking algorithms like

Davis-Putnam local search procedures like

GSAT

What’s so special about 4.3?



3

Random 3-SAT

Complexity peak coincides with solubility transition

l/n < 4.3 problems under-constrained and SAT

l/n > 4.3 problems over-constrained and UNSAT

l/n=4.3, problems on “knife-edge” between SAT and UNSAT



4

“Order parameter” for 3SAT [Mitchell, Selman, Levesque AAAI-92] = #clauses / # variables This predicts

satisfiability hardness of finding a model

slide thanks to Tuomas Sandholm


5slide thanks to Tuomas Sandholm


6

Generality of the order parameter The results seem quite general across model

finding algorithms Other constraint satisfaction problems have

order parameters as well



7

…but the complexity peak does not occur under all ways of generating the 3SAT instances



8

Iterative refinement algorithms for SAT



9

GSAT [Selman, Levesque, Mitchell AAAI-

92] (= a local search algorithm for model finding)

Incomplete (unless restart a lot)

200016001200800400

Avg. total flips

100 200

50 variables, 215 3SAT clauses

max-climbs

Greediness is not essential as long as climbs and sideways moves are preferred over downward moves.



11

BREAKOUT algorithm [Morris AAAI-93]

Initialize all variables Pi randomlyUNTIL currently state is a solution

IF current state is not a local minimumTHEN make any local change that reduces the total cost (i.e. flip one Pi)

ELSE increase weights of all unsatisfied clause by one

Incomplete, but very efficient on large (easy) satisfiable problems.

Reason for incompleteness: the cost increase of the current local optimum spills to other solutions because they share unsatisfied clauses.



12

Real-World Phase Transition Phenomena Many NP-hard problem distributions show

phase transitions - job shop scheduling problems TSP instances from TSPLib exam timetables @ Edinburgh Boolean circuit synthesis Latin squares (alias sports scheduling)

Hot research topic: predicting hardness of a given instance, & using hardness to control search strategy (Horvitz, Kautz, Ruan 2001-3)



13

Local search for SAT

Repair based methods Instead of building up a solution, take complete

assignment and flip variable to satisfy “more” clauses

Cannot prove UNSATisfiability Often will solve MAX-SAT problem


14

Papadimitrou’s procedure

T:= random truth assignment Repeat until T satisfies all clauses

v := variable in UNSAT clause T := T with v’s value flipped

Semi-decision procedureSolves 2-SAT in expected O(n^2) time


15

GSAT [Selman, Levesque, Mitchell AAAI 92]

Repeat MAX-TRIES times or until clauses satisfied T:= random truth assignment Repeat MAX-FLIPS times or until clauses satisfied

v := variable which flipping maximizes number of SAT clauses T := T with v’s value flipped

Adds restarts and greedinessSideways flips important (large plateaus to explore)


16

WalkSAT [Selman, Kautz, Cohen AAAI 94]


c := unsat clause chosen at random v:= var in c chosen either greedily or at random T := T with v’s value flipped

Focuses on UNSAT clauses


17

Novelty [McAllester, Selman, Kautz AAAI 97]


c := unsat clause chosen at random v:= var in c chosen greedily If v was last var in clause flipped and rand(1)<p then

v:= var in c with 2nd highest score T := T with v’s value flipped

Encourages diversity (like Tabu search)


18

Other local search methods

Simulated annealing Tabu search Genetic algorithms Hybrid methods

Combine best features of systematic methods (eg unit propagation) and local search (eg quick recovery from failure)


19

Exotic methods

Quantum computing DNA computing Ant computing …


20

Beyond the propositional

Linear 0/1 inequalities Quantified Boolean satisfiability Stochastic satisfiability Modal satisfiability …


21

Linear 0/1 inequalities

Constraints of the form: Sum ai . Xi >= c

SAT can easily be expressed as linear 0/1 problem X v -Y v Z => X + (1-Y) + Z >= 1 …


22

Linear 0/1 inequalities

Often good for describing problems involving arithmetic Counting constraints Objective functions

Solution methods Complete Davis-Putnam like methods Local search methods like WalkSAT(PB)


23

Quantified Boolean formulae

Propositional SAT can be expressed as: Exists P, Q, R . (P v Q) & (-Q v R) & …

Can add universal quantifiers: Forall P . Exists Q, R . (P v Q) & (-Q v R) & ..

Useful in formal methods, conditional planning …


24

Quantified Boolean formulae

SAT of QBF is PSPACE complete Can be seen as game

Existential quantifiers trying to make it SAT Universal quantifiers trying to make it UNSAT

Solution methods Extensions of the DPLL procedure Local search methods just starting to appear


25

Stochastic satisfiability

Randomized quantifier in addition to the existential With prob p, Q is True With prob (1-p), Q is False Can we satisfy formula in some fraction of possible worlds?

Useful for modelling uncertainty present in real world (eg planning under uncertainty)


26

Randomized Algorithms

A randomized algorithm is defined as an algorithm where at least one decision is based on a random choice.

slide thanks to Russ Greiner and Dekang Lin


27

Monte Carlo and Las Vegas

There are two kinds of randomized algorithms: Las Vegas: A Las Vegas algorithm always

produces the correct answer, but its runtime for each input is a random variable whose expectation is bounded.

Monte Carlo: A Monte Carlo algorithm runs for a fixed number of steps for each input and produces an answer that is correct with a bounded probability One sided Two sided



28

Local Search Summary

Surprisingly efficient search technique Wide range of applications Formal properties elusive Intuitive explanation:

Search spaces are too large for systematic search anyway. . .

Area will most likely continue to thrive



29

Pure WalkSat

PureWalkSat( formula ) Guess initial assignment

While unsatisfied do Select unsatisfied clause c = ±Xi v ±Xj v ±Xk

Select variable v in unsatisfied clause c

Flip v



30

Example:



31

Mixing Random Walk with Greedy Local Search

Usual issues: Termination conditions Multiple restarts

Value of p determined empirically, by finding best setting for problem class



32

Finding the best value of p

WalkSat[p]: W/prob p, flip var in unsatisfied clause W/prob 1-p, make a greedy flip to minimize # of unsatisfied

clauses Q: What value for p? Let: Q[p, c] be quality of using WalkSat[p] on problem c.

Q[p, c] = Time to return answer, or = 1 if WalkSat[p] return (correct) answer within 5

minutes and 0 otherwise, or = . . . perhaps some combination of both . . .

Then, find p that maximize the average performance of WalkSat[p] on a set of challenge problems.



33

Experimental Results: Hard Random 3CNF

Time in seconds Effectiveness: prob. that random initial assignment leads to a

solution. Complete methods, such as DP, up to 400 variables

Mixed Walk better than Simulated Annealing better than Basic GSAT better than Davis-Putnam



34

Overcoming Local Optimum and Plateau

Random restarts Simulated annealing Mixed-in random walk Tabu search (prevent repeated states) Others (Genetic

algorithms/programming, . . . )



35

Other Techniques

random restarts: restart at new random state after pre-defined # of local steps. [Done by GSAT]

tabu: prevent returning quickly to same state. Implement: Keep fixed length queue (tabu list). Add most recent step to queue; drop oldest step. Never make step that's on current tabu list.

Example: without tabu:

flip v1, v2, v4, v2, v10, v11, v1, v10, v3, ... with tabu (length 5)—possible sequence:

flip v1, v2, v4, v10, v11, v1, v3, ... Tabu very powerful; competitive w/ simulated annealing or

random walk (depending on the domain)


600.325/425 declarative methods - j. eisner 1 random 3-sat sample uniformly from space of all...

Documents

declarative methods

tuomas sandholm600

wellslide thanks

modelslide thanks

unsatslide thanks

eisnerslide thanks

instancesslide thanks

escaping slide thanks