Statistical Regimes Across Constrainedness Regions

Carla P. Gomes, Cesar FernandezBart Selman, and Christian Bessiere

Cornell UniversityUniversitat de Lleida

LIRMM-CNRS

CP 2004Toronto

Motivation

Bring together recent results on:

• Typical Case Analysis

• Randomized Complete Search Methods

• Heavy-Tailed Phenomena

• Random CSP Models

Typical Case Analysis: Beyond NP-Completeness

Constrainedness

Com

puta

tion

al C

ost (

Mea

n)

% o

f so

lvab

le in

stan

ces

Phase TransitionPhenomenon:Discriminating “easy” vs.“hard” instances

Hogg et al 96

Exceptional Hard Instances

Seem to defy the “easy-hard” pattern:

– such instances occur in the under-constrained area;

– they are considerably harder than other similar instances and even harder than instances from the critically constrained area.

Gent and Walsh 94Hogg and Williams 94Smith and Grant 97

Are Exceptionally Hard Instances Truly Hard?

• Different algorithms encounter different exceptionally hard instances.

• ``Hardness'' of exceptionally hard instances

not necessarily hardness of the instances, but rather a the combination of the instance with the details of the search method;

Gent and Walsh 94Hogg and Williams 94Selman and Kirkpatrick 96Smith and Grant 97

Randomized Backtrack Search

What if we introduce a tiny element of randomness into the search heuristic – e.g., by breaking ties randomly --- and run this (still complete) randomized search procedure on the same instance over and over again?

Study of runtime distributions of a randomized backtrack search

on the same instance : Way of isolating the variance caused

solely by the algorithm Gomes et al CP 97

Time: >20003011 >20007

Easy instance – 15 % preassigned cells

Gomes, et al 97

Extreme Variance in Runtimeof Randomized Backtrack Search

Heavy-tailed distributions

0,]Pr[ 2

CsomeforxCexX

Exponential decay for standard distributions, e.g. Normal, Logonormal,

exponential:

Heavy-Tailed Power Law Decay e.g. Pareto-Levy:

0,]Pr[ xCxxX

Normal

(Frost et al 97; Gomes et al 97 ,Hoos 1999,Walsh 99,)

Heavy-tailed Dist.

Visualization of Heavy-tailed Phenomenon(Log-Log Plot of Tail o Distribution)

Visualization of Heavy-tailed Phenomenon(Log-Log Plot of Tail o Distribution)

Normal(2,1000000)

Normal(2,1)

1-F

(x)

Unso

lved f

ract

ion

Runtime (Number of backtracks) (log scale)

O,1%>200000

50%

2

Median=2

Formal Results

Abstract Search Tree Models with provably heavy-tailed behavior (Chen, Gomes, Selman 2001)

Generalization and Assignment of Semantics to the Abstract Search Tree Models

(Williams, Gomes, Selman 2003)

Provably Polytime Restart Strategies

(Williams, Gomes, Selman 2003)

What about concrete CSP models?(so far no good characterization of

runtime distributions of concrete CSP models)

Research Questions:

1. Can we provide a characterization of heavy-tailed behavior: when it occurs and it does not occur?

2. Can we identify different tail regimes across different constrainedness regions?

3. Can we get further insights into the tail regime by analyzing the concrete search trees produced by the backtrack search method?

Concrete CSP ModelsComplete Randomized Backtrack Search

Outline of the Rest of the Talk

• Random Binary CSP Models• Encodings of CSP Models• Randomized Backtrack Search Algorithms• Search Trees• Statistical Tail Regimes Across Cosntrainedness

Regions– Empirical Results– Theoretical Model

• Conclusions

Binary Constraint Networks

• A finite binary constraint network P = (X, D,C)

– a set of n variables X = {x1, x2, …, xn}– For each variable, set of finite domains

D = { D(x1), D(x2), …, D(xn)}– A set C of binary constraints between pairs of variables;

a constraint Cij, on the ordered set of variables (xi, xj) is a subset of the Cartesian product D(xi) x D(xj) that specifies the allowed combinations of values for the variables xi and xj.

– Solution to the constraint networkinstantiation of the variables such that all constraints are satisfied.

Random Binary CSP Models

Model B < N, D, c, t >

N – number of variables; D – size of the domains; c – number of constrained pairs of variables;

p1 – proportion of binary constraints included in network ;c = p1 N ( N-1)/ 2;

t – tightness of constraints;p2 - proportion of forbidden tuples; t = p2 D2

Model E <N, D, p>

N – number of variables; D – size of the domains: p – proportion of forbidden pairs (out of D2N ( N-1)/ 2)

(Achlioptas et al 2000)

(Gent et al 1996)

N – from 15 to 50; (Xu and Li 2000)

Encodings

• Direct CSP Binary Encoding• Satisfiability Encoding (direct encoding)

Walsh 2000

Backtrack Search Algorithms

• Look-ahead performed::– no look-ahead (simple backtracking BT);– removal of values directly inconsistent with the last instantiation

performed (forward-checking FC);– arc consistency and propagation (maintaining arc consistency, MAC).

• Different heuristics for variable selection (the next variable to instantiate):– Random (random);– variables pre-ordered by decreasing degree in the constraint graph (deg);– smallest domain first, ties broken by decreasing degree (dom+deg)

• Different heuristics for variable value selection:– Random– Lexicographic

• For the SAT encodings we used the simplified Davis-Putnam-Logemann-Loveland procedure: Variable/Value static and random

Inconsistent Subtrees

Bessiere at al 2004

Distributions

• Runtime distributions of the backtrack search algorithms;

• Distribution of the depth of the inconsistency trees found during the search;

All runs were performed without censorship.

Main Results

1 - Runtime distributions2 – Inconsistent Sub-tree Depth

Distributions

Dramatically different statistical regimes across the constrainedness

regions of CSP models;

Runtime distributions

Distribution of Depth of Inconsistent Subtrees

Applet

Applet

Depth of Inconsistent Search Tree vs. Runtime Distributions

Other Models and More Sophisticated Consistency Techniques

Other Models and More Sophisticated Consistency Techniques

BT MAC

Heavy-tailed and non-heavy-tailed regions.As the “sophistication” of the algorithm increases the heavy-tailed region extends to the right, getting closer to the phase transition

Model B

SAT encoding: DPLL

Theoretical Model

Depth of Inconsistent Search Tree vs. Runtime Distributions

Theoretical Model

X – search cost (runtime);ISTD – depth of an inconsistent sub-tree;

Pistd [IST = N]– probability of finding an inconsistent sub-tree of depth N during search;

P[X>x | N] – probability of the search cost being larger x, given an inconsistent tree of depth N

Depth of Inconsistent Search Tree vs. Runtime Distributions:

Theoretical Model

See paper for proofdetails

Regressions for B1, B2, K

Regression for B1 and B2 Regression for k

Validation: Theoretical Model vs. Runtime Data

α= 0.26 using the model;α= 0.27 using runtime data;

Summary of Results

1 As constrainedness increases change from heavy-tailed to a non-heavy-tailed regime

Both models (B and E), CSP and SAT encodings, for the different backtrack search strategies:

Summary of Results

2 Threshold from the heavy-tailed to non-heavy-tailed regime

– Dependent on the particular search procedure;

– As the efficiency of the search method increases, the extension of the heavy-tailed region increases: the heavy-tailed threshold gets closer to the phase transition.

Summary of Results

3 Distribution of the depth of inconsistent search sub-trees

Exponentially distributed inconsistent sub-tree depth (ISTD) combined with exponential growth of the search space as the tree depth increases implies heavy-tailed runtime distributions.

As the ISTD distributions move away from the exponential distribution, the runtime distributions become non-heavy-tailed.

Research Challenges

How to exploit these results in terms of the design of more efficient search procedures?

– Randomization and restart strategies;– Search heuristics:– Look ahead and look back strategies;

Very exciting and promising research area !

Demos and papers:

www.cs.cornell.edu/gomes/http://fermat.eup.udl.es/~cesar/

www.cs.cornell.edu/selman/ http://www.lirmm.fr/~bessiere/