Constraint-based problem solving
Model problem specify in terms of constraints on acceptable solutions define variables (denotations) and domains define constraints in some language
Solve model define search space / choose algorithm
– incremental assignment / backtracking search
– complete assignments / stochastic search
design/choose heuristics
Verify and analyze solution
Model problem specify in terms of constraints on acceptable solutions define variables (denotations) and domains define constraints in some language
Solve model define search space / choose algorithm
– incremental assignment / backtracking search
– complete assignments / stochastic search
design/choose heuristics
Verify and analyze solution
Constraint-based problem solving
ConstraintSatisfactionProblem
Constraint satisfaction problem
A CSP is defined by a set of variables a domain of values for each variable a set of constraints between variables
A solution is an assignment of a value to each variable that
satisfies the constraints
Example: Crossword puzzles
CSP, binary CSP, SAT, 3-SAT, ILP, ...
Model and solve in one of these languages Model in one language, translate into
another to solve
Options
Article of faith
“Constraints arise naturally in most areas of human endeavor. They are the natural medium of expression for formalizing regularities that underlie the computational and physical worlds and their mathematical abstractions.”
P. Van Hentenryck and V. Saraswat
Constraint Programming: Strategic Directions
Reducibility
NP-Complete
3-SAT
ILP
CSP binary CSP
(0,1)-ILP
CSP, binary CSP, SAT, 3-SAT, ILP, ...
Model and solve in one of these languages Model in one language, translate into
another to solve
Options
Importance of the model
“In integer programming, formulating a ‘good’ model is of crucial importance to solving the model.”
G. L. Nemhauser and L. A. Wolsey
Handbook in OR & MS, 1989
“Same for constraint programming.”
Folk Wisdom, CP practitioners
Measures for comparing models
How easy is it to
–write down,
–understand,
–modify, debug,
–communicate? How computationally difficult is it to solve?
Constraint systems/languages
System LanguageILOG Solver C++ libraryCosytec CHIP C++ libraryEclipse Logic programmingOz Functional programming
Abstractions
Capture commonly occurring constraints special propagation algorithms e.g. alldifferent,
cardinality,
cumulative, ... User-defined constraints
full power of host language
CSP versus ILP
OO databases versus Relational databases C++ versus C C++ versus Fortran C++ versus Assembly language
Computational difficulty?
What is a good model depends on algorithm Choice of variables defines search space Choice of constraints defines
how search space can be reduced how search can be guided
Improving model efficiency
Given a model:
Add/remove variables, values, constraints (keep the denotation of the variables)
Use/translate to a different representation(change the denotation of the variables)
Adding redundant constraints
Improve computational efficiency of model by adding “right” constraints symmetries removed dead-ends encountered earlier in search process
Three methods: add hand-crafted constraints during modeling apply a consistency algorithm before solving learn constraints while solving
Examples:
Adding hand-crafted constraints: crossword puzzles
Applying a consistency algorithm: dual representation
Adding redundant variables
Variables that are abstractions of other variables e.g, decision variables
Suppose x has domain {1,…,10}
Add Boolean variable to represent decisions
(x < 5), (x 5) Variables that represent constraints
Translations between models
Improve computational efficiency by completely changing the model change denotation of variables
– e.g, convert from non-binary to binary
aggregate variables– e.g, timetabling multiple sections of a course
Not much as has been done on either the theory or the practice side
Conversion to binary
Any CSP can be converted into one with only binary constraints
Two techniques known dual graph method (Dechter & Pearl, 1989)
hidden variable method (Peirce, 1933; Dechter 1990)
Translations are polynomial if constraints are represented extensionally
(x1 x2 x6) (x1 x3 x4) (x4 x5 x6) (x2 x5 x6) Non-binary CSP: Boolean variables: x1, …, x6
constraints: one for each clause
3-SAT example
C1(x1, x2, x6) = {(0,0,1), (0,1,0), (0,1,1),
(1,0,0), (1,0,1), (1,1,0), (1,1,1)}
C2(x1, x3, x4) = ...
Dual graph representation
y1 = (x1, x2, x6) y4 = (x2, x5, x6)
y2 = (x1, x3, x4) y3 = (x4, x5, x6)
x2, x6
x5, x6
x4
x6x1
Hidden variable representation
x1 x2 x3 x4 x5 x6
h1
h2 h3
h4
Non-binary constraint:
Augmented constraint:
Binary constraints:
Hidden variable representation
C1(x1, x2, x6) = {(0,0,1), (0,1,0), (0,1,1),
(1,0,0), (1,0,1), (1,1,0),
(1,1,1)}
C1(x1, x2, x6, h1) = {(0,0,1,0), (0,1,0,1), (0,1,1,2),
(1,0,0,3), (1,0,1,4), (1,1,0,5), (1,1,1,6)}
R1(x1, h1) = {(0,0), (0,1), (0,2), (1,3), (1,4), (1,5), (1,6)}
R2(x2, h1) = {(0,0), (1,1), (1,2), (0,3), (0,4), (1,5), (1,6)}
R3(x6, h1) = {(1,0), (0,1), (1,2), (0,3), (1,4), (0,5), (1,6)}
x1
x2
x3
h1
h2
h3
Dual exponentially worse
FC on non-binary:
O(n) consistency checks
FC on dual:
O(2n) consistency
checks
x1, x1 x2, x1 x2 x3, ..., x1 … xn-1 xn
x1
x2
x3
h1
h2
h3
Dual exponentially better
FC on non-binary:
O(n2n) consistency checks
FC on dual:
O(n2) consistency
checks
x1 … xn-1, x1 … xn-2 xn, …, x2 … xn
Worst case:
Best case:
Bounds on performance
FC(non-binary) FC+(hidden) dk
FC+(hidden) FC (non-binary) dn
Crossword puzzles
1 2 3
6
4
7
5
8
10
9
20
11
22
12
21
13
17
14
181615
23
19
aaardvarkabackabacusabaftabaloneabandon...
Mona Lisamonarchmonarchymonarda...zymurgyzyrianzythum
Crossword puzzles
puzzle orig. cc’s dual cc’s n m
1 52 292 4 3
2 138 3,267 6 8
3 185 11,404 8 14
4 208 10,481 12 14
5 12,509 18,686 10 19
6 12,636 19,876 14 21
7 556,660 28,962 14 23
Crossword puzzles
puzzle orig. cc’s dual cc’s n m
8 5,780,710 27,656 20 30
9 * 47,167 26 41
10 * 224,258 18 35
11 * 29,777 24 40
12 * 729,125 18 38
13 28,446,460 50,281 28 52
14 * 72,307 34 65
Model of random non-binary CSP
n variables each with domain size d
m constraints each with k variables, chosen at random each with t tuples, chosen at random
Order of magnitude curves(n
um
ber
of
const
rain
ts)
/ (n
um
ber
of
vari
able
s)
(number of tuples in constraints) / (maximum tuples)
n = 20d = 2k = 3
n = 20d = 10k = 3
n = 20d = 10k = 5
n = 20d = 2k = 5
Random 3-SAT
clauses/variables
costratio
0.25 0.9
0.5 1.5
1 2.5
2 4.2
4 31
8 33
16 38
Given start state, goal state, and actions: determine a plan (a sequence of actions)
Box1
Project: Planning
Box2Box2
Contrasts
CP philosophy/methodology emphasis on modeling, domain knowledge general purpose search algorithm
– backtracking with constraint propagation
Successful e.g., can solve practical scheduling problems
Contrasts
Planning philosophy/methodology emphasis on minimal model
– just representation of actions
special-purpose search algorithms Not as successful
has not solved many practical problems
Tradeoffs
Robust CSP model needed for each new domain can require much intellectual effort
Less work needs to be done on algorithms many general purpose constraint solvers available
CSP model of planning
State-based model model each state by a collection of variables constraints enforce valid transitions between states
Example: logistics world variable for each package, truck, plane domains of packages: all locations, trucks, planes domains of trucks, planes: all locations
CSP model: constraints (I)
Action constraints model the effects of actions patterned after explanation closure axioms
State constraints variables within a state must be consistent
Improving model efficiency
Can add/remove/aggregate/decompose variables domain values constraints
Here: added hidden variables added redundant & symmetry-breaking constraints
CSP model: constraints (II)
Symmetric values constraints break symmetries on values variables can be
assigned Action choice constraints
break symmetries on equivalent permutations of actions
CSP model: constraints (III)
Domain constraints restrictions on original domains of variables
Capacity constraints bounds on resources
Distance constraints bounds on steps needed for a variable to change
from one value to another
Solving the CSP model
Given an instance of a planning problem generate a model with one step in it instantiate variables in the initial and goal states search for a solution (GAC+CBJ) repeat, incrementing number of steps, until plan
is found Properties:
forwards, backwards, or middle out planner sound, complete, guaranteed to terminate
Experiments
Five test domains from AIPS’98 Five planners:
CPlan Blackbox (Kautz & Selman) HSP (Bonet & Geffner) IPP (Koehler & Nebel) TLPlan (Bacchus & Kabanza)
Setup: machines: 400MHz Pentium II’s resources: 1 hour CPU time, 256 Mb memory
Gripper problems
CPlan BBox HSP IPP
1 0.0 0.1 0.0 0.0
2 0.0 5.7 0.1 0.4
3 0.1 0.1 7.9
4 0.2 0.2 100.4
5 0.3 0.3
6 0.5 0.4
20/20 2/20 20/20 4/20
Logistics problems
CPlan BBox HSP IPP
1 0.1 1.5 0.6
2 0.1 4.3 552.2
3 0.9
4 0.2
5 1.7 148.0 1.2 2.5
6 18.9
28/35 8/35 5/35 10/35
Mystery problems
CPlan BBox HSP IPP
1 0.0 0.1 0.1 0.1
2 0.0 4.2 7.1 11.4
3 0.0 0.4 0.4 0.9
4 0.0 1.2 0.4
5 0.0 11.4 7.6
6 3.1 134.0
30/30 19/30 16/30 24/30
MysteryPrime problems
CPlan BBox HSP IPP
1 0.1 0.6 0.1 2.3
2 0.1 4.4 13.9 23.9
3 0.2 0.6 0.5 4.4
4 0.2 0.6 0.6 2.5
5 0.2 7.2
6 2.3 201.8
35/35 18/35 25/35 21/35
Grid problems
CPlan BBox HSP IPP
1 0.7 8.1 1.1 3.0
2 33.4 4.3 9.4
3
4 1773.3 57.0
5
Related work
Planning as a CSP satisfiability (Kautz & Selman)
ILP (e.g., Bockmayr & Dimopoulos)
Adding declarative knowledge to improve efficiency hand-coded (e.g, Kautz & Selman, Bacchus &
Kabanza) automatically derived (e.g., Fox & Long, Nebel et al.)
Project: Resource-constrained instruction scheduling (I)
Modern architectures (VLIW) allow instruction level parallelism multiple functional units
Compiler to generate code that takes advantage of parallelism resource-constrained scheduling task
a: j i + hb: k i + gc: l j + 1
1: a, b2: c
Project: Resource-constrained instruction scheduling (II)
Current methods: heuristic recent: integer linear programming approach
Project: investigate constraint programming approach GNU compiler, MERCED chip
Project: Theoretical analysis of alternative models
Improve computational efficiency by completely changing the model change denotation of variables
– e.g, convert from non-binary to binary
aggregate variables– e.g, timetabling multiple sections of a course
Not much as has been done on either the theory or the practice side
Conclusions
Importance of the model ease of modeling is important form of model is important
Constraint programming advantages succinctness, declarativeness of models flexibility in specifying model on some classes of problems: speed
Conclusions
Advantages shared with CSP-like approaches expressiveness of modeling language declarativeness of models independence of model and solving algorithm
Advantages over other CSP-like approaches succinctness of models robustness: scales well, not as brittle speed
Future Work
Approximate planning heuristic CSP models solve same CSP models using local search
Alternative CSP models action-based models vs state-based models finding the right model can be key to solving
difficult combinatorial problems