Non-binary constraints

Toby WalshDept of CSUniversity of YorkEngland

Non-binary constraints

• BibliographyBacchus & van Beek, AAAI-98Chen, PhD thesis, UAlberta, 2000Dechter, AAAI-90Mohr & Masini, ECAI-88Regin, AAAI-94Stergiou & Walsh, AAAI-99 & IJCAI-99….

Non-binary constraints

• Major contribution to competitiveness of CP– despite academic focus on binary

CSPs!

• Some classic examples: all-different, cardinality, ...

Non-binary constraints

• Compact & declarative problem specifications– that often run with minimal “tweaking”

• Efficient algorithms for enforcing high levels of consistency– Regin’s specialized algorithm for a k-ary

all-different constraint– O(k^2d^2) compared compared to

O(d^k) for a generic algorithm

Non-binary constraints

• Comparison with binary constraints:– binary models without additional vars

(binary decompositions)– binary models with additional hidden

vars (binary encodings)

• Theoretical and empirical results

Binary decompositions

• Certain non-binary constraints decompose into binary constraints on same vars

• Sometimes called “network decomposable”

Binary decompositions

• Two examples:– all-different(x1,x2,x3) = x1<> x2,

x1<>x3, x2<>x3– monotone(x1,x2,x3) = x1 < x2 < x3

• One non-example:– even(x1+x2+x3)

Binary decompositions

• Theoretical comparison direct– compare pruning of vars in binary

decomposition with that in non-binary

• Empirical experiments reinforce theory– decomposing non-binary constraints can

add orders of magnitude to solution cost

Binary consistencies

• Arc-consistency (AC)– any assignment can be extended to a

second

• Path-consistency (PC)– any pair of assignments can be extended to

a third

• Strong path-consistency (strong PC)– AC and PC

Binary consistencies

• Path-consistency rarely used in practice– time complexity

– adds new (binary) constraints

• Path-consistency can be achieved with a (generalized) arc-consistency algorithm– see later

Binary consistencies

• Between AC and PC– inverse and singleton consistencies– just prune values (no additional

binary constraints)

• Experiments suggest can be useful– good pruning/time

Binary consistencies

• Path inverse consistency (PIC)– any assignment can be extended to

two more vars

• Neighbourhood inverse consistency (NIC)– any assignment can be extended to

all vars in immediate neighbourhood

Binary consistencies

• Singleton arc-consistency (SAC)– after any assignment, problem can be made

AC

• Restricted path-consistency (RPC)– problem is AC, and if any assignment is

consistent with just a single value for a 2nd var then any 3rd var has a consistent value

Binary consistencies

• Simple relationships between them

strongPC > SAC > PIC > RPC > AC

NIC > PIC

NIC ~ SAC

NIC ~ strongPC

Non-binary consistencies

• Generalized arc-consistency (GAC)– any assignment can be extended to

all other vars in same constraint

• Efficient algorithms exist for GAC on certain non-binary constraints– e.g. Regin’s all-different algorithm

Non-binary consistencies

• GAC v AC

{1,2}

{1,2} {1,2}

x1

x2 x3

x1<>x2, x1<>x3, x2<>x3 ACall-different(x1,x2,x3) not GAC

Non-binary consistencies

• GAC can be used to enforce path-consistency– given a binary CSP– construct a ternary CSP’– CSP is PC iff CSP’ is GAC

Non-binary consistencies

• Given CSP has:– vars xi with values a,b,c,...– binary constraints R(a,b)

• Construct CSP’ with:– vars xij with values <a,b>,...

– ternary constraints R’(<a,b>,<b,c>,<c,a>) iff R(a,b) & R (b,a) & R(c,a)

Non-binary consistencies

• Forward checking – forward checking (FC) on binary

constraints enforces limited form of AC– most recently instantiated var and all

futures vars are made AC

• Choices about how to generalize this to non-binary constraints

Non-binary consistencies

• nFC0– any constraint with one un-

instantianted var made AC

• nFC1– apply one pass of AC to each

constraint and projection involving current var and one future var

• … nFC5

Binary decompositions

• Upper and lower bound on FCnFC1 on non-binary > FC on

decomposition > nFC0 on non-binary

• Gaps can be exponentialConsider n-ary all-different with n-1 valuesnFC1 takes (n-1) branchesFC on decomposition takes (n-1)! branches

Binary decompositions

• GAC lower boundGAC on non-binary > AC on decompositionGap again can be exponentialBut if we decompose too much, GAC=AC!

• GAC upper boundIn general, GAC ~ NIC, GAC ~ PIC ..BUT if decomposition to clique, NIC > GAC

Binary decompositions

• Tighter results provable for stricter classes

• Tree decomposable constraints– constraint graph is tree

• Triangle preserving constraints– non-binary constraints on all triangles

Binary decompositions

• Tree decomposable constraints– e.g. monotone(x1,x2,x3)– GAC=AC– not surprising as AC enough to solve!

• Decomposition here doesn’t lose us anything– but even one cycle is enough for GAC>AC

Binary decompositions

• Triangle preserving decompositione.g. all-different(x1,x2,x3), quasigroups, ...GAC > PIC, gap can again be exponentialGAC ~ SAC, strongPC

• PIC is very strong consistency to be achieving at each node– GAC can do even better than this!– decomposition carries a very large price

Binary decompositions

• Experimental results– quasigroup completion– quasigroup existence

• Quasigroup is a Latin square– completion is completing partially

filled square– existence is finding one with

additional properties

Binary decompositions

• Modelling the quasigroup problem– n^2 vars, each with domain of size n

• Non-binary model– 2n all-different constraints (one for

each row and column

• Binary decomposition– 2n cliques of not-equals constraints

Binary decompositions

• Quasigroup completion– Gomes & Selman report “heavy-tailed”

distributions

• Maintaining AC on binary decomposition– problems often take long time to solve

• Maintaining GAC on all-different– almost all problems trivial

Binary decompositions

• Quasigroup existence– best paper at IJCAI-93– of interest to design theory

• Open results first proved by computer– in some cases, only ever proved by

computer

• Maintaining GAC very competitive– compared to specialized model finders like

FINDER, SEM

Intermission

• Mid-point of talk

• Short commercial breakCSPLib: a benchmark library for

constraintshttp://csplib.cs.strath.ac.uk

CSPLib: a benchmark library for constraints

http://csplib.cs.strath.ac.uk

A Why build it?

A What it contains?

A How to add to it?

Why build CSPLib?

• Benchmarking– shift field from random to more

realistic problems

• Modelling• Show-case• Challenges

Why not build CSPLib?

• Over-fitting– CSPLib needs to be large & extensible

• Unrepresentative– CSPLib needs your “realistic” problems

• No standard modelling language– CSPLib uses natural language

• Hard work– CSPLib needs to be a “community” effort

What does CSPLib contain?

• Over 20 problem types in 5 categories– bin packing (e.g. vessel loading)– scheduling (e.g. basketball tournament)– frequency assignment (e.g. Golomb rulers)– combinatorics (e.g. perfect square)– puzzles (e.g. nonagrams)

• Other useful links– solvers, other benchmark libraries, ...

What does CSPLib contain?

• Example entry:– NL specification– results– references– (data) for

instances– code– anything else of

use

“A Golomb ruler is a set of m integers,0=a1<a2<..<am such that the m(m-1)/2differences, aj-ai distinct. Such a ruleris of length am. The objective is to find the smallest length ruler…”

How to add to CSPLib?

• Send us your entries. NOW!– it takes just 1/2 hour

• Everyone has at least one problem to contribute– get others to solve your open problems– a problem shared is a problem solved

• Email problems to csplib@cs.strath.ac.uk

End of intermission

Now back to ...

Binary encodings

Binary encodings

• Not all non-binary constraint decompose into binary constraints– on the same set of variables

• Consider again– even(x1+x2+x3)

• But binary CSPs NP-complete

Binary encodings

• Every non-binary constraint can be encoded into binary constraints– using polynomial number of

additional (hidden) variables

• Two popular encodings– hidden variable encoding– dual encoding

Binary encodings

• Hidden variable encoding– add hidden var for each non-binary

constraint

• Dual encoding– add hidden var for each non-binary

constraint– throw away original variables

Binary encodings

• Dual encoding– consider c1:even(x1+x2),

c2:odd(x2+x3){00,11}

{01,10}

c1

c2

R21 R21=<00,01> or <11,10>

Binary encodings

• Hidden variable encoding– consider c1:even(x1+x2),

c2:odd(x2+x3)

{0,1}

{00,11}

{01,10}

{0,1} {0,1}x1 x2 x3

c1

c2

r1

r1

r2

r2r1=<0*,0> or <1*,1>r2=<*0,0> or <*1,1>

Binary encodings

• Hidden var encoding -> dual encoding– compose relations, discard original

vars

{0,1}

{00,11}

{01,10}

{0,1}x1 x3

c1

c2

R21

R21 = r2 + r1

Binary encodings

• Double encoding– dual + hidden var encoding– original vars + hidden vars– all constraints of dual and of hidden

• Also called “combined encoding”

Binary encodings

• Theoretical analysis complicated by:– encoding “builds” in GAC for hidden

vars– must translate between (original and

hidden) vars– pruning in dual can infer large arity

nogoods in original

Binary encodings

• Hidden var encoding

FC on hidden ~ nFC0 on original

each can be exponentially better than the other

• FC+ propogates through hidden vars

FC+ on hidden = nFC1 on original

Binary encodings

• Hidden var encoding

AC on hidden = GAC on original

• Before looking for efficient (specialized) GAC algorithm– try AC on hidden var encoding

Binary encodings

• No point doing NIC on hidden var encodingNIC on hidden = AC on hiddendue to star shaped topology of constraint

graph

• Higher consistencies remain distinctstrongPC on hidden > SAC on hidden > NIC, AC on hidden

Binary encodings

• Dual encodingFC on dual ~ nFC0 on originaleach can be exponentially better than

the other

• Dual better for tight constraints– domains for hidden vars then small

Binary encodings

• Dual encodingAC on dual > GAC on original

• BUT domains of hidden vars very large when non-binary constraints loose– AC on dual prohibitively expensive

Binary encodings

• Dual v hidden encodingBT on dual = FC on hiddenAC on dual > AC on hiddenSAC on dual > SAC on hiddenstrongPC on dual = strongPC on hidden

• Path consistency is enough to get through the hidden star!

Binary encodings

• Experimental results– crossword puzzles– Golomb rulers– random CSPs– random 3-SAT

Binary encodings

• Crossword puzzles• “Original” model

– vars for letters, domains {A-Z}

• Dual model– vars for words, domains = dictionary

• Dual at least 1,000 times faster on larger problems

Binary encodings

• Golomb ruler– set of ticks, xi on a ruler– all inter-tick distances, dij different

• Ternary constraints, dij=xi-xj– encoded using hidden var/double

encoding

• Competitive with non-binary model– runs on any binary CSP solver!

Binary encodings

• Random CSPs and SAT– Bacchus and van Beek plot contours to

show constraint tightness where encoding pays

– gives good predictions for performance on structured problems like crosswords, …

– number of satisfying tuples in constraint is most important factor in such predictions

Conclusions

• Non-binary v binary decompositions– GAC on non-binary can be stronger

than PIC on decomposition

• Non-binary v binary encodings– GAC on non-binary = AC on hidden– AC on dual > GAC on non-binary

Conclusions

• Non-binary v binary decompositions– decomposition can add significantly

to search cost

• Non-binary v binary encodings– encoding pays in practice on tight

constraints

Future directions

• Modelling with non-binary constraints– many more models possible– role of auxiliary vars and implied

constraints

• Different levels of consistency within a non-binay model– e.g. GAC on all-different, but bounds

consistency on arithmetic constraints

Future directions

• Mixed models– e.g. dual encoding for only some of

the (non-binary) constraints

• Optimization– objective function typically non-binary