Towards theoretical frameworks for comparing constraint satisfaction models and algorithms

Peter van Beek, University of WaterlooCP 2001 · Paphos, CyprusNovember 2001

2

Acknowledgements

Joint work with:

Fahiem Bacchus

Xinguang Chen

Grzegorz Kondrak

Toby Walsh

3

Constraint programming methodology

Model problem·specify in terms of constraints on acceptable solutions

·define/choose constraint model: variables, domains, constraints

Solve model·define/choose algorithm·define/choose heuristics

Verify and analyze solution

4

Previous work

Many models, algorithms, and heuristics proposed

Choice of model, algorithm, and heuristic can greatly influence efficiency

(e.g., Nadel, 1990; Ginsberg et al., 1990;

Frost and Dechter, 1994; Tsang et al., 1995 Smith et al., 2000; Beacham et al., 2001)

5

How to compare?

Empirical studies·benchmark sets·random problems

Theoretical studies·worst case·average case

(e.g. Brown & Purdom, 1981;

Nadel, 1983; Wilf, 1984)

·pair-wise comparisons-partial orders-bounded worse

Performance measures

·nodes visited·constraint checks·CPU time

6

Part I

Comparing backtracking algorithms

7

Some backtracking algorithms

Chronological Backtracking (BT)

Backjumping (BJ)(Gaschnig, 1978)

Conflict-Directed Backjumping (CBJ)(Prosser, 1993)

Forward Checking (FC)(Haralick & Elliott, 1980; McGregor, 1979)

Maintaining arc consistency (MAC)(Haralick & Elliott, 1980; McGregor, 1979; Sabin & Freuder, 1994)

8

Example: 4-queens

4

3

2

1

x1 x2 x3 x4

xi xj and

| xi - xj | | i - j |

9

Search tree for 4-queens

x1

x2x3

x4

1 2 3 4

(1,1,1,1) (4,4,4,4)(2,4,1,3) (3,1,4,2)

10

4

3

2

1

{x1 1}

Backjumping

Q

Q{x1 1, x2

3}

x1

x2

x1

x2

{x1 1, x2 3 , x4

2}

Q

x3

x1 x3 x4x2

11

“New” algorithms: BJ1, …, BJn

BJk :·allowed to backjump at most k times from a leaf·after that it must chronologically backtrack

Special cases:·BJ1 equivalent to BJ·BJn equivalent to CBJ

12

k-consistency

Given ·any k-1 distinct variables,·any consistent assignment to those variables,·any additional variable x

there exists a consistent assignment to x

4

3

2

1

x1 x2 x3 x4

2-consistent ?

Q

Q1-consistent ?

3-consistent ?strongly 2-

consistent

yes

yes

no

13

Maintaining consistency

{ x1 1, x2

4 }

4

3

2

1

x1 x2 x3 x4

Q

Qx3 x4

|x3 - x4| 1

Binary constraints

Unary constraintsx3 1, 3, 4

x4 1, 2, 4

Induced CSP

14

Maintaining consistency

4

3

2

1

x1 x2 x3 x4

Q

Q

{ x1 1, x2

4 }

x3 x4

|x3 - x4| 1

Binary constraints

Unary constraintsx3 1, 3, 4

x4 1, 2, 4

FC: strong 1-consistency

15

Maintaining consistency

Q

Q

{ x1 1, x2

4 }

x3 x4

|x3 - x4| 1

Binary constraints

Unary constraintsx3 1, 3, 4

x4 1, 2, 4

4

3

2

1

x1 x2 x3 x4

MAC: strong 2-consistency

Q

Q

16

“New” algorithms: MC1, …, MCn

MCk :·maintains strong k-consistency on induced CSP at each node in backtrack tree

Special cases:·MC1 equivalent to FC·MC2 equivalent to MAC for binary CSPs

17

Comparing backtracking algorithms

Proposal:for all CSP models for all variable orderings

algorithm a1 algorithm a2

Desirable features:·still holds if use best possible CSP model for a2

·still holds if use best possible variable ordering for a2

18

Theoretical methodology

Using local consistency concepts:·formulate necessary and sufficient conditions for a node to be visited by a backtracking algorithm

Using conditions:·construct partial orders of the algorithms

19

4

3

2

1

x1 x2 x3 x4

{x1 1}

{x1 1, x2

4}?

Example sufficient condition: FC

If a node is consistent and its parent is strongly 1-consistent, then FC visits the node

Q

Q

20

Example necessary condition: MAC-CBJ

If MAC-CBJ visits a node, then it is consistent and its parent is strongly 2-consistent

4

3

2

1

x1 x2 x3 x4

{x1 1}

{x1 1, x2

4}?

Q

21

Example conclusion: MAC-CBJ vs FC

FC visits p

MAC-CBJ visits p

p is consistent and parent(p) strongly 1-consistent

p is consistent and parent(p) strongly 2-consistent

22

Partial order for nodes visited

MCk+1-CBJ

MCk-CBJ

MC1-CBJ

MCn-CBJ

MCk+1

MCk

MC1

MCn

BJk+1

BJk

BJ1

BJn

BT

...

...

...

...

...

...

less than or equal

23

Partial order for nodes visited

MCk+1-CBJ

MCk-CBJ

MC1-CBJ

MCn-CBJ

MCk+1

MCk

MC1

MCn

BJk+1

BJk

BJ1

BJn

BT

...

...

...

......

...

incomparable

24

Partial order for nodes visited

BT

MCk+1-CBJ

MCk-CBJ

MC1-CBJ

MCn-CBJ

MCk+1

MCk

MC1

MCn

BJk+1

BJk

BJ1

BJn

...

...

...

......

...

incomparable

25

Partial order for nodes visited

MCk+1-CBJ

MCk-CBJ

MC1-CBJ

MCn-CBJ

MCk+1

MCk

MC1

MCn

BJk+1

BJk

BJ1

BJn

BT

...

...

...

...

...

...

incomparable

26

Partial order for nodes visited

MCk+1-CBJ

MCk-CBJ

MC1-CBJ

MCn-CBJ

MCk+1

MCk

MC1

MCn

CBJBT

...

......

...

27

Closer look: nodes visited, binary CSPs

MAC-CBJ

FC-CBJ

MAC

FC

CBJ

BJ2

BJ

BT

less than or equal

28

Closer look: constraint checks, binary CSPs

MAC-CBJ

FC-CBJ

MAC

FC

CBJ

BJ2

BJ

BT

less than or equal

at most O(f) more

O(nd)

O(n2d2)

O(nd)

29

Experiments: lookahead

MAC vs FC Faster? CommentsMcGregor (1979) FC 3 fasterHaralick & Elliott (1980) FC 3 fasterSabin & Freuder (1994) MAC much betterBacchus & van Run (1995) FC 3 20 fasterBessiere & Regin (1996) MAC much betterLarrosa (2000) both much better

Path vs MAC Faster? CommentsBacchus & van Run (1995) MAC much better

FC

MAC

O(nd)

Path

BT

O(nd)

O(nd)

30

Example: FC vs MAC

Adapted from J. Ullman (1988); suggested by D. De Haan

Odd-even relation

x y1 21 42 12 33 23 44 14 3

variables: x1, …, xn, n odd

domains: {1, 2, 3, 4}

x1 x2 xn...

FC: 2n+1 - 4

MAC: 4

31

Increasing lookahead

Problem MC2 MC3 MC4

1 > 3600 262 602 794 40 1403 7 3 34 1065 21 25 34 17 26 > 3600 125 37 58 6 48 17 3 29 163 95 210 87 4 2

Time (sec.) to solve 6 x 6 puzzlesMC4

MC3

MC2

MC1

32

Experiments: lookback

FC vs FC-CBJ Faster? CommentsProsser (1993) FC-CBJ three times betterFrost & Dechter (1994) FC-CBJ somewhat betterBacchus & van Run (1995) FC-CBJ slightlySmith & Grant (1995) FC-CBJ sometimes much betterBayardo & Schrag (1996, 1997) FC-CBJ much betterLi & Anbulagan (1997) FC as good or better

FC-CBJFC

33

Experiments: lookahead vs lookback

Increasing the level of consistency decreases effectiveness of CBJ

·Prosser (1993)·Bacchus & van Run (1995) ·Bessiere & Regin (1996)

MAC much better than FC-

CBJ

MAC better than MAC-CBJ

MAC-CBJMAC

FC-CBJFC

O(nd)

CBJBT

O(nd)

“CBJ becomes useless”

· Jussien, Debruyne, Boizumault (2000)MAC-DBT much better than MAC (binary)

·Chen & van Beek (2001)MGAC-CBJ much better than MAC (non-binary)

34

Theory: lookahead vs lookback

MCk+1-CBJ

MCk-CBJ

MCk+1

MCk

BJk+1

BJk

BJk+1 better only if there exists a (k+1)-level backjump that is not a chronological backtrack

Number of (k+1)-level backjumps number of k-level backjumps

Therefore, as k increases, effectiveness of backjumping decreases

35

Part II

Comparing CSP models

36

Conversion to binary

Any non-binary CSP can be converted into one with only binary constraints

·dual graph method (Dechter & Pearl, 1989; Rossi et al. 1989)

·hidden variable method (Peirce, 1933; Rossi et al. 1989; Dechter 1990)

37

Crossword puzzles

1 2 3

6

4

7

5

8

10

9

20

11

22

12

21

13

17

14

181615

23

19

aaardvarkabackabacusabaftabaloneabandon...

Mona Lisamonarchmonarchymonarda...zymurgyzyrianzythum

38

Non-binary model (original)

1 2 3

6

4

7

5

8

10

9

20

11

22

12

21

13

17

14

181615

23

19

variables: one for each

unknown letter (cell)

domains: ‘a’, …, ‘z’

constraints: contiguous

letters must form words in dictionary

39

Dual model (binary)

1 2 3

6

4

7

5

8

10

9

20

11

22

12

21

13

17

14

181615

23

19

variables: one for each unknown word across and down

domains: words from

dictionary

constraints intersecting words must agree on common letter

40

Hidden model (binary)

1 2 3

6

4

7

5

8

10

9

20

11

22

12

21

13

17

14

181615

23

19

variables: one for each unknown letter (cell) and

one for each unknown word across and down

domains: letters, words

constraints: letter and word

variable must agree

41

Comparing models

One proposal:for all backtracking algorithms for all variable orderings

model m1 model m2

Objections:·effectiveness of a model is algorithm dependent

·models may contain different variables

Desirable features:·meets objections·still holds if use best possible variable ordering for m2

Another proposal:given backtrack algorithm for all v2 for m2

there exists a v1 for m1 s. t. model m1 model m2

42

Theoretical methodology

Bound difference in number of nodes visited·compare pruning power of local consistencies of alternative models

(e.g., Debruyne & Bessiere, 1997;

Stergiou & Walsh, 1999; Prosser et al.,

2000; Walsh, 2000)·establish correspondence between nodes

Bound difference in cost at each node:·local consistency·variable ordering

43

Some relationships

MGAC-dual MGAC-hiddenMGAC-orig

FC-dual FC-hiddenFC-orig

BT-dual BT-hiddenBT-orig

polynomial not polynomial

44

Part III

Comparing variable ordering heuristics

Future work

45

Conclusion

Technical:·comparing backtracking algorithms

partial order·comparing CSP models

bounded worse relation on algorithm-model combinations

Big picture:·theory can guide, inform experimentation