thursday, 07 january 2016edward tsang (copyright)1 constraint satisfaction for decision making...

Friday 21 April 2023Edward Tsang (Copyright) 1

Constraint Satisfaction for Decision Making

Professor Edward TsangUniversity of Essex

URL: http://www.bracil.net/CSP/


Constraint Satisfaction

A non-technical introduction

PR CS LS F Friday 21 April 2023Edward Tsang (Copyright) 3

Sample Constraint Applications

• Resources Allocation– Staff rostering, timetabling– British Telecom’s work

force scheduling– British Airway’s flight

scheduling

• Transportation– ILOG Dispatcher for

vehicle routing– Train and bus scheduling– Collective transportation

• Logistics– Satellite scheduling– Radio links assignment

(military & mobiles)– Telephone Network

routing– Airport / container port

• Industrial Scheduling– Just-in-time scheduling– Hardware configuration– Car sequencing


Car Sequencing Problem

OptionsABSCD…

30 2030 40Total:120Production:

ABS area: 3/5 CD area: 2/3


Airline Applications

Constraint satisfaction and optimization are behind

many applications in airline operations; e.g.

British Airways uses ECLiPSe (IC-Parc) for

scheduling aircraft to serve their flights and ILOG

Solver for aircraft stand allocation at airports.


Transportation

Constraint satisfaction is one of the core technologies in transportation optimization. For example Guided Local Search was used in ILOG Solver’s vehicle routing package, Dispatcher. Cairo/Line schedule for collective transportation.


BT’s Workforce Scheduling

BT has many jobs to be done in UK every day. It has to

schedule a large number of teams to serve these jobs, subject to time, skill and

other constraints. Saving of 0.5% could mean Millions of

Pounds per year. Guided Local Search achieved the best results in one of BT’s

challenge problems.Technicians Jobs


CP in Steel Industry

IBM helped Korea’s biggest steel manufacturer to apply

constraint technology to schedule its production to

meet orders. This include the allocation of existing stocks to orders, subject to various

constraints, such as size, quality and colour, in order to minimize waste and cost.

Orders Steel Slabs


What is Constraint Satisfaction?• Constraint satisfaction is a decision problem

– You are given a number of decisions to make– For each decision, you are given all the choices– Decisions constrain each other

• Your task is to make those decisions without violating any of the constraints

• Sometimes you want the “best” solution– If so, you have a (constrained) optimisation problem

PR CS LS F

The Constraint Satisfaction Problem

• Constraint satisfaction is a decision problem

• Task: make decisions without violating constraints

• Sometimes you want the “best” solution

October 2003Edward Tsang (Copyright) 10

Variables(Decisions)

Domains (values available)

Constraints On assignments

x1

x2

x3

x4

X

X

X

X


Why constraint satisfaction?

• It is a very general problem (seen everywhere)– Mainly logistics, scheduling, resources allocation

• Specialized methods available

• Now multi-million Pounds/Dollars business

• Scientific challenges: – Combinatorial explosion (fundamental problem)– Modelling (engineering problem)


Combinatorial Explosion of the Car Sequencing Problem

• Schedule 30 cars:– Search space: 30 factorial 1032 leaf nodes

• Generously allow:– Explore one in every 1010 leaf nodes!

– Examine 1010 nodes per second!

• Problem takes over 32 thousand years to solve!!!– 1032 ÷ 1010 ÷ 1010 ÷ 60 ÷ 60 ÷ 24 ÷ 365 31,710

• How to contain combinatorial explosion?


Use The Force

• Constraints brought the problem here• Constraints guide us to solutions• Experts will follow the lead by constraints• See Guided Local Search later• The Incentive Method was used in financial

forecasting and bargaining• Reference to Daoism (Taoism)http://www.bracil.net/edward/library/Zhuangzi-cook

http://www.bracil.net/edward/library/Zhuangzi-cook


Constraint Satisfaction

A more technical account


Constraint Satisfaction:What is it all about?

• Many problems involve finding combinations of values to variables, e.g.– Telecom services scheduling– Deciding where to put warehouses

• Combinatorial problems are hard in nature• How big a problem can one solve?• Constraint research aims to extend our boundaries

in these problems– It’s all about algorithms, efficiency, problem solving …


Combinatorial Explosion

1

2

3

4

5

6

7

8

A B C D E F G H

1 2 4 8 16 32 64 128

1019

• Put 1 penny in square 1• 2 pennies in square 2• 4 pennies in square 3, etc.• Even the world’s richest man

can’t afford it– 1019 p = £100,000,000 Billion

0

500

1000

1500

2000

2500

Squares

1 2 3 4 5 6 7 8 9 10 11

Pennies


Problem Formulation(Modelling)

Is my problem a constraint satisfaction problem?

If so, I can apply established techniques


What is constraint satisfaction?

Given:• Z: a set of variable (Decisions)• D: mapping each variable x to a domain Dx (fixed

choices)• C: constraints restricting the values that variables

can take simultaneouslyTask:• Assign one value to each variable satisfying all

the constraints


1

2

3

4

5

6

7

8

A B C D E F G H

The 8-Queens Constraint Satisfaction Problem

• Task: to put 8 queens onto the board such that no queen attacks others

• No two queens appear in the same row, column and diagonal

Sample Solution

Constraint Techniques Overview

1. Problem Reduction– To reduce domain size

– To add or tighten constraints

– Aims:• To reduce a problem

to easier ones

• To detect dead-ends

• To achieve backtrack-free search

2. Search

• Systematic Search– Lookahead

– Learning nogoods or backtrack intelligently at dead-ends

– Ordering heuristics

Aim: to exhaustively search space

• Stochastic methods– To repair solutions heuristically

Aim: to find solutions fast, sacrificing completeness

3. (Solution Synthesis)

Constraint satisfaction

Constrained optimisation


1

2

3

4

5

6

7

8

A B C D E F G H

Backtracking Search In The 8-Queens Problem

• Place one queen per row

• Place one queen at a time

• Examine each column

Backtrack at dead-ends

X

Complete search, till solution found, or “no solution” is concluded


1

2

3

4

5

6

7

8

A B C D E F G H

Forward Checking Search

• Problem reduction – a major technique

• Combined with search methods

• Reduce domain of future variables

• Detect dead-ends – To backtrack early

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X X

X

X

X

X

X

X

X

X

X

XDead-end detected after Queen 4 – no legal space for row 6, backtrack…


Constraint Optimization

• Constraint Satisfaction & Optimization– Some solutions are better than others

• Partial Constraint Satisfaction– Accept best solution when no solution is found

• Relevant techniques:– Complete: Branch & Bound– Incomplete: HC, SA, Tabu, NN, GA, GLS


Problem Reduction for Problem Reduction for Constraint SatisfactionConstraint Satisfaction

Motivations:

1. Reduce problem to easier problems

2. Detect unsatisfiability


Problem Reduction Overview

• Node-consistency (NC)

• Arc-consistency (AC)

• Path-consistency (PC)

• (strong) k-consistency

• Directional Arc-consistency (DAC)

• Other Consistency Properties (not covered)– Directional PC, Adaptive consistency, ...


Node-consistency

• Principle of NC:– If unary constraint Cx exists for variable x– Remove any value v from Dx if v does not

satisfy Cx

• E.g. Dx: days of the week; Cx: this job must be done in weekdays

• Simple pre-processing strategy• Cheap to compute: O(n)


Maintaining AC, Example

• Variables: x, y, z

• Domains: {1, 2, 3, 4}

• Constraints: x < y ; y < z

x 1 2 3 4

y 1 2 3 4

z 1 2 3 4

• x<y means <x,4> not supported by y and <y,1> not supported by x

X

• y<z means <y,4> not supported by z and <z,1> & <z,2> not supported by y

X X

X X

• Re-check x<y would delete <x,3> as now (with <y,4> gone) it has no support from y

X


Example: AC but Unsatisfiable

• This problem is AC

• But unsatisfiable

• Can we detect unsatisfiability?

X

Y Z

{r, g}

{r, g} {r, g}


Path-consistency

• Principle: – Tightening constraints– By constraint composition

• Algorithms: PC-1, …, PC-4

• Complexity manageable: O(a3n3)

• Is effort justifiable?


Binary Constraint Representation

• A Binary constraint may be represented by a matrix, e.g.:

• Dx = Dy = Dz = {1, 2, 3}• Cxy : X + Y must be even• Cxz : at least one of X and Z

must be equal to 2

x

y z

{1, 2, 3}

{1, 2, 3} {1, 2, 3}Ev

en(X

+Y)

Even(Y+Z)

X=2 Z=2

010

111

010

101

010

101

,,, ZXZYYXCCC


Constraint Composition

• Constraint Cxy & Cyz could tighten Cxz

• Cxz = Cxz Cxy * Cyz

x

y z

{1, 2, 3}

{1, 2, 3} {1, 2, 3}Ev

en(X

+Y)

Even(Y+Z)

X=2 Z=20 1 0

1 1 10 1 0

1 0 10 1 01 0 1

1 0 10 1 01 0 1

*=

1 0 10 1 01 0 1

0 0 00 1 00 0 0

==0 1 01 1 10 1 0


PC Detecting Unsatisfiability• Given (<x,r><y,g>),

which satisfies Cx,y

• It has no compatible value in z

• Hence not PC

x

y z

{r, g}

{r, g} {r, g}

01

10,,, zxzyyx

CCC

01

10

01

10

01

10

00

00

10

01

01

10

yzzxyxyxCCCC

,,,,*


Example: PC but Unsatisfiable

• Problem is PC

• But unsatisfiable

• Can we detect unsatisfiability?

Z

{r, g, b}

{r, g, b}

X

{r, g, b}

W

Y

{r, g, b}


Problem Reduction, Summary

• Aim: reduce problem to one that is easier to solve, or detect dead-ends

• Concept is simple, procedures could be complex• General Concept: k-consistency

– NC1-C, AC2-C, PC3-C for binary CSPs

• NC & AC potentially reduce domains• PC potentially tightens binary constraints• k-C for k>2 potentially tightens k-1 constraints


Complete Search Algorithms Complete Search Algorithms for Constraint Satisfactionfor Constraint Satisfaction

Lookahead

Gather-information-while-searching


Complete Search Strategies Overview

• Preprocessing– Problem reduction; problem transformation

• General Search– Chronological backtracking

• Lookahead– Forward Checking, DAC- & AC-Lookahead

• Gather-information-while-searching– DDBT, Learning Nogoods, Backmarking

• Hybrids, e.g. FC-BM-CBJ


Pre-processing – before search begins

• Problem reduction– 1-consistency is always worth achieving– How about 2-consistency? 3-consistency?

• Problem transformation– Should redundant constraints be added?

• With A < B, B < C, would adding A < C help?

– Should the problem be decomposed into sub-problems?

• Some sub-problems may be tractable • E.g. cycle-cutset


1

2

3

4

5

6

7

8

A B C D E F G H

Backtracking Search In The 8-Queens Problem

• Place one queen per row

• Place one queen at a time

• Examine each column

Backtrack at dead-ends

X

Complete search, till solution found, or “no solution” is concluded


Lookahead Algorithms

• Lookahead so as to detect failure asap

• Reduce remaining problem

• More reduction requires more computation

• Does marginal gain justify marginal cost?

• MAC-Lookahead is effective in general– Key: to record support in maintaining AC


Forward Checking

• Algorithm FC:– Label one variable at a time

– After each label <x,vx> is committed to:• examine every future variable y

• remove every value vy from Dy such that <y,vy > is incompatible with <x,vx>

– Backtrack if any future domain becomes empty

• Found to be quite effective in general


1

2

3

4

5

6

7

8

A B C D E F G H

Forward Checking Search

• Problem reduction – a major technique

• Combined with search methods

• Reduce domain of future variables

• Detect dead-ends – To backtrack early

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X X

X

X

X

X

X

X

X

X

X

XDead-end detected after Queen 4 – no legal space for row 6, backtrack…


Stochastic SearchStochastic SearchAn element of randomness and statistics

Hill Climbing: Min-conflict, GSAT*

Metaheuristic Search: GENET, GLS*, GGA*

* Note: not in Tsang 1993


Why Stochastic Search?

• Schedule 30 jobs to 10 machines:– Search space: 1030 leaf nodes

• Generously allow:– Explore one in every 1010 leaf nodes!– Examine 1010 nodes per second!

• Problem takes 300 years to solve!!!

• How to contain combinatorial explosion?


Background: Local Search

• Ingredients:– Cost function– Neighbourhood

function– [Optional] Strategy

for visiting neighbours

• e.g. steepest ascent

• Problems:– local optimum– Plateau– When to stop?

• Ok with satisfiability • But not optimization

local max.global max.

Cost function

plateau

neighbourhood

Assume maximization problem


HC Example: 2-opting for TSP

• Candidate tour: a round trip route

• Neighbour: exchange two edges, change directions accordingly

AB

C

E D

AB

C

E D


List reversing 2-Opting

• List representation:– A list could represent cities in sequence

• 2-Opting can be seen as sub-list reversing– Easy to implement

1 3 654 8 2 7

Breaking points

1 3 684 5 2 7


Min-conflict Heuristic Repair

• Start with a random complete assignment– May initialise with min-conflict heuristic

• Repeat until all constraints are satisfied or run out of resources:– Randomly pick a variable x that is in conflict– Pick value v in domain of x such that

• <x, v> violates the least number of constraints

• break ties randomly


1

2

3

4

5

6

7

8

A B C D E F G H

MC Heuristic Repair, Example• Start with random

assignments• C2 attacks G6

D8 attacks E7• Randomly pick one,

say, E7, to repair

Count number of conflicts in each square

Randomly pick a square with least attacks, say, B7 1 1 3 3 1 2 2 3

Repeat repair

X

X2 1 1 2 2 4 3 2

X

Solution found


Meta-heuristics

Hill Climbinge.g. 2-Opt in TSP

• Changing hill climbing behaviour

• mainly to escape local optima

Tabu Search GENET, GLSSimulated Annealing

(GGA)

thursday, 07 january 2016edward tsang (copyright)1 constraint satisfaction for decision making...

Documents

constraint satisfactiona

constraint technology

decision problemtask

decision problemyou

number of decisions

scheduling aircraft

transportation optimization

best results