cs236501 introduction to ai tutorial 5 automatic generation of heuristic functions tutorial 5...

CS 236501Introduction to AI

CS 236501Introduction to AI

Tutorial 5Automatic Generation of Heuristic Functions

Tutorial 5Automatic Generation of Heuristic Functions

Problem DefinitionProblem Definition

• Most search algorithms need a heuristic function to guide them in the search space

• Writing a good heuristic function is hard:– Extensive knowledge about the problem is needed– The distance to the goal may be hard to estimate

(especially when variables have nominal values)– It is hard to measure how informative a heuristic

function is– We mostly want admissible heuristics

• We sometimes want a heuristic function for an unknown problem, or a problem that changes with time

Apr 21, 2023

Intro. to AI – Tutorial 5 – By Haggai Toledano 2

Proposed SolutionProposed Solution

• Automatically generate a heuristic function– Should be a problem and domain independent method– Can be state specific– Should produce informative, preferably admissible

heuristics– Should be fast in practice and preferably have low

theoretical complexity

• Such heuristics generally use a simplified version of the original problem, the (optimal) solution to which serves as the heuristic value for the original states

• Most ideas come from the field of automated planning

Apr 21, 2023


A useful formal definition of search problems


• A search problem P=<S,A,I,G> can be defined in the following terms:– S – The search space. S can be defined as a set of

variables V={v1,v2,…,vn} with value domains D={d1,d2,…,dn} such that each variable vi has an associated domain di, and a state s∊S is an element of the space defined by d1 X d2 X … X dn (alternatively this can be viewed as a complete assignment for the variables, i.e. s∊S={<vi,di>}i=1..n).

– I – The initial state. I is an element of S.– G – The goal state. G is a partial element of S, i.e.

G⊆{<vi,di>}i=1..n. When a state s∊S contains G, we say that s is in a goal state.

Apr 21, 2023




– A – The set of allowed actions (or operators). This is a function from S to S.An action <pre⊆{<vi,di>}i=1..n,eff⊆{<vi,di>}i=1..n>∊A takes as input a state s∊S, and if it meets the preconditions pre (i.e. the variables that pre specifies have the values that pre specifies), then the effect eff takes place (i.e. the variables that eff specifies are set to the values that eff specifies for them). Variables not specified in pre are “don’t cares” and the values of variables not in eff are copied from the input state to the output state.

– The set A*(I) is the feasible state space.– Note: Each action a∊A is a symbolic manipulation on

the values of the variables of a state s∊S.

Apr 21, 2023


RelaxationRelaxation

• We will consider problem relaxation– A relaxation P’ of problem P is achieved by deleting

some constraints present in P– Other methods include: Over constraining, Analogy,

and Probabilistic models

• Example: P = 8-Puzzle– Goal: Order tiles– Action: Swap(x, y) when

tile x is H/V-adjacent totile y, and either x is emptyor y is empty

Apr 21, 2023


1 2 3

4 5 6

7 8

8-Puzzle – The original problem8-Puzzle – The original problem

• The optimal solution of the original (non-relaxed) problem is:

Apr 21, 2023


h*=9

2 4

1 5 3

7 8 6

8-Puzzle – Relaxation I8-Puzzle – Relaxation I

• Tiles can move to any location (whether occupied by another tile or not)– This is the number of misplaced tiles heuristic

Apr 21, 2023


2

1 5 3

7 8 6

h=5

4

8-Puzzle – Relaxation II8-Puzzle – Relaxation II

• Tiles can move to any adjacent location (whether occupied by another tile or not)– Summing over all tiles, this is Manhattan Distance

Apr 21, 2023


h=7

2

1 5 3

7 8 6

4

8-Puzzle – Relaxation III8-Puzzle – Relaxation III

• Tiles can move to the empty location (even if it is not adjacent to them)– Optimal solution: Sort (place a misplaced target tile in

the empty place; prefer the target of that place)

Apr 21, 2023


h=5

2

1 5 3

7 8 6

4

Relaxations – UsageRelaxations – Usage

• Receive a problem P=<S,A,I,G>• Create P’=<S’,A’,I’,G’>, the relaxed version of P• Run your favorite heuristic algorithm on P

– For each s∊S, starting with I, get the successors succs

of s by applying all applicable actions from A.

– For each t∊succs compute the optimal solution of P’ from t’ to G’. The solution value (length, cost etc.) will be denoted by h(t).

– Use h(succs) as the heuristic function for the search algorithm.

• Since P’ is only a symbolic manipulation of P, it is problem and domain independent

Apr 21, 2023


h+ Relaxationh+ Relaxation

• A general method for creating a certain type of relaxations:– State variables accumulate values instead of merely

changing their value – never forget achieved values

• Simplifies the original problem – less constraints: – Allows “jumps” between previously unlinked states– In effect: Adds edges to the states transition graph– Preserves preconditions– Computation: Uninformed, complete search, e.g. BFS

• Monotonic: The original actions are still applicable to the states they were applicable to beforehand

• Admissible

Apr 21, 2023


8-Puzzle – h+8-Puzzle – h+

Apr 21, 2023


X2 4

1 5 3

7 8 6

2X2X 4

1 5 3

7 8 6

24X2X 4X

1 5 3

7 8 6

24X12X 4X

1X 5 3

7 8 6

24X12X 34X

1X 5 3X

7 8 6

24X124X

34X

1X 5 3X

7 8 6

0

3

1 2

54

4

8-Puzzle – h+8-Puzzle – h+

Apr 21, 2023


24X124X

34X

1X 5 36X

7 8 6X

24X124X

34X

14X 5 36X

7 8 6X

24X124X

34X

14X 5 36X

7 8 6X

h+ = 7h* = 9

6 7

4

h+ Relaxation – Other problemsh+ Relaxation – Other problems

• Lightz Out

Apr 21, 2023


..

On

Off

On, Off

..On

OffP

P+

Goal!


• City Map– I = <Boston>– succ(I) = {<Quebec>, <New York>}– succ+(I)= {<Boston, Quebec>,<Boston, New York>}– succ+(<Boston, New York>) =

<Boston, New York, Quebec>)

Apr 21, 2023


Boston

QuebecNew York

Boston

Quebec

New York


• Missionaries and Cannibals– State: <ML,CL,MR,CR,B>∊NxNxNxNx{right,left}, where

N={0,1,2,3}– I = <3,3,0,0,L> ; G = <0,0,3,3>– A = {CCR = [move 2 cannibals from L to R] =

<pre={CL≥2,B=left,MR≥CR+2},

eff={CL=CL-2,CR=CR+2,B=right}>, MMR, CML, … }

– CCR(I=<3,3,0,0,L>) = <3,1,0,2,R>

– CCR+ = <pre={CL≥2,B=left,MR≥CR+2},

eff={CL=CL U CL-2,CR=CR U CR+2,B=B U right}>

– CCR+(I+=<{3},{3},{0},{0},{L}>)= <{3},{1,3},{0},{0,2},{L,R}>

Apr 21, 2023


h+ Relaxation: Complexityh+ Relaxation: Complexity

• Although the +-relaxed problem seems simpler to solve, its complexity might still be too high

• Proof (Bylander ’94):– Propositional planning is a type of search problem– PLANMIN(k) is the decision problem of determining

whether an instance of a propositional planning problem is satisfiable in k or less actions

– PLANMIN(k)+ is PLANMIN(k) restricted to positive

postconditions (can only add facts, can’t delete them)– PLANMIN(k) is PSPACE complete

– PLANMIN(k)+ is NP complete (reduction from 3SAT)

• Solution: Approximate h+

Apr 21, 2023


h+ Approximation I: hmaxh+ Approximation I: hmax

• hmax – Find the longest path to any goal value for the relaxed problem– Build a layered planning graph:

• Start from the initial state• At each layer apply all applicable actions

– The first layer to hold all of the goal variables in their goal values is the approximated solution length

• Properties:– Not very informative (too optimistic; no mutex

detection)– Admissible

Apr 21, 2023


8-Puzzle – hmax8-Puzzle – hmax

Apr 21, 2023


X2 4

1 5 3

7 8 6

245X

2X 4X

1 5X 3

7 8 6

245X

1245X

2345X

1X123458X

3X

7 8X 6

123458X

1245X

12345X

1234578X

123458X

1234568X

7X12345678X

6X

123458X

1245X

12345X

1234578X

123458X

1234568X

7X12345678X

6X

0 1 2

3

h+ = 7h* = 9

hmax = 3

hmax

= max(2(1), 1(2), 2(3), 3(4), 0(5),

3(6), 0(7), 0(8)) = 3

h+ Approximation II: haddh+ Approximation II: hadd

• hadd – Sum over the costs to achieve all goal variables values

• Compute just like hmax but sum the costs instead of finding their maximum

• Assumes total independence of goal variables• Properties:

– Informative in problems where the goal variables have little interaction with one another

– Not admissible – often over estimates

Apr 21, 2023


8-Puzzle – hadd8-Puzzle – hadd

Apr 21, 2023


X2 4

1 5 3

7 8 6

245X

2X 4X

1 5X 3

7 8 6

0 1 2

3

h+ = 7h* = 9

hmax = 3hadd = 11

hadd

= 2(1)+1(2)+2(3)+3(4)+0(5)+3(6)+0(7)+0(8) = 11

245X

1245X

2345X

1X123458X

3X

7 8X 6

123458X

1245X

12345X

1234578X

123458X

1234568X

7X12345678X

6X

123458X

1245X

12345X

1234578X

123458X

1234568X

7X12345678X

6X

h+ Approximation III: hFFh+ Approximation III: hFF

• hFF – Fast Forward: The length of a non-optimal solution to the relaxed problem:– Build the planning graph– Backchain some solution (i.e. necessary actions) from

the last (goal) layer to the first (initial state):• For each goal in layer i, set its preconditions as goals for

layer i-1• Select actions that were applied on layer i-1 and contain

the goals from layer i as their combined postconditions• Repeat until reaching the initial graph layer

– The solution’s length (i.e. the sum of the number of uniquely selected actions for each layer) is the heuristic value for the initial state

Apr 21, 2023


h+ Approximation III: hFFh+ Approximation III: hFF

• Properties:– Takes into account positive interactions– Not admissible– Rarely over estimates

– Uses the fact that while PLANMIN(k)+ is NPC, PLANSAT+

(relaxed satisfiablity) is in P (so if the goal cannot be reached – h+ is unsolvable; if the goal is reached – h+ is solvable and a witness can be constructed)

– Note that solving for the minimal number of actions in each layer is NPC (reduction from SC)

• The original FF algorithm uses enforced hill-climbing (use BFS to escape local minima)

Apr 21, 2023


h+ and its approximationsh+ and its approximations

• Problem:– Variables: x∊{x1,x2,x3}, y∊{y1,y2,y3} , z∊{z1,z2}

– Actions (<prepost>): a1=<x1x2>, a2=<x2x3,y2>, a3=<x2x3>, a4=<x2y2>, a5=<y2y3>, a6=<z1z2>

– Initial state: {x=x1, y=y1, z=z1}

– Goal state: {x=x3, y=y3, z=z2}

Apr 21, 2023


x1

y1

z1

x1x2 x1,x2

y1

z1,z2

x1,x2,x3

y1,y2

z1,z2

y2y3

x1,x2,x3

y1,y2,y3

z1,z2z1z2

x2x3

x2y2

hmax=max{2(x),3(y),1(z)}=3

hadd=sum{2(x),3(y),1(z)}=6

hFF=2(a1,a6)+2(a3,a4)+1(a5)=5

h+=1(a1)+1(a2)+1(a5)+1(a6)=4

x2x3,y2

= h*

8-Puzzle – hFF8-Puzzle – hFF

Apr 21, 2023


X2 4

1 5 3

7 8 6

245X

2X 4X

1 5X 3

7 8 6

0 1 2

3

h+ = 7h* = 9

hmax = 3hadd = 11hFF = 9

hFF = by a certain choice

of actions = 9

245X

1245X

2345X

1X123458X

3X

7 8X 6

123458X

1245X

12345X

1234578X

123458X

1234568X

7X12345678X

6X

123458X

1245X

12345X

1234578X

123458X

1234568X

7X12345678X

6X

8-Puzzle – hFF backchaining8-Puzzle – hFF backchaining

Apr 21, 2023


1(1)

2(2)

3(3)

4(4)

5(5)

6(6)

7(7)

8(8)

1(1)

2(2)

3(3)

4(1)

X(4)

5(5)

6(9)

X(6)

7(7)

8(8)

1(4)

X(1)

2(2)

3(6)

X(3)

4(2)

X(1)

X(5)

5(5)

6(9)

7(7)

8(8)

1(4)

2(1)

X(2)

4(3)

5(5)

6(9)

3(6)

7(7)

8(8)

Unique non-noop actions: 3+4+2 hFF = 9badgood

noopaction

1(4)

X(1)

2(2)

3(6)

X(3)

4(2)

X(1)

X(5)

5(5)

6(9)

7(7)

8(8)

Pattern Database RelaxationPattern Database Relaxation

• The Pattern Database (PDB) relaxation method maps a problem P to a relaxed problem P’ by ignoring some aspects of the states– Option I: Variables deletion: Use only a subset of the

variables from P (but take care to always include a goal variable in this subset)

– Option II: Values domain abstraction: Use only a subset of the domain for some variables

• PDB only clusters states that share some properties (rather than arbitrary states)

• We now have a set of patterns of states– The new state(+actions) space is homomorphic to the

original state space

Apr 21, 2023



Apr 21, 2023


Use this pattern


• How to solve using the state patterns:– For each state, optimally solve the abstracted problem

and use the solution length as the heuristic value of the original state

– Construct a database of solutions for all abstracted states in advance (rather than on the fly)

• This method guarantees admissibility: All of the abstracted solutions are shorter than the optimal solution of the original problem

Apr 21, 2023



Apr 21, 2023


Use this pattern

h* = 3

h = 2

8-Puzzle – PDB Relaxation8-Puzzle – PDB Relaxation

Apr 21, 2023


2 4

1 5 3

7 8 6

tile place

1 4

2 1

3 6

4 3

5 5

6 9

7 7

8 8

tile place

1 4

2 1

4 3

6 9

7 7

8 8

2 4

1

7 8 6

21

4 6

7 8

Goalh = 6

tile place

1 4

2 1

3 6

4 3

5 5

6 9

7 7

8 8

1 2 3

4 5 6

7 8 9

1 2 3

4 5 6

7 8 9

1 2 3

4 5 6

7 8 9

Example action:<5(5), X(2) → 5(2)> <X(2) → Ø> =eff Ø<2(5), X(6) → 2(6)> <2(5), X(6) → 2(6)>


• Representation is important for understanding the patterned state space – In the last example, the representation was: “Where is

tile X located”. Eliminating tiles 3 and 5 adds two extra empty places on the board

– Using the representation “What tile does place X hold” deletes places, creating strange actions like “place y holds x ↔ place y holds empty”, allowing multiple instances of a tile and allowing the empty board – all of these do not impact admissibility

– However “What tile does place X hold” is a more intuitive representation for values domain abstraction: We could place the empty tile on several placeholders simultaneously

Apr 21, 2023


tile place

1 1

2 2

3 3


• PDB is not necessarily surjective: The new state space often contains states that have no mapping from the original state space (may affect search speed but not admissibility)

• Example: 3-Puzzle

Apr 21, 2023


This state has no source in the original

state space

tile place

1 1

2 2

3 3

1

3

21 2

3 4

1 21 2

3 4

2 11 2

3 4


• Very important subject (which we will not discuss): How to choose which variables to delete– Always keep at least one goal variable– Possible strategy: Try to decompose the problem into

as many independent subproblems as possible while retaining as many variables as possible

Apr 21, 2023


SummarySummary

• The solution to a relaxed problem can be used as a heuristic for the original problem

• It is preferable to use an admissible heuristic when possible

• Some (crucial) aspects of the automated process of heuristic functions generation still need to be taken care of manually

Apr 21, 2023


cs236501 introduction to ai tutorial 5 automatic generation of heuristic functions tutorial 5...

Documents

ai tutorial

state ss

goal state

input state

state specificshould

output state

initial state

values of variables