Local SearchLocal Search
Introduction toIntroduction toArtificial IntelligenceArtificial Intelligence
COS302COS302
Michael L. LittmanMichael L. Littman
Fall 2001Fall 2001
AdministrationAdministration
Questions?Questions?
Reading: Boyan thesis for more Reading: Boyan thesis for more information on local search.information on local search.
http://www-2.cs.cmu.edu/afs/cs.cmu.http://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/jab/mosaic/thesis/edu/user/jab/mosaic/thesis/
chap3.ps.gz, appen2.ps.gzchap3.ps.gz, appen2.ps.gz
SearchSearch
Recall the search algorithms for Recall the search algorithms for constraint problems (CSP, SAT).constraint problems (CSP, SAT).
• Systematically check all states.Systematically check all states.• Look for satisfied state.Look for satisfied state.• In finite spaces, can determine if no In finite spaces, can determine if no
state satisfies constraints state satisfies constraints (complete).(complete).
Local SearchLocal Search
Alternative approach:Alternative approach:• Sometimes more interested in Sometimes more interested in
positive answer than negative.positive answer than negative.• Once we find it, we’re done.Once we find it, we’re done.• Guided random search approach.Guided random search approach.• Can’t determine unsatisfiability.Can’t determine unsatisfiability.
Optimization ProblemsOptimization Problems
Framework: States, neighbors Framework: States, neighbors (symmetric). Add notion of (symmetric). Add notion of scorescore for states: V(s)for states: V(s)
Goal: Find state w/ highest Goal: Find state w/ highest (sometimes lowest) score.(sometimes lowest) score.
Generalization of constraint Generalization of constraint framework because…framework because…
SAT as OptimizationSAT as Optimization
States are complete assignments.States are complete assignments.
Score is number of satisfied clauses.Score is number of satisfied clauses.
Know the maximum (if formula is Know the maximum (if formula is truly satisfiable).truly satisfiable).
Local Search IdeaLocal Search Idea
In contrast to CSP/SAT search, work In contrast to CSP/SAT search, work with complete states.with complete states.
Move from state to state Move from state to state remembering very little (in remembering very little (in general).general).
Hill ClimbingHill Climbing
Most basic approachMost basic approach• Pick a starting state, s.Pick a starting state, s.• If V(s) > max If V(s) > max s’ in N(s) s’ in N(s) V(s’), stop V(s’), stop• Else, let s = any s’ s.t.Else, let s = any s’ s.t.
V(s’) = max V(s’) = max s’’ in N(s) s’’ in N(s) V(s’’) V(s’’)• Repeat Repeat Like DFS. Stops at local maximum.Like DFS. Stops at local maximum.
N-queens OptimizationN-queens Optimization
Minimize violated constraints.Minimize violated constraints.
Neighbors: Slide in column.Neighbors: Slide in column.1
10
0
0
02
2
N-queens Search SpaceN-queens Search Space
How far, at most, from minimum How far, at most, from minimum configuration?configuration?
Never more than n slides from any Never more than n slides from any configuration.configuration.
How many steps, at most, before How many steps, at most, before local minimum reached?local minimum reached?
Maximum score is n(n-1).Maximum score is n(n-1).
Reaching a Local MinReaching a Local Min
1
10
0
00
22
1
00
0
10
11
00
0
10
00
1
Graph PartitioningGraph Partitioning
Separate nodes into 2 groups to Separate nodes into 2 groups to minimize edges between themminimize edges between them
A B
CD
Objective FunctionObjective Function
Penalty for imbalancePenalty for imbalance
V(x) = cutsize(x) + (L-R)V(x) = cutsize(x) + (L-R)22
State space? Neighbors?State space? Neighbors?
State SpaceState Space
0000: 16+00000: 16+0 1000: 4+21000: 4+2
0001: 4+30001: 4+3 1001: 0+31001: 0+3
0010: 4+10010: 4+1 1010: 0+31010: 0+3
0011: 0+20011: 0+2 1011: 4+21011: 4+2
0100: 4+20100: 4+2 1100: 0+21100: 0+2
0101: 0+30101: 0+3 1101: 4+11101: 4+1
0110: 0+30110: 0+3 1110: 4+31110: 4+3
0111: 4+20111: 4+2 1111: 16+01111: 16+0
0000: 16+00000: 16+0 1000: 4+21000: 4+2
0001: 4+30001: 4+3 1001: 0+31001: 0+3
0010: 4+10010: 4+1 1010: 0+31010: 0+3
0011: 0+20011: 0+2 1011: 4+21011: 4+2
0100: 4+20100: 4+2 1100: 0+21100: 0+2
0101: 0+30101: 0+3 1101: 4+11101: 4+1
0110: 0+30110: 0+3 1110: 4+31110: 4+3
0111: 4+20111: 4+2 1111: 16+01111: 16+0
Bumpy LandscapeBumpy Landscape
1
2
3
4
5
6
1 6 11 16
State
Sc
ore
Simulated AnnealingSimulated Annealing
Jitter out of local minima.Jitter out of local minima.
Let T be non-negative.Let T be non-negative.
Loop:Loop:• Pick random neighbor s’ in N(s)Pick random neighbor s’ in N(s)• Let D = V(s) – V(s’) (improve)Let D = V(s) – V(s’) (improve)• If D > 0, (better), s = s’If D > 0, (better), s = s’• Else with prob e Else with prob e D/TD/T , s = s’ , s = s’
TemperatureTemperature
As T goes to zero, hill climbingAs T goes to zero, hill climbing
As T goes to infinity, random walkAs T goes to infinity, random walk
In general, start with large T and In general, start with large T and decrease it slowly.decrease it slowly.
If decrease If decrease infinitelyinfinitely slowly, will find slowly, will find optimal solution.optimal solution.
Analogy is with metal cooling.Analogy is with metal cooling.
Aside: AnalogiesAside: Analogies
Natural phenomena inspire algsNatural phenomena inspire algs• Metal coolingMetal cooling• EvolutionEvolution• ThermodynamicsThermodynamics• Societal marketsSocietal markets• Ant coloniesAnt colonies• Immune systemsImmune systems• Neural networks, …Neural networks, …
Annealing ScheduleAnnealing Schedule
After each step, update TAfter each step, update T
• Logarithmic: TLogarithmic: Tii = = /log(i+2)/log(i+2)
provable properties, slowprovable properties, slow
• Geometric: TGeometric: Tii = T = T00 int(i/L)int(i/L)
lots of parameters to tunelots of parameters to tune• AdaptiveAdaptive
change T to hit target accept ratechange T to hit target accept rate
Lam ScheduleLam Schedule
Ad hoc, but impressive (see Boyan, Ad hoc, but impressive (see Boyan, pg. 190, 191). Parameter is run pg. 190, 191). Parameter is run time.time.
• Start target accept rate at 100%Start target accept rate at 100%• Decrease exponentially to 44% at Decrease exponentially to 44% at
15% through run15% through run• Continue exponential decline to 0% Continue exponential decline to 0%
after 65% through run.after 65% through run.
Locality AssumptionLocality Assumption
Why does local search work? Why Why does local search work? Why not jump around at random?not jump around at random?
We believe scores change slowly We believe scores change slowly from neighbor to neighbor.from neighbor to neighbor.
Perhaps keep a bunch of states and Perhaps keep a bunch of states and search around the good ones…search around the good ones…
Genetic AlgorithmsGenetic Algorithms
Search : Population genetics ::Search : Population genetics ::
State : individual ::State : individual ::
State set : population ::State set : population ::
Score : fitness ::Score : fitness ::
Neighbors : offspring / mutationNeighbors : offspring / mutation
? : sexual reproduction, crossover? : sexual reproduction, crossover
Algorithmic OptionsAlgorithmic Options
Fitness functionFitness function
Representation of individualsRepresentation of individuals
Who reproduces? *Who reproduces? *
How alter individuals (mutation)?How alter individuals (mutation)?
How combine individuals How combine individuals (crossover)? *(crossover)? *
RepresentationRepresentation
““Classic” approach: states are fixed-Classic” approach: states are fixed-length bit sequenceslength bit sequences
Can be more complex objects: Can be more complex objects: genetic programs, for example.genetic programs, for example.
Crossover:Crossover:• One point, two point, uniformOne point, two point, uniform
GA ExampleGA Example
110111011111 101110111313 0100010022 1001100199
11011101 10111011 10111011 10011001
11111111 10011001 10011001 10111011
11111111 00010001 10001000 10111011
reproduction
crossover
mutation
Generation to NextGeneration to Next
G is a set of N (even) statesG is a set of N (even) states
Let p(s) = V(s)/sum Let p(s) = V(s)/sum s’s’ V(s’), G’ = { } V(s’), G’ = { }
For k = 1 to N/2For k = 1 to N/2
choose x, y with prob. p(x), p(y)choose x, y with prob. p(x), p(y)
randomly swap bits to get x’, y’randomly swap bits to get x’, y’
rarely, randomly flip bits in x’, y’rarely, randomly flip bits in x’, y’
G’ = G’ U {x’, y’}G’ = G’ U {x’, y’}
GSATGSAT
Score: number of unsat clauses.Score: number of unsat clauses.• Start with random assignment.Start with random assignment.• While not satisfied…While not satisfied…
– Flip variable value to satisfy greatest Flip variable value to satisfy greatest number of clauses.number of clauses.
– Repeat (until done or bored)Repeat (until done or bored)
• Repeat (until done or bored)Repeat (until done or bored)
GSAT AnalysisGSAT Analysis
What kind of local search is this most What kind of local search is this most like? Hill climbing? Simulated like? Hill climbing? Simulated annealing? GA?annealing? GA?
Known to perform quite well for Known to perform quite well for coloring problems.coloring problems.
WALKSATWALKSAT
On each step, with probability p, do a On each step, with probability p, do a greedy GSAT move.greedy GSAT move.
With probability 1-p, “walk” (flip a With probability 1-p, “walk” (flip a variable in an unsatisfied clause).variable in an unsatisfied clause).
Even faster for many problems Even faster for many problems (planning, random CNF).(planning, random CNF).
WALKSAT AnalysisWALKSAT Analysis
What kind of local search is this most What kind of local search is this most like? Hill climbing? Simulated like? Hill climbing? Simulated annealing? GA?annealing? GA?
Known to perform quite well for hard Known to perform quite well for hard random CNF problems.random CNF problems.
Random k-CNFRandom k-CNF
Variables: nVariables: n
Clauses: mClauses: m
Variables per clause: kVariables per clause: k
Randomly generate all m clauses by Randomly generate all m clauses by choosing k of the n variables for choosing k of the n variables for each clause at random. Randomly each clause at random. Randomly negate.negate.
Satisfiable FormulaeSatisfiable Formulae
0
0.2
0.4
0.6
0.8
1
0 5 10
clauses/variables
Pro
b.
of
sa
tisfi
ab
le
0
0.2
0.4
0.6
0.8
1
0 5 10
clauses/variables
Pro
b.
of
sa
tisfi
ab
le
Explaining ResultsExplaining Results
Three sectionsThree sections• m/n < 4.2, under constrained: m/n < 4.2, under constrained:
nearly all satisfiablenearly all satisfiable• m/n > 4.3, over constrained: nearly m/n > 4.3, over constrained: nearly
all unsatisfiableall unsatisfiable• m/n , under constrainted: nearly all m/n , under constrainted: nearly all
satisfiablesatisfiable
Phase TransitionPhase Transition
Under-constrained problems are Under-constrained problems are easy, just guess an assignment.easy, just guess an assignment.
Over-constrained problems aren’t Over-constrained problems aren’t too bad, just say “unsatisfiable” too bad, just say “unsatisfiable” and try to verify via DPLL.and try to verify via DPLL.
Transition region sharpens as n Transition region sharpens as n increases.increases.
Local Search DiscussionLocal Search Discussion
Often the Often the secondsecond best way to solve best way to solve any problem.any problem.
Relatively easy to implement.Relatively easy to implement.
So, good first attempt. If good So, good first attempt. If good enough, stop.enough, stop.
What to LearnWhat to Learn
Definition of hill climbing, simulated Definition of hill climbing, simulated annealing, genetic algorithm.annealing, genetic algorithm.
Definition of GSAT, WALKSAT.Definition of GSAT, WALKSAT.
Relative advantages and Relative advantages and disadvantages of local search.disadvantages of local search.
Programming Project 1 Programming Project 1 (due 10/22)(due 10/22)
Solve Rush Hour problems.Solve Rush Hour problems.
Implement BFS, A* with simple Implement BFS, A* with simple blocking heuristic, A* with more blocking heuristic, A* with more advanced heuristic.advanced heuristic.
Input and output examples available Input and output examples available on course web siteon course web site
Speed and accuracy count.Speed and accuracy count.
Homework 4 (due 10/17)Homework 4 (due 10/17)
1.1. In graph partitioning, call a In graph partitioning, call a balancedbalanced state state one in which both sets contain the same one in which both sets contain the same number of nodes. (a) How can we choose an number of nodes. (a) How can we choose an initial state at random that is balanced? (b) initial state at random that is balanced? (b) How can we define the neighbor set N so that How can we define the neighbor set N so that every neighbor of a balanced state is every neighbor of a balanced state is balanced? (c) Show that standard GA balanced? (c) Show that standard GA crossover (uniform) between two balanced crossover (uniform) between two balanced states need not be balanced. (d) Suggest an states need not be balanced. (d) Suggest an alternative crossover operator that preserves alternative crossover operator that preserves balance.balance.
2.2. More soon…More soon…