Some material adopted from notes by Charles R. Dyer, University of

Wisconsin-Madison

Today’sclass:localsearch

•  Itera&veimprovementmethods– Hillclimbing– Simulatedannealing– Localbeamsearch– Gene&calgorithms

•  Onlinesearch

HillClimbing

•  Extendedthecurrentpathwithasuccessornodewhichisclosertothesolu&onthantheendofthecurrentpath

•  Ifourgoalistogettothetopofahill,thenalwaystakeasteptheleadsyouup

•  Simplehillclimbing–takeanyupwardstep

•  Steepestascenthillclimbing–considerallpossiblesteps,andtaketheonethatgoesupthemost

•  Nomemory

Hillclimbingonasurfaceofstates

HeightDefinedbyEvalua&onFunc&on

Hill‐climbingsearch• Ifthereexistsasuccessorsforthecurrentstatensuchthat–  h(s)<h(n)–  h(s)<=h(t)forallthesuccessorstofn

thenmovefromntos.Otherwise,haltatn• LooksonestepaheadtodetermineifasuccessorisbeLerthanthecurrentstate;ifso,movetothebestsuccessor.

• LikeGreedysearchinthatitusesh,butdoesn’tallowbacktrackingorjumpingtoanalterna&vepathsinceitdoesn’t“remember”whereithasbeen.

• IsBeamsearchwithabeamwidthof1(i.e.,themaximumsizeofthenodeslistis1).• Notcompletesincethesearchwillterminateat"localminima,""plateaus,"and"ridges."

Hillclimbingexample2 8 3 1 6 4 7 5

2 8 3 1 4 7 6 5

2 3 1 8 4 7 6 5

1 3 8 4

7 6 5

2

3 1 8 4 7 6 5

2

1 3 8 4 7 6 5

2 start goal

-5

h = -3

h = -3

h = -2

h = -1

h = 0 h = -4

-5

-4

-4 -3

-2

f(n) = -(number of tiles out of place)

Image from: http://classes.yale.edu/fractals/CA/GA/Fitness/Fitness.html

local maximum

ridge

plateau

ExploringtheLandscape

• LocalMaxima:peaksthataren’tthehighestpointinthespace

• Plateaus:thespacehasabroadflatregionthatgivesthesearchalgorithmnodirec&on(randomwalk)

• Ridges:flatlikeaplateau,butwithdrop‐offstothesides;stepstotheNorth,East,SouthandWestmaygodown,butasteptotheNWmaygoup.

Drawbacksofhillclimbing

•  Problems:localmaxima,plateaus,ridges•  Remedies:– Randomrestart:keeprestar&ngthesearchfromrandomloca&onsun&lagoalisfound.

– ProblemreformulaDon:reformulatethesearchspacetoeliminatetheseproblema&cfeatures

•  Someproblemspacesaregreatforhillclimbingandothersareterrible.

ExampleofalocalopDmum

1 2 5 7 4

8 6 3 4

1 2 3 8 7 6 5

1 2 5 7 4 8 6 3

2 5 1 7 4 8 6 3

1 2 5 8 7 4

6 3

-3

-4

-4

-4

0

start goal

HillClimbingand8Queens

Gradientascent/descent

•  Gradientdescentprocedureforfindingtheargxminf(x)

– chooseini&alx0randomly– repeat

•  xi+1←xi–ηf'(xi)

– un&lthesequencex0,x1,…,xi,xi+1converges•  Stepsizeη(eta)issmall(perhaps0.1or0.05)

Images from http://en.wikipedia.org/wiki/Gradient_descent

Gradientmethodsvs.Newton’smethod

• AreminderofNewton’smethodfromCalculus:xi+1←xi–ηf'(xi)/f''(xi)

• Newton’smethoduses2ndorderinforma&on(secondderiva&ve,or,curvature)totakeamoredirectroutetotheminimum.

• Thesecond‐orderinforma&onismoreexpensivetocompute,butconvergesquicker.

Contour lines of a function Gradient descent (green) Newton’s method (red)

Image from http://en.wikipedia.org/wiki/Newton's_method_in_optimization

Annealing

•  Inmetallurgy,annealingisatechniqueinvolvinghea&ngandcontrolledcoolingofamaterialtoincreasethesizeofitscrystalsandreducetheirdefects•  Theheatcausestheatomstobecomeunstuckfromtheirini&alposi&ons(alocalminimumoftheinternalenergy)andwanderrandomlythroughstatesofhigherenergy.•  Theslowcoolinggivesthemmorechancesoffindingconfigura&onswithlowerinternalenergythantheini&alone.

Simulatedannealing(SA)•  SAexploitstheanalogybetweenhowmetalcoolsandfreezesintoaminimum‐energycrystallinestructure&searchforaminimum/maximuminageneralsystem

•  SAcanavoidbecomingtrappedatlocalminima•  SAusesarandomsearchthatacceptschangesincreasingobjec&vefunc&onfandsomethatdecreaseit

•  SAusesacontrolparameterT,whichbyanalogywiththeoriginalapplica&onisknownasthesystem“temperature”

•  Tstartsouthighandgraduallydecreasestoward0

SAintuiDons

•  combineshillclimbing(efficiency)withrandomwalk(completeness)

•  Analogy:gekngaping‐pongballintothedeepestdepressioninabumpysurface–  shakethesurfacetogettheballoutofthelocalminima

–  nottoohardtodislodgeitfromtheglobalminimum

•  Simulatedannealing:–  startbyshakinghard(hightemperature)andgraduallyreduceshakingintensity(lowerthetemperature)

–  escapethelocalminimabyallowingsome“bad”moves–  butgraduallyreducetheirsizeandfrequency

Simulatedannealing•  A“bad”movefromAtoBisacceptedwithaprobability

‐(f(B)‐f(A)/T)e

•  Thehigherthetemperature,themorelikelyitisthatabadmovecanbemade

•  AsTtendstozero,thisprobabilitytendstozero,andSAbecomesmorelikehillclimbing

•  IfTisloweredslowlyenough,SAiscompleteandadmissible

Simulatedannealingalgorithm

Localbeamsearch

•  Basicidea– Beginwithkrandomstates

– Generateallsuccessorsofthesestates– Keepthekbeststatesgeneratedbythem

•  Providesasimple,efficientwaytosharesomeknowledgeacrossasetofsearches

•  Stochas3cbeamsearchisavaria&on:

– Probabilityofkeepingastateisafunc3onofitsheuris&cvalue

GeneDcalgorithms(GA)•  Asearchtechniqueinspiredbyevolu3on•  Similartostochas&cbeamsearch•  Startwithkrandomstates(theini3alpopula3on)•  Newstatesaregeneratedby“muta&ng”asinglestateor“reproducing”(combining)twoparentstates(selectedaccordingtotheirfitness)

•  Encodingusedforthe“genome”ofanindividualstronglyaffectsthebehaviorofthesearch

•  Gene&calgorithms/gene&cprogrammingarealargeandac&veareaofresearch

MaandPasoluDons

8Queensproblem

• Representstatebyastringof8digitsin{1..8}

• S=‘32752411’• Fitnessfunc&on=#ofnon‐aLackingpairs

• f(Sol)=8*7/2=28• F(S)=24

GeneDcalgorithms

•  Fitnessfunc&on:numberofnon‐aLackingpairsofqueens(min=0,max=(8×7)/2=28)

•  24/(24+23+20+11)=31%•  23/(24+23+20+11)=29%etc

GeneDcalgorithms

GApseudo‐code

AntColonyOpDmizaDon(ACO)Aprobabilis&csearchtechniqueforproblemsreducibletofindinggoodpathsthroughgraphs

InspiraDon•  Antsleavenest•  Discoverfood•  Returntonest,preferringshorterpaths

•  Leavepheromone trail

•  Shortestpathisreinforced

Tabusearch

•  Problem:Hillclimbingcangetstuckonlocalmaxima

•  Solu&on:Maintainalistofkpreviouslyvisitedstates,andpreventthesearchfromrevisi&ngthem

CLASSEXERCISE

•  WhatwouldalocalsearchapproachtosolvingaSudokuproblemlooklike?

3

1

3

2

Onlinesearch•  Interleavecomputa&on&ac&on(searchsome,actsome)

•  Explora&on:Can’tinferoutcomesofac&ons;mustactuallyperformthemtolearnwhatwillhappen

•  Compe&&vera&o:Pathcostfound/Pathcostthatwouldbefoundiftheagentknewthenatureofthespace,andcoulduseofflinesearch*Onaverage,orinanadversarialscenario(worstcase)

•  Rela&velyeasyifac&onsarereversible(ONLINE‐DFS‐AGENT)

•  LRTA*(LearningReal‐TimeA*):Updateh(s)(instatetable)basedonexperience

•  MoreabouttheseinchaptersonLogicandLearning!

Othertopics

•  Searchincon&nuousspaces– Differentmath

•  Searchwithuncertainac&ons– Mustmodeltheprobabili&esofanac&onsresults

•  Searchwithpar&alobserva&ons– Acquiringknowledgeasaresultofsearch

Summary:Informedsearch• Hill‐climbingalgorithmskeeponlyasinglestateinmemory,butcangetstuckonlocalop&ma.

• Simulatedannealingescapeslocalop&ma,andiscompleteandop&malgivena“longenough”coolingschedule.

• GeneDcalgorithmscansearchalargespacebymodelingbiologicalevolu&on.• Onlinesearchalgorithmsareusefulinstatespaceswithpar&al/noinforma&on.