heuristic search (informed search) - … search •the idea is to develop a domain specific...

1McIlraith&Allin,CSC384,UniversityofToronto,Winter2018

HeuristicSearch(InformedSearch)

HeuristicSearch

• Inuninformedsearch,wedon’ttrytoevaluatewhichofthenodesonthefrontier/OPENaremostpromising.Wenever“look-ahead”tothegoal.

E.g.,inuniformcostsearchwealwaysexpandthecheapestpath.Wedon’tconsiderthecostofgettingtothegoalfromtheendofthecurrentpath.

• Oftenwehavesomeotherknowledgeaboutthemeritofnodes,e.g.,goingthewrongdirectioninRomania.

HeuristicSearchMerit ofafrontier/OPENnode:differentnotionsofmerit.• Ifweareconcernedaboutthecostofthesolution,wemightwantanotionofmeritofhowcostlyitistogettothegoalfromthatsearchnode.

• Ifweareconcernedaboutminimizingcomputation insearchwemightwanttoconsiderhoweasyitistofindthegoalfromthatsearchnode.

• Wewillfocusonthe“costofsolution”notionofmerit.

HeuristicSearch

• Theideaistodevelopadomainspecificheuristicfunctionh(n).

• h(n)guesses thecostofgettingtothegoalfromnoden(thecostofcompletingthepaththatiscapturedbythestateofnoden).

• Therearedifferentwaysofguessingthiscostindifferentdomains.I.e.,heuristicsaredomainspecific.

“Asthecrowflies”– Straightlineheuristic

Onthemap,thenumbersbetweencitiesrepresentthedrivingdistancebetweencitiesonpotentiallywigglyroads,eventhoughtheyaredrawnasstraightlines.Contrastthistotheline-of-sight/``asthecrowflies”distancewhichignoreswigglesintheroad,cliffs,bridges,andassumesyoucanjustdriveinastraightlinefromonecitytoanother.

Planning a path from Arad to Bucharest, we can utilize the straight line distance from each city to our goal as a heuristic/guess of the actual distance. This lets us plan our trip by picking cities at each time point that minimize the distance to our goal.

Example:StraightLineDistance

HeuristicSearch

• Ifh(n1)<h(n2) thismeansthatweguessthatitischeapertogettothegoalfromn1 thanfromn2.

• Werequirethat• h(n)=0 foreverynodenwhosestatesatisfiesthegoal.• Zerocostofgettingtoagoalnodefromn.

Usingonlyh(n):Greedybest-firstsearch(GreedyBFS)

• Weuseh(n)torankthenodesonthefrontier/OPEN.• Alwaysexpandnodewithlowesth-value.

• Wearegreedilytryingtoachievealowcostsolution.

• However,thismethodignoresthecostofgettington,soitcanbeleadastrayexploringnodesthatcostalottogettobutseemtobeclosetothegoal:

→ stepcost=10

→ stepcost=100h(n3)=50h(n1)=70

[S][n3,n1][Goal, n1]

Usingonlyh(n):Greedybest-firstsearch(GreedyBFS).

→ step cost = 10

→ step cost = 100h(n3) = 50h(n1) = 70

(Greedy BFS is• Incomplete• not optimal)

• Weuseh(n)torankthenodesonthefrontier.• Alwaysexpandnodewithlowesth-value.

• Wearegreedilytryingtoachievealowcostsolution.

• However,thismethodignoresthecostofgettington,soitcanbeleadastrayexploringnodesthatcostalottogettobutseemtobeclosetothegoal:

Greedybest-firstsearchexample

Whenyou’reatSibiuandcontemplatingwhethertogotoFagarasorRV,theheuristicvalueofthesuccessornodes,i.e.,thehvalueguessofthecostis:h(Fagaras)=178andh(RV)=193),soFagaraslookslikethebetterchoice,but…

ActualCost(Arad-Sibiu-RV-Pitesli-Bucharest):140+80+97+101=140+278=418ActualCost(Arad-Sibiu-Fagaras-Bucharest): 140+99+211 =140+310 =450

A*search

• Takeintoaccountthecostofgettingtothenodeaswellasourestimateofthecostofgettingtothegoalfromn.

• Defineanevaluationfunctionf(n)f(n)=g(n)+h(n)• g(n)isthecostofthepathtonoden• h(n)istheheuristicestimateofthecostofgettingtoagoalnodefromn.

• Alwaysexpandthenodewithlowestf-valueonthefrontier.

• Thef-valueisanestimateofthecostofgettingtothegoalviathisnode(path).

A* examplef(n) =g(n)+h(n),

=actualcostton+heuristicestimateofcostfromntothegoal

A*search

• Takeintoaccountthecostofgettingtothenodeaswellasourestimateofthecostofgettingtothegoalfromn.

• Defineanevaluationfunctionf(n)f(n)=g(n)+h(n)• g(n)isthecostofthepathtonoden• h(n)istheheuristicestimateofthecostofgettingtoagoalnodefromn.

• Alwaysexpandthenodewithlowestf-valueonthefrontier.

• Thef-valueisanestimateofthecostofgettingtothegoalviathisnode(path).

Conditionsonh(n)

• Wewanttoanalyzethebehavioroftheresultantsearch.• Completeness,timeandspace,optimality?

• Toobtainsuchresultswemustputsomefurtherconditionsontheheuristicfunctionh(n)andthesearchspace.

Conditionsonh(n):Admissible

• Wealwaysassumethatc(n1→n2)≥ε >0.Thecostofanytransitionisgreaterthanzeroandcan’tbearbitrarilysmall.

• Leth*(n)bethecostofan optimalpath fromntoagoalnode(¥ ifthereisnopath).Thenanadmissible heuristicsatisfiesthecondition

h(n)≤h*(n)admissibleheuristichalwaysunderestimatesthetruecosttoreach

thegoal.i.e.,itisoptimisticJ

• Hence• h(g)=0,foranygoalnote,g• h*(n)=¥ ifthereisnotpathfromntoagoalnode

Consistency(akamonotonicity)

• Isastrongerconditionthanh(n)≤h*(n).

• Amonotone/consistent heuristicsatisfiesthetriangleinequality(forallnodesn1,n2):

h(n1)≤c(n1→ n2)+h(n2)

• Notethattheremightbemorethanonetransition(action)betweenn1andn2,theinequalitymustholdforallofthem.

• Notethatmonotonicityimpliesadmissibility.• (foralln1,n2)h(n1)≤c(n1→ n2)+h(n2)è (foralln)h(n)≤h*(n)

Intuitionbehindadmissibility

h(n)≤h*(n)meansthatthesearchwon’tmissanypromisingpaths.• Ifitreallyischeaptogettoagoalvian(i.e.,bothg(n)andh*(n)arelow),thenf(n)=g(n)+h(n)willalsobelow,andthesearchwon’tignoreninfavourofmoreexpensiveoptions.

• Thiscanbeformalizedtoshowthatadmissibilityimpliesoptimality.

Intuitionbehindmonotonicity

h(n1)≤c(n1→n2)+h(n2)

• Thissayssomethingsimilar,butinadditiononewon’tbe“locally”mislead.Seenextexample.

Consistencyè Admissible• Assumeconsistency:h(n1)≤c(n1→n2)+h(n2)Proveadmissible:h(n)≤h*(n)

Proof:If nopathexistsfromntoagoalthenh*(n)=¥ andh(n)≤h*(n)Else letnà n1à …à n*beanOPTIMALpathfromntoagoal.Notethecostofthispathish*(n),andeachsubpath (nià …à n*)hascostequaltoh*(ni).

Proveh(n)≤h*(n)byinductiononthelengthofthisoptimalpath.

BaseCase:n=n* [optimalpathlength=0]Byourconditionsonh,h(n)=0≤h(n*)=0InductionHypothesis:h(n1)≤h*(n1)h(n)≤c(n → n1)+h(n1)[consistency]

≤c(n→n1)+h*(n1) [defn h*]=h*(n)

Example:admissiblebutnonmonotonic

Thefollowingh isnotconsistent(i.e.,notmonotone) sinceh(n2)>c(n2→n4)+h(n4).Butitisadmissible.

→ stepcost=200→ stepcost=100

{S}→{n1[200+50=250],n2[200+100=300]}→{n2[100+200=300], n3[400+50=450]}→{n4[200+50=250],n3[400+50=450]}→{goal[300+0=300],n3[400+50=450]}

Wedofind theoptimalpathastheheuristicisstilladmissible.But wearemisleadintoignoringn2untilafterweexpandn1.

h(n2)=200

h(n4)=50

h(n1)=50

h(n3)=50g(n)+h(n)=f(n)

Example:admissiblebutnonmonotonic

→ stepcost=200→ stepcost=100

{S}→{n1[200+50=250],n2[100+200=300]}→{n2[100+200=300], n3[400+50=450]}→{n4[200+50=250],n3[400+50=450]}→{goal[300+0=300],n3[400+50=450]}

Wedofind theoptimalpathastheheuristicisstilladmissible.But wearemisleadintoignoringn2untilafterweexpandn1.

h(n2)=200

h(n4)=50

h(n1)=50

h(n3)=50g(n)+h(n)=f(n)

Thefollowingh isnotconsistent(i.e.,notmonotone) sinceh(n2)>c(n2→n4)+h(n4).Butitisadmissible.

“Asthecrowflies”– Straightlineheuristic

• Mostadmissibleheuristicsarealsomonotone.(Indeedit’shardtofindanadmissibleheuristicthatisnotmonotone!)

Consequencesofmonotonicity

1. Thef-valuesofnodesalongapathmustbenon-decreasing.

Let<Start→n1→n2…→nk>beapath.Weclaimthat

f(ni)≤f(ni+1)

Proof:f(ni)=c(Start→…→ni)+h(ni)

≤c(Start→…→ni)+c(ni→ni+1) +h(ni+1)[monotonicity]=c(Start→…→ni→ni+1)+h(ni+1)=g(ni+1)+h(ni+1)=f(ni+1).

Proof(2cases):• Ifn2wasonthefrontier/OPENwhenn1wasexpanded,

thenf(n1)≤f(n2)otherwisewewouldhaveexpandedn2.• Ifn2wasaddedtothefrontier/OPENaftern1’sexpansion,thenletn beanancestorofn2thatwaspresentwhenn1wasbeingexpanded(thiscouldben1itself).Wehavef(n1)≤f(n)sinceA*chosen1whilen waspresentinthefrontier/OPEN.Also,sincen isalongthepathton2,byproperty(1)wehavef(n)≤f(n2).So,wehavef(n1)≤f(n2).

----------------------------1) Thef-valuesofnodesalongapathmustbenon-decreasing.2) Ifn2isexpandedaftern1,thenf(n1)≤f(n2)

2. Ifn2isexpandedaftern1,thenf(n1)≤f(n2)(thef-valueincreasesmonotonically)

ConsequencesofmonotonicityCorollary: thesequenceoff-valuesofthenodesexpandedbyA*isnon-decreasing.I.e,Ifn2isexpandedafter (notnecessarilyimmediatelyafter)n1,thenf(n1)≤f(n2)

(thef-valueofexpandednodesismonotonic non-decreasing)Proof:• Ifn2wasonfrontier/OPENwhenn1wasexpanded,

thenf(n1)≤f(n2)otherwisewewouldhaveexpandedn2.• Ifn2wasaddedtofrontier/OPENaftern1'sexpansion,then

letn beanancestorofn2thatwaspresentwhenn1wasbeingexpanded(thiscouldben1itself).Wehavef(n1)≤f(n)sinceA*chosen1whilen waspresentonfrontier/OPEN.Also,sincen isalongthepathton2,byproperty(1)wehavef(n)≤f(n2).So,wehavef(n1)≤f(n2).

• Proof: Assumebycontradictionthatthereexistsapath<Start,n0,n1,ni-1,ni,ni+1,…,nk>withf(nk)<f(n)andni isitslastexpandednode.• ni+1mustbeonthefrontier/OPENwhilenisexpanded,so

a)by(1) f(ni+1)≤f(nk)sincetheyliealongthesamepath.b)sincef(nk)<f(n)(given)sowehavef(ni+1)<f(n)(froma)c)by(2) f(n)≤f(ni+1)becausenisexpandedbeforeni+1.

• Contradictionfromb&c!-----------------------------------------------------------

1) Thef-valuesofnodesalongapathmustbenon-decreasing.2) Ifn2isexpandedaftern1,thenf(n1)≤f(n2)3) Whennisexpandedeverypathwithlowerf-valuehasalreadybeenexpanded.

3. Whennisexpandedeverypathwithlowerf-valuehasalreadybeenexpanded.

Consequencesofmonotonicity4. Withamonotoneheuristic,thefirsttimeA*expandsa

state,ithasfoundtheminimumcostpathtothatstate.Proof:• LetPATH1 =<Start,n0,n1,…,nk,n> bethefirst pathtonfound.

Wehavef(path1)=c(PATH1)+h(n).• LetPATH2=<Start,m0,m1,…,mj,n> beanotherpathtonfound

later.wehavef(path2)=c(PATH2)+h(n).• Byproperty (3)anditscorollary,f(path1)≤f(path2)• hence:c(PATH1)≤c(PATH2)

1) Thef-valuesofnodesalongapathmustbenon-decreasing.2) Ifn2isexpandedaftern1,thenf(n1)≤f(n2)3) Whennisexpandedeverypathwithlowerf-valuehasalreadybeenexpanded.Corollary:thesequenceoff-valuesofthenodesexpandedbyA*isnon-decreasing.I.e,Ifn2isexpandedafter (notnecessarilyimmediatelyafter)n1,thenf(n1)≤f(n2)

(thef-valueofexpandednodesismonotonic non-decreasing)

ConsequencesofmonotonicityComplete.

• Yes,consideraleastcostpathtoagoalnode• SolutionPath =<Start→n1→…→G>withcostc(SolutionPath)• Sinceeachactionhasacost≥ε >0,thereareonlyafinitenumberofpaths

thathavecost ≤c(SolutionPath).• Allofthesepathsmustbeexploredbeforeanypathofcost>

c(SolutionPath).• SoeventuallySolutionPath,orsomeequalcostpathtoagoalmustbe

expanded.TimeandSpacecomplexity.

• Whenh(n)=0,foralln,hismonotone.(avery*un*informativeheuristic!!!)• A*becomesuniform-costsearch!

• Itcanbeshownthatwhenh(n)>0forsomen,thenumberofnodesexpandedcanbenolargerthanuniform-cost.

• Hencethesameboundsasuniform-costapply.(Theseareworstcasebounds).Stillexponentialunlesswehaveaverygoodh!

• Inrealworldproblems,werunoutoftimeandmemory!IDA*cansometimesbeusedtoaddressmemoryissues,butIDA*isn’tverygoodwhenmany cyclesarepresent.

ConsequencesofmonotonicityOptimality§ Yes,by(4)thefirstpathtoagoalnodemustbeoptimal.

CycleChecking§ Wecanuseasimpleimplementationofcyclechecking

(multiplepathchecking)---justrejectallsearchnodesvisitingastatealreadyvisitedbyapreviouslyexpandednode.Byproperty(4)weneedkeeponlythefirstpathtoanode,rejectingallsubsequentpaths.

4. Withamonotoneheuristic,thefirsttimeA*expandsastate,ithasfoundtheminimumcostpathtothatstate.

Searchgeneratedbymonotonicity

Insideeachcounter,thefvaluesarelessthanorequaltocountervalue!

• Foruniformcostsearch,bandsare“circular”.• Withmoreaccurateheuristics,bandsstretchoutmoretowardthegoal.

AdmissibilitywithoutmonotonicityWhen“h”isadmissiblebutnotmonotonic.

• TimeandSpacecomplexityremainthesame.Completenessholds.• Optimalitystillholds(withoutcyclechecking),butneedadifferent

argument:don’tknowthatpathsareexploredinorderofcost.

Proof(bycontradiction)ofoptimality(withoutcyclechecking):• Assumethegoalpath<S,…,G>foundbyA*hascostbiggerthanthe

optimalcost:i.e.C*(G)<f(G).• Theremustexistsanoden intheoptimalpaththatisstillinthe

frontier.• Wehave: f(n)=g(n)+h(n)≤g(n)+h*(n)=C*(G)<f(G)

• Therefore,f(n)musthavebeenselectedbefore GbyA*.contradiction!

Admissibilitywithoutmonotonicity

WhataboutCycleChecking?• Nolongerguaranteedwehavefoundanoptimalpathtoanodethefirst

time wevisitit.

• So,cyclecheckingmightnotpreserveoptimality.• Tofixthis:forpreviouslyvisitednodes,mustremembercostof

previouspath.Ifnewpathischeapermustexploreagain.

• contoursofmonotonicheuristicsdon’thold.

SpaceProblemswithA*

• A*hasthesamepotentialspaceproblemsasBFSorUCS

• IDA*- IterativeDeepeningA*issimilartoIterativeDeepeningSearchandsimilarlyaddressesspaceissues.

IDA*- IterativeDeepeningA*Objective:reducememoryrequirementsforA*• Likeiterativedeepening,butnowthe“cutoff”isthef-value(g+h)rather

thanthedepth• Ateachiteration,thecutoffvalueisthesmallestf-valueofanynodethat

exceededthecutoffonthepreviousiteration• Avoidsoverheadassociatedwithkeepingasortedqueueofnodes• Twonewparameters:

• curBound (anynodewithabiggerf-valueisdiscarded)• smallestNotExplored (thesmallestf-valuefordiscardednodesina

round)whenfrontier/OPENbecomesempty,thesearchstartsanewroundwiththisbound

• Easiertoexpandallnodeswithf-valueEQUALtothef-limit.Thiswaywecancompute“smallestNotExplored” moreeasily.

ConstructingHeuristics

BuildingHeuristics:RelaxedProblem

• Oneusefultechniqueistoconsideraneasierproblem,andleth(n)bethecostofreachingthegoalintheeasierproblem.

8-Puzzle

• CanmoveatilefromsquareAtoBif• Aisadjacent(left,right,above,below)toB• and Bisblank

8-Puzzlemoves(continued)• CanmoveatilefromsquareAtoBif

• Aisadjacent(left,right,above,below)toB• and Bisblank

• Canrelaxsomeoftheseconditions1. canmovefromAtoBifAisadjacenttoB(ignorewhetherornot

positionisblank)2. canmovefromAtoBifBisblank(ignoreadjacency)3. canmovefromAtoB(ignorebothconditions).

BuildingHeuristics:RelaxedProblem• #3“canmovefromAtoB(ignorebothconditions)”.

leadstothemisplacedtiles heuristic.• Tosolvethepuzzle,weneedtomoveeachtileintoitsfinalposition.• Numberofmoves=numberofmisplacedtiles.• Clearlyh(n)=numberofmisplacedtiles≤theh*(n)thecostofanoptimal

sequenceofmovesfromn.

• #1“canmovefromAtoBifAisadjacenttoB(ignorewhetherornotpositionisblank)”leadstothemanhattan distance heuristic.• Tosolvethepuzzleweneedtoslideeachtileintoitsfinalposition.• Wecanmoveverticallyorhorizontally.• Numberofmoves=sumoverallofthetilesofthenumberofverticaland

horizontalslidesweneedtomovethattileintoplace.• Againh(n)=sumofthemanhattan distances≤h*(n)

• inarealsolutionweneedtomoveeachtileatleastthatfarandwecanonlymoveonetileatatime.

Theoptimal costtonodesintherelaxedproblemisanadmissible heuristic fortheoriginalproblem!

ProofIdea:theoptimalsolutionintheoriginalproblemisasolutionforrelaxedproblem,thereforeitmustbeatleastasexpensiveastheoptimalsolutionintherelaxedproblem.

Soadmissibleheuristicscansometimesbeconstructedbyfindingarelaxationwhoseoptimalsolutioncanbeeasilycomputed.

Depth IDS A*(Misplaced)h1 A*(Manhattan)h210 47,127 93 3914 3,473,941 539 11324 --- 39,135 1,641

Leth1=Misplaced,h2=Manhattan• Doesh2always expandfewernodesthanh1?

• Yes!Notethath2dominatesh1,i.e.foralln:h1(n)≤h2(n).• Therefore,amongseveraladmissibleheuristictheonewithhighest

valueexpandsthefewestnodes.Isitthefastest?

Comparison ofIDSandA*(averagetotalnodesexpanded):

BuildingHeuristics:Patterndatabases.

•By searching backwards from these goal states, we can compute the distance of any configuration of these tiles to their goal locations. We are ignoring the identity of the other tiles.

•For any state n, the number of moves required to get these tiles into place form a lower bound on the cost of getting to the goal from n.

• Admissibleheuristicscanalsobederivedfromsolutiontosubproblems:Eachstateismappedintoapartialspecification,e.g.in15-puzzleonlypositionofspecifictilesmatters.

• Herearegoalsfortwosub-problems(calledCornerandFringe)of15-puzzle.

• NotethelocationofBLANK!

BuildingHeuristics:Patterndatabases.

• Theseconfigurationsarestoredinadatabase,alongwiththenumberofmovesrequiredtomovethetilesintoplace.

• Themaximum numberofmovestakenoverallofthedatabases canbeusedasaheuristic.

• Onthe15-puzzle• Thefringedatabaseyieldsabouta345folddecreaseinthesearchtreesize.

• Thecornerdatabaseyieldsabout437folddecrease.

• Sometimesdisjointpatterns canbefound,thenthenumberofmovescanbeadded ratherthantakingthemax(ifweonlycountmovesofthetargettiles).

LocalSearch

• Sofar,wekeepthepathstothegoal.• Forsomeproblems(like8-queens)wedon’tcareaboutthepath,weonlycareaboutthesolution.ManyrealproblemlikeScheduling,ICdesign,andnetworkoptimizationsareofthisform.

• Localsearch algorithmsoperateusingasinglecurrentstateandgenerallymovetoneighborsofthatstate.

• Thereisanobjectivefunction thattellsthevalueofeachstate.Thegoalhasthehighestvalue(globalmaximum).

• AlgorithmslikeHillClimbing trytomovetoaneighbourwiththehighestvalue.

• Dangerofbeingstuckinalocalmaximum.Sosomerandomnessisaddedto“shake”outoflocalmaxima.

• SimulatedAnnealing:Insteadofthebestmove,takearandommoveandifitimprovesthesituationthenalwaysaccept,otherwiseacceptwithaprobability<1.

• [IfinterestedreadthesetwoalgorithmsfromtheR&Nbook].

heuristic search (informed search) - … search •the idea is to develop a domain specific...

Documents

09 heuristic search

implementing fast heuristic search code -...

heuristic search techniques

slide 4.1 4 heuristic search - radford...

heuristic search. best first search a* heuristic search...

heuristic search-best-first-search

artificial intelligence 2.2 heuristic (informed) search...

heuristic search: gbfs, a*

4 heuristic search - computer science heuristic search 4.0...

analisis metode heuristic search

09 informed search - the university of edinburgh21/10/04...

heuristic search

chapter 4 informed search and exploration. outline informed...

d heuristic search

heuristic search techniques. contents several general...

heuristic search best first search

04 search heuristic

automatic planning chapter 7: heuristic...

heuristic search:...

ch4 heuristic search