CS 294-5: Statistical Natural Language Processing

Graphs vs trees up front; use grid too; discuss for BFS, DFS, IDS, UCSCut back on A* optimality detail; a bit more on importance of heuristics, performance dataPattern DBs?AnnouncementsHomework 1 (uninformed search) due tomorrowHomework 2 (informed and local search, search agents) due 9/15Part I through edX online, instant grading, submit as often as you like.Part II through www.pandagrader.com -- submit pdf

Project 1 out on Thursday, due 9/24AI in the news TechCrunch, 2014/1/25

AI in the news

General Tree SearchImportant ideas:FrontierExpansion: for each action, create the child nodeExploration strategy

Main question: which frontier nodes to explore?function TREE-SEARCH(problem) returns a solution, or failure initialize the frontier using the initial state of problem loop do if the frontier is empty then return failure choose a leaf node and remove it from the frontier if the node contains a goal state then return the corresponding solution expand the chosen node, adding the resulting nodes to the frontier

A note on implementationNodes havestate, parent, action, path-cost

A child of node by action a hasstate = result(node.state,a)parent = nodeaction = apath-cost = node.path-cost + step-cost(node.state,a,state)

Extract solution by tracing back parent pointers, collecting actions

Depth-First SearchDepth-First SearchSabdpacephfrqqcGaqephfrqqcGaSGdbpqcehafrqphfdbacerStrategy: expand a deepest node firstImplementation: Frontier is a LIFO stackSearch Algorithm Properties

Search Algorithm PropertiesComplete: Guaranteed to find a solution if one exists?Optimal: Guaranteed to find the least cost path?Time complexity?Space complexity?

Cartoon of search tree:b is the branching factorm is the maximum depthsolutions at various depths

Number of nodes in entire tree?1 + b + b2 + . bm = O(bm)

b1 nodeb nodesb2 nodesbm nodesm tiersDepth-First Search (DFS) Propertiesb1 nodeb nodesb2 nodesbm nodesm tiersWhat nodes does DFS expand?Some left prefix of the tree.Could process the whole tree!If m is finite, takes time O(bm)

How much space does the frontier take?Only has siblings on path to root, so O(bm)

Is it complete?m could be infinite, so only if we prevent cycles (more later)

Is it optimal?No, it finds the leftmost solution, regardless of depth or cost

Breadth-First Search

Breadth-First SearchSabdpacephfrqqcGaqephfrqqcGaSGdbpqcehafrSearchTiersStrategy: expand a shallowest node firstImplementation: Frontier is a FIFO queueBreadth-First Search (BFS) PropertiesWhat nodes does BFS expand?Processes all nodes above shallowest solutionLet depth of shallowest solution be sSearch takes time O(bs)

How much space does the frontier take?Has roughly the last tier, so O(bs)

Is it complete?s must be finite if a solution exists, so yes!

Is it optimal?Only if costs are all 1 (more on costs later)

b1 nodeb nodesb2 nodesbm nodess tiersbs nodesQuiz: DFS vs BFS

Quiz: DFS vs BFSWhen will BFS outperform DFS?

When will DFS outperform BFS?[Demo: dfs/bfs maze water (L2D6)]Video of Demo Maze Water DFS/BFS (part 1)

Video of Demo Maze Water DFS/BFS (part 2)

Iterative DeepeningbIdea: get DFSs space advantage with BFSs time / shallow-solution advantagesRun a DFS with depth limit 1. If no solutionRun a DFS with depth limit 2. If no solutionRun a DFS with depth limit 3. ..

Isnt that wastefully redundant?Generally most work happens in the lowest level searched, so not so bad!Finding a least-cost pathBFS finds the shortest path in terms of number of actions.It does not find the least-cost path. We will now covera similar algorithm which does find the least-cost path. STARTGOALdbpqcehafr29281823244151322Uniform Cost Search

Uniform Cost SearchSabdpacephfrqqcGaqephfrqqcGaStrategy: expand a cheapest node first:Frontier is a priority queue (priority: cumulative cost)SGdbpqcehafr39116411571381011171106391128821512Cost contours2Uniform Cost Search (UCS) PropertiesWhat nodes does UCS expand?Processes all nodes with cost less than cheapest solution!If that solution costs C* and arcs cost at least , then the effective depth is roughly C*/Takes time O(bC*/) (exponential in effective depth)

How much space does the frontier take?Has roughly the last tier, so O(bC*/)

Is it complete?Assuming best solution has a finite cost and minimum arc cost is positive, yes!

Is it optimal?Yes! (Proof next lecture via A*)

bC*/ tiersc 3c 2c 1Uniform Cost IssuesRemember: UCS explores increasing cost contours

The good: UCS is complete and optimal!

The bad:Explores options in every directionNo information about goal location

Well fix that soon!

StartGoalc 3c 2c 125Search Gone Wrong?

Tree Search vs Graph Search

Failure to detect repeated states can cause exponentially more work. Search TreeState GraphTree Search: Extra Work!

O(m)

O(2m)Failure to detect repeated states can cause exponentially more work. Search TreeState GraphTree Search: Extra Work!O(2m2)O(4m)

m=20: 800 states, 1,099,511,627,776 tree nodesGeneral Tree Searchfunction TREE-SEARCH(problem) returns a solution, or failure initialize the frontier using the initial state of problem loop do if the frontier is empty then return failure choose a leaf node and remove it from the frontier if the node contains a goal state then return the corresponding solution expand the chosen node, adding the resulting nodes to the frontier

General Graph Searchfunction GRAPH-SEARCH(problem) returns a solution, or failure initialize the frontier using the initial state of problem initialize the explored set to be empty loop do if the frontier is empty then return failure choose a leaf node and remove it from the frontier if the node contains a goal state then return the corresponding solution add the node to the explored set expand the chosen node, adding the resulting nodes to the frontier but only if the node is not already in the frontier or explored set

Tree Search vs Graph SearchGraph searchAvoids infinite loops: m is finite in a finite state spaceEliminates exponentially many redundant pathsRequires memory proportional to its runtime!

CS 188: Artificial Intelligence

Informed Search

Instructor: Stuart RussellUniversity of California, BerkeleyPlease retain proper attribution and the reference to ai.berkeley.edu. Thanks!

33Search Heuristics

A heuristic is:A function that estimates how close a state is to a goalDesigned for a particular search problemExamples: Manhattan distance, Euclidean distance for pathing10511.2

34Example: distance to Bucharest

h(x)

Example: Pancake Problem

Gates, W. and Papadimitriou, C., "Bounds for Sorting by Prefix Reversal.", Discrete Mathematics. 27, 47-57, 1979.

36Example: Pancake ProblemStep Cost: Number of pancakes flippedGates, W. and Papadimitriou, C., "Bounds for Sorting by Prefix Reversal.", Discrete Mathematics. 27, 47-57, 1979.

37Example: Pancake ProblemState space graph with step costs32433222434342A* expanded 8100 ; Path cost = 33UCS expanded 25263 . Path cost = 33Greedy expanded 10 . Path cost = 41[0, 7, 5, 3, 2, 1, 4, 6](7, 0, 5, 3, 2, 1, 4, 6)(6, 4, 1, 2, 3, 5, 0, 7)(3, 2, 1, 4, 6, 5, 0, 7)(1, 2, 3, 4, 6, 5, 0, 7)(5, 6, 4, 3, 2, 1, 0, 7)(6, 5, 4, 3, 2, 1, 0, 7)(0, 1, 2, 3, 4, 5, 6, 7)38Example: Pancake ProblemHeuristic: the number of the largest pancake that is still out of place4302333443444h(x)A* expanded 8100 ; Path cost = 33UCS expanded 25263 . Path cost = 33Greedy expanded 10 . Path cost = 41[0, 7, 5, 3, 2, 1, 4, 6](7, 0, 5, 3, 2, 1, 4, 6)(6, 4, 1, 2, 3, 5, 0, 7)(3, 2, 1, 4, 6, 5, 0, 7)(1, 2, 3, 4, 6, 5, 0, 7)(5, 6, 4, 3, 2, 1, 0, 7)(6, 5, 4, 3, 2, 1, 0, 7)(0, 1, 2, 3, 4, 5, 6, 7)39Effect of heuristicsGuide search towards the goal instead of all over the placeStartGoalStartGoalUninformedInformedGreedy Search

Greedy SearchExpand the node that seems closest(order frontier by h)What can possibly go wrong?1932531000176

Sibiu-Fagaras-Bucharest = 99+211 = 310Sibiu-Rimnicu Vilcea-Pitesti-Bucharest = 80+97+101=278Greedy SearchStrategy: expand a node that seems closest to a goal state, according to h

Problem 1: it chooses a node even if its at the end of a very long and winding road

Problem 2: it takes h literally even if its completely wrongbFor any search problem, you know the goal. Now, you have an idea of how far away you are from the goal.43Video of Demo Contours Greedy (Empty)

Video of Demo Contours Greedy (Pacman Small Maze)

A* SearchA* Search

UCSGreedyA*Notes: these images licensed from iStockphoto for UC Berkeley use.47Combining UCS and GreedyUniform-cost orders by path cost, or backward cost g(n)Greedy orders by goal proximity, or forward cost h(n)

A* Search orders by the sum: f(n) = g(n) + h(n)SadbGh=5h=6h=218112h=6h=0ch=73eh=11Example: Teg GrenagerSabceddGGg = 0 h=6g = 1 h=5g = 2 h=6g = 3 h=7g = 4 h=2g = 6 h=0g = 9 h=1g = 10 h=2g = 12 h=048Is A* Optimal?What went wrong?Actual bad goal cost < estimated good goal costWe need estimates to be less than actual costs!AGS13h = 6h = 05h = 7Admissible Heuristics

Idea: Admissibility

Inadmissible (pessimistic) heuristics break optimality by trapping good plans on the frontierAdmissible (optimistic) heuristics slow down bad plans but never outweigh true costsAdmissible HeuristicsA heuristic h is admissible (optimistic) if:

where is the true cost to a nearest goal

Examples:

Coming up with admissible heuristics is most of whats involved in using A* in practice.

4

15Optimality of A* Tree Search

Optimality of A* Tree SearchAssume:A is an optimal goal nodeB is a suboptimal goal nodeh is admissible

Claim:A will be chosen for expansion before B

Optimality of A* Tree Search: BlockingProof:Imagine B is on the frontierSome ancestor n of A is on the frontier, too (maybe A!)Claim: n will be expanded before Bf(n) is less or equal to f(A)

Definition of f-cost

Admissibility of h

h = 0 at a goal

Optimality of A* Tree Search: BlockingProof:Imagine B is on the frontierSome ancestor n of A is on the frontier, too (maybe A!)Claim: n will be expanded before Bf(n) is less or equal to f(A)f(A) is less than f(B)

B is suboptimalh = 0 at a goal

Optimality of A* Tree Search: BlockingProof:Imagine B is on the frontierSome ancestor n of A is on the frontier, too (maybe A!)Claim: n will be expanded before Bf(n) is less or equal to f(A)f(A) is less than f(B) n expands before BAll ancestors of A expand before BA expands before BA* search is optimal

Properties of A*Properties of A*bbUniform-CostA*UCS vs A* ContoursUniform-cost expands equally in all directions

A* expands mainly toward the goal, but does hedge its bets to ensure optimalityStartGoalStartGoal60Video of Demo Contours (Empty) -- UCS

Video of Demo Contours (Empty) -- Greedy

Video of Demo Contours (Empty) A*

Video of Demo Contours (Pacman Small Maze) A*

Comparison

GreedyUniform CostA*A* ApplicationsVideo gamesPathing / routing problemsResource planning problemsRobot motion planningLanguage analysisMachine translationSpeech recognition[Demo: UCS / A* pacman tiny maze (L3D6,L3D7)][Demo: guess algorithm Empty Shallow/Deep (L3D8)]

Video of Demo Empty Water Shallow/Deep Guess Algorithm

Creating Heuristics

Creating Admissible HeuristicsMost of the work in solving hard search problems optimally is in coming up with admissible heuristics

Often, admissible heuristics are solutions to relaxed problems, where new actions are available

15

366Example: 8 PuzzleWhat are the states?How many states?What are the actions?How many actions from the start state?What should the step costs be?

Start StateGoal State

Actions8 Puzzle IHeuristic: Number of tiles misplacedWhy is it admissible?h(start) =This is a relaxed-problem heuristic8Average nodes expanded when the optimal path has4 steps8 steps12 stepsUCS1126,3003.6 x 106TILES1339227

Start StateGoal StateStatistics from Andrew Moore8 Puzzle IIWhat if we had an easier 8-puzzle where any tile could slide any direction at any time, ignoring other tiles?

Total Manhattan distance

Why is it admissible?

h(start) =

3 + 1 + 2 + = 18Average nodes expanded when the optimal path has4 steps8 steps12 stepsTILES1339227MANHATTAN122573

Start StateGoal StateCombining heuristicsDominance: ha hc if

Roughly speaking, larger is better as long as both are admissibleThe zero heuristic is pretty bad (what does A* do with h=0?)The exact heuristic is pretty good, but usually too expensive!What if we have two heuristics, neither dominates the other?Form a new heuristic by taking the max of both:

Max of admissible heuristics is admissible and dominates both!Example: number of knights moves to get from A to Bh1 = (Manhattan distance)/3 (rounded up to correct parity)h2 = (Euclidean distance)/5 (rounded up to correct parity)h3 = (max x or y shift)/2 (rounded up to correct parity)

Semi-lattice: x