4/17: Plan Recognition

Several slides borrowed from Kautz’s overview talk

Plan/Goal Recognition• Recognize the plan and/or higher level goal(s) of the agent from partial

observations of the agent’s behavior (actions done; states of the world visited)

– Recognition can be used to either aid or thwart the agent’s plans and/or make sense of their actions

• Very active area of late with many shades..– Intention Recognition; Goal Recognition; Plan recognition; Activity Recognition;

Behavior Recognition• Applications include:

– Dialog Understanding (if you recognize an agent’s plan/goals) you can do a better job of understanding their actions

– Activity/Behavior recognition (recognize suspicious activities; recognize need to assist cognitively impaired people)

– Intrusion detection/avoidance– User intent recognition to provide intelligent user interfaces– Assistance (if you recognize what the agent is trying to do, you can help the

agent do it better/faster etc.)• But if you mis-recognize, you will have the woman and old man on the Bus scenario.

Levels of Recognition

Physical movement Movement sensor fires

Behaviors Running, grasping, lifting, …

Plans Getting a drink of water Describes conventional way of achieving a goal

Goals Quench thirst

Research efforts normally may start and end in the middle… e.g. from plansgoals orFrom movementbehaviors

Generally, the PR literature separates “Goals” from normal state variables (can think of goals as higher-order state variables… Thus the popularity of hierarchical models)

On the importance of pre-set goals in plan recognition In normal planning settings, we take the view that any set of

states can be designated goal states. So, with n state variables, we can have 2^2^n possible goal states.

In plan recognition, we have to assume that the agent is interested in a much smaller set of goals (than 2^2^n). If you allow all sets of states to be potentially goal states, then

there is nothing to recognize (whatever state you see the agent reaching is potentially the state the agent wanted to reach!)

The set of potential agent goals is circumscribed in two ways: Provide hierarchical plan libraries (e.g. HTN schemata). You

recognize the plan under observation by parsing it in terms of the HTN schema [e.g. Kautz’s original work; Charniak/Goldman work etc.]

Provide (hierarchical) goal libraries. You match the states encountered with the possible goals they are satisfying. (e.g. the Hong work on goal graphs; most of the work on activity recognition).

Planning vs. Plan Recognition• It is intuitively clear that a plan recognizer can benefit from the knowledge of “goals”

of the agent, as well as the domain model – But which domain model?

• The *real* domain model?• The model according to the agent acting• The model according to the recognizer• The that the recognizer thinks the agent has of the world?

– In theory, all of these could be different.. • If agent thinks that speaking loudly will help blind people understand better, then the agent may be

rational w.r.t. its model)• If the recognizer thinks that when people step out the front door, they just stand behind that door until

they decide to come in, then that is the model it will use – Plan recognition is an inherently multi-agent activity!

• Planning is also relevant in two ways– If the recognizer knows the agent’s goal and the domain model, then it can figure out what

the agent’s policy should be (notice that the agent may not be optimal even with respect to its potentially faulty model—so the plan that the recognizer comes up with may only be an approximation to what the agent actually does)

– If the recognizer wants to help or thwart the agent, it will need to make a plan based on its recognition of the agent’s goals as well as the preconditions of the agent’s actions

• You can facilitate or defeat the preconditions based on whether you want to help or thwart

Interactive Spelling Correction--and word completion

• Given a dictionary of words

• A partial or complete typing of a “word”

• Complete/Correct the word

• argmaxc P(c|w)

• argmaxc P(w|c) P(c) / P(w) • P(w|c) Error model

– What is the probability that you will type w when you meant c?

– Different kinds of errors (e.g. letter swapping) have different prob

– Consider edit distance

• P(c) “language model”

- How frequent is c in the language that is used?

http://norvig.com/spell-correct.html

In Auto-completion, you are trying to suggest most likely completion of the word you are typing…(even in face of typing errors—a la Ipod.)

Comparing Spelling correction to plan recognition…

• Dictionary of words ~ Library of Plans• Errors ~ “wrong plans”

– Is the passenger looking furtively around while going through security using a wrong normal travel plan or right “blow up the plane and not get caught” plan?

• Language Model ~ Plan/Goal Corpus.. (with what goals are more likely)

Dimensions of the behavior recognition problem

Keyhole versus interactive Keyhole

Determine how an agent’s actions contribute to achieving possible or stipulated goals

No model of the observer – fly on the wall Interactive

Actions performed by an agent to signal to another agent

Speech acts Model social conventions & agents’ models of other

agents

Dimensions of the plan recognition problem

Ideal versus fallible agents Mistaken beliefs

John drives to Reagan, but flight leaves Dulles. Cognitive errors

Distracted by the radio, John drives past the exit. Irrationality

John furiously blows his horn at the car in front of him.

Dimensions of the plan recognition problem

Reliable versus unreliable observations “There’s a 80% chance John drove to Dulles.”

Open versus closed worlds Fixed plan library? Fixed set of goals?

Metric versus non-metric time John enters a restaurant and leaves 1 hour later. John enters a restaurant and leaves 5 minutes later.

Single versus multiple ongoing plansAlso, does the PR agent observe actions or the state? Normally, only the state is observed. In computer game worlds etc where the agent initiates an action by pressing specific buttons, actions can also be observed

Dimensions of the plan recognition problem

Desired output: Set of consistent plans or goals? Most likely plan or goal? Most critical plan or goal? Interventions observer should perform to aid or

hinder the agent?Also, is the recognition done at the end of the observed behavior, or online.

In intrusion detection, you may want to know all possible consistent plans (so you can defeat all of them); in passive assistance, you may want to look at a set of likely plans (so you can ‘prefetch’ information to speedup their execution). In active assistance, you will likely need to stick to a single most likely plan/goal

Approaches to plan recognition Consistency-based

Hypothesize & revise Closed-world reasoning Version spaces

Probabilistic Stochastic grammars Pending sets Dynamic Bayes nets Layered hidden Markov models Policy recognition Hierarchical hidden semi-Markov models Dynamic probabilistic relational models Example application: Assisted Cognition

Can be complementary.. First pick the consistent plans, and check which of them is most likely (tricky if the agent can make errors)

Hypothesize & Revise

The Plan Recognition Problem C. Schmidt, 1978

Based on psychological theories of human narrative understanding

Mention of objects suggest hypothesis

Pursue single hypothesis until matching fails

Closed-world reasoning

A Formal Theory of Plan Recognition and its Implementation Henry Kautz, 1991

•Infers the minimum set(s) of independent plans that entail the observations

•Observations may be incomplete

•Infallible agent

•Complete plan library

Similar to Bottom-up HTN parsing..

Bottom-up HTN Parsing vs. Plan Recognition

• One way of doing HTN planning we discussed involved keeping incremental parses of the plans being generated by the underlying non primitive planner [Barrett & Weld; 1994]

• In a way the HTN parser was “recognizing” what the searcher below was doing

• However, it only cared to make sure that there is a non-empty set of partial parses

• We

Goal model &put the applications up front..

Goal Graph-based recognition

Separately identified “Goal” schemas At each level, state literals supporting goal schemas are identified An action done is “relevant” to a goal schema if its effect supports it directly or indirectlyThe “consistent” goal schemas are those for which all the observed actions are relevant. Output htese…. [Hong; JAIR 2000]

Can’t recognize “sequential” goals unless goal schemas are in CTL

Version Space Algebra

A sound and fast goal recognizer Lesh & Etzioni Programming by Demonstration Using Version Space Algebra Lau,

Wolfman, Domingos, Weld.

•Recognizes novel plans

•Complete observations

•Sensitive to noise

Approaches to plan recognition Consistency-based

Hypothesize & revise Closed-world reasoning Version spaces

Probabilistic Stochastic grammars Pending sets Dynamic Bayes nets Layered hidden Markov models Policy recognition Hierarchical hidden semi-Markov models Dynamic probabilistic relational models Example application: Assisted Cognition

Can be complementary.. First pick the consistent plans, and check which of them is most likely (tricky if the agent can make errors)

Next class