1

PLAN RECOGNITION &

USER INTERFACES

Sony Jacob

March 4th, 2005

2

AGENDA

Motivation– Examples

Introduction– Collaboration

Plan Recognition vs. Traditional Systems Plan Recognition System “Steve” Conclusions Discussion

3

EXAMPLE: TRADITIONAL SYSTEM

Microsoft Interactive help system Provides “jump-in” help instead of relevant on-going collaborative

help Unable to comprehend overall goals and does not use task model Most users are unable to use this feature successfully

4

EXAMPLE: PLAN RECOGNITION SYSTEM

Charles RichCandy SidnerNeal LeshAndrew GarlandShane BoothMarkus Chimani2004Mistubishi Electric Research Laboratory

5

EXAMPLE: PLAN RECOGNITION SYSTEM

Charles RichCandy SidnerNeal LeshAndrew GarlandShane BoothMarkus Chimani2004Mistubishi Electric Research Laboratory

6

MOTIVATION

Minimize amount of initiative required from user

Create simple and consistent interfaces

Guide user without limiting capability

7

COLLABORATION DIAGRAM

SHARED ARTIFACT(GUI)

GOALS

COMMUNICATION

PRIMIVITIVE

ACTIONS

PRIMIV

ITIV

E

ACTIO

NS

Model for Plan Recognition in user interfaces

8

COLLABORATION FRAMEWORK

Defined as an Interaction between “Agent” and User– Mutual Goals– Agent and user can perform actions– Agent uses Plan Tree

Hierarchal partially ordered representation of actions to achieve goals

Methods of interaction– Discussion between agent and user– User conveys intentions through actions– Agent solicits clarification

9

EXAMPLE: PLAN TREE

Charles RichNeal LeshAndrew Garland2002

Mistubishi Electric Research Laboratory

10

EXAMPLE: PLAN RECOGNIZER

Charles RichCandy SidnerNeal Lesh1998Mistubishi Electric Research Laboratory

11

RESPONSES

Possible actions for an agent– Move user to next step of task (goal oriented)– Confirm completion of a goal– Allow user initiative– Focus user to current goal– Explain steps needed for a task– Discover and report incorrect actions

12

RESPONSES CONTINUED

Traditional responses– Application dependent

If(user pressed button A)Call function A

Collaborative responses – Application independent

If(user completed a step in current task)Go to next step of task

13

INTERACTION

Traditional system– Limited range

1. Tutoring systems– Agent has majority of plan knowledge and initiative

2. Help system– User has majority of plan knowledge and initiative

– Turn based interaction User performs action and agent responds

14

INTERACTION CONTINUED

Collaborative System – Broad range

System can shift incrementally within this range– Examples mentioned in Traditional systems represent

extremes of this range Mode depends on current task

– Non-turn based interaction User may perform 0 or more actions followed by 1 or

more communications Agent may perform 0 or more actions followed by 1 or

more communications

15

COMMUNICATION

Traditional Communication– User initiates system actions through “commands”– Requires user to have knowledge of command

Collaborative Communication– On-going Discourse between agent and user

Define goals and how to achieve them Discuss task being performed

– Requires user to have common goal with agent

16

HELP SYSTEMS

Traditional help systems – Wizards, Tool-tips, Help Assistants– Attempt to compensate for lack of user knowledge– Requires separate interaction by user

Collaborative help system– Integrated as part of the interface– User knowledge level does not affect level of help

system interaction

17

EXAMPLE: WIZARDS

Wizards– Provide a guided interaction for user– Partially follows collaborative paradigm

Lacks versatility to allow user to take initiative Goals cannot be adjusted

18

ADVANTAGES OF DIAMOND HELP

Provides consistent interaction paradigm– Different applications of Diamond help will be

familiar to user– Appearance and operation remains the same

Used for appliances and control systems Possible expansion allows for speech-

enabled interaction– Agent speaks interaction and performs speech

recognition for user

19

PLAN RECOGNITION SYSTEM EXAMPLE: STEVE

Training agent– Steve (Soar Training Expert for Virtual Environments)

Virtual reality tutoring system, agent embodiment Uses plan recognition to guide user Actions taken by user are interpreted and compared to plan

tree Steve orients user towards goal based on plan tree Advises user when a deviation is made from the plan tree or

when help is needed for the next step

Demo– http://www.isi.edu/isd/VET/steve-demo.html

20

CONCLUSIONS

Effective Collaboration– Abstract representation of situation

Key to reuse and modularity of components

– Well-designed task model Hierarchy must model tasks which complete a goal or

sub-goal

– Focus must be maintained If goal is modified, focus “stack” must be adjusted

21

CONCLUSIONS CONTINUED

Complexity and scalability– Must be able to create abstract representations

for various tasks– Depends on modularity and reusability of

components – For complex interactions, must allow more direct

user actions– Require Sub-goals for top goals

22

DISCUSSION

Questions?

•How do we program a collaborative system?

•What are the drawbacks of creating this system?

•How does the agent realize a plan for a complex system?

•How would I implement this in my workplace environment?

23

PROGRAMMING TECHNIQUE

Object Oriented Plug-ins– Use separate components to construct desired

interface Composable and reusable components Abstract class definitions for dialogs Low-level functionality is controlled and monitored by

high-level plan recognition system

24

References

Note: Web addresses only http://km.aifb.uni-karlsruhe.de/ws/LLWA/abis/schneider.pdf

http://www.merl.com/papers/docs/TR98-23.pdf http://www.merl.com/reports/docs/TR2002-10.pdf

http://rpgoldman.real-time.com/papers/discex01pr.pdf

http://www.isi.edu/isd/VET/eca00.pdf