dialog management 1 intelligent robot lecture note
Post on 18-Dec-2015
226 views
TRANSCRIPT
Dialog Management
Dialogue System
• A system to provide interface between the user and a computer-based application
• Interact on turn-by-turn basis• Dialogue manager
► Control the flow of the dialogue► Main flow
◦ information gathering from user◦ communicating with external application◦ communicating information back to the user
► Three types of dialogue system◦ finite state- (or graph-) based◦ frame-based ◦ agent-based
3Intelligent Robot Lecture Note
Dialog Management
Dialog System Architecture
• Typical dialog system has following components► User Interface
◦ Input: Speech Recognition, keyboard , Pen-gesture recognition ..◦ Output: Display, Sound, Vibration ..
► Context Interpretation◦ Natural language understanding (NLU)◦ Reference resolution◦ Anaphora resolution
► Dialog Management◦ History management◦ Discourse management
• Many dialog system architectures are introduced. ► DARPA Communicator► GALAXY Communicator► etc.
4Intelligent Robot Lecture Note
Dialog Management
Dialog System Architecture
• The DARPA Communicator program was designed to support the creation of speech-enabled interfaces that scale gracefully across modalities, from speech-only to interfaces that include graphics, maps, pointing and gesture.
5
AT&T
CMU
MIT
CU
BBN
Bell Lab
SRIDARPA
Intelligent Robot Lecture Note
Dialog Management
Galaxy Communicator
• The Galaxy Communicator software infrastructure is a distributed, message-based, hub-and-spoke infrastructure optimized for constructing spoken dialogue systems.
• An open source architecture for constructing dialogue systems • History
► MIT Galaxy system► Developed and maintained by MITRE Corporation► Current version is 4.0
6Intelligent Robot Lecture Note
Dialog Management
CU Communicator
• Dialogue management in CU Communicator► Event-driven approach
◦ Current context of the system is used to decide what to do next◦ Do not need a dialogue script ◦ A general engine operates on the semantic representations and the current
context to control the interaction flow► Mixed-initiative approach
◦ Not separate “user initiative” and “system initiative”
9Intelligent Robot Lecture Note
Dialog Management
10
CMU Communicator
• Dialogue management in CMU Communicator► Frame-based approach
◦ Form-filling method◦ Not to specify a particular order in which slots need to be filled◦ Loosen the requirement for the system designed to correctly intuit the
natural order in which information is supplied► Agenda-based approach
◦ Treats the task as one of cooperatively constructing a complex data structure, a product
◦ Uses a product tree which is developed dynamically◦ Supports topic shifts
Intelligent Robot Lecture Note
Dialog Management
11
Queen’s Communicator
• Object-oriented architecture, distributed and inherited functionality: generic and domain-specific
• Uses discourse history and confirmation status to determine how to confirm (explicit or implicit)
Intelligent Robot Lecture Note
Dialog Management
Dialog System approaches
• There are many approaches to represent dialog► Frame based► Agent based ► Voice-XML based ► Information State approach
13Intelligent Robot Lecture Note
Dialog Management
14
Frame-based Approach
• Frame-based system► Asks the user questions to fill slots in a template in order to perform a
task (form-filling task)► Permits the user to respond more flexibly to the system’s prompts (as
in Example 2.)► Recognizes the main concepts in the user’s utterance
Example 1)• System: What is your destination?• User: London.• System: What day do you want to
travel?• User: Friday
Example 2) System: What is your destination? User: London on Friday around
10 in the morning. System: I have the following
connection …
Intelligent Robot Lecture Note
Dialog Management
15
Frame-based Approach
• Advantages► The ability to use natural language, multiple slot filling► The system processes the user’s over-informative answers and
corrections
• Disadvantages► Appropriate for well-defined tasks in which the system takes the
initiative in the dialog► Difficult to predict which rule is likely to fire in a particular context
• Related systems► CU Communicator► CMU Communicator
Intelligent Robot Lecture Note
Dialog Management
16
Agent-based Approach
• Properties► Complex communication using unrestricted natural language► Mixed-Initiative► Co-operative problem solving► Theorem proving, planning, distributed architectures► Conversational agents
• Examples
User : I’m looking for a job in the Calais area. Are there any servers?
System : No, there aren’t any employment servers for Calais. However, there is an employment server for Pasde-Calais and an employment server for Lille. Are you interested in one of these?
System attempts to provide a more co-operative response that might address the user’s needs.
Intelligent Robot Lecture Note
Dialog Management
17
Agent-based Approach
• Advantages► Suitable to more complex dialogues► Mixed-initiative dialogues
• Disadvantages► Much more complex resources and processing► Sophisticated natural language capabilities► Complicated communication between dialogue modules
• Related Works► TRAINS project► TRIPS project
Intelligent Robot Lecture Note
Dialog Management
18
• TRAINS (1995~1997)► CISD research group in University of Rochester
◦ http://www.cs.rochester.edu/research/cisd/projects/trains/► Task
◦ Finding efficient routes for trains► Goal
◦ Robust performance on a very simple task► Approach
◦ Speech Act, Plan reasoning► Demo
◦ http://www.cs.rochester.edu/research/cisd/projects/trains/movies/TRAINS95-v1.3-Pia.qt.gz
TRAINS project
Intelligent Robot Lecture Note
Dialog Management
TRIPS Project
• TRIPS► The Rochester Interactive Planning System
◦ http://www.cs.rochester.edu/research/cisd/projects/trips/► Goal
◦ An intelligent planning assistant (natural language + graphical display)
◦ Extending TRAINS system to several domain► Domains (supported currently)
◦ Pacifica - Evacuating people from an island◦ Airlift – Organization Airlift scheduling◦ TRIPS-911 – Managing the resources in small 911 emergency◦ Underwater Survey – Planning in collaboration with semi-
autonomous robot agents► Demo (Pacifica)
◦ http://www.cs.rochester.edu/research/cisd/projects/trips/movies/TRIPS-98_v4.0/200K/TRIPS-98_v4.0_200K.html
19Intelligent Robot Lecture Note
Dialog Management
21
VoiceXML-based System
• What is VoiceXML?► The HTML(XML) of the voice web.► The open standard markup language for voice application
• Can do► Rapid implementation and management► Integrated with World Wide Web► Mixed-Initiative dialogue► Able to input Push Button on Telephone► Simple Dialogue implementation solution
Intelligent Robot Lecture Note
Dialog Management
Dialogue by VoiceXML
• Most VoiceXML dialogues are built from ► <menu>► <form> form based dialog
• Form-based dialogue is similar to “Slot & Filling” system
• Limiting User’s Response► Goal
◦ Verification, and Help for invalid response◦ Good speech recognition accuracy
22Intelligent Robot Lecture Note
Dialog Management
23
Example - <Menu>
<vxml version="2.0" xmlns="http://www.w3.org/2001/vxml"> <menu> <prompt>Say one of: <enumerate/></prompt> <choice next="http://www.example.com/sports.vxml"> Sports scores </choice> <choice next="http://www.example.com/weather.vxml"> Weather information </choice> <choice next="#login"> Log in </choice> </menu> </vxml>
Browser : Say one of: Sports scores; Weather information; Log in.
User : Sports scores
Intelligent Robot Lecture Note
Dialog Management
24
Example – <Form>
<vxml version="2.0" xmlns="http://www.w3.org/2001/vxml"> <form id="login"> <field name="phone_number" type="phone"> <prompt> Please say your complete phone number
</prompt> </field> <field name="pin_code" type="digits">
<prompt>Please say your PIN code
</prompt> </field> <block>
<submit next=“http://www.example.com/servlet/login” namelist=phone_number pin_code"/> </block> </form> </vxml>
Browser : Please say your complete phone number
User : 800-555-1212
Browser : Please say your PIN code
User : 1 2 3 4
Intelligent Robot Lecture Note
Dialog Management
25
Information State Approach
• A method of specifying a dialogue theory that makes it straightforward to implement
• Consisting of following five constituents► Information Components
◦ Including aspects of common context◦ (e.g., participants, common ground, linguistic and intentional structure,
obligations and commitments, beliefs, intentions, user models, etc.)► Formal Representations
◦ How to model the information components◦ (e.g., as lists, sets, typed feature structures, records, etc.)
Intelligent Robot Lecture Note
Dialog Management
26
Information State Approach
► Dialogue Moves◦ Trigger the update of the information state◦ Be correlated with externally performed actions
► Update Rules◦ Govern the updating of the information state
► Update Strategy◦ For deciding which rules to apply at a given point from the set of applicable
ones
Intelligent Robot Lecture Note
Dialog Management
Reading Lists
• B. Pellom, W. Ward, S. Pradhan, 2000. The CU Communicator: An Architecture for Dialogue Systems, International Conference on Spoken Language Processing (ICSLP), Beijing China.
• Rudnicky, A., Thayer, E., Constantinides, P., Tchou, C., Shern, R., Lenzo, K., Xu W., Oh, A. 1999. Creating natural dialogs in the Carnegie Mellon Communicator system. Proceedings of Eurospeech, 531-1534.
• Ian M. O’Neill and Michael F. McTear. 2000. Object-Oriented Modelling of Spoken Language Dialogue Systems Natural Language Engineering, Best Practice in Spoken Language Dialogue System Engineering, Special Issue, Volume 6 Part 3.
• George Ferguson and James Allen, July 1998. TRIPS: An Intelligent Integrated Problem-Solving Assistant," in Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98), Madison, WI, 26-30, pp. 567-573.
33Intelligent Robot Lecture Note
Dialog Management
Reading Lists
• S. Larsson, D.R. Traum. 2001. Information state approach to dialogue management. Current and New Directions in Discourse & Dialogue, Kluwer Academic Publishers.
• S. Larsson, D.R. Traum. 2003. Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural Language Engineering.
34Intelligent Robot Lecture Note
Dialog Management
Reinforcement Learning
36
Inputs(Feature, Target Label)
Supervised LearningSystem
Outputs
Training Info = desired (target) outputs
Objective: To minimize error (Target Output – Actual Output)
Inputs(State, Action, Reward)
RLSystem
Outputs (“actions”)
Training Info = evaluations (“rewards”/”costs”)
Objective: To get as much reward as possible
Intelligent Robot Lecture Note
Dialog Management
Stochastic Modeling Approach
• Stochastic Dialog Modeling [E. Levin et al, 2000]► Optimization Problem
◦ Minimization of Expected Cost (CD)
► Mathematical Formalization◦ Markov Decision Process
– Defining State Spaces, Action Sets, and Cost Function– Formalize dialog design criteria as objective function
► Automatic Dialog Strategy Learning from Data◦ Reinforcement Learning
37
ii
iD CC Ci measures the effectiveness and the achievement of application goal
Intelligent Robot Lecture Note
Dialog Management
Mathematical Formalization
• Markov Decision Process (MDP)► Problems with cost(or reward) objective function are well modeled as
Markov Decision Process.► The specification of a sequential decision problem for a fully
observable environment that satisfies the Markov Assumption and yields additive rewards.
38
Dialog Action(Prompts, Queries, etc.)
Dialog Manager
Environment (User, External DB or other Servers)
Dialog StateCost(Turn, Error, DB Access, etc.)
Intelligent Robot Lecture Note
Dialog Management
Dialog as a Markov Decision Process
39
User
SpeechUnderstanding
SpeechGeneration
StateEstimator
DialogPolicy
Optimize
k
kk rR
us
ua
ma
ua~
ma~
duum ssas ~,~,~~
MDP
usergoal
userdialog act
noisy estimate ofuser dialog act
dialoghistory
machinestate
machinedialog act
ReinforcementLearning
Reward
),( mm asr
ms~
ds
[S. Young, 2006]
Intelligent Robot Lecture Note
Dialog Management
Month and Day Example
• State Space► State St represents all the knowledge of the system at time t
(values of the relevant variables).◦ St=(d, m) where d=-1,…,31 and m=-1,..,12
◦ 0 : not yet filled◦ -1 : completely filled◦ (0,0) = Initial State ◦ (-1,-1) = Final State
40Intelligent Robot Lecture Note
Dialog Management
Month and Day Example
• State Space
41
--
Day:1-
Day:31-
Day:30-
Day:1Month:1
Day:30Month:12
Day:31Month:12
-Month:1
-Month:12
-Month:11
--
1 (initial) + 12(months) + 31(days) + 365(dates) + 1(final)
Total Dialog State : 410 statesIntelligent Robot Lecture Note
Dialog Management
Month and Day Example
• Action Set► At each state, the system can choose an action at.
◦ Dialog Actions– Asking the user for input, providing a user some output, confirmation,
etc.
42
Which month? (Am)Which day? (Ad)Which date? (Adm)Thank you. Good Bye.(Af)
St
Intelligent Robot Lecture Note
Dialog Management
Month and Day Example
• State Transitions► When an action is taken the system changes its state.
43
New state might depend on external inputs:Not Deterministic
Transition Probability: PPTT(S(St+1t+1|S|Stt,a,att))
--
-Month: 12
SYSTEM : Which month?
-Month: 11
-Month: 1
Intelligent Robot Lecture Note
Dialog Management
Month and Day Example
• Action Costs and Objective Function► A cost Ct is associated to action at at state St.
44
Cost Distribution: PPcc(C(Ctt|S|Stt,,aatt))
slots unfilled nsinteractio *#Errors*#*#C feiD
--
-Month: 12
SYSTEM : Which month?
-Month: 11
-Month: 1
Intelligent Robot Lecture Note
Dialog Management
Month and Day Example
45
Optimal strategy is the one that minimizes the cost.
Strategy 1 is optimal if wi + P2* we - wf > 0 Recognition error rate is too high
Strategy 1.
Strategy2.
Strategy 3.
Good Bye.
--
--
DayMonth
Which date ? Good Bye.
--
DayMonth
Day-
Which day ? Which month?
--
Good Bye.
--
--
2 11 **C fi
0 P*2* 3 23 **C fei
Strategy 3 is optimal if 2*(P1-P2)* we - wi > 0 P1 is much more high than P2 against a cost of longer interaction
0 P*2* 2 12 **C fei
Intelligent Robot Lecture Note
Dialog Management
Policy
• The goal of MDP is to learn a policy, π : S→A► But we have no training examples of form <s,a>► Training examples are of form <s,a,s’,r>► For selecting it next action at based on the current observed state st.
46
S0 S1 S2 …a0
r0
a1
r1
a2
r2
Goal : Learn to choose actions that maximize the reward function.
)10 (where ...0
22
1
iit
ittt rrrr
discount factor
Intelligent Robot Lecture Note
Dialog Management
Policy
• Discounted Cumulative Reward► Infinite-Horizon Model
◦ γ=0 : Vπ(st) =rt
– Only immediate reward considered.◦ γ closer to 1 : Delayed Reward
– Future rewards are given greater emphasis relative to the immediate reward.
• Optimal Policy (π*)► Optimized policy π that maximize Vπ(s) for all state s.
47
)10 (where ...)(0
22
1
iit
itttt rrrrsV
sfor )(argmax* sV
Intelligent Robot Lecture Note
Dialog Management
Q-Learning
• Define the Q-Function.► As evaluation function.
• Rewrite the optimal policy.
• Why is this rewrite important?► It shows that if the agent learns the Q-function instead of the V*
function.◦ It will be able to select optimal actions even when it has no knowledge of
the function r and δ.
48
)),((),(),( * asVasrasQ
),(maxarg)(
)),((),(maxarg
*
**
asQs
asVasr
a
a
Intelligent Robot Lecture Note
Dialog Management
Q-Learning
• How can Q be learned?► Learning the Q function corresponds to learning the optimal policy.
◦ The close relationship between Q and V*
► It can be written recursively as
◦ This recursive definition of Q provides the basis for algorithm that iteratively approximate Q.
► It can updates the table entry for Q(s,a) following each such transition, according to the rule.
49
)',(axm)('
* asQsVa
)','(ˆaxm),(ˆ'
asQrasQa
)'),,((axm),(),('
aasQasrasQa
Intelligent Robot Lecture Note
Dialog Management
Q-Learning
• Q-Learning algorithm for deterministic MDP.
50Intelligent Robot Lecture Note
Dialog Management
Action Selection in Q-Learning
• How actions are chosen by the agent.► To select the action that maximize the Q hat function.
◦ Thereby exploiting its current approximation Q hat.◦ Biased to previously trained Q hat function.
► Probability Assigning◦ Actions with higher Q hat values are assigned higher probabilities.◦ But every action is assigned a nonzero probability.
◦ k > 0 is a constant that determines how strongly the selection favors actions with high Q hat values.
– Larger values of k will assign higher probabilities to actions with above average Q hat.
– Causing the agent to exploit what it has learned and seek actions it believes will maximize its reward.
◦ k is varied with the number of iterations.– Exploitation vs. Exploration
51
j
asQ
asQ
ij
i
k
ksaP
),(ˆ
),(ˆ
)|(
Intelligent Robot Lecture Note
Dialog Management
Example-based Dialogue Modeling
• Limitation of Rule-based Dialogue Modeling► For the situation-action rule, there are about possible 213 states of
EPG domain.◦ Problem
– Much Human Efforts– Inconsistency– Unreliability
• How to automatically design situation-based rules► We have developed example-based dialogue modeling.
◦ Using dialogue examples indexed from dialogue corpus.◦ It is more effective and domain portable.
– Because it is able to automatically generate system responses from dialogue example.
52Intelligent Robot Lecture Note
Dialog Management
53
• Dialogue Example Database► Semantic-based indexing of dialogue examples
◦ Lexical-based example database needs much more examples.◦ The SLU results is the most important index key.
► Automatically indexing from dialogue corpus.
Example-based Dialogue Modeling
Utterance 그럼 SBS 드라마는 언제 하지 ?
Then, when do this SBS dramas start?
Dialog Act Wh-question
Main Action Search_start_time
Component Slots [channel = SBS, genre = 드라마 ]
Discourse History [1,0,1,0,0,0,0,0,0]
System Action Inform(date, start_time, program)
Input : User Utterance
Output : System Concept
Index Keys
Intelligent Robot Lecture Note
Dialog Management
54
Example-based Dialogue Modeling
• Utterance Similarity► When the retrieved dialogue examples are not unique
◦ We choose the best one using the utterance similarity measure.
► How to define the similarity measure for dialogue system.◦ Lexico-Semantic Similarity
– Morpheme Similarity between utterances with the semantic slots using normalized edit distance.
◦ Discourse History Similarity– The cosine similarity between the slot-filling vectors
– The value 1 if the slot is filled until a current dialogue state.– The value 0 otherwise.
Intelligent Robot Lecture Note
Dialog Management
55
• Example of Utterance Similarity► Lexico-Semantic Representation
► Utterance Similarity Measure
Example-based Dialogue Modeling
User Utterance그럼 SBS 드라마는 언제 하지 ?Then, when do this SBS dramas start?
Component Slots [channel = SBS, genre = 드라마 (drama)]
Lexico-Semantic Representation
그럼 [channel] [genre] 는 언제 하 지Then, when do the [channel] [genre] start
그럼 [channel] [genre] 는 언제 하 지
Slot-Filling Vector : [1,0,1,0,0,0,0,0,0][date] [genre] 는 몇 시에 하 니
Slot-Filling Vector : [1,0,0,1,0,0,0,0,0]
Current User Utterance
Retrieved Examples
Lexico-Semantic Similarity
Discourse History Similarity
Intelligent Robot Lecture Note
Dialog Management
56
Strategy of EBDM
DialogueCorpus
Dialogue Example DB
DomainExpert
User’s Utterance
AutomaticIndexing
Retrieval
DiscourseHistory
Query Generation
DialogueExamplesTie-breaking
Lexico-semantic Similarity Discourse history Similarity
Utterance Similarity
SemanticFrame
Best DialogueExample
User Intention
System Responses
DialogueCorpus
DialogueCorpus
Dialogue Example DB
DomainExpert
User’s Utterance
AutomaticIndexing
Retrieval
DiscourseHistory
Query Generation
DialogueExamplesTie-breaking
Lexico-semantic Similarity Discourse history Similarity
Utterance Similarity
Lexico-semantic Similarity Discourse history Similarity
Utterance Similarity
SemanticFrame
Best DialogueExample
User Intention
System Responses
Intelligent Robot Lecture Note
Dialog Management
• Generic Dialogue Modeling► By automatically constructing the dialogue example database from the
dialogue corpus
• Easy Development of an effective and practical dialogue system► Need a small amount of dialogue corpus.
• High Domain Portability► Can be applied to various domains with low cost.
◦ Goal-oriented dialogue system– EPG, Navigation, Weather Information Center
◦ Chat Agent
57
Advantages of EBDM
Intelligent Robot Lecture Note
Dialog Management
58
Case Study I :Example based Multi-domain
Dialogue System Development
Intelligent Robot Lecture Note
Dialog Management
59
POSSDM
• The Basic Idea► Situation-based dialogue management
◦ State-free dialogue management based on the current situation of dialogue
– Dialogue Situation is the dialogue information state.
– Including user intention, semantic frame, and discourse history
► Object-oriented architecture◦ Improving a domain portability
– Separation of domain-independent and domain-dependent dialogue modules.
► Example-based dialogue modeling◦ To generate the system responses according to the current situation using
generic dialogue modeling
Intelligent Robot Lecture Note
Dialog Management
61
ChatDEDB
ChatDialog Corpus
Domain Spotter
Chat Expert
Discourse History Stack
• previous user utterance• previous dialog act and semantic frame• previous scenario session
Frame-SlotExtraction
Dialog Act Identification
DiscourseInference
USER : 어제 여친이랑 싸웠어 .
USER : 어제 여친이랑 싸웠어 .
• Agent = Chat• Domain = Friend
• Dialog Act = statement-non-opinion
• Main Action= Fight• Date = 어제
Retrieved Dialog Examples
• Calculate utterance similarity
System Response SYSTEM : 왜 ? 무슨 일 있
어 ?
SYSTEM : 왜 ? 무슨 일 있어 ?
ChatMeta-Rule
XML Rule Parser
• When no example is retrieved, meta-rules are used.
Agent Spotter
Chat Expert
Intelligent Robot Lecture Note
Dialog Management
62
EPGDEDB
EPGDialog Corpus
EPG Expert
Discourse History Stack
• previous user utterance• previous dialog act and semantic frame• previous slot-filling vector
Frame-SlotExtraction (EPG)
Dialog Act Identification
DiscourseInference
USER : TV 에서 지금 뭐 하지 ?
USER : TV 에서 지금 뭐 하지 ?
• Agent = Task• Domain = EPG
• Dialog Act = Wh-question
• Main Action= Search_Program• Start_Time = 지금
Retrieved Dialog Examples
• Calculate utterance similarity
System Response
EPGMeta-Rule
XML Rule Parser
• When no example is retrieved, meta-rules are used.
Domain SpotterAgent Spotter
SYSTEM : 현재 “ KBS” 에서는 “해피선데이”가 , “MBC” 에서는 “일요일 일요일 밤에”가 , “SBS”에서는 “일요일이 좋다”가 방송 중 입니다 .
SYSTEM : 현재 “ KBS” 에서는 “해피선데이”가 , “MBC” 에서는 “일요일 일요일 밤에”가 , “SBS”에서는 “일요일이 좋다”가 방송 중 입니다 .
Web Contents
DatabaseManager
TV ScheduleDatabase
Goal-oriented Dialog Expert
Intelligent Robot Lecture Note
Dialog Management
63
Experiment and Result
• Dialog Corpus & Experiment Setup► # of Chat Corpus = 2377 user utterance in 10 domains► # of Goal-Oriented Dialog Corpus = 513 user utterances in EPG and
Navigation domains► Avg. # word per utt. = 3.22► Distribution of the domain in the dialog corpus
Intelligent Robot Lecture Note
Dialog Management
64
Experiment and Result
• Spotting Evaluation► 10-fold cross validation using Maximum Entropy Classifier
► For the baseline performance of the domain spotter, we evaluated only using the TF*IDF weighting alone.
Feature Set Accuracy (%)
Baseline (Only Linguistic Features) 96.69
Semantic Features+ Dialog Act 97.39
+ Main Action 98.09
Feature Set Accuracy(%)
Baseline (TF*IDF) 72.88
Linguistic Features 77.47
Semantic Features 77.92
Keyword Features+2-best keyword 78.87
+2-best domain class 86.18
Intelligent Robot Lecture Note
Dialog Management
65
Experiment and Result
• Dialog Modeling Evaluation► Human Evaluation: 4 test volunteers ( 422 user utterances )► EMR designates the average ratio of the example match type for user
utterance input.► STR designates the average success turn rate of the response correctness.► The exact match means that the dialog examples were successfully retrieved
when using all indexing keys.► The partial match means that the dialog examples were retrieved when
using parts of indexing keys after the failure of the exact match query.
Intelligent Robot Lecture Note
Dialog Management
66
Experiment and Result
• Dialog Modeling Evaluation► Example Matching Rate (EMR) and Success Turn Rate (STR)
► Goal-oriented dialog evaluation of UMDM
Example Match Type EMR STR
Exact Match 0.60 0.69
Partial Match 0.36 0.52
No Example 0.04 0.06
Evaluation Goal-Oriented Dialog
Success Turn Rate 0.75
Task Completion Rate 0.81
Intelligent Robot Lecture Note
Dialog Management
68
Case Study II : Statistical Dialog System Design
68Intelligent Robot Lecture Note
Dialog Management
Air Travel Information System (ATIS)
• ATIS dialog system helps the user to find flight information in an efficient way.
► The efficiency here involves:◦ The duration of the dialog◦ The cost of external resources ◦ The effectiveness of the system output to the user
► Objective function
69
ssoorrii F)N(fNNC
<Ni> = The expected length of the whole interaction in number of turns<Nr> = The expected number of tuples retrieved from the database during the sessionf(No) = The data presentation cost function with No
Fs = An overall task success measure0 if N*< No
k*N if N*>No
f(No)=1 - no info. was given0 - otherwiseFs=
N* is the reasonable value for data presentation(small for voice based system, higher for display)Intelligent Robot Lecture Note
Dialog Management
The Actions in ATIS
• Greeting : This is ATIS Travel service. How can I help you ?
• Constraining:
• Releasing constraints:
• Database retrieval
• Output data: There are 58 flights: Flight 111 leaves…, Flight 222 …
• Closing: Thank you for using ATIS. Good Bye.
70
Where are you departing from? (Constrain ORIGIN)
What is the airline? (Constrain AIRLINE)
…What time are you leaving? (Constrain DEPARTUE_TIME)
There are no flight with AA. Do youwant to see flight with other airlines? (Relax AIRLINE)
...
Intelligent Robot Lecture Note
Dialog Management
The State in ATIS
• State Space► The state included three templates
◦ A template is a set of keyword-value pairs.
71
User Query
History of system actions
Data retrieved
Representing accumulated information from the user
ORIGIN: XDESTINATION: XAIRLINE: X
The number of data tuples retrieved
Recording a partial history of actions.
GREETINGCONSTRAININGRELAXATION…
Ndata: YIntelligent Robot Lecture Note
Dialog Management
User Model
• Simulated User► Assumption
◦ The user response depends only on the current system action and not the state.► Parameterized the simulated user in the following way
◦ 1) Response to Greeting– P(n), n=0,1,2, … the # of attributes specified by the user in a single utterance.– P(attribute) (e.g. ORIGIN, DESTINATION, AIRLINE, …)– P(Value|attribute) (e.g. P(Boston|ORIGIN), P(Delta|AIRLINE))
◦ 2) Response to Constraining Questions
– P(kR|kG): The prob. of the user specifying a value for attribute kR when asked for the value of attribute kG. P(airline|departue_time)
– P(N|kG): The prob. of providing N additional unsolicited attributes in the same response.
◦ 3) Response to a Relaxation Prompt
– P(yes|kG)=1-P(no|kG): The prob. of accepting(or rejecting) the proposed relaxation of attribute kG.
► We can obtain these probability distributions from dialog corpus.
72Intelligent Robot Lecture Note
Dialog Management
Incrementally More Complex Strategies
73
Closing Cost1 = 14051.
Cost2 = 469.24
Greeting Retrieval Output Closing2.
RetrievalGreeting Output Closing
ConstrainToo muchdata
Cost3 = 231.95
3.
Greeting Retrieval Output Closing
Constrain Too muchdata
Release
No data
Cost4 = 123.93
4.
Intelligent Robot Lecture Note
Dialog Management
The Learned Optimal Strategy
74
Enoughconstraints
Greeting Constrain Retrieval
ConstrainRelease
OutputClosing
Too muchdata
No data
*Same strategy was independently handcraftedin many DARPA ATIS cites:BBN, CMU, AT&T…
Intelligent Robot Lecture Note
Dialog Management
Example of Dialog
75
Untrained System
Trained SystemIntelligent Robot Lecture Note
Dialog Management
Reading Lists
• R. S. Sutton, and A. G. Barto. 1998. Reinforcement Learning: An Introduction. MIT Press
• S. Young. 2006. Reinforcement Learning for Spoken Dialog Systems: Using POMDPs for Dialog Management. SLT
• L. P. Kaelbling, M. L. Littman, and A. W. Moore. 1996. Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research 4:237-285
• E. Levin, R. Pieraccini, and W. Eckert. January 2000. A Stochastic Model of Human-Machine Interaction for Learning Dialogue Strategies. IEEE Transaction on Speech and Audio Processing. 1:11-23
76Intelligent Robot Lecture Note
Dialog Management
Reading Lists
• Cheongjae Lee, Sangkeun Jung, Jihyun Eun, Minwoo Jeong, Gary Geunbae Lee. 2005. Example and situation based dialog management for spoken dialog system. Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop.
• Cheongjae Lee, Sangkeun Jung, Jihyun Eun, Minwoo Jeong, and Gary Geunbae Lee. 2006. A Situation-based Dialogue Management using Dialogue Examples. Proceedings of the 2006 IEEE international conference on acoustics, speech and signal processing.
• Cheongjae Lee, Sangkeun Jung, Minwoo Jeong, and Gary Geunbae Lee. 2006. Chat and Goal-Oriented Dialog Together: A Unified Example-based Architecture for Multi-Domain Dialog Management. Proceedings of the IEEE/ACL 2006 workshop on spoken language technology.
77Intelligent Robot Lecture Note