cs#599:#natural#language# dialogue#systems#...
TRANSCRIPT
CS 599: NATURAL LANGUAGE DIALOGUE SYSTEMS
Lecture 1: Overview of Dialogue Research and Dialogue Systems
David Traum Ins.tute for Crea.ve Technologies University of Southern California
[email protected] h=p://www.ict.usc.edu/~traum
1/17/13 Traum – CS599
Outline
• Basic Terms & Overview of Dialogue Research • Example Systems • Dialogue System Components • Aspects of Dialogue System Genres • (if .me) focus on MRE virtual humans • Brief overview of Special Topics
NL Dialogue Overview
• Communication involving: – Multiple contributions, – Coherent Interaction – More than one participant
• Interaction modalities: – Input: Speech, typing, writing, menu, gesture – Output: Speech, text, graphical display/
presentation, animated body
1/17/13 Traum – CS599
Natural Language Dialogue: Par.cipa.ng in Conversa.on
• Understanding Human Language – What does a person say? – What does the speech mean? – In context of current interac.on
• What did the person try to accomplish? • In terms the virtual human can understand
• IntegraIng Language & Managing Dialogue – How does speech affect virtual human?
• What new informa.on is provided? • What updates have to be done? • What opportuni.es are opened for addressing vhuman goals? • What new obliga.ons and threats must be managed? • How is this informa.on communicated to other modules
– (e.g., planning, emo.on)?
• Producing Language – Deciding when to speak (or listen or act) – Deciding what to say
• choosing the appropriate meaning – Deciding how to say it
• so partner can understand it • So expression seems natural
Dialogue terms
• Dialogue Modelling – Formal characterization of dialogue, evolving
context, and possible/likely continuations • Dialogue system
– System that engages in a dialogue (with a user) • Dialogue Manager
– Module of a system concerned with dialogue modelling and decisions of how to contribute to dialogue
– Cf speech recognizer, domain reasoner, parser, generator, tts,…
1/17/13 Traum – CS599
Dialogue Research Methods • Empirical inves.ga.on
– Gather corpora of in-‐domain dialogue • Natural human-‐human conversa.on • Roleplay • Wizard of OZ • Human-‐machine
– Annotate meanings • Theory development
– How to account for observa.ons • Construc.on of conversa.onal agents
– Ability to converse and act in a specific domain – Integra.on with other abili.es (percep.on, cogni.on, emo.on)
• Empirical evalua.on of systems – Component and integrated performance tes.ng (task comple.on, F-‐
score, appropriateness,…) – Compared to human-‐human – User experience
NA
TUR
AL
CO
NTR
OLLED
Methodology
Theory
Data
Computational Model
Behavior
Corpus
Training Set
Test Set
Artificial Systems
Annotation Scheme
People
CONTEXT
Learn
Implement
Act Record
Observe
Analyze
Motivate
Introspect
Roles for Dialogue Systems
• Information provider • Advisor • Service provider • Collaborative partner • Tutor • Instruction-giver • Conversational Partner • Competitor • Antagonist
1/17/13 Traum – CS599
Spiral methodology: • For a given system, start with simple version
• Then Add – more robustness, – more accurate model of phenomena, – more complex phenomena handled, – more complex tasks handled
Dialogue Systems: State of the Art • Deployed Commercial Systems
– Call routing/call center first contact – Voice menus – Simple information tasks (Siri) – in car navigation & services
• Useful systems – Command & control – Language tutoring – Immersive Training
• Advanced Research Prototypes – Collaborative systems – Adaptive systems – Multi-modal & Robot systems – Companion Systems – Non-cooperative role-playing agents
1/17/13 Traum – CS599
Rochester: TRAINS-93
(Allen et al 1994)
1/17/13 Traum – CS599
MERL: Sample Collagen System:
VCR Help (Rich, Sidner & Lesh 2001)
1/17/13 Traum – CS599
NASA Clarissa System (Rayner et al 2005)
1/17/13 Traum – CS599
U Pittsburgh: ITSpoke (Litman & Silliman 2004)
1/17/13 Traum – CS599
Siri
1/17/13 Traum – CS599
MRE (2001) SASO (2004)
SASO-EN (2007)
ICT Virtual Human Negotiation: Capability-advancing prototypes#
Decision-making
Non-cooperative Negotiation
Multi-party negotiation Persuasion and Conflict resolution
MRE (2003)
Multi-party Interaction
SASO-4 (2011)
ICT Videos
• Twins • Moleno • TACQ • DCAPS
1/17/13 Traum – CS599
Towards a typology of Dialogue Systems • Genres
– What do conversants talk about? – What kinds of things do they say? – Ini.a.ve, exper.se, rules, conven.ons
• Architectures – How many and which kinds of modules (interfaces, processing) – What kinds of informa.on APIs between modules – What are the best trade-‐offs to make to support genre interac.ons?
• Dialogue Knowledge Acquisi.on – Created
• Programmed • Authored
– Learned • From what kinds of data • What kinds of algorithms
SDS Components
• Architecture • Back-end/Domain Reasoners • Input Interface (Audio, Keyboard,etc) • Interpretation (internal representation) • Dialogue Management • Generation • Output Interface
1/17/13 Traum – CS599
Dialogue Modules & Architecture#• Standard Pipeline Architecture#
Automatic Speech
Recognition (ASR)
Natural Language
Understanding (NLU)
Natural Language Generation
(NLG)
Speech Synthesis
(TTS)
Semantic Representation Text
Speech
Dialogue Manager
(DM)
Text
Speech
Semantic Representation
Interpretation: Speech Recognition • Phases
– Signal Processing – Acoustic Model, tri-phones – Language Model (N-grams)
• Issues – Small or large vocabulary – N-gram or grammar-based language model – Integrated or pipelined understanding – Output (concepts, n-best word list, lattice) – Unified or State-specific recognizers
1/17/13 Traum – CS599
Interpretation: Parsing/Semantic Representation • Tasks
– Retrieval/Classification – Understanding/Extraction
• Output – Response – (aspects of) Meaning (e.g., semantic roles, speech acts, parse)
• Styles – Key-word – Language model – Grammar-based – Concept-based (semantic parser) – Expectation-driven
• Spoken Dialogue vs. Written text – Utterance length, grammaticality, interactivity, repairability,
transience, …
1/17/13 Traum – CS599
Dialogue Management Tasks
• Maintaining & Updating Context • Deciding what to do next • Interface with back-end/task model • Provide expectations for interpretation
1/17/13 Traum – CS599
(Cf. Larsson 2005: Interfaces vs Simulations) 1/17/13 Traum – CS599
Generation & Synthesis • Generation
– Output • Text • Prosodic cues • multimodal generation
– Method • Fixed text • Template-based • Sentence Planning & Realization
– Grammar-based – Statistical Language model
• Synthesis – Voice Clip, or TTS – TTS or Concept to Speech
1/17/13 Traum – CS599
Using Data
• Corpus Collection – Human-Human – Wizard of OZ – Human-System
• Annotation – Coding Scheme – Coding
• Automatic • Tool-assisted • Inter-coder Reliability (Kappa)
1/17/13 Traum – CS599
Evaluation
• Objective Metrics – Task success – Resources used (time, turns, attention,..)
• Subjective Evaluation • Issues
– On-line vs off-line – Black Box vs. Glass Box – Class of User (Expert, Novice) – Feedback into system design
1/17/13 Traum – CS599
Specialty Topics for Dialogue Systems • turn-‐taking • mixed-‐ini.a.ve • referring in dialogue • grounding and repair • dialogue act modeling • dialogue act recogni.on • prosody and informa.on structure • Argumenta.on & persuasion • incremental speech processing • mul.-‐modal dialogue • mul.-‐party dialogue (3 or more
par.cipants) • Tutorial dialogue • Mul.-‐task dialogue
• embodied conversa.onal agents • human-‐robot dialogue interac.on • dialogue tracking in other
language-‐processing systems (machine transla.on, summariza.on/extrac.on)
• non-‐coopera.ve dialogue systems (nego.a.on, decep.on)
• affec.ve dialogue systems • dialogue with different user
popula.ons (children, elderly, differently abled)
• Enculturated Dialogue Agents • Dialogue “in the wild” • Long-‐term Dialogue Companions
1/17/13