1 spoken dialogue systems dialogue and conversational agents (part iv) chapter 19: draft of may 18,...
TRANSCRIPT
1 Spoken Dialogue Systems
Dialogue and Conversational Agents (Part IV)
Chapter 19: Draft of May 18, 2005
Speech and Language Processing: An Introduction to Natural Language Processing, Computational
Linguistics, and Speech RecognitionDaniel Jurafsky and James H. Martin
2 Spoken Dialogue Systems
Remaining Outline
Advanced: Plan-Based Dialogue AgentsModels of Discourse Structure
Do we have them?Grosz & Sidner ’86
What identifies discourse structure to Hearers?Textual cuesSpoken cues
How can we produce appropriate discourse structure in TTS systems?Can we identify discourse structure automatically, from speech?
3 Spoken Dialogue Systems
Is there structure in this discourse?
A beautiful mallard spotted the dove I was feeding.
The duck dove supply is small this year.That dove was history in a minute.Well, to recover from this horrible scene, I went
to the park snack bar for a cup of cocoa.To my surprise, I ran into a friend from back
home. When I told her of my recent experience she
questioned my sanity.
4 Spoken Dialogue Systems
Is this a reasonable structure?
A beautiful mallard spotted the dove I was feeding.
The duck dove supply is small this year.
That dove was history in a minute.Well, to recover from this horrible scene, I went
to the park snack bar for a cup of cocoa.To my surprise, I ran into a friend from back
home. When I told her of my recent experience she
questioned my sanity.
5 Spoken Dialogue Systems
This?
A beautiful mallard spotted the dove I was feeding.
The duck dove supply is small this year.That dove was history in a minute.Well, to recover from this horrible scene, I went
to the park snack bar for a cup of cocoa.
To my surprise, I ran into a friend from back home.
When I told her of my recent experience she questioned my sanity.
6 Spoken Dialogue Systems
This?
A beautiful mallard spotted the dove I was feeding.
The duck dove supply is small this year.That dove was history in a minute.
Well, to recover from this horrible scene, I went to the park snack bar for a cup of cocoa.
To my surprise, I ran into a friend from back home.
When I told her of my recent experience she questioned my sanity.
7 Spoken Dialogue Systems
What information do we use in segmenting a discourse?
‘Topic’ coherence?Repeated reference?‘Cue’ phrases?????
8 Spoken Dialogue Systems
Structures of Discourse Structure (Grosz & Sidner ‘86)
A leading theory of discourse structureBased upon Speaker intentions and Speaker and Hearer attentional stateIdentifies a few, general relations that hold among Speaker intentionsIdentifies a model of attentional state
Three components:Linguistic structureIntentional structureAttentional structure
9 Spoken Dialogue Systems
Linguistic Structure
What is actually said or writtenHow is the linguistic structure represented?
Assume discourse is segmented into Discourse Segments (DS)
– What is the basic unit of analysis?– Do we all segment alike?– Do we all use the same cues?
10 Spoken Dialogue Systems
Linguistic Structure of Discourse D
S1: A beautiful mallard spotted the dove I was feeding.
The duck dove supply is small this year.That dove was history in a minute.
S2: Well, to recover from this horrible scene, I went to the park snack bar for a cup of cocoa.
To my surprise, I ran into a friend from back home.
When I told her of my recent experience she questioned my sanity.
11 Spoken Dialogue Systems
Intentional Structure
Discourse purpose (DP): basic purpose of the Speaker in producing the discourseDiscourse segment purposes (DSPs): the Speaker’s purpose in producing the segmentSegments are related to one another by their purposes:
Satisfaction-precedence: DSP1 must be satisfied before DSP2Dominance: DSP1 dominates DSP2 if fulfilling DSP2 constitutes part of fulfilling DSP1
12 Spoken Dialogue Systems
Linguistic Structure of Discourse D
DSP1: Describe murder of dove by duck.S1: A beautiful mallard spotted the dove I was feeding.The duck dove supply is small this year.That dove was history in a minute.
DSP2: Describe meeting of old friend.S2: Well, to recover from this horrible scene, I went to the
park snack bar for a cup of cocoa.To my surprise, I ran into a friend from back home. When I told her of my recent experience she questioned my
sanity.
13 Spoken Dialogue Systems
DSP2: Describe recovery process.S2:
DSP3: Describe snackS3: Well, to recover from this horrible scene, I went
to the park snack bar for a cup of cocoa.DSP3: Describe meeting old friend.
S4: To my surprise, I ran into a friend from back home. DSP5: Describe friend’s reaction
S5: When I told her of my recent experience she questioned my sanity.
14 Spoken Dialogue Systems
Attentional State: The Focus Stack
Stack of focus spaces, each containing objects, properties and relations salient during each DS, plus the DSPState changes: transition rules controlling the addition/deletion of focus spaces
Information at lower levels may or may not be available at higher levelsFocus spaces are pushed onto the stack when
– A new DS is begun
15 Spoken Dialogue Systems
– An embedded DS (e.g. a DS dominated by another DS) is begun
Focus spaces are popped when they are completed
State of focus stack models felicitous reference, coherence in discourse
S2: DSP2, scene, Speaker, snack_barCocoa, friend, home,sanity
S1: DSP1, duck, dove, Speaker, duck_dove_supply
16 Spoken Dialogue Systems
Limits of the Theory
Assumes discourses are task-orientedAssumes a single, hierarchical structure shared by S and HQuestions:
Do people really build such structures when they converse? Use them in interpreting what others say?How could they do it?
17 Spoken Dialogue Systems
How might people recognize discourse structure?
Linguistic markers?tense and aspectcue phrases
Inference of Speaker intentions?Inference from task structure?Intonational Information?
18 Spoken Dialogue Systems
Acoustic and Prosodic Cues to Discourse Structure
Intuition:Speakers vary acoustic and prosodic cues to convey variation in discourse structureSystematic? In read or spontaneous speech?
Evidence: Observations from recorded corporaLaboratory experimentsMachine learning of discourse structure from acoustic/prosodic features
19 Spoken Dialogue Systems
Prosodic Correlates of Discourse/Topic Structure
Pitch rangeLehiste ’75, Brown et al ’83, Silverman ’86, Avesani &
Vayra ’88, Ayers ’92, Swerts et al ’92, Grosz & Hirschberg’92, Swerts & Ostendorf ’95, Hirschberg & Nakatani ‘96
Preceding pauseLehiste ’79, Chafe ’80, Brown et al ’83, Silverman ’86,
Woodbury ’87, Avesani & Vayra ’88, Grosz & Hirschberg’92, Passoneau & Litman ’93, Hirschberg & Nakatani ‘96
20 Spoken Dialogue Systems
RateButterworth ’75, Lehiste ’80, Grosz & Hirschberg’92,
Hirschberg & Nakatani ‘96
AmplitudeBrown et al ’83, Grosz & Hirschberg’92, Hirschberg &
Nakatani ‘96
ContourBrown et al ’83, Woodbury ’87, Swerts et al ‘92
21 Spoken Dialogue Systems
Issues
Do we find significant and reliable cues to discourse structure in prosodic variation
When tested against an independent theory of discourse structure?In spontaneous as well as read speech?
Are Hearers interpretations of discourse structure influenced by intonational variation?
22 Spoken Dialogue Systems
Grosz & Hirschberg ‘92
Small corpus of read AP newswireRead by professional speakerLabeled for discourse structure from text alone or from text and speechPre-ToBI labeledAcoustic-prosodic features extracted for each intermediate (level 3) phrase
– Pitch range and change from prior phrase– Intensity (rms) and change in db from prior phrase– Preceding and subsequent pause– Speaking rate
23 Spoken Dialogue Systems
Analysis of phrases in different segment positions: SBEG, SF, parentheticals, quoted speech
ANOVA’s and t-tests on means
Results:Direct quotes: larger pitch rangeParentheticals: smaller range, neg change from prior phrase, neg change in db, faster rateSBEG: larger range, louder, greater preceding pause, less subsequent pauseSF: greater subsequent pause
24 Spoken Dialogue Systems
Machine learning experiments identified:SBEG with 91.5% est. accuracy (x-validation)SF, 92.5%Attributive tags, 96.9%Direct quotations, 86.4%Indirect quotations, 88.5%Parentheticals, 89.2%
Conclusion: Acoustic/prosodic information is available to permit Hearers to identify discourse structure…
25 Spoken Dialogue Systems
Summary of Dialog in general
The Linguistics of ConversationBasic Conversational Agents
ASRNLUGenerationDialogue Manager
Dialogue Manager DesignFinite StateFrame-basedInitiative: User, System, Mixed
VoiceXMLInformation-State
Dialogue-Act DetectionDialogue-Act Generation
EvaluationUtility-based conversational agents
MDP, POMDPAdvanced: Plan-Based Dialogue Agents