Rough schedule• Multimodal, multi-party dialogue [30 min]• D’Homme, SIRIDUS [10 min]

– dialogues with networked devices in a smart house • SRI demo (DM) , (IBL robot (JB))

– negotiative dialogue– menu-based natural command dialogue (SL)– future work on TrindiKit (SL)

• Discussion: Dialogue genres [20 min]– reconfigurability (DM)– mixing infoseeking+command dialogue , switching domains dynamically (GoDiS demo)– more on building systems (incrementally) (SL)

• GoDiS and VoiceXML (SL): [10 min]• General discussion (everyone): [20 min]

– NL generation from infostates • generation from DRSs (JB)• generation of focus (SL)

– Logics & proofs of dialogue, e.g.completeness of update rules– Real, practical applications: what are the problems– grammar based vs. statistical approaches (DM)

Future work on TrindiKit

ESSLLI 2001

GUI (Graphical User Interface)

• Currently being developed• Purpose

– simplifying building of DS – pedagogical purposes– debugging

• Gives overview of – information state at various levels of dialogue– update rules– interactions between modules, infostate, and

resources

• Eventually, build systems using GUI

Extended formalism

• improved infostate path language• additional constructs for writing

update algorithms– inspiration from SOAR, OZ, …

• extended typechecking & debugging facilities

Adding more readymade modules & interfaces

• include more powerful NL interpretation and generation modules– e.g. HPSG

• Build hook modules for various useful programs – planners, reasoning engines etc.– Any OAA module can be accessed from TrindiKit, so

not limited to Prolog programs

• Resource interfaces for – SQL databases, – uPnP-specified devices, – etc.

Adding more speech modules

• New products come out all the time• Actually using them in a system can be

tricky and take lots of time• ”Open Source”; users contribute

– modularity promotes reuse – send us a link to your TrindiKit® software!– requires general solutions and explaining

how module works

Future work on GoDiS

ESSLLI 2001

• Integrate with VoiceXML?• Extended dialogue capabilities: Current

– collaborative negotiation– multiple simultaneous plans– information sharing between plans

• Extended dialogue capabilities: Future– talking to autonomous robots– noncollaborative negotiation– complex planning tasks

• Implement more domains– no limits…

Typology of dialogues & dialogue systems

Typology of dialogues & dialogue systems

• to what extent are dialogue types related by subsumption? To what extent are they orthoginal?– infoseeking < info exchange– instuctional, command < planning– negotiation

• meaning / dialogue strategy / domain • collaborative / noncollaborative• argumentative / non-argumentative

Typology of dialogues & dialogue systems

• Can dialogue types be defined in terms of theories / system features needed to handle them?– infostate contents– moves, rules

• Approach: build systems incrementally– succesively increase scope of dialogue

phenomena handled by system – reuse of system components over dialogue

types

Negotiative dialogue

some definitions and ideas

Problem with current GoDiS

• assumes database always returns exactly one post (price info); not generally true

• but we want to be able to – talk about several flights, – allowing the user to ask questions about

them, – deciding on one of them, and then– getting price information

• Requires negotiation

Negotiation vs. acceptance

• Clark’s ladder:– 1. A attends to B’s utterance– 2. A percieves B’s utterance– 3. A understands B’s utterance (grounding)– 4. A accepts or rejects B’s utterance

• Sidner and others sees negotiative dialogue as proposals and acceptance/rejections of proposals– this means that all dialogue is negotiative– all assertions (and questions, instructions etc.) are

proposals• But some dialogues are negotiative in another sense, by

explicitly containing discussions about different solutions to a problem, and finally deciding on one– Negotiation is not Clark’s level 4

Two senses of “negotiation”

• Negotiation in Sidner’s sense– A: I want to go to Paris [propose]– B(1): OK [accept]– B(2): Sorry, there are no flights to Paris [reject]

• Negotiation in our sense– U: flights to paris on september 13 please [answer]– S: there is one flight at 07:45 and one at 12:00

[propose]– U: what airline is the 12:00 one [ask]– S: the 12:00 flight is an SAS flight [answer]– U: I’ll take the 12:00 flight please [accept]

Optimistic approach to acceptance

• DPs assume their utterances are accepted (and integrated into SHARED)– If A asks a question with content Q, A will put Q

topmost on SHARED.QUD

• If addresse indicates rejection, backtrack– using the PRIVATE.TMP field

• No need to indicate acceptance explicitly; it is assumed

• The alternative is a pessimistic approach– If A asks a question with content Q, A will wait for an

acceptance (implicit or explicit) before putting Q on top of QUD

Negotiativity

• Negotiation is a type of problem-solving (cf. Di Eugenio et. al., Coconut)

• Negotiation: DPs discuss several alternative solutions before choosing one of them

• Negotiation does not imply conflicting goals – perhaps not 100% correspondence to everyday use of

the word “negotiation”, but useful to keep collaborativity as a separate dimension from negotiation

• Both AOD and IOD can be negotiative– in a flight information service, the user does not

become obliged to fly anywhere; so it’s IOD– but several different flights may be discussed

Negotiation tasks• Some factors influencing negotiation

– distribution of information between DPs – whether DPs must commit jointly (e.g. Coconut) or one DP can

make the comittment (e.g. flight booking)

• We’re initially trying to model negotiation in flight booking– sample dialouge

• U: flights to paris on september 13 please• S: there is one flight at 07:45 and one at 12:00• U: what airline is the 12:00 one• S: the 12:00 flight is an SAS flight• U: I’ll take the 12:00 flight please

– Sys provides alternatives, User makes the choice– Sys knows timetable, User knows when he wants to travel etc.

Degrees of negotiativity

• non-negotiative dialogue: only one alternative is discussed

• semi-negotiative dialogue: a new alternative can be introduced by altering parameters of the previous alternative, but previous alternatives are not retained

• negotiative dialogue: several alternatives can be introduced, and old alternatives are retained and can be returned to

Semi-negotiative dialogue

• Does not require keeping track of several alternatives

• Answers must be revisable; this can be done using reraising of answered questions

• Correction of optimistic assumption of acceptance not necessarliy distinguished from revision

• Example: Swedish SJ system (Philips): ”Do you want an earlier or later train?”

Issues Under Negotiation i negotiative dialogue

• IUN is a question e.g. what flight to take

• In an activity, some questions are marked as negotiable issues; other questions are assumed to be non-negotiable

• Needs a new IS field: SHARED.IUN of type assocset(question,set(answer))

Alternatives in negotiation

• Alternatives are alternate answers to an IUN• a proposal is the introduction of a new possible

answer to IUN• An IUN is resolved when an answer to it is

given, i.e. when an alternative is accepted• Alternatives and information about them is

optimistically assumed to be accepted• Alternatives are needed whenever database

search can return more than one result

General and specific information

• General information concerns all alternatives, and is collected in an initial information-seeking dialogue (e.g. flights to paris)– e.g. x.dest(x,Paris)

• Specific information concerns specific alternatives (e.g. flight f345 leaves at 10:45)

• Specific info usually results from a database search whose input is general info; does this motivate separate fields in IS?

Example

• IUN is x.sel_flight(x) (“which is the chosen flight”?)

• A: flight to paris, december 13– answer(x.dest(x,paris)) etc.;

• B: OK, there’s one flight leaving at 07:45 and one at 12:00– propose(f1), propose(f2),– answer(dep_time(f1,07:45)),

answer(dep_time(f2,12:00))

• A: I’ll take the 07:45 one– answer(sel_flight(X), dep_time(X, 07:45)),– after contextual interpretation: answer(sel_flight(f1))

PRIVATE =

PLAN =

AGENDA = { findout(?x.sel_flight(x)) }

SHARED =

findout((?x. ccn(x))book_ticket

COM = dep_time(f1,0745),dep_time(f2,1200)dest(paris), ...

QUD = <> LM = {propose(f1), propose(f2),

answer(dep_time(f1,07:40),...}

BEL = {flight(f1), dep_time(f1,0745), ... } TMP = (same structure as SHARED)

IUN = < ?x.sel_flight(x f1, f2 } >

B: OK, there’s one flight leaving at 07:45 and one at 12:00

VoiceXML

• form-filling paradigm– forms, consiting of slots+values– menus, special case of forms

• speech recogntion grammars defined with scope over– slot– form– document

• Why is VoiceXML interesting for us?– becoming industry standard – supports plan constructs similar to GoDiS– multiple active grammars allow behaviour reminiscent of

question and task accommodation

Dialogue capabilities

• VoiceXML: if input matches several fields, the first is chosen– GoDiS can ask clarification question

• VoiceXML: user initiated subdialogues cannot share information with main dialog, or other subdialogues– the default in GoDiS is that all information is

shared between subdialogues

• unclear how to implement several simultaneously active plans

Architectural issues

• information state is global and keeps plan separate from accumulated propositions– VoiceXML based on forms, which can be

seen as local information states• VoiceXML mixes dialogue knowledge, domain

knowledge, and language knowledge in a single specification– GoDiS keeps them separate, enabling easier

reconfiguration and plug-and-play – when implement a new menu-driven system, there is

no need to reimplement general dialogue strategies

future issues

• Open question whether VoiceXML can extend to more complex dialogue, e.g. negotiation– but the information state approach is ideal for

complex dialogues– GoDiS is currently being extended to handle

collaborative negotiation

• Can GoDiS be combined with VoiceXML?– have GoDiS use extended VoiceXML

specifications,– or use standard VoiceXML specs in a smarter

way

generating focus from information states

Accommodation

• Generality– no need to distinguish requested and non-requested

(but relevant) information– single rule for integrating answers

• Theoretically motivated concept, with independent justification

• Easy to implement, given information state approach

• Gives useful side-effects (e.g. task accommodation leads to loading a plan)– supports the idea of accommodation as opposed to

“direct interpretation”

Extensions

• Questions can also be reaccommodated– if the user answers a question which has

already been answered:– remove proposition from shared beliefs, and

put question back on QUD

• Extend to domain where there are many plans containing a question matching a given answer– e.g. menu-based dialogue

• Focus intonation and QUD

Focus and information states

• Focal Question Presupposition (FQP) (based on Ginzburg and others):– An utterance with narrow focus on a

constituent presupposes a function/question obtained by abstracting over the focussed constituent

• Example:– “Jill likes BILL” [like(jill, bill)] presupposes

“Who does Jill like?” [?x.like(jill, x)]

Focal Question Accommodation

FQuAcc: interpretation version of FQP

“When an utterance occurs which focally presupposes Q, and Q is not topmost on QUD, make Q topmost on QUD”

pre:

eff:

in(SHARED.LM, Move)

foc-presupp(Move, Q)

~fst(SHARED.QUD , Q) push(SHARED.QUD, Q)

Interpreting utterances with focus

• Example:A: Are you FLYING to london?

?(to-city(london)&transport(fly)) presupposes Q’ = ?(x.to-city(london)&transport(x)), i.e. “how are you getting to london”

B1: YesB2: No, I’m taking a TRAIN?B3: No [Q’ still on QUD!]

Generating utterances with focus

• Generation version of FQP:– When generating Q, and there is a Q’

on QUD such that Q’ is an abstraction of Q over constituent C (Q=Q’(C)), put narrow focus on C

Generating clarifications with focus (cautious grounding)

• Example 1:– U: I want a flight please– S: What city do you want to go to? (Q’)– U: London– S: So you’re flying to LONDON? (Q) [Q=Q’(london); Q’ still on QUD, so put focus on “London”]

• Example 2:– U: I want to go to London– S: How do you want to travel? (Q’)– U: A flight please– S: So you’re FLYING to London? (Q)

Generating questions with focus II

• If – a question is to be asked, and – a parallell question has previously been answerd

(there is a parallel information P in SHARED.COM), – put focus on the part that differentiates Q1 and P

• Example:– S: What CITY do you want to go to?– U: Paris....– S: What city do you want to go FROM?– ?x.to-city(x) parallell to ?x.from-city(x)

Generating questions with focus II (cont’d)

• Extension to focal question presupposition: – A question Q with narrow focus on C

presupposes a parallell question Q’’ which differs from Q by having C replaced by some B

• Does accommodation apply?• What about propositions?• What, exactly, does “parallell” mean? (cf.

Pulman 1998 for a formal account of parallellism for propositions)