Domain Act Classification using Domain Act Classification using a Maximum Entropy modela Maximum Entropy model

Lee, Kim, SeoLee, Kim, Seo(AAAI unpublished)(AAAI unpublished)

Yorick WilksYorick Wilks

Oxford Internet InstituteOxford Internet Institute

andand

University of SheffieldUniversity of Sheffield

www.dcs.shef.ac.ukwww.dcs.shef.ac.uk

Why are we reading this Why are we reading this unpublished paper?unpublished paper?

It proposes a pretty clear ML model using a standard It proposes a pretty clear ML model using a standard method (ME) but which is novel in its application to method (ME) but which is novel in its application to dialogue and it is easy to see how to do better than dialogue and it is easy to see how to do better than them and gain some publishable traction.them and gain some publishable traction.

Basically it tries to learn over DAs (Dialogue Acts) as Basically it tries to learn over DAs (Dialogue Acts) as well as conceptual content--of very much the type well as conceptual content--of very much the type we propose.we propose.

It gets better DA figures than Webb by ML over both It gets better DA figures than Webb by ML over both at once.at once.

It suggests figures would be even better if they had It suggests figures would be even better if they had measured the DEPENDENCE between the two.measured the DEPENDENCE between the two.

Sample of the annotation they Sample of the annotation they need for their classifier.need for their classifier.

When is the changed date?When is the changed date? [System: ask_ref+change-date][System: ask_ref+change-date]

It’s December 5th.It’s December 5th. [User: response+change-date [User: response+change-date

{date=December 5th}]{date=December 5th}]

Types of information annotatedTypes of information annotated

System or UserSystem or User Speech/Dialogue Act (from a set of 11, e.g. Speech/Dialogue Act (from a set of 11, e.g.

ask_if=YNQ)ask_if=YNQ) Concept (from domain set: e.g. change-date, Concept (from domain set: e.g. change-date,

information-object, which function as n-place information-object, which function as n-place predicates)predicates)

Objects that are values of the predicate Objects that are values of the predicate variable, e.g date=5 December)variable, e.g date=5 December)

ALSO actions in domain tied to instantiated ALSO actions in domain tied to instantiated predicates (e.g. Timeble:Insert:Date)predicates (e.g. Timeble:Insert:Date)

The overall classification taskThe overall classification task To derive a general classifier assigning speech acts

AND concepts at once, treating them as independent. EVEN THOUGH they can be seen not to be I.e. SA/DAs based on local evidence and sequence

AND Concepts based on local evidence and sequence Big fat ME expression to do these all in one. DA element not very different from Webb method:

both used lexical evidence, POS tags and n-grams.

Key example of combination of Key example of combination of local/global and SA/C local/global and SA/C

information.information. When is the changed date?When is the changed date? [System: ask_ref+change-date][System: ask_ref+change-date]

It’s December 5th.It’s December 5th. [User: response+[User: response+change-datechange-date {date=December 5th}] {date=December 5th}] Rather thanRather than [User: inform+information-object {date=December [User: inform+information-object {date=December

5th}]5th}] BUT THIS CANT BE DONE WITHOUT LINKING BUT THIS CANT BE DONE WITHOUT LINKING

SPEECH ACTS AND CONCEPTSSPEECH ACTS AND CONCEPTS

ResultsResults

SA/DA rising to 93% precision after 1000 SA/DA rising to 93% precision after 1000 turns;turns;

Concepts rising to 90% slightly laterConcepts rising to 90% slightly later DA set seems very small (how compare DA set seems very small (how compare

Webb and DAMSL? His figures less Webb and DAMSL? His figures less good).good).

What can we take from this?What can we take from this?

Cf. old arguments about limits on DA Cf. old arguments about limits on DA accuracy without semantic content. accuracy without semantic content.

Cf. Interactions local/global in JelinekCf. Interactions local/global in Jelinek BUT THEY DON’T ACUALLY DO IT, SO BUT THEY DON’T ACUALLY DO IT, SO

WHY THE BETTER FIGURES?WHY THE BETTER FIGURES?