dialogue manager senior companion university of sheffield march 2008
TRANSCRIPT
Senior Companion Dialogue Manager 2-tier planning where
High-level plans built using the Cognitive Model Topic specific plans built using Dialogue Action Forms
DAFS Dialogue Action Forms Augmented Transition networks Represent domain knowledge Modularizable to encapsulate sub-topics within a conversation
domain
Back channel response while working on more complete system utterances (hm, uh-huh)
DialogueManager
Natural Language
Understanding
Dialoguehistory
Ontology Photoapp
Generation
NLU output
of user utterance
Semantic content of
system utterance
+
emotion tags
CognitiveModel
Dialogue Manager Architecture
DialogueManager
•Dialogue Acts•Named Entities
•Person•Family Relations
•Sister•Mother•grandmother
•Location•Celebration event
•Birthday•Anniversary•Wedding•Christmas
•Pronoun references•Semantic categories for
•User input objects (nouns)•User input actions(verbs)
•Original user input string•Modalities for user input
NLP output•Emotion Tags•Planning•System Model
Cognitive Model
•Hierarchy of Semantic tags for Photos•User instances
Ontology
•Record of User and systemutterances
•Photo/s id•Photo tags•Photo attributes
Photo App
DialogueHistory
DM input closeup
DialogueManager
•Number of output Utterancesfor each:•Semantic Representation
•objects•actions
•Dialogue Act•Emotional content•Output modality
•Expected user response category
Cognitive Model
Fission/Generation
Speech Analysis
Dialogue History
OntologyPhoto App
DM output closeup
Dialogue Action Forms
GUI editor for creating DAFsComposed of nodes and arcs containing
tests and actionsDAFs pre-stacked, but can be overidden
by matching indexing terms (semantic classes, significant words)Essential for mixed-initiative conversation
SC Dialogue Manager Stack
goodbye
greeting
Run Greeting DAFPop greeting DAFPush photo DAF
goodbye
photo
System start
goodbye
location
event
people
date
special memory
goodbye
location
event
date
special memory
Run photo DAFRun people DAFPop people DAF
People DAF
SC Output (1) Natural Language Generation
Currently choosing from a selection of pre-defined surface forms. A Generation module to be developed (CU)
To be shared with the Health and Fitness Companion To include input from the Cognitive model for emotion and
conversational style (based on user profile) Will contain lists of alternative question realisations derived from
corpora
TTS Loquendo TTS
Avatar
AAA, CrazyTalk, Nabaztag (2-D)
Current Dialogue Manager Architecture
InputQueue
Input Manager
NLU Photo Watcher
DAF EngineDAFStack
OutputQueue
Output Manager NLG
Output of ASRIn the format of Galaxy Communicator frame.For example:{ c output
:hypothesises ( {c hypothesis …} {c hypothesis …} … ) }
{c hypothesis:ROName “Language Model Name”:Likelyhood -159.035736:Confidence 0.758829:String “nice one”:NumOfWords 2:WordsInfo: ( {c word …} {c word …} ) }
{c word:String “nice”:AcousticScore 3.791455:Confidence 0.750377:StartPoint 0:EndPoint 42:Language: “en-gb”}
NLU
InputA textual string. For example: “Kate is the mother of Marry”.
Output1. GATE Annotation Set.A snippet output for the sentence: “Kate is the mother of Mary.”[AnnotationImpl: id=0; type=Token; features={category=NNP, kind=word, orth=upperInitial, length=4, string=Kate}; start=NodeImpl: id=0; offset=0; end=NodeImpl: id=1; offset=4, AnnotationImpl: id=1; type=SpaceToken; features={kind=space, length=1, string= }; start=NodeImpl: id=1; offset=4;end=NodeImpl: id=2; offset=5, AnnotationImpl: id=2; type=Token; features={category=VBZ, kind=word, orth=lowercase, length=2, string=is}; start=NodeImpl: id=2; offset=5; end=NodeImpl: id=3; offset=7, AnnotationImpl: id=3; type=SpaceToken; features={kind=space, length=1, string= }; start=NodeImpl: id=3; offset=7; end=NodeImpl: id=4; offset=8……]
NLU
2. A populated ontology in RDF format
A snippet output for the sentence: “Kate is the mother of Mary.”……………………… <owl:FunctionalProperty rdf:ID="hasFather"> <rdf:type rdf:resource="http://www.w3.org/2002/07/owl#ObjectProperty"/> <rdfs:range rdf:resource="#Man"/> <rdfs:domain rdf:resource="#Person"/> <rdfs:subPropertyOf rdf:resource="#hasParent"/> </owl:FunctionalProperty> <Person rdf:ID="someone2"> <hasMother> <Person rdf:ID="someone1"> <hasName rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >Kate</hasName> <hasSex rdf:resource="#female"/> </Person> </hasMother> <hasName rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >someone2</hasName> </Person></rdf:RDF>
DAFs
DAFs are either triggered byBeing on the top of the stack Through index term matching
Each DAF has an associated set of terms associated with it (semantic categories)
Dialogue Manager Strategy
Very simple for now, discusses variety of things in and about the photos People - their names, ages and relationship to the
user Location of the photo and when it was taken and
for what purpose
DM strategy at present connects all photos that contain the same person and discusses the similarities and differences.
DAFs
Created using a GUI editor
Set of nodes and arcsA node represents a dialogue stateEach arc contains a test and an associated
action
The main knowledge base will be an ontology which will hold all the world knowledge. There is another ontology which holds the current session knowledge. We chose ontologies to represent the knowledge because in the same file we have a structure, instances and logical rules.
The tests invoke rules predefined in the ontology. If the rule evaluates to true, then the relative action is invoked, otherwise not. Like this, a person creating the rule needs not know anything on programming.
The rules will look like ...
Person(?p) ^ hasAge(?p,?age) ^ greaterThan(?age,200) -> TooOld(?p)
They can be written and tested in Protege using a graphical interface.
DAF tests invoke protégé Rules
SWRL http://protege.cim3.net/cgi-bin/wiki.pl?SWRLLanguageFAQ
SWRL is an acronym for Semantic Web Rule Language. It is intended to be the rule language of the Semantic Web. SWRL is based on the OWL Web Ontology Language. It allows users to write rules to reason about OWL individuals and to infer new knowledge about those individuals.
•SWRL rule contains an antecedent part, which is referred to as the body, and a consequent part, which is referred to as the head.
•Both the body and head consist of positive conjunctions of atoms
•SWRL rules are written in terms of OWL classes, properties and individuals
•SWRL supports Open World Reasoning
•One of the most powerful features of SWRL is its ability to support user-defined
• built-ins. A built-in is a predicate that takes one or more arguments and evaluates
• to true if the arguments satisfy the predicate.
・ Person(?p) ^ hasAge(?p, ?age) ^ swrlb:greaterThan(?age, 17) -> Adult(?p)
Not all user input can be structured
If the NLU module cannot fit data into a structured form of the type we are expecting then the NLU module outputs an Annotation Set which has coordinates and properties for
those coordinates.
Dialogue Manager uses the Knowledge Base to
Access individual pieces of KnowledgeStatic system knowledgeDynamic user instances
Make inferences about the knowledge *relies on the knowledge being structured
NLG
InputThe input of NLG is a logical representation which has three properties: domain, type and Value (optional). To generate a sentence of requesting some general information of a photo,following representation will be sent to NLG:
Domain: PHOTOType: REQUEST_GENERALValue: EMPTY
NLG
OutputA predefined utterance from a template file. Below is a snippet of the template file.(The $ will be replaced by the value specified in the logical representation.)
………………<Template domain="PHOTO">
<Sentences type="REQUEST_AGE"><Sentence>How old is $?</Sentence><Sentence>What is $'s age?</Sentence><Sentence>What is the age of $?</Sentence><Sentence>$ is how old?</Sentence></Sentences>
<Sentences type="REQUEST_RELATION"><Sentence>How do you know $?</Sentence><Sentence>What is your relationship with $?</Sentence></Sentences>……………..