Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 137–139,Seoul, South Korea, 5-6 July 2012. c©2012 Association for Computational Linguistics

A Mixed-Initiative Conversational Dialogue System for Healthcare

Fabrizio Morbini and Eric Forbell and David DeVault and Kenji Sagae andDavid R. Traum and Albert A. Rizzo

Institute for Creative TechnologiesUniversity of Southern California

Los Angeles, CA 90094, USA{morbini,forbell,devault,sagae,traum,rizzo}@ict.usc.edu

Abstract

We present a mixed initiative conversationaldialogue system designed to address primar-ily mental health care concerns related tomilitary deployment. It is supported by anew information-state based dialogue man-ager, FLoReS (Forward-Looking, RewardSeeking dialogue manager), that allows bothadvanced, flexible, mixed initiative interac-tion, and efficient policy creation by domainexperts. To easily reach its target populationthis dialogue system is accessible as a web ap-plication.

1 Introduction

The SimCoach project is motivated by the challengeof empowering troops and their significant others inregard to their healthcare, especially with respect toissues related to the psychological toll of militarydeployment. SimCoach virtual humans are not de-signed to act as therapists, but rather to encourageusers to explore available options and seek treatmentwhen needed by fostering comfort and confidence ina safe and anonymous environment where users canexpress their concerns to an artificial conversationalpartner without fear of judgment or possible reper-cussions.

SimCoach presents a rich test case for all compo-nents of a dialogue system. The interaction with thevirtual human is delivered via the web for easy ac-cess. As a trade-off between performance and qual-ity, the virtual human has access to a limited set ofpre-rendered animations.

The Natural Language Understanding (NLU)module needs to cope with both chat and military

Figure 1: Bill Ford, a SimCoach character. SimCoachvirtual humans are accessible through a web browser.The user enters natural language input in the text fieldon the bottom of the screen. The simcoach responds withtext, speech and character animation. The text area to theright shows a transcript of the dialogue.

slang and a broad conversational domain. The dia-logue policy authoring module needs to support non-dialogue experts given that important parts of the di-alogue policy are contributed by experts in psycho-metrics and mental health issues in the military, andothers with familiarity with the military domain.

The dialogue manager (DM) must be able to takeinitiative when building rapport or collecting the in-formation it needs, but also respond appropriatelywhen the user takes initiative.

2 Supporting Mixed Initiative Dialogues

There is often a tension between system initiativeand performance of the system’s decision-makingfor understanding and actions. A strong system-initiative policy reduces the action state space since

137

user actions are only allowed at certain points inthe dialogue. System initiative also usually makesit easier for a domain expert to design a dialoguepolicy that will behave as desired.1 Such systemscan work well if the limited options available to theuser are what the user wants to do, but can be prob-lematic otherwise, especially if the user has a choiceof whether or not to use the system. In particular,this approach may not be well suited to an appli-cation like SimCoach. At the other extreme, somesystems allow the user to say anything at any time,but have fairly flat dialogue policies, e.g., (Leuski etal., 2006). These systems can work well when theuser is naturally in charge, such as in interviewinga character, but may not be suitable for situationsin which a character is asking the user questions, ormixed initiative is desired.

True mixed initiative is notoriously difficult for amanually constructed call-flow graph, in which thesystem might want to take different actions in re-sponse to similar stimuli, depending on local utili-ties. Reinforcement learning approaches (Williamsand Young, 2007; English and Heeman, 2005) canbe very useful at learning local policy optimizations,but they require large amounts of training data and awell-defined global reward structure, are difficult toapply to a large state-space and remove some of thecontrol, which can be undesirable (Paek and Pierac-cini, 2008).

Our approach to this problem is a forward-lookingreward seeking agent, similar to that described in(Liu and Schubert, 2010), though with support forcomplex dialogue interaction and its authoring. Au-thoring involves design of local subdialogue net-works with pre-conditions and effects, and also qual-itative reward categories (goals), which can be in-stantiated with specific reward values. The dialoguemanager, called FLoReS, can locally optimize pol-icy decisions, by calculating the highest overall ex-pected reward for the best sequence of subdialoguesfrom a given point. Within a subdialogue, authorscan craft the specific structure of interaction.

Briefly, the main modules that form FLoReS are:• The information state, a propositional knowl-

1Simple structures, such as a call flow graph (Pieraccini andHuerta, 2005) and branching narrative for interactive games(Tavinor, 2009) will suffice for authoring.

edge base that keeps track of the current stateof the conversation. The information state sup-ports missing or unknown information by al-lowing atomic formulas to have 3 possible val-ues: true, false and null.• A set of inference rules that allows the sys-

tem to add new knowledge to its informationstate, based on logical reasoning. Forward in-ference facilitates policy authoring by provid-ing a mechanism to specify information stateupdates that are independent of the specific di-alogue context.2

• An event handling system, that allows the in-formation state to be updated based on user in-put, system action, or other classes of author-defined events (such as system timeouts).• A set of operators. Operators represent lo-

cal dialogue structure (trees), and can also bethought of as reusable subdialogues. Each statewithin the subdialogue can include a rewardfor reaching that state. Rewards are functionsof the goals of the system, and are the mainmethod used to decide what to do when there ismore than one applicable operator. Operatorshave preconditions and effects. Effects specifychanges to the information state. The precondi-tions define when an operator can be activated.

3 Sample Dialogue

In this demo, the user will interact with the Sim-Coach character Bill Ford, using a standard webbrowser and typing text. The virtual human, drivenby FLoReS, will respond using pre-rendered anima-tions encoded as H.264 video, delivered via a stan-dard web server. Table 1 shows an excerpt from asample conversation with Bill Ford that illustratessome of the features of this dialogue manager.

The excerpt starts from a rapport buildingsmalltalk sub-dialogue on the topic of barbecuingwhich is interrupted by a user question about con-fidentiality. The system responds to the user inter-ruption and then re-starts the interrupted smalltalkbecause it is still the most valuable conversation con-tinuation available at that moment.

2For example: every time the user says that s/he has night-mares we want to update the information state to include thats/he also has sleeping problems.

138

Dialogue transcript Notes

Ask anybody about me, andthey’ll tell you that I love toBBQ

BBQ SmalltalkThe character is equippedwith a few operators forsmalltalk about a few topics.BBQ is one of them.

Is this conversation se-cret?

We don’t share your infowith anyone who can per-sonally identify you. Thetechs can see what we say,but just to tell that the site isworking. But they have noidea who said it, just whatwas saidDid that help you?

Yes it did.

Great.

Confidentiality QA

Here the system is inter-rupted by a user question andit decides that answering it isthe best course of action.

Like I was saying, I love toBBQ

BBQ Smalltalk After answering the question,the best course of action is toawaken the paused operatorabout the BBQ smalltalk.

What is PTSD?

PTSD, or post-traumaticstress disorder is an anxietycondition associated withserious traumatic events.It can come with survivorguilt, reliving the trauma indreams, numbness, and lackof involvement with reality.

What is PTSD QA

Again the BBQ smalltalk isinterrupted by another ques-tion from the user.

So, is PTSD somethingyou’re worried about. I onlyask, because you’ve beenasking about it. ...

PTSD Topic Interest QA

After answering the secondquestion the system decidesto ignore the paused operatorand load a follow-up operatorrelated to the important topicraised by the user’s question.The selection is based on theexpected reward that talkingabout PTSD can bring to thesystem.

Table 1: An excerpt of a conversation with Bill Ford thatshows opportunistic mixed initiative behavior.

Next, the user asks a question about the impor-tant topic of post-traumatic stress disorder (PTSD).That allows operators related to the PTSD topic tobecome available and at the next chance the most

rewarding operator is no longer the smalltalk sub-dialogue but one that stays on the PTSD topic.

4 Conclusion

We described the SimCoach dialogue system whichis designed to facilitate access to difficult health con-cerns faced by military personnel and their fami-lies. To easily reach its target population, the sys-tem is available on the web. The dialogue is drivenby FLoReS, a new information-state and plan-basedDM with opportunistic action selection based on ex-pected rewards that supports non-expert authoring.

Acknowledgments

The effort described here has been sponsored by theU.S. Army. Any opinions, content or informationpresented does not necessarily reflect the position orthe policy of the United States Government, and noofficial endorsement should be inferred.

ReferencesM.S. English and P.A. Heeman. 2005. Learning mixed

initiative dialogue strategies by using reinforcementlearning on both conversants. In HLT-EMNLP.

Anton Leuski, Ronakkumar Patel, David Traum, andBrandon Kennedy. 2006. Building effective questionanswering characters. In Proceedings of the 7th SIG-dial Workshop on Discourse and Dialogue, pages 18–27.

Daphne Liu and Lenhart K. Schubert. 2010. Combin-ing self-motivation with logical planning and inferencein a reward-seeking agent. In Joaquim Filipe, AnaL. N. Fred, and Bernadette Sharp, editors, ICAART (2),pages 257–263. INSTICC Press.

Tim Paek and Roberto Pieraccini. 2008. Automatingspoken dialogue management design using machinelearning: An industry perspective. Speech Commu-nication, 50(89):716 – 729. Evaluating new methodsand models for advanced speech-based interactive sys-tems.

Roberto Pieraccini and Juan Huerta. 2005. Where do wego from here? Research and commercial spoken dia-log systems. In Proceedings of the 6th SIGdial Work-shop on Discourse and Dialogue, Lisbon, Portugal,September.

Grant Tavinor. 2009. The art of videogames. New Di-rections in Aesthetics. Wiley-Blackwell, Oxford.

J.D. Williams and S. Young. 2007. Scaling POMDPs forspoken dialog management. IEEE Trans. on Audio,Speech, and Language Processing, 15(7):2116–2129.

139