Natural Language Processing In GamesNatural Language Processing In Games

Michael MateasMichael MateasGeorgia Institute of TechnologyGeorgia Institute of Technology

College of Computing & LCCCollege of Computing & LCC

Andrew SternAndrew SternInteractiveStory.netInteractiveStory.net

www.interactivestory.netwww.interactivestory.net

egl.gatech.eduegl.gatech.edu

grandtextauto.orggrandtextauto.org

Natural language processing (NLP) and games Natural language processing (NLP) and games

• We’ll explore why you’d want NLP in gamesWe’ll explore why you’d want NLP in games

• We'll discuss the issues involved with NLP in We'll discuss the issues involved with NLP in games, using a case study of the NLP in the games, using a case study of the NLP in the interactive drama interactive drama FaçadeFaçade (to be released next (to be released next month!)month!)

• We’ll suggest future applications and research We’ll suggest future applications and research directions for NLP in games directions for NLP in games

Immediate potential uses of NLP in gamesImmediate potential uses of NLP in games

• NPC conversations in RPG / MMPORPG / Adventure NPC conversations in RPG / MMPORPG / Adventure gamesgames• Avoid lock-step, menu-based dialog trees

• Already a lot of texting in MMPORPGs

• In FPS gamesIn FPS games• Communication with NPC comrades

• Already a lot of texting in multi-player FPS

• In RTS gamesIn RTS games• High-level commands to units

Medium-term uses of NLP in gamesMedium-term uses of NLP in games

• In MMPORPGs, listen in on player-to-player In MMPORPGs, listen in on player-to-player conversationconversation• Robust rule-based parsing and statistical parsing

• But need a game master AI that shapes world events

• Mobile phone games (e.g. Vivienne virtual girlfriend)Mobile phone games (e.g. Vivienne virtual girlfriend)

• New genres: e.g. interactive dramaNew genres: e.g. interactive drama

The technologies of NLPThe technologies of NLP

• Natural language understanding (NLU) – given natural language input, Natural language understanding (NLU) – given natural language input, extract meaningextract meaning• Involves syntax, semantics, pragmatics

• May include added step of speech recognition

• Conversation management – given an ongoing conversation, figure out Conversation management – given an ongoing conversation, figure out what state the conversation is in and where it is goingwhat state the conversation is in and where it is going• Tracking the conversational state & expectations

• Deciding what to say next

• Natural language generation (NLG) – given a meaning, generate text that Natural language generation (NLG) – given a meaning, generate text that expresses that meaningexpresses that meaning• Turning a formal meaning representation into character-specific dialog

• Generating facial expressions, gestures, etc. that accompany dialog

(Rare) examples of game NLP(Rare) examples of game NLP

• Text adventures – (brittle) parser maps text to verbs Text adventures – (brittle) parser maps text to verbs

• SeamanSeaman, , BabyzBabyz, , FurbyFurby – relatively simple – relatively simple stimulus/response conversations, simple word stimulus/response conversations, simple word associationassociation

• Lifeline, Rebel Moon RevolutionLifeline, Rebel Moon Revolution – commands – commands

• Constructive dialog interfaceConstructive dialog interface

Game designers and NLPGame designers and NLP

• Game designers are (rightly) suspicious of NLUGame designers are (rightly) suspicious of NLU• Text-based adventure games: “I don’t understand”

• AI complete problem – need the entirety of common sense to understand language?

• Language input (even speech) may break up the action

• Dialog management is typically accomplished with finite state Dialog management is typically accomplished with finite state machines, dialog trees machines, dialog trees • For example, the buying and selling dialogs of shopkeepers

• Natural language generation has rarely (never?) been triedNatural language generation has rarely (never?) been tried• Dialog is canned – enormous effort goes into writing thousands of lines of

character dialog

• But the dialog is (hopefully) well written – complete authorial control

Problems of open-ended NL inputProblems of open-ended NL input

• NLU is a notoriously difficult, AI-complete problemNLU is a notoriously difficult, AI-complete problem

• There will be NLU failures, so why do it?

• Why not use dialog menus, or menus of story actions, or physical action?

• Text inputText input

• Requires keyboard

• Speech inputSpeech input

• Requires training your voice to work well at all

• User-independent, emotional speech recognition technology not there yet

Why do NLU in games?Why do NLU in games?

• Explicit choices (menus)…Explicit choices (menus)…• Foreground the boundaries of the experience

• Weight all choices equally

• Become unmanageable for broad action spaces

• Not natural!

• Far more expressiveness for playerFar more expressiveness for player

• When characters respond to what When characters respond to what I saidI said, they finally seem , they finally seem alivealive• But risky – when they don’t respond appropriately, seem more mechanical

Deeper, more personal games / stories require languageDeeper, more personal games / stories require language

Why conversation management?Why conversation management?

• Conversation doesn’t just stay on one trackConversation doesn’t just stay on one track

• Actually combines multiple simultaneous conversations / levels

• Even simple conversations should have thisEven simple conversations should have this

• Shopkeepers, for example, currently force you to walk through lock-step, modal interactions

• Trees and state machines don’t scale for representing Trees and state machines don’t scale for representing complex, multi-threaded conversationscomplex, multi-threaded conversations

Why natural language generation?Why natural language generation?

• Unwieldy to pre-write all responsesUnwieldy to pre-write all responses• Tension between authorial burden vs. authorial

control

• But, it’s the farthest awayBut, it’s the farthest away• Personality-specific, natural-sounding dialog

generation

• Speech synthesis with emotion

Case study of advanced NLP in games:FaçadeCase study of advanced NLP in games:Façade

• Dramatic story world with personality-rich, emotional characters

• The player plays a protagonist within the story, first-person point of view – most interaction with the characters is through dialog

• The player experiences a story with a dramatic arc

Façade NLP requirementsFaçade NLP requirements

• Support broad range of language relevant to story domain Support broad range of language relevant to story domain

• Not a narrow range of language specific to a task domain

• Extract interesting player intentionsExtract interesting player intentions

• Not distinguish “correct” and “incorrect” utterances

• Understanding sensitive to story & character contextUnderstanding sensitive to story & character context

• ABL characters have their own internal lives

• Not the same as a chatterbotNot the same as a chatterbot

Code support for NLP requirementsCode support for NLP requirements

• Rule language for matching surface text featuresRule language for matching surface text features• Semantic parsing

• Support deep and shallow rules existing side-by-sideSupport deep and shallow rules existing side-by-side

• Reaction selection framework supports conversation Reaction selection framework supports conversation managementmanagement• Includes ABL meta-behaviors for incorporating reactions into the

current character goals

Two phases of the Façade NLUTwo phases of the Façade NLU

NLU: Surface text to discourse acts

Conversation management: Discourse acts to reactions

Surface textDiscourse acts (~25)

“You two look so happy in this wedding picture”

text pattern feature

feature feature

AgreeDisagreePraiseRefer to…

Context:Affinity Game

Context:Global

Proposer Proposer Priority map

Proposer Proposer Priority map

Selector}

Façade architectureFaçade architecture

Drama Manager(sequences

beats)Bag of beats

beatbeat

beat

beat

beatbeat

Desired valuearc(s)

Selected beat

History

timePrevious action

beat beat beat beat

Activity not part of a beat

Currentstory

values

Player

Story World

TripGrace

Recognizers

Natural Language Processing

surface text

discourse acts

discourse acts

reactions

ABL Agents

Focus on what language does:PragmaticsFocus on what language does:Pragmatics

““it’s not a problem”it’s not a problem”

• Syntax – produce a parse tree of the sentenceSyntax – produce a parse tree of the sentence

• Semantics – represent the formal meaning of the sentenceSemantics – represent the formal meaning of the sentence exists(x).Situation(x) ^ ~Problem(x)exists(x).Situation(x) ^ ~Problem(x)

• Pragmatics – the conversational “move” of the sentence Pragmatics – the conversational “move” of the sentence Agree(<character>)Agree(<character>)

Example discourse actsExample discourse acts

Representation of Discourse Acts Pragmatic Meaning of Discourse Acts

(DAAgree ?char) Agree with a character. (e.g. “certainly”, “sure thing”, “I would love to” )

(DADisagree ?char) Disagree with a character. (e.g. “No way”, “Fat chance”, “Get real”, “Not by a long shot”)

(DANegExcl ?char) A negative exclamation, potentially directed at a character. (e.g. “Damn”, “That really sucks”, “How awful”, “I can’t stomach that”)

(DAThank ?char) Thank a character (e.g. “Thanks a lot”)

(DAAlly ?char) Ally with a character. (e.g. “I like you”, “You are my friend”, “I’m here for you”)

(DAExplain ?char ?adj) Explain something simple about a character. (e.g. “You’re afraid”, “Trip is controlling”, “Grace is angry”

(DAExplainRel ?char1 ?char2 ?rel)

Explain about a relationship between characters. (e.g. “Grace doesn’t love Trip”, “Trip is cheating on Grace”)

The text pattern rule languageThe text pattern rule language

• Embed a text pattern language in Jess, a forward Embed a text pattern language in Jess, a forward chaining rule languagechaining rule language

• Text pattern language supports regular Text pattern language supports regular expressions plus occurs expressions expressions plus occurs expressions

(defrule simple-hello-rule (template (toc hello)) => (assert (iGreet)))

(defrule character-grace (template (toc grace)) => (assert (iCharacter grace)))

(defrule greet-discourse-act (iGreet) (iCharacter ?x) => (assert (DAGreet ?x)))

Positional factsPositional facts

Positional facts add recursion (parsing)Positional facts add recursion (parsing)

(defrule positional_Is (template (toc (tor am are is seem seems sound sounds look looks) [like]))=> (assert (iIs)))

(defrule positional_PersonPosDesc (template (tor buddy clever comrade confidant friend genius pal)) => (assert (iPersonPosDesc)))

(defrule Praise_you_are_PersonPos (template ({iPersonPosDesc} | (you [{iIs}] [a | my] {iPersonPosDesc})))=> (assert (iPraise)))

Other featuresOther features

• Wordnet expansions and stemmingWordnet expansions and stemming

• Simple anaphora resolutionSimple anaphora resolution• Keep facts around representing current references

• Anaphora resolution quickly becomes complicated

• Term retraction – useful for idioms and handling double Term retraction – useful for idioms and handling double negativesnegatives• -(toc (piece of cake)) => (assert DAAgree)

• -(toc ({ipNegate} {ipNegativeWord})) => (assert DAPositive)

NLU rule idiomsNLU rule idioms

We organize our rules in tiers (salience)We organize our rules in tiers (salience)

1.1. Generally useful patterns and synonymsGenerally useful patterns and synonyms

2.2. Idioms, retracting idiomatic textIdioms, retracting idiomatic text9000 idioms harvested from collections, ~1000 appropriate9000 idioms harvested from collections, ~1000 appropriate

3.3. Handle double negatives using retractionHandle double negatives using retraction

4.4. Combination (“robust parsing”) and discourse act Combination (“robust parsing”) and discourse act keyword rules keyword rules

Ungrammatical (non-sensical) inputsUngrammatical (non-sensical) inputs

• Template rules tend to permissively accept surface textTemplate rules tend to permissively accept surface text

• E.g. “you sounds buddy”

• Prefer matching some story-specific discourse act, versus Prefer matching some story-specific discourse act, versus matching nothingmatching nothing

• Reactions that move story forward or reveal character backstory are more interesting than “huh?”

• Support the story, not the meta-game of breaking the NLUSupport the story, not the meta-game of breaking the NLU

More expressive than chatterbotsMore expressive than chatterbots

Façade NLU Façade NLU Chatterbots (e.g. AIML)Chatterbots (e.g. AIML)

Multiple levels of intermediate representations

Tend to directly map surface text to response

Operates in context of rich character and story state(conversation management)

Minimal state

Author can control matching order

Matching order fixed by implementation

Rich support for rule chaining (positional facts)

Minimal support for recursion

Façade NLU ConclusionsFaçade NLU Conclusions

• In playtests, we map appropriate discourse acts for 50%+ of In playtests, we map appropriate discourse acts for 50%+ of inputsinputs• Our bottlenecks so far have been in conversation management &

generation

• With current rule base (compiles > 7000 jess rules), real-time With current rule base (compiles > 7000 jess rules), real-time performance adequate (< 300 milliseconds)performance adequate (< 300 milliseconds)• Uses Rete nets to avoid the naïve strategy of retesting all the

complicated conditions of our thousands of rules all the time

• Unrestricted natural language input is necessary for deeper Unrestricted natural language input is necessary for deeper games about people – relationships, emotion, psychologygames about people – relationships, emotion, psychology

• Unrestricted natural language input is (barely) viable todayUnrestricted natural language input is (barely) viable today

Façade dialog managementFaçade dialog management

• A A contextcontext defines a currently active dialog state (like a node defines a currently active dialog state (like a node of a dialog FSM)of a dialog FSM)

• Within a context, Within a context, proposerproposer rules suggest possible responses rules suggest possible responses depending on what the player says (like the arrows out of a depending on what the player says (like the arrows out of a dialog FSM state)dialog FSM state)

• Unlike FSMs, Unlike FSMs, multiple simultaneous responsesmultiple simultaneous responses can be can be proposed – our dialog management framework has various proposed – our dialog management framework has various mechanisms for choosing between themmechanisms for choosing between them

• Unlike FSMs, Unlike FSMs, multiple dialog contextsmultiple dialog contexts can be simultaneously can be simultaneously active, and competeactive, and compete

Representing responsesRepresenting responses

• Proposers propose possible responses in the given contextProposers propose possible responses in the given context

• Responses are described by response descriptors

• Characters are responsible for figuring out the details of Characters are responsible for figuring out the details of performing the response (actual words spoken, etc.)performing the response (actual words spoken, etc.)

Specific context

Global context Proposer Proposer

Proposer Proposer type: txnOutReaction character: gracepriority: 40attitude: negativeaffinityChange: leanTPAcollapseDA: strongDisagreementrawDA: <DA we’re responding to>

}Response descriptor

…

…

State examined by reaction proposersState examined by reaction proposers

• Proposers look at the discourse act recognized by NLU Proposers look at the discourse act recognized by NLU as as well as character and game statewell as character and game state to propose responses to propose responses

• Example Example FaçadeFaçade state examined by proposer rules: state examined by proposer rules:

• Who is the primary speaking character right now

• Has a response already been used up (keeps track of used up response descriptions)

• Has the main conflict of the current story beat been posed

• Which objects are associated with which story topics

• History of responses (topic progressions)

Example proposer ruleExample proposer rule

Let primaryChr = the primary character speaking nowLet nonPrimaryChr = the non-primary characterLet rawDA = the discourse act recognized by NLULet pri = the priority with which a response is proposed

Rule MildDisagreement in ArtistAdvertising beat If the ArtistAdversting conflict has been posed and we haven’t already used up this txnOutReaction and (Disagree directed at primaryChr or no character (pri = 20) or MaybeUnsure directed primaryChr or no character (pri = 15) or Agree directed at nonPrimaryChr (pri= 10) or Express laughter directed at nonPrimaryChr(pri = 10) or Thank directed at nonPrimaryChr (pri = 10)) Then propose (TxnOutReaction responder: grace, priority: pri, attitude: negative, affinityChange: leanToTPA, collapseDA: mildDisagreement, da: rawDA)

ContextsContexts

• A context contains reaction proposersA context contains reaction proposers• By activating and deactivating contexts over time, the author can change the

reactions to the same discourse acts at different points in the experience

• Multiple contexts can be active at the same timeMultiple contexts can be active at the same time• Useful for simultaneously proposing specific reactions alongside broader, more

general reactions

• The game/story logic is responsible for deciding which contexts The game/story logic is responsible for deciding which contexts are activeare active• In a long-term conversation or game arc, how you respond to the same player

input needs to change over time

Priority mapping and selectionPriority mapping and selection

• By default, prefer specific reactions over general By default, prefer specific reactions over general onesones• “nice couch” during Discuss Redecorating beat

• But multiple discourse acts are generated by But multiple discourse acts are generated by player’s textplayer’s text• “yeah, I remember your wedding”

• Context priority mappers allow important general Context priority mappers allow important general reactions to trump less important specific onesreactions to trump less important specific ones• Secondary reactions

Conversation management conclusionsConversation management conclusions

• Rule-based architecture avoids manually unwinding Rule-based architecture avoids manually unwinding finite state machinesfinite state machines• Can handle multiple simultaneous contexts, and multiple simultaneous

interpretations

• Somewhere describe loop between game logic and phase 2 (for us, this loop is managed by beats – handlers are really part of narrative intelligence)

• Combined with character dialog logic, creates the experience of an ongoing conversation

• Interesting alternative – have system combine multiple Interesting alternative – have system combine multiple simple conversation graphs into a larger multi-simple conversation graphs into a larger multi-threaded conversationthreaded conversation• Rob Zubek’s work at Northwestern

Live NLP demoLive NLP demo

Natural language generationNatural language generation

• Currently, no examples of NLG in gamesCurrently, no examples of NLG in games

• Academic NLGAcademic NLG• Symbolic approaches to individual sentences and sentence planning

• Statistical approaches to summarization and translation

• But little work on rich, personality-specific, emotion But little work on rich, personality-specific, emotion laden dialogladen dialog

• NLG requires deep(er) formal knowledge representationNLG requires deep(er) formal knowledge representation

NLG conclusionsNLG conclusions

• Won't be able to do NLG for game characters without a Won't be able to do NLG for game characters without a focused 5+ year research effortfocused 5+ year research effort

• Situation is far too complex / rich

• Game research that pushes NLG will need to use simpler Game research that pushes NLG will need to use simpler situation + simpler languagesituation + simpler language

• Examples: children on playground, cavemen, …

For more info(including these slides)For more info(including these slides)

interactivestory.netinteractivestory.netFaçade project site

egl.gatech.eduegl.gatech.eduExperimental Game Lab(NLP research should be starting up soon)

grandtextauto.orggrandtextauto.orgGroup blog on games, interactive drama, IF, new media

Conversation management systemConversation management system

Story specific

Global Proposer Proposer

Proposer ProposerPriority

reordering…

…

Proposer Proposer…

Conversation-specific

Priorityreordering Selector

Game AI(includes character AI)

Response Description

Modify Conversation Management