beyond believable agents - employing ai for improving game like simulations for tactical training
TRANSCRIPT
BEYOND BELIEVABLE AGENTS EMPLOYING AI FOR IMPROVING GAME LIKE SIMULATIONS FOR TACTICAL TRAINING
Mirjam Palosaari Eladhari, 10th of May 2016, Spelvetenskapligt Seminarium, Göteborgs Universitet
ABOUT ME➤ Game designer, researcher, and indie developer.
➤ Research associate at Institute of Digital Games in Malta, and at Dept. of Computer and System Sciences at Stockholm University
➤ Recently founded Otter Play - one person indie studio
➤ Current interests: AI Based Game Design, Story Making Games, and co-creation of social intelligent agents.
➤ Past: Game Programmer 2000-02, Liquid MediaTech Lead, Zero Game Studio, Interactive Institute 2002 - 04Then: 10+ years of game research & faculty work, mostly in Game AI & Design.
OVERVIEW➤ Introduction
➤ Believable Agents
➤ approaches
➤ opportunites
➤ AI in military training scenarios, ex.
➤ The gamist problem
➤ What the AI Game programmers Guild said
➤ AI Based Game Design
➤ Discussion: What are the learning goals of training simulations, and how can they be reached? (with or without AI)
KRIEGSSPIEL (1812)Kriegsspiel, from the German word for wargame, was a system used for training officers in the Prussian army. The first set of rules was created in 1812 and named Instructions for the Representation of Tactical Maneuvers under the Guise of a Wargame. It was originally produced and developed further by Lieutenantvon Reiswitz of the Prussian Army.
http://www.boardgamestudies.info/pdf/issue3/BGS3Hilgers.pdf
Taktischer Kriegsspielapparat, von Domänen- und Kriegsrat Georg Leopold Baron von
Reiswitz für Friedrich Wihelm III entworfen und 1812 angefertigt. Siftung Preußischer
Schlösser und Gärten Berlin-Brandenburg (Foto Roman März, Berlin).
TURING’S PAPER MACHINE 1962
Games and AI - historically intertwined.
Many game AI programmers considers game AI to be game design.
ARTIFICIAL INTELLIGENCE (AI)
“The subfield of computer science concerned with the concepts and methods of symbolic inference by computer and symbolic knowledge representation for use in making inferences. AI can be seen as an attempt to model aspects of human thought on computers. It is also sometimes defined as trying to solve by computer any problem that a human can solve faster.”
The Free On-line Dictionary of Computing, © 1993-2007 Denis Howe
INTELLIGENT AGENTS
In artificial intelligence, an intelligent agent (IA) is an autonomous entity which observes through sensors and acts upon an environment using actuators (i.e. it is an agent) and directs its activity towards achieving goals (i.e. it is "rational", as defined in economics). (Russell & Norvig 2003, chpt. 2)
AI BASED GAME DESIGN
Diagram is a joint effort of Josh McCoy, Anne Sullivan, Gillian Smith, and me (2011, Santa Cruz)
BELIEVABLE AGENTS (BATES 1994)➤ "one that provide the illusion of life, and thus permits the audiences suspension of
disbelief."
➤ "an interactive analog of the believable characters discussed in the Arts."
➤ "Emotion is one of the primary means to achieve believability, this illusion of life, because it helps us know that characters really care about what happen in the world, that they truly have desires".
➤ “Believability places a variety of demands on an interactive character. These include the appearance of reactivity, goals, emotions and situated social competence, among others.”
J. Bates. The role of emotions in believable agents. Technical Report CMU-CS-94-136, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, April 1994.
COMMON TECHNOLOGIES FOR BELIEVABLE AGENTS
BELIEVABLE CHARACTERS: STATE OF THE ART
• Use of Planning models- reactive & traditional
• Plan-based representations of physical, social and language actions (domain specific hierarchical planning)
• Emotion modelling with convergence on cognitive appraisal architectures
• FSMs (used now again for dialogue- moving towards performance of interactive dialogue)
BELIEVABLE CHARACTERS: STATE OF THE ART
• Mood modelled as a function of emotions (sums, averages) and with decay
• Modelling mood through spreading activation. When a character is reminded of something that is emotionally significant, there is an emotional echo
• Episodic memory (temporal query), evaluative labelling, generalisation.
COMMON APPROACHES FOR BELIEVABLE AGENTS
➤ Planning
➤ BDI - Belief, Desire, Intention
➤ Reactive planning
➤ Hierarchical task networks (HTNs)
➤ Finite State Machines (FSMs) and Behaviour Trees
➤ Various architectures for appraisal and decision making for Intelligent Virtual Agents and Synthetic Humans
PLANNING
Planning is the process of generating a sequence of actions that will achieve a goal.
Planning is the process of generating (possibly partial) representations of future behavior prior to the use of such plans to constrain or control that behavior. The outcome is usually a set of actions, with temporal and other constraints on them, for execution by some agent or agents.
STRIPS (Stanford Research Institute Problem Solver) First automated planner, 1971.Language with the same name, used for the inputs to this planner. This language is the base for most of the languages for expressing automated planning problem instances in use today.
THE BELIEF-DESIRE-INTENTION (BDI) SOFTWARE MODEL
BDI Agents are
➤ Situated - they are embedded in their environment
➤ Goal directed - they have goals that they try to achieve
➤ Reactive - they react to changes in their environment
➤ Social - they can communicate with other agents (including humans)
Wooldridge, M. (2000). Reasoning About Rational Agents. The MIT Press. ISBN 0-262-23213-8. http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=3533.
BELIEFS - “I BELIEVE”
➤ the informational state of the agent - in other words its beliefs about the world (including itself and other agents). Beliefs can also include inference rules, allowing forward chaining to lead to new beliefs. Typically, this information will be stored in a database (sometimes called a belief base), although that is an implementation decision.
➤ Using the term belief - rather than knowledge - recognises that what an agent believes may not necessarily be true (and in fact may change in the future).
DESIRES - “I WANT”
➤ Desires (or aims) represent the motivational state of the agent. They represent objectives or situations that the agent would like to accomplish or bring about. Examples of desires might be: find the best price, go to the party, or become rich.
INTENTIONS - “I’M GOING TO…”
Intentions represent the deliberative state of the agent: what the agent has chosen to do. Intentions are desires to which the agent has to some extent committed (in implemented systems, this means the agent has begun executing a plan - towards a goal).
Plans are sequences of actions that an agent can perform to achieve one or more of its intentions.
Mirjam Eladhari, [email protected] Högskolan på Gotland Institutioner för teknik, konst och nya medier
Gotland University, Department of Technology, Art and New Media
BLACK AND WHITE
OFFICE PLANT #1, Marc Bohlen and Michael Mateas, Carnegie Mellon University, March 1998
Mirjam Eladhari, [email protected] Högskolan på Gotland Institutioner för teknik, konst och nya medier
Gotland University, Department of Technology, Art and New Media
BLACK AND WHITE
BLACK AND WHITE AI DEV STRATEGY
To make a plausible agent, there must be an explanation of why he is in that particular mental state.
In particular, if an agent has a belief about an object, that belief must be grounded in his perception of that object: creatures in Black & White do not cheat about their beliefs
– their beliefs are gathered from their perceptions, and there is no way a creature can have free access to information he has not gathered from his senses.
REACTIVE PLANNING
A group of techniques for action selection by autonomous agents.
Differ from classical planning in two aspects:
➤ operate in a timely fashion and hence can cope with highly dynamic and unpredictable environments.
➤ compute just one next action in every instant, based on the current context.
Reactive planners often (but not always) exploit reactive plans, which are stored structures describing the agent's priorities and behaviour.
REACTIVE PLAN REPRESENTATION
➤ Finite State Machines (FSMs) and Behavior trees
➤ Fuzzy approaches (Utility Systems)
➤ Connectionists approaches
FINITE STATE MACHINES
➤ set of states and transitions between these states.
➤ Transitions are condition action rules.
➤ In every instant, just one state of the FSM is active, and its transitions are evaluated. If a transition is taken it activates another state. That means, in general transitions are the rules in the following form: if condition then activate-new-state.
Example: Isla, D.: Handling complexity in Halo 2. Gamastura online (2005) http://www.gamasutra.com/gdc2005/features/20050311/isla_pfv.htm
Reactive Planning
Traditional - often used
BEHAVIOR TREES
➤ Reactive decision making and control of the virtual creatures
➤ Behavior trees replace the often intangible growing mess of state transitions of finite state machines (FSMs) with a more restrictive but also more structured traversal defining approach.
➤ Behavior trees are formed by hierarchically organizing behavior sub-trees consisting of nodes.
➤ Fuzzy Logic: conditions, states and actions are no more boolean or "yes/no" instead: but are approximate and smooth. Behaviour will transition smoother, especially in the case of transitions between two tasks.
➤ Trade-off: It is slower.
Zadeh, L.A. (1965). "Fuzzy sets". Information and Control 8 (3): 338-353
FUZZY APPROACHESReactive Planning
Related to ‘utility systems’
CONNECTIONISTS APPROACHES
➤ Reactive plans can be expressed by connectionist networks like artificial neural networks or free-flow hierarchies.
➤ units with several input links that feed the unit with "an abstract activity" and output links that propagate the activity to following units.
➤ Each unit itself works as the activity transducer. (Typically, the units are connected in a layered structure.)
Reactive Planning
My ‘mind module’ is an ex of this
IVAS AND SYNTHETIC HUMANS
➤ Agents built to interact in real world (as opposed to a fictional world), so rules and laws are known
➤ Agents mimicking humans, modeled from psychological and cognition theories, such as the OCC model
➤ Big 5 and affect theory.
➤ Example: psychSim (Stacy Marcella et. al)
BELIEVABLE AGENTS -
OPPORTUNITIESMichael Mateas, Elizabeth Andre, Ruth Aylett, Mirjam
Eladhari, Richard Evans, Ana Paiva, Mike Preuss, Michael Young
at a Dagstuhl Seminar on AI for games
BELIEVABLE CHARACTERS: OPPORTUNITIES
• Most agent learning focuses on easy-to-evaluate criteria
– Learning should be personality-specific – Online learning must maintain believability while doing
exploration (converge in relatively few exposures)
BELIEVABLE CHARACTERS: OPPORTUNITIES
• Combine statistical language model for style with semantic / symbolic content models
• Dynamically generated character soundtrack communicating character & social state
• Two directions for more information-rich signals from – Player: high-bandwidth naturalistic interaction (gestures, gaze); new
communication modes (biometrics, out-of-band interactions like music selection, player modeling)
– Purposefully set up choices for characters to allow them
• to express personality through choice
• and conversely set up player choices that give information about player
BELIEVABLE CHARACTERS: OPPORTUNITIES
• Take advantage of low-cost Mocap to “correlate” prosodics & gestures features in a generative models (has to be parameterized by emotion & personality and social context)
• Using explainable AI to drive interface elements (text explanations) or thought bubbles
EXPRESSION: CHALLENGES
• Rich emotional state currently not expressed
• Language is the elephant in the room
• Multi-modal expression often results in inconsistencies between modes
• Handcrafted currently works best
EXPRESSION: OPPORTUNITIES
• Go beyond naturalistic world simulation affordances to express character state
– reifying state in characters and objects, – behaviour explanation (HUD) – South Park and other stylised comic-inspired forms , music,
abstract visuals
AI FOR MILITARY TRAINING
AI IN VBS
IMMERSE PROJECT
An interaction with one of the virtual characters, while an activity continues to take place in the background.
IMMERSE
➤ IMMERSE is a projected funded under the DARPA Strategic Social Interaction Modules program. The goal of the project is to produce a game-based training environment that teaches people how to be “good strangers”, supporting the practice of skills necessary to have successful social interactions when in novel culture and language contexts.
➤ Virtual dramatic scenarios, realized within the Unity game engine, in which the player interacts with computer-controlled autonomous characters in high-consequence social environments, learning how to quickly recognize and navigate the social interactions norms in these environments.
IMMERSE
➤ The simulation is embedded in a 3D virtual environment modeled in the Unity Game Engine. The social simulation is implemented by integrating a real-time behavior language with a rule-based artificial intelligence system specifically designed for turn- based social interactions. The player interacts with the environment and other embodied virtual agents through custom automated speech recognition and full body gestures via input from a Microsoft Kinect.
➤ Pedagogical goals are embedded into each scenario, along with specialized pedagogical coaching through specialized virtual agents, to help focus the trainees attention on the most salient points in the simulation and to work on specific problem areas identified by the player model.
Expressive Intelligence Studio, UC Santa Cruz
FINAL REPORT
http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA625663
Report contains example scenarios!
BUT… all these agent technologies, approaches and applications,
can’t solve all problems
THE GAMER MODE PROBLEM
Anders Frank, 2010:
“at different times players may focus on the rules, the fiction or on both during game play. In military education with games, this poses a problem when the learner becomes too focused on the rules, trying to win at any price rather than taking the representation and what it implies in terms of permissible behaviour seriously.”
Frank, 2010, Swedish Defence University, Gamer mode: Identifying and managing unwanted behaviour in military educational wargaming
http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A768474&dswid=4699
A Grail of AI research in its infancy:
The Automated Game Master
WHAT THE HIVE MIND OF THE AI GAME
PROGRAMMERS’ GUILD SAID.
(here only sharing those parts that people were OK with being shared outside the
workshop)
SUMMARY
➤ Manual paper gaming can be an alternative.
➤ Maps are important!
➤ VBS is very expensive.
➤ Little sharing of knowledge because of monetary and national interests. (Big contracts, national security)
➤ Physical tasks are hard to simulate - focus on strategic decision making (at least until we have better VR/AR)
➤ Scenario construction: Rules for game play MUST match training goals
AI BASED GAME DESIGN
Diagram is a joint effort of Josh McCoy, Anne Sullivan, Gillian Smith, and me (2011, Santa Cruz)
Metaphor: adapt game mechanics to the goals of the game. The learning goals of the training scenario. Build as true a representation of the real situation as possible.
AI BASED GAME DESIGN
➤ Diagram is a joint effort of Josh McCoy, Anne Sullivan, Gillian Smith, and me (2011, Santa Cruz)
Metaphor: adapt game mechanics to the goals of the game. The learning goals of the training scenario. Build as true a representation of the real situation as possible.
AI BASED GAME DESIGN
➤ Diagram is a joint effort of Josh McCoy, Anne Sullivan, Gillian Smith, and me (2011, Santa Cruz)
Be cautions using off-the shelf game play from existing genres such as FPSes. Players likely to go into gamer mode if controls and GUI are similar to what they play for entertainment. Difficulty: if the game-engine used IS an FPS engine. Then it will be conscious effort to change those parts.
Metaphor: adapt game mechanics to the goals of the game. The learning goals of the training scenario. Build as true a representation of the real situation as possible.
AI BASED GAME DESIGN
➤ Diagram is a joint effort of Josh McCoy, Anne Sullivan, Gillian Smith, and me (2011, Santa Cruz)
Consider the AI in the same manner: “known” NPC behaviour creates a feeling of game-ness,
unrealness. Adapt it to the scenario.
Be cautions using off-the shelf game play from existing genres such as FPSes. Players likely to go into gamer mode if controls and GUI are similar to what they play for entertainment. Difficulty: if the game-engine used IS an FPS engine. Then it will be conscious effort to change those parts.
WHAT I DO: CO-CREATION OF SOCIAL INTELLIGENT AGENTS
➤ Agents with personality, moods and emotional memory
➤ Created by players & system in collaboration
➤ In play: agents enacted, allowing for reflection and perspective change
My past work of this sort: PataphysicInstitute (2010)
C2Learn suite of games (2014)C2Learn.eu
ENDING TALK -> DISCUSSION
➤ What are the current most important goals for military training?
➤ How can these challenges be met - with or without AI?
