Squad Tactics and Artificial Intelligence

Matt MitchellDavid PennengaJohn Formica

Correct Movement

Topics to cover Levels of threat Straying from the team Use of caution Team context Speed of movement

Levels of Threat

The more targets, the greater the possibility of obstructing line of sight (LOS) How to fix this?

If the player is looking in the direction of NPC Move to the angle of

LOS that has less targets

Levels of Threat (cont.)

If the player is looking away from NPC NPC should move

in opposite direction that the player is looking

Levels of Threat (Cont.)

If above heuristics fail, prioritize obstruction Threat level of enemy

A player is more likely to target: enemies with greater threat levels▪ Obstruct view of enemies with lesser threat

levels Groups of enemies over individuals▪ Opt for obstructing view of smaller groups or

individuals▪ Threat level also should be considered

Straying from the team

NPCs should stay close to the team Set maximum radius from team

Why? Straying from the team could:▪ Alert enemy combatants▪ Distract player from game▪ Trying to find lost teammates

▪ Cause failure to assist player

Use of Caution

Teammates should avoid risking the welfare of the team

NPCs should not be the first to engage an enemy Explore a room Move over a hill or around a corner This could undermine the player’s play

style

Team Context

Take different play styles of the player into consideration Some games allow player to set play styles▪ Aggressive vs passive▪ E.G. Republic Commando▪ “search and destroy”▪ “form up”▪ “hold position”

Move at the same pace as player But not obstructing LOS Consider tactically sound behavior▪ We will cover more about this shortly

Correct Behavior

Types of behavior Use of cover Selective firing reloading

Use of Cover

Path-finding algorithm Prefer cover and the shadows▪ Causes player to believe in tactical prowess

of NPCs▪ Adds to realism of game play▪ Prevents accidentally alerting NPCs

Take player cues If NPC behavior does not match player behavior▪ E.G. Speed of player movement could cancel

priority for cover in path-finding algorithm if player appears to be in a rush

Selective Firing

Only fire when within range and in view Controlled bursts▪ Maintain accuracy▪ Conserve ammunition

Reaction times▪ Should be manipulated if individual combat-

related AI is too powerful▪ Balance high accuracy with low reaction times▪ Certain manipulations also make contextual sense:

Rainbow Six style squad games

▪ Team reaction time should not be lower than enemy team’s reaction time

Reloading

Cycling aggressive NPCs Reloading NPCs fall back or take cover▪ Return when finished

Stagger fire of NPCs So NPCs do not run out of ammunition at

the same time “taking turns” at attacking ensures that

player will most likely always have supporting fire

Supporting the Player

Close support of the player Protecting the rear Reporting to the player Selecting a target Responding to orders

Reporting to the Player

Protect player’s sides Report threats via audio or visual cues▪ Messages are a staple of squad game play▪ Do not overwhelm player▪ Reporting threats while in combat is not usually

necessary

Pick appropriate engagement times▪ Not in a fire fight▪ Do not aggravate more enemies than already

fighting

Selecting a target

Chose targets other than the player’s Chose targets farthest from the player’s

LOS NPCs are there to complement game

play not hinder it Also should take threat level into

account Player might need assistance▪ Health of player’s target versus player’s own

health Consider higher concentration of targets

The player is most important Weapon selection

NPC weapon should not be more powerful than player’s unless dictated by player

Weapon and item selection Allow player first choice▪ Implemented by tracking player proximity to items▪ If the player approaches items and walks away

then NPCs may pick up items Reaction time

Allow player to attack enemies first▪ Unless team is under attack

Implementation

Line of sight avoidance hierarchy Each child avoids the line of sight of the parent Allows for easy restructuring in the event of

casualties▪ Node replacement algorithms

Threat awareness Use LOS algorithm to find and report targets▪ Maintain group awareness of threats by “looking around”

so that NPC LOS updates, player awareness of environment▪ Used in some groups’ Robocode this semester

Environment awareness Identify cover/restricted areas/straying from player

Implementation (cont.)

NPCs reference a team knowledge base that governs NPC behavior

Finding available NPCs Availability = (1+N)(1+O)(1+P)+(Q*[infinity])▪ N= # enemies in covering area (integer value)▪ O= # enemies within range (integer value)▪ P= # enemies threatening team (integer value)▪ Q= supporting another teammate (boolean value)

The above formula▪ Provides a return value of greatest value in the event that the

NPC in query is supporting a teammate (optimal behavior)▪ If the NPC is not supporting teammate, it provides a finite

return value which can easily be compared with the availability of other NPCs

Example of Correct NPC Behavior

Rainbow Six: Raven Shield Team of AI finish a mission

http://www.youtube.com/watch?v=-9f3cXBB-Nc

Visible in this video Maintaining constant awareness of environment Reaction time alterations for context▪ Very fast for mimicking elite soldier behavior

“Stick to the shadows” path-finding Controlled fire NPCs stay close to the team Appears to use LOS dominance hierarchy

Team AI and Emergent Maneuvers

Decentralized approach Squad members exchange info equally▪ Messages!

No commanding force▪ Instead, equally shared knowledge bases for reference

of each member Squad behavior is an extension of individual AI

Centralized approach Leader receives and interprets info from soldiers Orders soldier behavior▪ Differing strength of orders▪ This will be covered shortly

The Decentralized Approach Pros

Extension of individual AI (easier to implement)▪ All AI simply must communicate with each other either directly (messages)

or indirectly (communicating with an abstract knowledge base) Robustly handles situations▪ Emergent behavior facilitates problem solving without having to

specifically code for given tasks Deals well with variations in capabilities of teammates▪ Individual AI is permitted to interpret or act upon supplied info differently

▪ hence this approach being an extension of individual AI

Easily combined with scripted AI▪ Same reason as above

Cons Weak at autonomy or tight coordination▪ Lack of specific plans for problems hinders team cohesion

Cannot deal with individual strengths/weaknesses

Squad Maneuvers

Tightly coordinated Members rely on detailed, repeatedly

rehearsed drills, and continuous flow of info▪ In CSE terms, events are anticipated and

coded specifically to be solved Loosely coordinated

Does not rely on planned maneuvers Relies more on verbal communications

Emergent Behavior

Fire and maneuver behavior individual behavior of either firing or moving

▪ Communicate this within the squad▪ Emergent behavior of position cycling

Goal behavior achieved via communication Stay close to cover

▪ Communicate info of cover whereabouts Prevent blocking LOS of others

▪ Communicate direction of LOS Take weapon capabilities into account

▪ Damage, area of effect, etc. Maintain group cohesion

▪ NPC location and intentions Spread out

▪ Communicate distance from other NPCs Keep LOS on targets

▪ Target locations, if the target is moving in the direction of another NPC

Squad Member’s Mental Picture

For each member Current position and activity Claimed destination position LOS

For each opponent Last known position/state Estimated current position Members engaging opponent Members able to observe opponent LOS

For other hazards/threats Known/estimated position Damage radius

Messages

Why so preferred? Model communication latency by queuing messages Present the message in game play Can filter messages for player benefit▪ Prioritize info of messages

Sends to dead members assumed alive for realism▪ Only serves a purpose when player witnesses it

Can use scripted entities to direct members▪ The content of messages serves a purpose whether the

source of the info is adaptable AI or not Accommodates human members

Ambush Example

Goals Wait for enemy to enter kill zone Pull back to predefined rally point after

engagement Fall back when discovered

How does a decentralized approach handle ambushing?

Ambush Example (cont.)

Decentralized approach… Returns fire well Can prefer being near rally point BUT▪ Cannot reach unanimous behavior on

attacking because each agent will carry out its own behavior▪ Cannot orchestrate movement well because

there is no central executive force to create a consensus of behavior

Planned Maneuvers

Centralized approach styles Authoritarian command style▪ Focuses team performance by forcing agents

to perform commands▪ Cannot be ignored by individual AI

Coaching command style▪ Advises, rather than forces, agents▪ Allows for agents to override commands

▪ Problems could arise from agents wanting to always perform their own behavior

The Squad’s Situation

Things to consider Force ratio ▪ ratio of allies to enemies

Line of fire ratio ▪ ratio of # of enemies in allies’ line of fire to

the opposite Consider Boolean states to make

judgment calls Knowledge bases First order logic conjunctions

A Good Maneuver

Chosen path is a good combination of Short travel time Concealment from pursuing enemies Sufficient space to maneuver

Maneuvering

Progress should be monitored Arriving at position informs the squad Provides individuals with a new

command Pros of this approach

Allows for changes in squad behavior mid-maneuver▪ E.G. a member is trapped in a spot which

forces a reevaluation of the path finding algorithm and sends it far away from the squad

Pulling Back

Use maps or an alteration of the A* function

New cost function considers Nodes that can be fired at from

preceding nodes Nodes that provide insufficient space to

bypass Nodes that are a bad position from

which to fight The time and spatial complexity of

this new function force it to be applied to the squad aggregate for practicality

Pulling Back (cont.)

To properly pull back Mark each position just before a bend in

the path that blocks the LOS of many preceding positions on the path

Features sufficient room for members to bypass it▪ If first choice of bypass is obstructed some

how, a second choice should be available Is a good position from which to fire▪ Pulling back across an open field versus a

rocky hillside

Command Hierarchies

Chain of command: Commander -> captain -> sergeant -> soldier More subdivisions are allowed if necessary

Creates an organized and efficient way of commanding soldiers

Example: Outflanking Maneuvers

Strategic Decision Process Check if maneuver is possible/appropriate▪ Doors are accessible (referencing door IDs so

there are no duplicate actions) Compare▪ Distance between opponent and door with

distance between soldier and door Maneuver organization

Consider distance to closest door so maneuver is as fast as possible

Example: Covering Exits Maneuver

Strategic decision process Try to determine whether the maneuver

is possible/advantageous▪ Check if ally must cross a specific room in

order to get from their room to a target

Example (cont.)

Maneuver organization Find how many soldiers are needed to

cover exits Order soldiers to cover designated

positions Decide on a room to search Order soldiers to move to reachable

doors of the first room When all of the above have been

completed, a search command is sent to squad

Formations

Types of formations Line Left flank Right flank Column Box Wedge Vee

Facing in Formations

Different facing applies to different tactics Defense vs offense LOS Friendly fire Project more firepower

Staggered line Outward facing vs inward facing

Ordering

Mixed unit type ordering Weaker/longer-ranged units in the back of

the line or center of a box Faster units placed at flanks

Closest position Calculating the best match of unites to

positions▪ Sort units based on their minimum distance to

the closest position▪ Iterate through the sorted list▪ Assign each unit the closest unused position

Unit mobility

Direction may play a role if turning is difficult

Create formation positions that avoid crossing paths of units Use movement vector and parallel paths

Arriving time Fix variations in arrival time by making

all units’ paths lead to the center of the formation and order is determined by arrival time (illustration on next slide)

Unit Mobility (cont.)

Unit Mobility (cont.)

Spacing distance Must be scaled so that the size difference in

units is considered Ranks

Each formation has a maximum occupant limit▪ Start new formation once reached

Playbook If certain places are reserved for specific units

restrictions must be coded▪ If no match found for a position, slot may be skipped

(hole is made)

Moving Out

When do the units fall into formation?

Upon initial movement Might delay movement Might cause unit backtracking

Just before destination Might give player impression that

nothing is happening Might be best choice for coordinated

attack

Moving out (cont.)

do formation at destination Faster movement/no delays Formations made as distance to

destination decreases Not as effective while en route Not as organized upon arrival

Moving out (cont.)

Group path finding and movement Calculate path for lead, others apply

formation offsets to path Flocking style where group follows

slightly behind lead in rough formation When mixed units move as a group,

differences in locomotion can break formation integrity▪ Solution: all units move like the slowest unit▪ Problem -> faster units might appear to move in slow

motion

Moving out (cont.)

React to ambush Spread out to avoid area of effect or

converge to chase down an enemy After passing obstacle or attack

regroup in the same fashion that the group was originally formed Reassign positions to make up for lost

units if necessary

Influence vectors

Algorithms capable of tactical assessments in a dynamic environment

Modification of influence mapping Cells of influence are calculated from unit

positions outward Total influence: a factor which assesses bot

tactical situation in the engagement area Each “cell” superimposed on the influence

map has a calculated value associated with agent and environmental qualities

Genetic Programming

Learning algorithms that give an NPC the ability to adapt to opponents’ playing styles in real time

Pros Provide more challenge as opponents

because they are unpredictable Facilitate the discovery of new strategies

Cons Developers cannot test what behaviors

the NPCs will exhibit

Genetic Programming (cont.) Entire team of NPCs view as one chromosome

Each agent derived from a different part of the chromosome▪ Creates heterogeneous teams

Five node sets, total of fifty nodes used in evolution

Genetic Programming (cont.)

Fitness calculation Takes into consideration the game’s

duration and remaining health of team and enemy agents

Selection Elitism: m copies of the best n

chromosomes from each generation are copied directly into the next generation

Roulette wheel selection▪ Any chromosomes selected are subject to

randomness

Example of capabilities

http://www.youtube.com/watch?feature=player_embedded&v=4ErEBkj_3PY