squad tactics

SQUAD TACTICSAn overview of current and

potential techniques for squad based tactics in games.

Bryan Auslanderbla204@lehigh.edu

Introduction I am originally from New Jersey. Fourth year CSB student. Planning to be here a fifth year obtain a

masters in computer science. Spent the last 2 semesters working on a

case based reasoning system (RetaliCase) for Unreal Tournament bots. (More on this later.)

Agenda Introduction to squad tactics. Brief overview of waypoints and squad

tactics derived directly from these. Decentralized Team AI based on

emergent behavior. Planned squad tactics through centralized

squad leader/system. RetaliCase development and overview.

Halo 3 examples from developers and in game footage will be used throughout presentation.

Squad Tactics Introduction Deviation between true AI systems and

the illusion of intelligence is most evident. Difficulty to program. CPU time. Unpredictable behavior.

Chris Butcher and Jamie Griesemer (designers on Halo) comment:

“If your looking for tips about how to make the enemies in your game intelligent we don’t have any for you we don’t know how to do that, but it sounds really hard.”

Squad Tactics in Sports Games The Madden NFL is a

widely praised learning squad AI. 11 players on each team

to coordinate. Each player has different

attributes. Coordinate blocking to

allow running plays. Learns how to predict

and counter players play choices.

Can choose a wide variety of offensive plays based on situation.

Top: Madden 07 Bottom: Madden 08

Video: http://www.youtube.com/v/qNoxHk53uNw&rel=1

Madden AI to good? Accused of cheating:

Game more difficult than previous ones. AI seems to make miraculous comebacks.

Other ways Madden makes AI seem better. Higher difficulties increase AI player attributes

while decreasing player attributes.

First Person Shooters Large area of focus for squad tactics.

Military applications. Used to simulate Large dynamic and uncertain

environment. Limited information. Imperfect communication.

Popular gaming category. Halo Rainbow 6 Bio Shock.

Brief Intro to Waypoints. Waypoints are commonly used in many

games. This is going to be covered extensively in another

class NPCs use them to navigate the environment. Points are nodes in a node graph which is a

representation of all paths NPC can travel. Greatly simplifies AI movement.

Use path finding like A* to get around. Nodes can be used for other applications

besides just path finding. Store precalculated visibility information. Tactical value can be derived.

Supreme Commander

Rainbow Six 3 Mission Planning

Squad Tactics Through Waypoints.

For entire node graph store visibility and connectivity as a bit string. Used to find safe

nodes. Good attack positions. Multiple team mates

sharing information increases knowledge of good and bad nodes.

Coordinate Flanking.

Squad Tactics Through Waypoints: Pinch Points

In FPS need to anticipate opponent.

Pinch Points are places to launch an ambush. Single exit from a

room. End of hallway. Coordinated effort in

a multiple exit room

Squad Tactics Through Waypoints: Issues

Node graphs can become large so may need to only store part of it at a time. Processing issues. Memory issues More complex.

Requires proper node locations. Less of a problem with experience. No good automatic node placement

algorithms available yet.Sensory calculations can’t be done on the fly. “If you want to run the AI logic every frame then you will bring down the whole frame rate. Raycasts account for 60% of the AI computation budget.”

Decentralized Squad Tactics

No squad Leader. Each squad member

can have its own unique AI.

Team members communicate with each other.

Squad tactics are an emergent behavior.

Decentralized Halo video.

Decentralized Squad Tactics features

Easy extension to individual AI.

Can handle a wide variety of situations.

Can effectively use each team members capabilities.

Easy to combine with scripted action’s.

Decentralized squad AI individual behavior.

Simple Example: Fire and Maneuver

Squad members move and announce movements and actions to squad.

When enemy found decides whether to engage threat based on: Number of squad members engaging. Number of squad members moving. Engaging an enemy too long.

Results in Fire and Maneuver behavior: Squad members are able engage multiple enemies effectively. Don’t stand still to long. Can switch targets easily. Allows for individual member specialties to be used. Can easily handle squad member deaths.

Halo vs swarm

Example Two: Squad assaulting hostile positoin.

Each squad member chooses next position. Based off

individual preferences

Intentions of other team members

Can make heavy use of waypoints as described before for faster planning.

Ineffective Example: Ambushing

Wait till enemy arrives in kill zone. Engage them then pull back to rally point. If discovered return fire and head back to rally

zone.

Ambush Problems Difficult for squad to decide on action

since no leader. Would have to put squad to sleep till enemy in kill zone.

Squad can not react if the enemy decides to pull back before reaching kill zone.

Squad would never create L shaped ambush on its own.

Decentralized Conclusions Advantages:

Does not require major addition to individual AI. Can make team members act to take advantage

of their specialties. Easy to Combine scripting to create custom

scenarios. Disadvantages:

Can not easily obtain unanimous decisions. Movement can become difficult when avoiding

each other paths. Can be memory intensive for each member to

maintain its own state.Halo: [Maintaining full state]too complicated – O(n2) for n=25. instead actors only track characters that are important to them, n around 3 for friends, 5 for enemies

Squad Tactics:Centralized Planned Maneuvers

As is true in the military squad AI can not assess its situation and decide the best course of action without a leader and predefined courses of action.

Adding a centralized planning element does not replace the individual AI, but extends it.

"There is nothing inevitable about military victory, even for forces of apparently overwhelming strength. The Greeks at Marathon, Alexander against the Persian Empire, the success of the colonists against the British in the American Revolution, Napoleon over the Austrians in Italy... all offer dramatic evidence to the contrary. In the absence of inspired military leadership... the more powerful side wears down the weaker." - Bevin Alexander

Comparison of Decentralized vs. Centralized

Command Styles Authoritarian

Squad members always obey and execute commands. Yields tightly coordinate maneuvers. Allows squad member sacrifices for overall squad. Ineffective when squad member view conflicts with squad level.

Ex: Sniper visible to squad member, but conflicts with orders Coaching

Squad AI assigns tasks to members. Squad members tries to execute it and reports when it can’t so

it can be reassigned. Yields looser coordination due to members being easily

distracted. By mixing command styles advantages from both can be

obtained.

Command Style Examples Authoritarian

When command is issued unit must perform.

Coaching Command is

issued then up to NPCs to decide how to proceed.

How to represent situations

Situations can be expressed in abstract features. Features can be used to choose best maneuver.

Situations features can be represented through influence map as above.

Choosing a Maneuver: Fuzzy Logic

Choosing a Maneuver AI keeps track of permissible maneuvers. Evaluates each ones fitness. Chooses maneuver with highest fitness.

Fitness can be represented with fuzzy rules that express situations the maneuver is applicable in. Situations can have multiple rules to represent

different situations Closely resemble our own rules of thumb.

Halo: Behaviors are made obvious:“Initially nobody noticed so we had to keep adding clues to make it more obvious. By the time we shipped we had made it so every single grunt runs away every single time an Elite is killed.”

Bounding Overwatch Pullback Manuever

Pull back while maintaining cover fire. Members in line

provide suppressive fire.

Lead member runs to back.

Repeats until they have reached destination.

Pullback Maneuver Details Path used to pullback requires finding a

path that Has short travel times Provides concealment from enemies Gives sufficient space to leapfrog teammates.

Searching for this path is CPU intensive so squad level decision so save CPU time.

A maneuver is performed in several steps Preparation. Execution Finalization

Pullback Sequence Chart

Centralized Squad Conclusions Advantages

Good for synchronization and coordination Resembles human tactic and rules of thumbs. Can include create many premade plans.

Disadvantages Can not easily adopt to small variations.

Ignoring a sniper because it goes against plans. Can sometimes ignore squad member strengths. Requires a leader to analyze the situation

Requires lots of communication and processing to keep state current.

Halo: Hide weaknesses with design.• Design levels to showcase AI.• Design behaviors to cover up weaknesses

• Ex: AI gets more accurate the longer a player is out of cover.

Halo 3 Videos Thank you to Jim Blaney and Tabin

Ahmad for assisting me in creating these clips. Clip 1 Clip 2 Clip 3

Halo AI problems.

CONSIDER MOVING CLIPS T APPROPRIATE PLACES IN THE PRESENTATION

RetaliCase Case based reasoning system applied to

a reinforcement learning algorithm to control a team of unreal tournament bots.

Retaliate originally created by Megan Vasta Was presented in class before Bots would learn strategies during game

based of current state and score (Reinforcement Learning).

System proved very effective, capable of defeating the previous research project “HTN bots” in only two games.

Retaliate Findings. Retaliate’s biggest

weakness is maintaining memory between different opponents. Retaliate must unlearn

a case to learn a new one.

Can not identify previous opponents.

Can take a while to learn a strategy.

-10

0

10

20

30

40

50

60

Time

Scor

e

RETALIATE HTNbots

-10

0

10

20

30

40

50

60

Time

Scor

e

RETALIATE HTNbots

RetaliCase Conception Retaliate could be improved to respond to

changing opponents. Could store previous knowledge (qtables) for

future use. Could be more competitive against new

opponents by finding a similar situation. Case-Based reasoning is ideally suited for

this task. Store plan information. Can retrieve data based on in game situation. Could be built right on top of Retaliate.

RetaliCase Tactical View Uses 18 calculated features about current

state to save and retrieve cases Location of Bots compared to domination

locations. Average time a domination point is owned by

Retaliate or Enemy Team score and enemy score differences.

These features are stored in the case based system for each case.

A case about the current situation is also maintained and continuously updated.

Simple RetaliCase ChartRetaliate

Case List

Maintains

RetaliCase

Save Case

Evaluates Case

Load Case

No Yes (winning)

Store Case

Save Case

Case

Retreive and Load

Case

Retrieves Case

Yes (Losing)

Loads case into retaliate

No (do nothing)

RetaliCase Storage Cases are saved

during every successful game.

At the start of every game cases are read into memory. Each case takes up very

little memory and comparisons are fast.

On larger projects however may need a database to speed up case retrieval and minimize memory usage

RetaliCase Results Results so far

Loading good cases has shown increases in performance.

Retalicase seems to be performing better then Retaliate in start of game.

Discovered the need to make sure cases are mature and have had a chance to have been modified before saving/loading. Poor cases were being saved. Bad case loading loops were occuring. Hindered learning.

RetaliCase Demonstration Possible demonstration.

Summary Squad tactics are starting to move toward true AI. Squad tactics are applicable to variety of genres

and applications. Makes heavy use of other technologies

Waypoints Scripting Languages

Increasing research and development going into organizing and controlling squads. Centralized squads Decentralized squads.

True learning AI systems being created and tested all the time (ex. RetaliCase).