influence maps

Lecture 17 – Influence Maps

CSci 4511 – Intro to AI3.21.11baylor

University of Minnesota

We begin our story with a bunch of problems…

Terrain Reasoning

• Old fashioned war• Strategies:

– Strengthen front– Attack flank

• Pincer– Ambush rear

Terrain Reasoning




• But where are those?

Problem 1. Finding Flanks





Group units close to each other

Problem 1. Finding Flanks (contd.)





Draw line through cores(so can tell who’s behind or side)






Perpendicular line is the FrontSides = Flanks, Behind = Rear

Flank

Flank

Rear

Fron

t






How do we make a computer understand that?

Flank

Flank

Rear

Fron

t

MOUT• Don’t get to play tanks and trenches much

anymore– Stupid nuclear bombs

• MOUT – Military Operation in Urban Terrain– Cities– Filled with hiding spots– And civilians

• And enemies who look like civilians• We care because of the CNN effect. And stuff

• Modern training is for individual combat, non-lethal combat, checkpoints, etc

Problem 2. City Defense

building,1 door,2 windows

wall,2m / 7ft

wall,1m / 3ft

deepwater

bridge w.1m / 3ft walls

stairs,from water

?Where do we put:• Snipers• Sentries• Sentry guns• Heavy weapons• Traps• Ambush

Image courtesy of William van der Sterren



wall,2m / 7ft

wall,1m / 3ft

deepwater


stairs,from water

?

Sniperlong range view,enclosed space,

Checkpointchokepoint

Bad for Sniperlong range view but exposed

We can see this but how can a computer “see” it?

Terrain Reasoning

• These problems are instances of terrain reasoning

• Computers can’t “see”– Sees raw data, not groupings, relationships– Humans very good at this

• Understanding the space is important

Problem 3. Path Finding

Plenty of algorithms for finding paths…

Path Finding (contd.)

• Plenty of things should influence path• Some are semi-permanent

– Terrain type– Light (for games where you sneak around)– Sound (ditto)

• Some are dynamic– Enemies– Buildable/destroyable walls (RTSes, Tower Defense,

Red Faction)– Fog of War– Treasure– Mobile Targets (e.g., ball in soccer)


Plenty of algorithms for finding paths…But they don’t take into account things like risk

Building

• Several game types require building– Most obviously Real Time Strategies (RTSes)

• Factors– Place walls where they safeguard the most /

most important items– Place buildings near defense– Place buildings near resources

Placing farms near each other make them easier to defendwith walls, nearby troops

Problem 4. Building location

Building

• Several game types require building– Most obviously Real Time Strategies (RTSes)

• Factors– Place walls where they safeguard the most /

most important items– Place buildings near defense– Place buildings near resources

• Also, Area of Effect (AoE) attacks


…but also make themmore vulnerable toAoE attacks

Problem 5. Bomb placement

Building (contd.)

• Lots of games need to be able to place things in a good place– So they need to understand their environment

• Includes types that haven’t been made yet– Urban renewal games– Lemonade stands– Parties (e.g., Hello Kitty Party Pals)– Shopping games (PaperDoll Tower Defense)– Synergy games

• Clownfish & Anemones, Cinnabon & Weight Watchers

6. Ever-Popular Real Estate Game!

• We want to build a house

• Close to roads good• Elves are good, but

not too close• Don’t build near

haunted castles

Where should we build?

How Do We Solve These?

7. One More Example…

• Should we attack that army?

Us

Them

Lots more ofThem

Attenuated Threat Radius

• Assume each unit is worth a certain amount (danger/attack strength)

• Each has a “threat radius”– Represents reach of unit (if ranged)– Also spots unit can reach quickly

• Threat drops off with distance

Note fancy big words!

Red TankBase Value: 10Propagation Divisor: 2Propagation Distance: 3


• Add unit values to map, Sum values

Us

Them

Lots more ofThem


• Should we attack that army?

Us

Them

Lots more ofThem

One ofours


• Quantitative representation– Computer can

reason with this• Computationally

simple and fast– Just adds and

multiplies– Linear growth in

computation time

8. Reminder: Go is Hard

• 10761 games– vs. 1040 vs. 10120

• Champion: Not Yet• Brute Force does

not scale very well

for computers

Influence Maps

• Created in 1969 by Albert Zobrist– Computers didn’t “understand” spatial patterns

• This capability is important to human problem solving

– Looking to capture human-like “visual organization”• Divide the world into a grid• Each piece adds a number to the grid• Values “flow” through the grid

– Like oil/water (borders) or electric charge (intensity)• Add it all up and you understand the world

– Note: You still have to decide how to use the info

Multiple Maps• Can have more than one type of influence

– Represent different concerns / factors– Can use all of them at the same time

• Sum across influence types / maps• Can select relevant maps at run time

– Can be specific to goal, agent, state, personality, etc.• All started with Go

– Goals: Pieces grouped into armies, influence of armies, segmentation of board

– Maps: Borders, # empty squares in segment (safety), breathing spaces in chain, # stones in chain, # neighbors

Zobrist, A. L., "A model of visual organization for the game of GO," AFIPS Conf. Proc., 1969

Anything You Want

In GO• Influence• Concentration• Tension• Instability

For an RTS• Borders• Resources• Defensibility• Line of fire• Fog of war

Types of Influence Maps• Passability• Resources• Exploration• Threat• Light level• Historical• Emotion (fear, beauty, etc.)• Anything you make up• Desirability Map

– Summation of the above

Light Map, Action Quake 2

Propagation• Can propagate however you want

– Predefined actor halo, rules– Auto-propagation - linear, exponential, Gaussian, etc.

• Can have different “conductance” for each tile– Walls or mountains could block– Roads or supply lines might spread influence more

Roadsspreadinfluence

Lakesblock it(for tanks)

Problems Revisited

6. Ever-Popular Real Estate Game!

Elf

Evil Castle

Road that looks like water

Place A Grid Over The Map

Convert Map Into Something Easy For Us To Reason With

O O O O OO O

O O O

O O O OO

R R RR

RR

RE

XO

Convert Grid To Numbers

1 1 1

1

1 1

1

0 0 0 0 0

00

0000

0

00

00

0 0

Normal land is worth 0Road is worth 1

Live close to elves, but not too close

1

-1-1

-1

-1-1

1100

1

1

1

11

00

0 0

00

100

Stay away from evil castles

-2-2

-2

-2 -2

-2

-2

-2

-1 -1 -1 -1

-1

-1

-1

0 0 0 0 0

0

0

0

0

Each Object Affects The Area Around It

-2-2

-2

-2 -2

-2

-2

-2

-1 -1 -1 -1

-1

-1

-1

0 0 0 0 0

0

0

0

0

Squares next to castle worth -2

Squares next to those squares worth -1

Add The Values Together

0 0

0 0

0 0

0

1 1 1

1 -2 -1

-2 -2 -1 -1

-2

-2 -2 -1 2

-2

We Build At The Highest Number0 0

0 0

0 0

0

1 1 1

1 -2 -1

-2 -2 -1 -1

-2

-2 -2 -1 2

-2Right here

We Build At The Highest Number

Right here


…but also make themmore vulnerable toAoE attacks

Problem 5. Bomb placementProblem 4. Building location

Shortest distance doesn’t use cover, good way to get shot

A* runs a lot faster when you do it this way


Waypoints are marked as being visible to the enemy, factors into the node cost

A lot more work for A*

Note: Color, like danger, fades with distance

Problem 1. Finding Flanks




How a human sees the situation

Flank

Flank

Rear

Fron

t

0 = border / no man’s land

Ferretman, AI Game Programming Wisdom volume 1



wall,2m / 7ft

wall,1m / 3ft

deepwater


stairs,from water

?

Sniperlong range view,enclosed space,

Checkpointchokepoint

Bad for Sniperlong range view but exposed

We can see this but how can a computer “see” it?

What Makes A Good Sniping Spot?• Inconspicuous• Overlooking distant locations• Overlooking key locations• Target locations lack cover• Hard to reach from target locations• Nearby cover for sniper• Hard to surprise• Flank protected• Have freedom of movement• “Focus”

Data We Need

• Line of sight• Travel time from one point to another• Distance from one point to another• Which part of area do we belong to

– Are we in a specific room, courtyard, etc.

Waypoint relations• (potential) “interaction” or influence

between two waypoints• directional, leveled [0..1]

Examples• nearby cover from threat• hard to reach from target• overlooking key locations• hard to surprise

Case: Tactical Assessment

0

0.5

1

1.5

0 100 200

Relation: nearby cover from threat• the amount of nearby cover

from threat and nearby positions


0.0

0.5

1.0

weapon performance

protection

2/9 = 0.22


nearbycoverfromseen threatseast ofposition

Focus• concentration of relations in certain

direction (sphere sector)–If long LoS in one direction but

not another, focus is high–If long LoS in all directions, no focus

• Useful for determining flank protection


No focus

Partial focus (blocked 3 directions, open 3 directions)


focuseast(E,NE,SE, Eup, Edown)


localproperty:light level

Terrain Characteristics Used

• William’s formula for CGF for AQ2– 17 ingredients– Checks in 26 directions

• Specifics for this example (map)– 480 waypoints– 10 seconds to calculate on a PIII/500

• Done off-line– 24KB of sniping ratings

Technical Notes (so far)

Making Your Own Formula

• How William created the CGF for AQ2 formula– 10% Analysis – 40% Experimenting– 50% Tuning

• Store Sniping behavior at each waypoint– Time spent sniping

• Time spent at location• Time spent zoomed in• Shots fired

– Damage done while sniping– Damage taken while sniping

• Breakdown by– Sniping spot– Weapon used– Direction faced

Adding History


positivesnipingexperience,after40 rounds,non-directional


snipingeast,usinggeometry+ physics+ learning


trafficdata (all teams)after40 rounds


snipingeast,usinggeometry+ physics+ learning+ tactical

game playanalysis

waypoints

line of sight / fire

level design terrain analysis game round analyse activity + rerank locationsgame round

game round

paths + travel time

sniper spotanalysis [~20 s]

sniperactivity

snipingratings

sniping activitytracking [ 0.0s]

reinforcementlearning [0.1s]

traffic +kills ...

sniperactivityphysics

Results


Bonus Problems!

9. Track and Predict

• Target runs behind something and now we can’t see him

• Want to predict where he’s going to pop out


house

window

AI actor

threat

door

Situation (top view)

Slides stolen directly from William

?

?


Where to look for the threat? What to aim for?

!?


Option: assume threat static after moving out of sight(often incorrect)

!?


Option: assume threat continues last observed movement(incorrect after a little while)

The waypoint grid(overlay)


The waypoint grid(AI “view”)


Extrapolate movement along waypoints(valid paths, movement speed known).

Threat no longer “leaves” the house!


Use known line-of-sight for “negative”observations.Exclude some likely threat movements.


9. Track and Predict• How can we use the waypoint graph?

– Restrict guesses to valid movements– Estimate the threats’ movement speed– Exclude visible potential movements

that were not observed

• Predict a likely path– Path best aligned with last observed direction– Using per waypoint traffic statistics– Towards most attractive nearby objective– Towards strongest defensive position

• WWiD (unrelated to terrain reasoning)– Could ask “what would i do in their situation?”


Where to aim?• Where you think the threat will likely reappear

Good guesses• Visible spot closest to threat• First visible spot on predicted path• If you think he’s coming back for you…

– First visible spot on shortest path back to you• Any nearby portal (door, window, hole, ...)

1

2

3

Threat turns to attack (1) or moves to nearest attack position (2), or continues movement (3)


10. Grenade Handling

• Can’t (or don’t want to) shoot at enemies? Throw grenades!

• Best part – grenades bounce!• Humans good at this, AI bad• You do not want to compute this at run-time

– If you’re going to rebound, there are sooomany places you can throw

– Physics are imprecise so you gotta Monaco it• Precompute, store results in terrain map

– Mark the good spots in your map

Information to track to determine value of spot as a grenade tossing spot

AI is much easier with good debugging tools

Have no direct shot at enemy. But we could throw a grenade from the green spot

We know we can hit this because we pre-computed it long before the game began

Hey, how’d he do that?

Grenades: A Warning

• To throw grenades, need to know– Where to stand– Point to aim at

• Grenades and other bouncy things require you to be very precise– Clean ray trace is not good enough

• If you are just a little off, very bad things can happen

It’s very important to be very precise. Or very bad things could happen

good

bad

oops, we clipped the wall

11. Team Maneuvers

• Choosing an avenue of approach• Bounding overwatch

– One person moves, the other covers him• Keeping clear lines of fire• Keeping to cover• Flanking

11. Team Maneuver

how toattack thethreats asa team?

1

23

11. Team Maneuver

Approach• Initially, sequentially for each member

– evaluate nearby waypoints (within r ) for position improvement

– claim best position p– move up

to position p• Repeat on

memberinitiative

Things To Evaluate For Each Waypoint

• Travel time to / from threat, power-up, ...• Nearby cover and flank protection• Tactical rating of position• Min projected distance to team member

– Spread out a bit• Keep out of friendly line-of-fire• Maintain visual contact with enemy

11. Team Maneuver

how toattack thethreats asa team?

1

23

11. Team Maneuver

evaluate foreach waypoint:-distance to

threat-spread-nearby cover-not claimed-etc.

and claimbest position…

1

23

11. Team Maneuver

member1claims positionand will move up

1

23

11. Team Maneuver

member2evaluatesand picksbest positiongivenexisting andclaimedpositions

12

3

11. Team Maneuver

same formember3

however, inthe search radius, thebest spotmight not bethat good(LOF blocked)…

12

3

11. Team Maneuver

upon arrival,member3indeed seesa blockedline-of-fire

he quicklyexecutesthe sameprocedurefrom hisnew spot…

!#%&!12

3

11. Team Maneuver

now member3moves upagain toa good position…

in an obstaclerich environment,you’ll get flankingfor free…

12

3

Questions?

influence maps

Documents