A HUMAN –LEVELA HUMAN –LEVELARTIFICIAL ARTIFICIAL

INTELLIGENCEINTELLIGENCEAPPLICATIONAPPLICATION

More precisely called

Branch of AI behind it

are Interactive games an area of Human-level AI research ?

is AI used in Interactive games ?

Picture Courtesy : Google Images

Human -like attributes expected Human -like attributes expected in a human-level ai system…in a human-level ai system…

are Interactive games an area of Human-level AI research ?

is AI used in Interactive games ?

Search

Planning

Logic

Focus : Game Tactics

A case study : the basicsA case study : the basics

How AI is used to enhance Game Tactics

How AI is used to enhance Game Tactics

AI tools used

Evolutionary computation &

Reinforcement Learning

Real-time Strategy Games

Genetic Algorithm

A learning technique with a mathematical reward function.

A learning technique with a mathematical reward function.

• Player needs to control armies to defeat all opposing forces in a virtual battlefield.

• Key to winning lies in efficiently collecting and managing resources., and appropriately allocating these resources over various action elements.

• Famous examples : Age Of Empires , World of Warcraft .Picture Courtesy : http://www.igniq.com/images/age_of_empires_3

Improve

Weaponry Attack

• AI in RTS games determines all decisions of the computer opponents.

• Encoded in the form of scripts. Called STATIC SCRIPTS

I don’t care about available

resources. Attack at earliest !!!Ha Ha Ha!!

I have to first well develop my army,

then only I can attack. This will

take a while.

HUMAN

AI

Picture Courtesy : World Of Warcraft

I have suffered heavy losses. Now I need to increase my

strength first. Small attacks are

of no use.

AI is gathering resources and preparing for

heavy assault.

HUMAN

AI

Picture Courtesy : World Of Warcraft

)()(

)(

1,,1,,

1,,

isisiaia

iaiai

SSSS

SSR

iaS ,

isS ,

winbSS

S

lostbSS

S

R

LsLa

La

LsLa

La

,max

,min

,,

,

,,

,

C end is a parameter and is set less than 0.5.

Contribution of State Reward is kept larger than Global Reward.

P max and R max are the maximum penalty and maximum reward respectively.

}{1)1(

1

}{)1(

max

max

bRb

bRC

b

bRCR

bRb

RbC

b

RbCP

Wi

endend

iendend

Evolutionary State Based Tactics Generator (ESTG)

Genetic Algorithm Application !!!

Counter Strategies are “played” against training scripts , only the fittest are allowed to the next generation.

Chromosome EncodingEA works with a population of chromosomes . Each represents a static strategy .

The chromosome is divided into the m states .

Start State 1 State 2 State m End

States include a state marker followed by the state number and a series of genes.

Chromosome Encoding

A Gene

Parameter values

4 types of genes

Partial example of a chromosome .

Chromosome Encoding

Fitness Function

b

MM

M

bMM

M

C

C

F

sa

a

sa

aT

,max

,minmax

Fitness Function

Genetic Operators

Genetic Operators

KT: State-based Knowledge Transfer

The possible tactics during a game mainly depend on the available units and technology, which in RTS games typically depend on the buildings that the player possesses.

Thus, we distinguish tactics using the Wargus states .

All genes grouped in an activated state (which includes at least one activated gene) in the chromosomes are considered to be a single tactic.

tactics

Extracting Tactics for a state

Performance of Dynamic Scripting Experiment Scenario

Performance Analysis

The three bars that reached 100 represent runs where no RTP was found (e.g., dynamic scripting was unable to statistically outperform the specified opponent).

The opponent strategies

Ave

rag

e R

TP

valu

eRTP is the number of the first game in which the adaptive agent outperforms the static agent.

low RTP value indicates good efficiency for dynamic scripting

Where we stand Where we stand today………today………

Achieved

Achieved

Achieved

Achieved

Achieved

Achieved

Not Achieved

Not Achieved

Picture Courtesy : Prince Of Persia , Google Images

DrawbacksDrawbacks

Giving undue advantages to AI agents.

Future – Scope:Future – Scope:

• Removing the “cheating” factor from Interactive games.

• Introduction of Creativity in AI agents.

• Capability of AI agents to reason with human-like Common Sense.

Ponsen,M. & Spronck,P.(2006). Automatically Generating Game Tactics via Evolutionary Learning.

Spronck,P. , Sprinkhuizen Kuyper,I. & Postma,E. (2004).Online adaptation of game opponent AI with dynamic scripting.

Sutton,R., & Barto,A.(1998). Reinforcement learning : an introduction.