N-GRAM PREDICTIONAND

BAYES INFERENCE27/10/2014

AI for Games Seminarby Andrea Tucci

Slides by Andrea Tucci - @andreatux

OUTLINE

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

➔ Context➔ Action Prediction

◆ Techniques◆ N-Grams

● Window Size◆ Considerations

➔ Decision Learning◆ Techniques◆ Naive Bayes Classifiers◆ Bayes Networks

● Bayes Inference● Probability Propagation● Games Application

OUTLINE

OUTLINE

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

➔ Conclusion➔ Literature and Further Readings➔ Questions

OUTLINE - 2

OUTLINE

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Learning in games○ more dynamism, basing on the player○ online and offline○ build a bot from user data/decision○ difficult to debug

Uses● Parameter Modification● Action Prediction● Decisions● ...more

Context

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Study the player’s behaviour, guess next move

● Adaptive AI: the player cannot use the same technique

● Same challenge, different experience

● A little bit of randomness is needed…○ otherwise the player will predict the bot!

Action Prediction

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Probability○ store data of the player’s choices○ pick the most predictable

+ very easy to implement- the player can soon learn the mechanism

● String Matching○ a string is used to store the player’s decisions○ to predict the next move, the string is parsed○ search for identical situation in the string○ prediction is based on history of choices

Techniques

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

Example:

● In a RPG context the player can perform a physical attack and 3 spells: Fire, Blizzard, Thunder.○ Encoding:

■ Physical Attack: “A”■ Fire : “F”■ Blizzard: “B”■ Thunder “T”

● During a battle we store his/her decisions○ “AAFBAFBAATTFBBAAFB”

Techniques - 2

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Next move?

● Search in the string what the player did in the past, after performing a Fire spell followed by a Blizzard spell.

● AAFBAFBAATTFBBAAFB (?)○ A physical attack is more predictable

● Window size : how many moves should be looked up○ is 2 in the example above○ important for the performance of the prediction

Techniques - 3

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Record the probabilities for each move given all combination of choices for the previous “N” moves.

[Millington]

● “N” = window size + 1○ 3-Gram keeps track the probabilities for a window

size of 2

● Update the list when new actions are performed

● Predict the one with higher probability

N-Grams

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

Example: Left or Right?

LRRLLLRLLRLRL..(?)3-Gram predictor

N-Grams - 2

->R ->L

LL 2/3 1/3

LR 1/4 3/4

RL 1/3 2/3

RR 0/1 1/1

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

Easy to predict things like...

N-Grams - 3

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Determine the accuracy of the prediction

● Increasing the window size means better prediction○ ...until it get worse

● Optimal window size should not be too large○ Long sequences tend to have too much randomness

N-Grams - Window Size

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

Accuracy of N-Grams for the Left-Right game.[Millington]

N-Grams - Window Size

ACTION PREDICTION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● The more the window increase in size, the more memory is required to store probabilities

● Best performance are achieved when sequences are “known”○ otherwise learning takes a while…

● Very suitable for combat games, computational linguistics, computational biology and in general pattern-based actions

Considerations

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Learn how to make decisions

● From a set of possible actions, AI learn which one apply basing on observation data

● Weak and strong supervision○ give feedback to the AI choices

● Huge amount of information in games

Decision Learning

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Decision Tree Learning

● Artificial Neural Networks

● Naive Bayes Classifiers

● Reinforcement Learning

Techniques

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Decide an action from previous observation○ can be other players decisions, in the same context

● Basing on evidence, gives the probability of “how good” is executing an action

● Bayes rule

Naive Bayes Classifiers

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Core concept: express the conditional probability of A given B, in terms of its inverse, B given A.○ the probability that it rained can be expressed as the

observation that the ground is wet

Example: in a soccer game, should we shoot at goal?

● Base our decision on a training set● Try to find decision patterns● Data in the example is labeled

○ this help to generalise similar context

Naive Bayes Classifiers - 2

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

Naive Bayes Classifiers - Example

Shoot? Distance Direction Markers

Yes Medium Central Low

Yes Large Central None

No Short Central High

No Large Angular High

Yes Short Angular None

No Medium Angular High

Yes Medium Central Medium

No Short Central High

Yes Medium Angular Low

[...] [...] [...] [...]

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

Should we shoot if...○ distance = medium○ direction = angular○ markers = none

P ( shoot? | dist, dir, mark) = P(shoot? | dist) * P(shoot? | dir) * P(shoot? | mark)

● Applying the bayes rule, we have an indication based on the data collected○ Google Docs

● Player should shoot!

Naive Bayes Classifiers - Example ctd.

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Reason under uncertainty○ AI reasons in a human like way, no cheats

● Arrange variables and their relationship in a “belief network”○ acyclic graph○ variables are encoded as node○ relationships are arcs○ variable probability distribution encoded in a table

● Inference using probabilistic reasoning○ Causal Inference: given A → B, we know A○ Diagnostic Inference: given A → B, we know B○ Intercausal Inference: given A → C and B → C, we know C

Bayes Networks

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

Bayes Networks - 2

Away from home, our neighborhood tell us that he heard our alarm rang.. What happened?

● P(A) = P(B) * P(E) * P(A|B,E)

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● If the news inform us on an earthquake… the probability that a burglary occurred as well, are low!○ B → A, E → A. P(E) changed and P(B) changed as

well

Bayes Inference

P(B) P(E) P(A|B,E) P(A) N. P(A)

T=.001 T=.002 .95 .000002 .000795

T=.001 F=.998 .94 .000938 .372814

F=.999 T=.002 .29 .000579 .230127

F=.999 F=.998 0.001 .000997 .396264

.002516 1

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

Determining the probability of a burglary after the news

Bayes Inference - 2

P(B) P(E) P(A|B,E) P(A) N. P(A)

T=.001 T=1 .95 .00095 .003269

T=.001 F=0 .94 0 0

F=.999 T=1 .29 .28971 .996731

F=.999 F=0 .001 0 0

.29066 1

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Find the probability distribution over a variable X given the evidence e○ P(X | e) ?

● Inference can be exact or approximated

● Computing the probability of a node: summing the probabilities of the parent nodes○ chain rule

Bayes Inference - 3

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● The belief of each node is updated by prior or posterior evidence updates

Probability Propagation

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Visibility○ “fog of war”

● Dependency graphs○ RTS technology graph

■ infer existence of some technologies by the presence of others

■ which dependencies a player is attaining■ etc

● Detect Intruder

Bayes Networks in games

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Learning can be used for many reasons in game development

● Main question.. do we need it?○ balance effort of building a learning system with

outcomes● Offline learning is often preferred

● Learning a whole new behaviour is not so easy

● Adaptive AI can be the future of gaming○ ideally no repetitive challenges

Conclusion

DECISION LEARNING

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

● Artificial Intelligence for Games [Ian Millington, John Funge]● AI Games Programming Wisdom [Steve Rabin]● Teaching Bayesian behaviours to video game characters [Ronan Le

Hy, et al]● MIT open courseware● PR-OWL● Bayesian Networks [Judea Pearl, Stuart Russell]● Design and Development of a Compound DSS for Laboratory

Research [Tomáš Hujer]● Bayesian Artificial Intelligence [Kevin B. Korb, Ann E. Nicholson]● Player Prediction techniques for AI in video games [Michael

Webersdorfer]● Adaptive AI for fighting games [Ricciardi, Thill

Literature and Further Readings

CONCLUSION

N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux

Questions?

That’s it!Thank you for your

attention