peter van emde boas: imperfect information games; looking for the right model. imperfect information...
Post on 18-Dec-2015
215 views
TRANSCRIPT
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
IMPERFECT INFORMATION GAMES
looking for the right Model
Peter van Emde Boas
ILLC-FNWI-Univ. of AmsterdamBronstee.com Software & Services B.V.
References and slides available at: http://turing.science.uva.nl/~peter/teaching/thmod02.html
© Games Workshop© Games Workshop
Algemeen Wiskunde Colloquium KdV-UvA
Feb 27 2002
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Topics• Games, Computation and Computer
Science
• Game Representations
• Decision Problems on Games
• Backward Induction and PSPACE (The Holy Quadrinity)
• The INIGMA Project
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
First things First
© Peter van Emde Boas
A previous appearance ; Nov 21 1979 @ UvAothers: Sep 25 1974, Mar 24 1982, Dec 04 1985
© Peter van Emde Boas © Peter van Emde Boas
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
First things First
Know thy Audience .....© Peter van Emde Boas
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
First things First
© Peter van Emde Boas © Peter van Emde Boas
The Horror of attending a talk..... ; Dec 13 1976 @ VUA
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
GAMES AND COMPUTATIONGAMES AND COMPUTATION
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Games ???Past (1980) position of Games in Mathematics & CS:
Study object for a marginal part of AI(Chess playing programs)
Recreational Mathematics (cf. Conway, Guy &Berlekamp Theory)
Game Theory (Economy): von Neumann, Morgenstern, Aumann,
Games in Logic: Determinacy in set theory, Ehrenfeucht-Fraise Games, ....
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Computer Science• Computation Theory• Complexity Theory• Machine Models• Algorithms• Knowledge Theory• Information Theory• Cryptography, Systems and Protocols
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
• Evasive Graph properties (1972-74)• Information & Uncertainty (Traub ea. - 1980+)• Pebble Game (Register Allocation, Theory 1970+)• Tiling Game (Reduction Theory - 1973+)• Alternating Computation Model (1977-81)• Interactive Proofs /Arthur Merlin Games (1983+)• Zero Knowledge Protocols (1984+)• Creating Cooperation on the Internet (1999+)• E-commerce (1999+)• Logic and Games (1950+)• Language Games, Argumentation (500 BC)
Games and Computer Science
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Game Theory
• Theory of Strategic Interaction
• Attributes– Discrete vs. Continuous ( state space )– Cooperative vs. Non-Cooperative
( pay-off )– Perfect Information vs. Imperfect Information
( Information sets ); Knowledge Theory
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
PARTICIPANTS & MOVES
• Single player - no choices• Single player - random moves• Single player - choices : Solitaire• Two players - choices• Two players - choices and random moves• Two players - concurrent moves• More Players - Coalitions
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
COMPUTATION
• Deterministic
• Nondeterministic
• Probabilistic
• Alternating
• Interactive protocols
• Concurrency
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
COMPUTATION• Notion of Configurations: Nodes• Notion of Transitions: Edges• Non-uniqueness of transition:
Out-degree > 1 - Nondeterminism• Initial Configuration : Root • Terminal Configuration : Leaf• Computation : Branch Tree • Acceptance Condition:
Property of trees
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Linking Games and Computations
• Single player - no choices : Routine : Determinism• Single player - choices : Solitaire (Puzzle) :
Nondeterminism• Two players – choices : Finite Combinatorial Games :
Alternating Computation• Single player - random moves : Gambling :
Probabilistic Algorithms• Two players - choices and random moves :
Interactive Proof Systems• Several players & Coalitions - group moves :
Multi Prover Systems
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
GAME REPRESENTATIONSGAME REPRESENTATIONS
2 / 0
5 / -71 / 4
-1 / 4
3 / 1
-3 / 21 / -1R
D
O S
1/-1
1/-1
-1/1
-1/1
© Donald Duck 1999 # 35
Strategic Format Game Graph Naive Format
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
© Games Workshop
URGATOrc Big Boss
© Games Workshop
THORGRIMDwarf High King
Introducing the Opponents
Games involve strategic interaction ......Games involve strategic interaction ......
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Game Trees (Extensive Form - close to ComputationExtensive Form - close to Computation)
Root
Thorgrim’s turn
Urgat’s turnTerminal node:
Non Zero-Sum Game:Pay-offs explicitly designated at terminal node
2 / 0
5 / -71 / 4
-1 / 4
3 / 1
-3 / 21 / -1
Pay - offs
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Bi-Matrix Games
© Games Workshop © Games Workshop© Games Workshop© Games Workshop
© Games Workshop© Games Workshop
Runesmith Dragon SquiggOgre
R
D
O S
1/-1
1/-1
-1/1
-1/1
A Game specified by describing A Game specified by describing the Pay-off Matrix ....the Pay-off Matrix ....
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Von Neumann’s Theorem
( )/2 :+ ( )/2+© Games Workshop © Games Workshop© Games Workshop© Games Workshop
© Games Workshop© Games Workshop
R D SO
R
D
O S
1/-1
1/-1
-1/1
-1/1
Mixed Strategy Mixed Strategy Nash EquilibriumNash Equilibrium; ; no player can improve his pay-off by deviation.no player can improve his pay-off by deviation.
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
A Game
© Donald Duck 1999 # 35
Starting with 15 matchesplayers alternatively take1, 2 or 3 matches away untilnone remain. The playerending up with an oddnumber of matches winsthe game
A Game specified by describing A Game specified by describing the rules of the game ....the rules of the game ....
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Questions about this Game
• What if the number of matches is even?
• Can any of the two players force a win by clever playing?
• How does the winner depend on the number of matches
• Is this dependency periodic? If so WHY?
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
WHY WORRY ABOUT MODELS?
Algorithmic problem
Instances
Solutions
InstanceFormat
Question
Instance Size
Algorithm Space/TimeComplexity
The rules of the game called “Complexity Theory”
MachineModel
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Format and Input SizeThink about simple games like Tic-Tac-ToeNaive size of the game indicated by measures like:
-- size configuration ( 9 cells possibly with marks)-- depth (duration) game (at most 9 moves)
The full game tree is much larger : ~986410 nodesSize of the strategic form beyond imagination.....
What size measure should we use for complexitytheory estimates ??
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
The Impact of the FormatThe gap between the experienced size and the size of the game tree is Exponential !
Another Exponential Gap between the game tree and the strategic form.
These Gaps are highly relevant for Complexity!
The Challenge: Estimate Complexity of Game Analysis in terms of Wood Measure.
Wood Measure : configuration size & depth
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Decision Problems on Decision Problems on GamesGames
from H.W. Lenstra: Aeternitatem Cogita
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Decision Problems on Games
• Which Player wins the game– Winning Strategy ?
• End-game Analysis• Termination of the Game• Forcing States or Events
– Safety (no bad states)– Lifeness (some good state will be reached)
• Power of Coalitions• Game Equivalence (when are two games the
same?)
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Backward Induction and its Backward Induction and its ComplexityComplexity
2 / 0
5 / -71 / 4
-1 / 4
3 / 1
-3 / 21 / -1
2 / 0
3 / 1
1 / 4-3 / 2
1 / 4
2 / 0
5 / -71 / 4
-1 / 4
3 / 1
-3 / 21 / -1
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Backward Induction
Root
Thorgrim’s turn
Urgat’s turn
Terminal node:
Non Zero-Sum Game:Pay-offs computed for allgame nodes including the Root.
2 / 0
5 / -71 / 4
-1 / 4
3 / 1
-3 / 21 / -1
2 / 0
3 / 1
1 / 4-3 / 2
1 / 4
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Backward Induction2 / 0
5 / -71 / 4
-1 / 4
3 / 1
-3 / 21 / -1
2 / 0
3 / 1
1 / 4-3 / 2
1 / 4
At terminal nodes: Pay-off as explicitly given
At Thorgrim’s nodes: Pay-off inherited from Thorgrim’s optimal choiceAt Urgat’s nodes: Pay-off inherited from Urgat’s optimal choiceAt Probabilistic nodes: Pay-off evaluated by averaging
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Backward Induction in PSPACE?
The Standard Dynamic Programming Algorithm forBackward Induction uses the entire ConfigurationGraph as a Data Structure: Exponential Space !
Instead we can Use Recursion over Sequences of Moves:
This Recursion proceeds in the game tree from theLeaves to the Root.
Pitfalls: Draws possible? Terminating Game?Loops ?
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Backward Induction in PSPACE?The Recursive scheme combines recursion(over move sequence) with iteration (over locallylegal moves).
Space Consumption =O( | Stackframe | . Recursion Depth )
| Stackframe | = O( | Move sequence | + | Configuration| )
Recursion Depth = | Move sequence | =O( Duration Game )
Hence: Polynomial with Respect to the Wood Measure !
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
REASONABLE GAMESFinite Perfect Information (Zero Sum) Two Player Games
(possibly with probabilistic moves)
Structure: tree given by description, where deciding properties like:is p a position ?, is p final ? is p starting position ?, who has to move in p ?, generation of successors of p are all trivial problems .....
The tree can be generated in time proportional to its size.....
Moreover the duration of a play is polynomial.
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
THE HOLY QUADRINITY
FINITE COMBINATORIAL
GAMES
QUANTIFIEDPROPOSITIONAL
LOGIC (QBF) : ALTERNATION
PSPACE
UNRESTRICTEDUNIFORM
PARALLELISM
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Known Hardness results on Games in Complexity Theory (1980+)
• QBF ( PSPACE ) ( the “mother game” )• Tiling Games (NP, PSPACE, NEXPTIME,....)• Geography (PSPACE)• HEX (generalized or pure) (PSPACE)• Checkers, Go (PSPACE-hard)• Pebbling Game (PSPACE) (solitaire game!)• Block Moving Problems (PSPACE)• Chess (EXPTIME) (repetition of moves ! )
The Common View is that Games Characterize PSPACE
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
The InIGMA ProjectThe InIGMA Project
© Games Workshop © Games Workshop© Games Workshop© Games Workshop
© Games Workshop© Games Workshop
Runesmith Dragon SquiggOgre
R
D
O S
1/-1
1/-1
-1/1
-1/1
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Imperfect Information makes life more complex !
Examples of games where analyzing thePerfect Information version is easier than theImperfect version.
Neil Jones produces such Example in 1978I.E., perfect FAT in P and Imperfect IFATwhich is PSPACE hard......
How to compare two versions of a game?
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Matching Pennies
L
R
l r
1 / -1
1 / -1-1 / 1
-1 / 1
L R
1 / -1 -1 / 1 1 / -1
l lr r
-1 / 1
In the Game tree Urgat has a winning StrategyIn the Matrix Form nobody has a winning strategy
So Tree is incorrect representation of the game. Why ?
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
INFORMATION SETS
L
R
l r
1 / -1
1 / -1-1 / 1
-1 / 1
L R
1 / -1 -1 / 1 1 / -1
l lr r
-1 / 1
When Urgat has to Move he doesn’t know Thorgrim’s move.Information sets capture this lack of Information.Kripke style semantics.Strategies must be UniformUrgat has no winning Uniform Strategy.
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Urgat doesn’t know the position he is in !
Matrix Games areImperfect Information Games
Thorgrim’s Choice of strategy
Urgat’s Choice of strategy
Pay-off phase
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Modified combat of champs
D R
1 / -1 1 / -1
o os s
-1 / 1
W NW
1 / -1 -1 / 1 1 / -1
o os s
-1 / 1
D R
The squigg scares the dragon only after a sulfur bath.....
1 / -1
?
?
?
?
?
?
Backward Induction on Uniform Strategies
?
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Imperfect Information Version of the same game ?
D R
1 / -1 1 / -1
o os s
-1 / 1
W NW
1 / -1 -1 / 1 1 / -1
o os s
-1 / 1
D R
1 / -1
?
?
?
?
?
?
D R1 / -1
1 / -1 1 / -1
o os s
-1 / 1
W NW
1 / -1 -1 / 1 1 / -1
o os s
-1 / 1
D R
1 / -1
1 / -1
-1 / 1 -1 / 1
-1 / 1
-1 / 1
-1 / 1
?
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Imperfect Information makes life more complex !
Imperfect Information Game <==> Extension of Perfect Information Game Graph with information sets and Uniform moves ???
Analysis remains in P !!be it O(v.e) rather than O(v+e)
So something else is going on...
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Imperfect Information Games
Adaptation of BI on Graphs:-- Simple games no longer are determinated-- Information sets capture uncertainty-- Uniform strategies are required
HOWEVER.....-- Nodes may belong to multiple information sets: disambiguation causes exponential blow-up in size....-- Earlier algorithms become incorrect if used on nodes without disambiguation
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Neil Jones’ example (1978)
Game played on (Deterministic) Finite Automaton
Some states are selected to be winning for ThorgrimPlayers choose in turns an input symbol (I.E. the next transition)
Just a pebble moving game on a game graph;
This can easily be analyzed in Polynomial time.(even in linear time, if done efficiently...)
GAME FAT: Finite Automaton Traversal
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Neil Jones’ example (1978)
Consider the version of FAT where Thorgrim doesn’t observe Urgat’s moves:
Thorgrim can’t see where the pebble moves.
By a simple reduction from the problem to decide whether a given regular expression describes the language {0,1}* (shown to be PSPACE-complete by Meyer and Stockmeyer) this version is proven to be PSPACE-hard.
GAME IFAT: Imperfect Finite Automaton Traversal
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Jones’ ReductionFor a given regular expression R first construct its NFA : M(R)
Next consider the following game:
Each turn Thorgrim chooses an input symbol: 0 or 1; next Urgat chooses a legal transition in M(R) . Thorgrim can’t observe the state of M(R) after the transition ...... !!!
Thorgrim decides when to end the game.
Urgat wins if an accepting state is reached at the end of the game;otherwise Thorgrim wins the game
Thorgrim’s winning strategies correspond to input words outsideL(R) , the language described by R;So Thorgrim wins the game iff L(R) {0,1}*
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Jones’ Example ?Question: in which sense is IFAT animperfect information version of FAT ?
Alternating choices between input symbols and transitions is irrelevant difference;introducing new states <q,s> for old statesq and input symbols s both players choose transitions...
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Jones’ Example ?What are the configurations in IFAT ??
in FAT the states in the FA are adequate representations of the game configurations.
in IFAT the states are inadequate; configurations are to be placed in an information set with all other configurations where (according to Thorgrim) the game could be...and that depends on the input symbols processed so far.
Compare with subset construction for transforming anNFA into a DFA. These subsets could be adequate.....
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Jones’ Example ?These subsets could be adequate.....
SNAG: the subset construction increases the sizeof the FA exponentially!
The jump of complexity from P to PSPACE is betterthan we could have predicted; the naive graph basedbackward induction yields an EXPTIME algorithm....
STILL: The subset construction does not yield the Kripkemodel with Information sets.
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
What is the Kripke Model?
A candidate Kripke Model is the productof the Automaton and its Deterministicversion obtained by the subset construction:
{<q, A> | q A } with <q,A> ~ <q’,A> whenboth q and q’ A .Uniform strategies correspond to input symbols (as should be the case).
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
The Punch lineAdding Imperfect Information in Jones’example hardly increases the size of thegame in the Wood Measure, but increasesthe game graph exponentially.
By coincidence, for the Perfect Informationversion the wood measure and the size of thegame graph are proportional.
So again: Complexity with respect towhich measure.....???!!!
Peter van Emde Boas: Imperfect Information Games; looking for the right model.
Conclusion
Imperfect Information Games can be
harder to analyze !!!
But doing the comparison is non trivial,
since it has everything to do with
(succinct) game representations