1/38 game-tree search over high-level game states in rts games alberto uriarte and santiago...
TRANSCRIPT
![Page 1: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/1.jpg)
1/38
Game-Tree Search over High-Level Game States in RTS Games
Alberto Uriarte and Santiago Ontañón
Drexel UniversityPhiladelphia
October 6, 2014
![Page 2: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/2.jpg)
2/38
Outline
Motivation High-level Abstraction in RTS Games Mapping Low-Level and High-Level States High-level Game-Tree Search Evaluation Conclusions
![Page 3: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/3.jpg)
3/38
Motivation
RTS properties Simultaneous moves “Real-time” Partially observable Non deterministic
Game Stochastic Incomplete information
Real-time and simultaneous
Chess NO NO NO
Go NO NO NO
Backgammon YES NO NO
Poker YES YES NO
StarCraft YES YES YES
![Page 4: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/4.jpg)
4/38
Game complexity
State-Space ComplexityNumber of legal game positions reachable from the initial position of the game.
Game State-space
Tic-tac-toe 103
Reversi (Othello) 1028
Chess 1047
Go 10171
StarCraft
![Page 5: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/5.jpg)
5/38
Game complexity
State-Space ComplexityNumber of legal game positions reachable from the initial position of the game.
Game State-space
Tic-tac-toe 103
Reversi (Othello) 1028
Chess 1047
Go 10171
StarCraft 101685
StarCraft map: 128x128Maximum number of units: 400
Considering only unit positions:(128x128)400=16384400≈101685
![Page 6: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/6.jpg)
6/38
Motivation
Game-Tree ComplexityNumber of leaf nodes in the minimal solution depth of a full-width search tree.
EstimationUsing the branching factor (b) and the depth (d) of a game: bd
Game Branching Depth
Chess 35 80
Go 360 200
StarCraft
![Page 7: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/7.jpg)
7/38
Motivation
Game-Tree ComplexityNumber of leaf nodes in the minimal solution depth of a full-width search tree.
EstimationUsing the branching factor (b) and the depth (d) of a game: bd
Game Branching Depth
Chess 35 80
Go 360 200
StarCraft 3050 - 30200 36000
Units: 50 – 200Actions per unit: 30Branching factor: 3050 - 30200
Length of a game: 25 minutes25 min x 60 sec x 24 iteration per sec == 36000
![Page 8: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/8.jpg)
8/38
High-level Abstraction in RTS games
Levels of decisions Strategy. The whole army and buildings. Tactics. Group of units. Reactive Control. One unit.
We focused on tactical decisions!!
![Page 9: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/9.jpg)
9/38
High-level Abstraction in RTS games
Two different abstractions:1. Map abstraction
![Page 10: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/10.jpg)
10/38
High-level Abstraction in RTS games
Two different abstractions:1. Map abstraction
Perkins’ algorithm to decompose a map into regions and chokepoints.
![Page 11: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/11.jpg)
11/38
High-level Abstraction in RTS games
Two different abstractions:1. Map abstraction
New regions around chokepoint influence
![Page 12: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/12.jpg)
12/38
High-level Abstraction in RTS games
Two different abstractions:1. Map abstraction
All distances are pre-computed
![Page 13: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/13.jpg)
13/38
High-level Abstraction in RTS games
Two different abstractions:2. Unit group abstraction
Hit Points (shield) Position Order:
move, attack, stop, patrol repair, build, siege
Size Damage (points and type)
![Page 14: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/14.jpg)
14/38
High-level Abstraction in RTS games
Two different abstractions:2. Unit group abstraction
Player. Which player controls this group Type. Type of units in this group Size. Number of units forming this group Region. Which region is this group in Order. Which order is currently performing
N/A, Move, Attack, Idle Target. The ID of the target region End. In which game frame is the order
estimated to finish
![Page 15: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/15.jpg)
15/38
High-level Abstraction in RTS games
We only consider military units and main bases
![Page 16: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/16.jpg)
16/38
High-level Abstraction in RTS games
We only consider military units and main bases
![Page 17: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/17.jpg)
17/38
High-level Abstraction in RTS games
Abstract state
Player Type Size Region Order Target End
1 Base 1 1 N/A - -
1 Tank 2 2 Move 3 230
1 Vulture 4 3 Idle - 400
![Page 18: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/18.jpg)
18/38
High-level Abstraction in RTS games
Abstract state. Possible Actions
Player Type Size Region Order Target End
1 Base 1 1 N/A - -
1 Tank 2 2 Move 3 230
1 Vulture 4 3 Idle - 400
![Page 19: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/19.jpg)
19/38
High-level Abstraction in RTS games
Abstract state. Possible Actions
Player Type Size Region Order Target End
1 Base 1 1 N/A - -
1 Tank 2 2 Move 3 230
1 Vulture 4 3 Idle - 400
Move to 1. Move to 3. Idle
![Page 20: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/20.jpg)
20/38
High-level Abstraction in RTS games
Abstract state. Possible Actions
Player Type Size Region Order Target End
1 Base 1 1 N/A - -
1 Tank 2 2 Move 3 230
1 Vulture 4 3 Idle - 400
Move to 2, 4, 5 or 6. Idle
![Page 21: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/21.jpg)
21/38
High-level Abstraction in RTS games
Abstract state. Possible Actions
Player Type Size Region Order Target End
1 Base 1 1 N/A - -
1 Tank 2 2 Move 3 230
1 Vulture 4 3 Idle - 400
Branching factor: (1)×(2+1)×(4+1) = 15
![Page 22: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/22.jpg)
22/38
High-level Abstraction in RTS games
SparCraft (Churchill, Saffidine, and Buro 2012)
![Page 23: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/23.jpg)
23/38
Mapping Low-Level and High-Level States
Squad groups don’t have to be the same as abstract groups!!
![Page 24: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/24.jpg)
24/38
High-level Game-Tree Search
1. State forwarding (simulator)We estimate in which game frame the group finish their order. Moving: velocity + distance to regionAttack: DPS between groups
Next player to move: the one with the group with the smallest “end” frame.
![Page 25: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/25.jpg)
25/38
High-level Game-Tree Search
1. State forwarding (simulator)We estimate in which game frame the group finish their order. Moving: velocity + distance to regionAttack: DPS between groups
2. State evaluation
![Page 26: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/26.jpg)
26/38
High-level Game-Tree Search
Alpha-BetaMCTS
![Page 27: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/27.jpg)
27/38
High-level Game-Tree Search
Alpha-BetaMCTS
ABCD (Churchill, Saffidine, and Buro 2012) UCTCDMCTSCD
![Page 28: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/28.jpg)
28/38
High-level Game-Tree Search
MCTSCD
![Page 29: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/29.jpg)
29/38
High-level Game-Tree Search
![Page 30: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/30.jpg)
30/38
High-level Game-Tree Search
![Page 31: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/31.jpg)
31/38
Evaluation settings
Games limited to 20 minutes (28,800 frames)Search called every 400 frames (pause game)No fog of war (future work)
![Page 32: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/32.jpg)
32/38
Evaluation settings
Games limited to 20 minutes (28,800 frames)Search called every 400 frames (pause game)No fog of war (future work)
ABCD parameters Maximum depth: 3 Children limited to 10,000 nodes Time limited to 30 seconds Simultaneous move: Alt policy
![Page 33: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/33.jpg)
33/38
Evaluation settings
Games limited to 20 minutes (28,800 frames)Search called every 400 frames (pause game)No fog of war (future work)
MCTSCD parameters Tree policy: e-greedy (e=0.2) Default policy: random move selection Simultaneous move: Alt policy Tree policy depth: limited to 10 2,000 playouts limited to 7,200 game
frames (5 minutes)
![Page 34: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/34.jpg)
34/38
Results
![Page 35: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/35.jpg)
35/38
Results
![Page 36: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/36.jpg)
36/38
Results
![Page 37: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/37.jpg)
37/38
Conclusions and Future Work
Conclusions Future work
• New MCTS algorithm for simultaneous actions (MCTSCD)
• High level abstraction in the right direction to be able to apply game-tree search algorithms
• Improve the game tree search algorithm • different bandit strategies • deal with partial observability
• More abstractions and their tradeoffs
• Improve the game simulator by learning during the course of a game
![Page 38: 1/38 Game-Tree Search over High-Level Game States in RTS Games Alberto Uriarte and Santiago Ontañón Drexel University Philadelphia October 6, 2014](https://reader036.vdocument.in/reader036/viewer/2022062713/56649cc55503460f9498e3a9/html5/thumbnails/38.jpg)
38/38
Game-Tree Search over High-Level Game States in RTS Games
Alberto Uriarte [email protected] Ontañón [email protected]