Research Related to Real-Time Strategy Games
Robert Holte
November 8, 2002
No Games Knowledge, but…
• Single-agent search = sequential decisions with additive costs– path-finding occurs in RTS games– are there other occurrences ?– transfer from 1-agent to 2-agent or N-agent games ?– is a 2-agent game with a deterministic opponent a
single-agent search problem ?
• Path-finding research group (P. Yap, Yngvi, …)
and …
• Combinatorial optimization = resource allocation with complex constraints– combinatorial auctions– “boolean programming”– hill-climbing, ESG, a bit of CPLEX
and …
• Machine Learning– opponent-modelling (poker)– user-modelling (adaptive games, personalization)– learning to make good moves (M. Mueller)
• Interactive Systems– mostly-automatic evaluation methodology
• Abstraction in Poker
Using Abstraction to Speed Up Search
Search = path-finding in a graph
GIVEN:– A graph– Two nodes in the graph (“start”, “goal”)
FIND:– a shortest path from start to goal
Labyrinth at the gardens of Versailles(could be a computer game!)
“Implicit” Graphs
Instead of an adjacency list/matrix,Define a graph by a successor function, succ,
and one or more “seed” nodes.Nodes of the graph = transitive closure of succ applied to the seed nodes
An edge exists from n1 to n2 iff n2 succ(n1)
A graph can be exponentially larger than its description.
Towers of Hanoi puzzlefor D disks:
– graph has 3D nodes– description is O(D2)
3-disk State Space
Speed up Search by Using Extra Information
• Solution “skeleton” = chain of subgraphs leading from start to goal– Refinement: search in the first subgraph until you reach
the second, then search in the second until you reach the third, etc.
– very fast but path found not guaranteed to be optimal
• Heuristic = h(n) = estimate of the distance from node n to the goal node.– if it never overestimates, it can eliminate (prune) parts of
the graph and still guarantee finding an optimal path
My Research
Aim:to generate heuristics or skeletal solutions automatically
Method:1. Create an abstraction of the given graph (simplified
version of your problem that “approximately” preserves “important” features of the problem).
2. Find a solution path in the abstract graph.3. Use the abstract solution as a skeletal solution, or use
its length as a heuristic.
Example: Towers of Hanoistart
goal
Abstraction:“Ignore the smallest
disk”start
goal
Abstract Solution
start
goal
Key Techniques
For Automatically Generating Abstractions– explicit graphs: STAR abstraction
– implicit graphs: domain abstraction
– select among abstractions using Korf & Reid’s method for predicting search time
Search Methods– refinement: AltO
– heuristic: Hierarchical A*, pattern databases
– anytime combination of refinement and heuristic search
STAR abstraction
Abstract state = connected subgraph
(all nodes within distance R of a given node)
• Fully automatic (but need to amortize cost)
• Fine control over granularity (R)
• Can be applied recursively (abstraction hierarchy)
• Guarantees refinement will succeed• Produces a consistent heuristic
AltO Refinement Technique
Conduct abstract search in the opposite direction to the base-level search;
Use the whole abstract search graph for refinement.
• 12% faster than standard refinement and produces better solutions (20% shorter)
• Robust – not sensitive to the abstract solution found, and not very sensitive to the radius of abstraction
• 10-40 times faster than A* (same abstraction) and solutions usually within 30% of optimal
Pattern Database Successes• Joe Culberson & Jonathan Schaeffer (1994).
– 15-puzzle– PDB + Manhattan Distance (MD) reduced MD-
alone search tree 1000-fold
• Rich Korf (1997)– PDBs enabled optimal solutions of Rubik’s Cube
• Stefan Edelkamp (2001)– PDBs competitive with best planners
• Istvan Hernadvolgyi– PDBs reduced macro-table solution lengths from 90
to 50
Example: 8-puzzle
1 2
3 4 5
6 7 8
Domain = blank 1 2 3 4 5 6 7 8
181,440 states
“Patterns”created by domain mapping
1 2
3 4 5
6 7 8
This mapping produces9 patterns
Domain = blank 1 2 3 4 5 6 7 8Abstract = blank
corresponding patternoriginal state
Pattern Database
Pattern
Distance to goal 0 1 1 2 2 2
Pattern
Distance to goal 3 3 4
Calculating h(s)
Given a state in the original problem
Compute the corresponding pattern
and look up the abstract distance-to-goal
8 1 4
3 5
6 7 2
2
Heuristics defined by PDBs are consistent, not just admissible.
Domain Abstraction
1 2
3 4 5
6 7 8 6 7 8
30,240 patternsDomain = blank 1 2 3 4 5 6 7 8Abstract = blank 6 7 8
Automatic Creation of Domain Abstractions
• Easy to enumerate all possible domain abstractions
• They form a lattice, e.g.
is “more abstract” than the domain abstraction above
Domain = blank 1 2 3 4 5 6 7 8Abstract = blank
Domain = blank 1 2 3 4 5 6 7 8Abstract = blank
8-puzzle: A* vs. PDB size#
node
s ex
pand
ed (
A*)
pattern database size (# of abstract states)
Hierarchical A*Unlike pattern databases, each h(n) is computed on
demand.
If h(n) is not already known compute it by searching at the abstract level.
The search at an abstract level can be informed by a heuristic defined by a higher abstract level.
* many abstract searches for one base-level search* naïve implementation is 10 times slower than
blind base-level search.
Speed versus Optimality
• Refinement– very fast (when refinability guaranteed)– suboptimal paths
• A*– much slower– optimal paths
• See also: bidirectional search, -admissible A*
Hybrid of HA* and Refinement (1)
Although the algorithms seem very different there are actually only two differences.
(1) The relative weight given to g and h.HA*: g + h
Refinement: h (use g to break ties)
Generalization: *g + (1- )*h ( 0 1 )
(2) What to do if a node n is encountered for which h(n) is not known from the first abstract search ?
HA*: compute h(n) by doing another abstract search
Refinement: ignore n
Generalization: choose randomly between the two strategies with probability P ( 0 P 1 )
Hybrid of HA* and Refinement (2)
2-d Space of Algorithms
0 0.5 1.0
= weight given to g
1.0
P
0.5
0.0
A*
Refinement
GraphTraverser
OptimalRefinement