starcraft learning algorithms by logan yarnell, steven raines, and dean antel

StarCraft Learning Algorithms

By Logan Yarnell, Steven Raines, and Dean Antel

Abstract

The opening or initial build order during a StarCraft match is an integral part of a player’s strategy. If an opening is naively chosen it makes a player vulnerable to tactics that can end the game very quickly. We examine three algorithms for choosing a “good” opener: Bayesian network, a genetic algorithm, and a pathfinding algorithm.

About StarCraft

Real-time strategy game

Similar to a complex digital version of chess but in real-time

Expert level gameplay is very complex, often requiring over 300 actions per minute.

Considered a national sport in South Korea

About StarCraft

Playing Starcraft requires simultaneously managing several distinct competencies:

Strategy

Production

Economy

Recon

Tactics

Basic Actions

Build buildings

Build units

Attack the enemy

Gather resources/manage economy

Genetic Algorithm

Modifies some set of “solutions” to a problem in a way similar to natural selection

Has mutation, crossover, and elitism functions

Solution = a build order

Each build order is given a fitness value determined by how successful it is

The number of mutations the build has gone through is also recorded

If a build order has gone through 10 mutations and it’s fitness < 5, elitism removes that build order

Genetic Algorithm

Most important part of genetic algorithm = mutate function

Sometimes the only part implemented if computational resources are limited

Fitness value determination is also important

Considered factors in determining a build’s fitness: time taken, amount of resources used, whether the match was won or lost, number of units/structures that survived…

In the end only win/loss rate was chosen to affect fitness

Genetic Algorithm

Mutate function randomly chooses to add a new unit, reorder the build, or remove a unit

isBuildValid method checks to make sure the mutate function doesn’t produce a useless build order by placing a unit before it’s prerequisite(s)

The newly mutated build is then written to a .txt file with NumOfMutations incremented

A match is run with that build, if it wins the fitness value is incremented, if it looses the fitness is not altered

There is an element of randomness, but complexity is more or less constant

Genetic Algorithm

Genetic Algorithm

Bayesian Network

Use knowledge from earlier games to predict what type of units the opponent is likely to have.

Requires mining a large amount of data from replays

Scouting is essential to make this work effectively

Difficult due to incomplete information available during matches

Pathfinding

From what is seen, create a shortest possible build. Relies on scouting.

Dijkstra’s algorithm O(n) complexity.

O(|V|^2)

O(|E|+|V|log|V|)

Intersection O(n^2)

Pathfinding (Continued)

Conclusions

Understanding the complexity of the environment is a key element to creating a learning algorithm for it.

Complementary not competitive.

Why 380?

There are many ways to implement a StarCraft bot that include various algorithms for the different aspects of gameplay:

Finite state machines

Scripting

Dynamic scripting

Probabilistic inference

Influence maps

Neural networks

Swarm intelligence

Potential fields

Genetic programming

Future Work

Enhance unit combat

Increase the amount of units considered in build orders

Optimize building efficiency to allow multiple buildings to be constructed at the same time.

Add dynamic scouting

Questions

What is the computational complexity (O(n)) of Dijkstra’s algorithm without optimization?

Is this a constraint satisfaction problem, or an optimization problem?

What probabilistic relationship is there between nodes in a Bayesian network that represents a StarCraft tech tree?

Answers

N^2

Optimization

Conditional dependence

Sources

http://www.sscaitournament.com/

http://web.engr.oregonstate.edu/~tgd/publications/aiide2011-starcraft-model.pdf

http://arxiv.org/pdf/1111.3735.pdf

http://plankter.se/wp-content/uploads/2014/12/making_and_acting_on_predictions_on_scbw_2014.pdf