november 10, 2009introduction to cognitive science lecture 17: game-playing algorithms 1 decision...

November 10, 2009 Introduction to Cognitive Science Lecture 17: Game-Playing Algorithms

1

Decision TreesDecision Trees

Many classes of problems can be formalized as Many classes of problems can be formalized as search problemssearch problems in in decision treesdecision trees..

We are going to look at two different applications of We are going to look at two different applications of search trees:search trees:

First, we will solve the First, we will solve the n-queens problemn-queens problem using using backtracking search.backtracking search.

Second, we will discuss how you can write a program Second, we will discuss how you can write a program that beats you at your favorite that beats you at your favorite board gameboard game..


2

Decision TreesDecision Trees

Example I:Example I: The n-queens problem The n-queens problem

How can we place n queens on an nHow can we place n queens on an nn chessboard n chessboard so that no two queens can capture each other?so that no two queens can capture each other?

QQQQ

xx

xx

xx

xx

xx

xx

xxxx

xx

xxxx

xx

xx

xx

xxxx

xx

xx

xx

xxxx

xx

xx

xx

xx

xx

xx

A queen can move any A queen can move any number of squares number of squares horizontally, vertically, and horizontally, vertically, and diagonally.diagonally.

Here, the possible target Here, the possible target squares of the queen Q are squares of the queen Q are marked with an marked with an xx..


3

Decision TreesDecision TreesLet us consider the Let us consider the 4-queens4-queens problem. problem.

Question:Question: How many possible configurations of 4 How many possible configurations of 44 4 chessboards containing 4 queens are there?chessboards containing 4 queens are there?

Answer:Answer: There are 16!/(12! There are 16!/(12!4!) = 4!) = (13(131414151516)/(216)/(2334) = 134) = 1377554 = 1820 possible 4 = 1820 possible configurations.configurations.

Shall we simply try them out one by one until we Shall we simply try them out one by one until we encounter a solution?encounter a solution?

No, it is generally useful to think about a search No, it is generally useful to think about a search problem more carefully and discover problem more carefully and discover constraints constraints on on the problem’s solutions.the problem’s solutions.

Such constraints can dramatically speed up the search Such constraints can dramatically speed up the search process.process.


4

Decision TreesDecision TreesObviously, in any solution of the n-queens problem, Obviously, in any solution of the n-queens problem, there must be there must be exactly one queen in each columnexactly one queen in each column of of the board. the board.

Otherwise, the two queens in the same column could Otherwise, the two queens in the same column could capture each other.capture each other.

Therefore, we can describe the solution of this Therefore, we can describe the solution of this problem as a problem as a sequence of n decisionssequence of n decisions: :

Decision 1: Place a queen in the first column.Decision 1: Place a queen in the first column.Decision 2: Place a queen in the second column.Decision 2: Place a queen in the second column.......Decision n: Place a queen in the n-th column.Decision n: Place a queen in the n-th column.


5

Backtracking in Decision TreesBacktracking in Decision Trees

There are problems that require us to perform an There are problems that require us to perform an exhaustive searchexhaustive search of all possible sequences of of all possible sequences of decisions in order to find the solution. decisions in order to find the solution.

We can solve such problems by constructing the We can solve such problems by constructing the complete decision treecomplete decision tree and then find a path from its and then find a path from its root to a leave that corresponds to a solution of the root to a leave that corresponds to a solution of the problem (breadth-first search often requires the problem (breadth-first search often requires the construction of an almost complete decision tree).construction of an almost complete decision tree).

In many cases, the efficiency of this procedure can be In many cases, the efficiency of this procedure can be dramatically increased by a technique called dramatically increased by a technique called backtracking backtracking (depth-first search). (depth-first search).


6


Idea:Idea: Start at the root of the decision tree and move Start at the root of the decision tree and move downwards, that is, downwards, that is, make a sequence of decisionsmake a sequence of decisions, , until you either reach a solution or you enter a until you either reach a solution or you enter a situation from where no solution can be reached by situation from where no solution can be reached by any further sequence of decisions.any further sequence of decisions.

In the latter case, In the latter case, backtrack to the parentbacktrack to the parent of the of the current node and take a different path downwards current node and take a different path downwards from there. If all paths from this node have already from there. If all paths from this node have already been explored, backtrack to its parent.been explored, backtrack to its parent.

Continue this procedure until you Continue this procedure until you find a solutionfind a solution or or establish that establish that no solution existsno solution exists (there are no more (there are no more paths to try out). paths to try out).


7


QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

QQ

place 1place 1stst queen queen

place 2place 2ndnd queen queen

place 3place 3rdrd queen queen

place 4place 4thth queen queen

empty boardempty board


8

Two-Player Games with Complete TreesTwo-Player Games with Complete Trees

Example II:Example II: Let us use similar search algorithms to Let us use similar search algorithms to write “intelligent” programs that write “intelligent” programs that play gamesplay games against a against a human opponent.human opponent.

Just consider this extremely simple (and not very Just consider this extremely simple (and not very exciting) game:exciting) game:

• At the beginning of the game, there are seven coins At the beginning of the game, there are seven coins

on a table. on a table.

• One move consists of removing 1, 2, or 3 coins. One move consists of removing 1, 2, or 3 coins.

• Player 1 makes the first move, then player 2, then Player 1 makes the first move, then player 2, then player 1 again, and so on. player 1 again, and so on.

• The player who removes all remaining coins wins.The player who removes all remaining coins wins.


9

Two-Player Games with Complete TreesTwo-Player Games with Complete Trees

Let us assume that the computer has the first move. Let us assume that the computer has the first move. Then, the game can be described as a Then, the game can be described as a series of series of decisionsdecisions, where the first decision is made by the , where the first decision is made by the computer, the second one by the human, the third computer, the second one by the human, the third one by the computer, and so on, until all coins are one by the computer, and so on, until all coins are gone.gone.

The The computercomputer wants to make decisions that wants to make decisions that guarantee its victoryguarantee its victory (in this simple game). (in this simple game).

The underlying assumption is that the The underlying assumption is that the humanhuman always always finds the finds the optimal moveoptimal move..


10

Two-Player Games with Complete TreesTwo-Player Games with Complete Trees77

66

55

44

33 22 11

11 22 33

22 11

CC CC

1144

22

CC

33

33

55

22

CC

HH

CC

HH

CC

11

33 22 11

11 22 33

22 11

HH HHHH

33

11

11

44

33 22 11

11 22 33

22 11

HH HHHH

33

44

33 22 11

11 22 33

22 11

CC CCCC

33

HH


11

Two-Player Games with Complete TreesTwo-Player Games with Complete TreesThe previous example showed how we can use a The previous example showed how we can use a variant of backtracking to variant of backtracking to • determine the computer’s best move in a given determine the computer’s best move in a given situation and situation and• prune the search tree, that is, avoid unnecessary prune the search tree, that is, avoid unnecessary computation. computation.

However, for more interesting games such as chess However, for more interesting games such as chess or go, it is or go, it is impossibleimpossible to check every possible to check every possible sequence of moves. sequence of moves.

The computer player then only looks ahead a certain The computer player then only looks ahead a certain number of moves and number of moves and estimatesestimates the chance of the chance of winning after each possible sequence.winning after each possible sequence.


12

Two-Player GamesTwo-Player Games

Therefore, we need to define a Therefore, we need to define a static evaluation static evaluation function e(p)function e(p) that tells the computer how favorable that tells the computer how favorable the current game position p is from its perspective.the current game position p is from its perspective.

In other words, the value of e(p) will be In other words, the value of e(p) will be positivepositive if a if a position is likely to result in a win for the computer, position is likely to result in a win for the computer, and and negativenegative if it predicts its defeat. if it predicts its defeat.

In any given situation, the computer will make a move In any given situation, the computer will make a move that that guaranteesguarantees a maximum value for e(p) after a a maximum value for e(p) after a certain number of moves.certain number of moves.

Again, the opponent is assumed to make Again, the opponent is assumed to make optimal optimal movesmoves..


13


For example, let us consider For example, let us consider Tic-Tac-ToeTic-Tac-Toe (although it (although it would still be possible to search the complete game would still be possible to search the complete game tree for this game).tree for this game).

What would be a suitable evaluation function for this What would be a suitable evaluation function for this game?game?

We could use the We could use the number of linesnumber of lines that are still open that are still open for the computer (for the computer (XX) minus the ones that are still open ) minus the ones that are still open for its opponent (for its opponent (OO).).


14


e(p) = 8 – 8 = 0e(p) = 8 – 8 = 0

XX

OO XX

e(p) = 6 – 2 = 4e(p) = 6 – 2 = 4

OO OO XX

XX OO

XX

e(p) = 2 – 2 = 0e(p) = 2 – 2 = 0

shows the weak-shows the weak-ness of this e(p)ness of this e(p)

How about these?How about these?

OO OO XX

XX

XX

e(p) = e(p) =

XX XX

OO OO OO

XX

e(p) = - e(p) = -


15



16


In summary, we need In summary, we need three functionsthree functions to make a to make a computer play our favorite game:computer play our favorite game:

• a function that generates all possible moves in a a function that generates all possible moves in a given situation, given situation,

• a static evaluation function e(p), anda static evaluation function e(p), and

• a tree search algorithm.a tree search algorithm.

This concept underlies most game-playing programs.This concept underlies most game-playing programs.

It is It is very powerfulvery powerful, as demonstrated, for example, by , as demonstrated, for example, by the program Deep Blue beating the world chess the program Deep Blue beating the world chess champion Gary Kasparov in 1997.champion Gary Kasparov in 1997.

november 10, 2009introduction to cognitive science lecture 17: game-playing algorithms 1 decision...

Documents

decision n

n queens

nqueens problem

decision trees example

decision trees idea

search problems

n n chessboard

gameplaying algorithms