Poki: The Poker Agent

Greg PriebeZak Knudson

Overview

• Texas Hold’em poker• Architecture and Opponent

Modeling of Poki• Improvements from past Poki• Betting strategies w/ analysis of

hand strengths, weighting, and probabilities.

Texas Hold’em

• Each player gets 2 “hole” cards

• Community cards:– 3 “flop” cards– 1 “turn” card– 1 “river” card

• Rounds of betting after each set of cards

Poki Program Architecture

Opponent Modeling

• Weight table• No modeling• Generic opponent

modeling• Specific opponent

modeling

The Old Way

• Specific opponent modeling• Betting frequency table

– Keeps track of actions taken in different contexts

– Predicts median hand strength

• Left out many relevant details

New and Improved

• Specific opponent modeling• Detailed• Dynamic learning/adaptation• Prelim. Neural net study

Poki’s Neural Net

Some Results

• Training Data• Results from

80-90 percent• Confusion

matrix• Error prone?

In Field Predictions

#holdem1: 24% increase

In Field Performance

Small Bets Won

• Pro: .05 - .10• Old opponent

modeling: .09• New opponent

modeling: .22

Betting Strategy

• Pre-flop and Post-flop strategies are significantly different.

- Pre: little information available to influence the betting decision.

- Post: uses opponent models, private hand, and game context to generate an action.

Pre-flop Betting Strategy* - pocket pair hand (two cards of the same rank)s - suited hand o - offsuit hand

Post-flop: Basic Betting Strategy• Compute Poki’s effective hand strength

(EHS).• Using the game context, betting rules,

and formulas to translate the EHS into a probability triple{ Pr(fold), Pr(call), Pr(raise) }.

• Generate a random number and use it to choose an action from the probability distribution.

Hand Strength

• Probability that a given hand is better than that of an active opponent.

Hand Potential• With 2 cards yet to be revealed, we

want to know the impact.• Positive potential (Ppot) – chance hand

improves to win.• Negative potential (Npot) – chance hand

ends up losing.• Calculated by enumerating over all

possible hole cards, and over all possible board cards.

• This is an expensive process.

Hand potential example

Effective Hand Strength

• Combines hand strength and potential to give Poki’s own relative strength against an opponent.

• Pr(win) = pr(ahead) * pr(opp doesn’t improve) + Pr(behind) * pr(we improve)

= HS * (1 – Npot) + (1 – HS) * Ppot

Weighting the Enumerations

• Probability of hands played to a particular point will vary.

• To account for this, Poki maintains a weight table for each opponent.

Conclusions

• Need opponent modeling • Complex system • Betting strategies and determining hand

strength• Reweight helps to determine

opponents likely action