poki: the poker agent greg priebe zak knudson. overview texas hold’em poker architecture and...
Post on 24-Jan-2016
220 Views
Preview:
TRANSCRIPT
Poki: The Poker Agent
Greg PriebeZak Knudson
Overview
• Texas Hold’em poker• Architecture and Opponent
Modeling of Poki• Improvements from past Poki• Betting strategies w/ analysis of
hand strengths, weighting, and probabilities.
Texas Hold’em
• Each player gets 2 “hole” cards
• Community cards:– 3 “flop” cards– 1 “turn” card– 1 “river” card
• Rounds of betting after each set of cards
Poki Program Architecture
Opponent Modeling
• Weight table• No modeling• Generic opponent
modeling• Specific opponent
modeling
The Old Way
• Specific opponent modeling• Betting frequency table
– Keeps track of actions taken in different contexts
– Predicts median hand strength
• Left out many relevant details
New and Improved
• Specific opponent modeling• Detailed• Dynamic learning/adaptation• Prelim. Neural net study
Poki’s Neural Net
Some Results
• Training Data• Results from
80-90 percent• Confusion
matrix• Error prone?
In Field Predictions
#holdem1: 24% increase
In Field Performance
Small Bets Won
• Pro: .05 - .10• Old opponent
modeling: .09• New opponent
modeling: .22
Betting Strategy
• Pre-flop and Post-flop strategies are significantly different.
- Pre: little information available to influence the betting decision.
- Post: uses opponent models, private hand, and game context to generate an action.
Pre-flop Betting Strategy* - pocket pair hand (two cards of the same rank)s - suited hand o - offsuit hand
Post-flop: Basic Betting Strategy• Compute Poki’s effective hand strength
(EHS).• Using the game context, betting rules,
and formulas to translate the EHS into a probability triple{ Pr(fold), Pr(call), Pr(raise) }.
• Generate a random number and use it to choose an action from the probability distribution.
Hand Strength
• Probability that a given hand is better than that of an active opponent.
Hand Potential• With 2 cards yet to be revealed, we
want to know the impact.• Positive potential (Ppot) – chance hand
improves to win.• Negative potential (Npot) – chance hand
ends up losing.• Calculated by enumerating over all
possible hole cards, and over all possible board cards.
• This is an expensive process.
Hand potential example
Effective Hand Strength
• Combines hand strength and potential to give Poki’s own relative strength against an opponent.
• Pr(win) = pr(ahead) * pr(opp doesn’t improve) + Pr(behind) * pr(we improve)
= HS * (1 – Npot) + (1 – HS) * Ppot
Weighting the Enumerations
• Probability of hands played to a particular point will vary.
• To account for this, Poki maintains a weight table for each opponent.
Conclusions
• Need opponent modeling • Complex system • Betting strategies and determining hand
strength• Reweight helps to determine
opponents likely action
top related