Download - Lecture 2 - Probability
![Page 1: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/1.jpg)
Intro to Probability
![Page 2: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/2.jpg)
History of Probability
• Until the 16th Century, nobody put together a
systematic analysis of probability.
• Cardano, an eminent Mathematician (and
compulsive gambler) wrote “A Book on Games of
Chance” in 1526.
• He also included chapters on effective cheating
strategies.
2
![Page 3: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/3.jpg)
Basics
• If you have five things to choose from, and only one
of them is right, you have a 1-in-5 chance of getting
it right.
‣ Also 1/5
‣ 20%
‣ 0.2
• If X represents “choosing right” we can say
‣ P(X) = 0.2 (or 20%, 1/5 etc)
3
![Page 4: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/4.jpg)
Monty Hall
• Last lecture we talked about the “Monty Hall”
problem
• There are 3 possible doors - behind one of them is
a car, and behind two are donkeys.
• The aim is to win the car.
4
![Page 5: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/5.jpg)
The Twist
• After you pick a door, the gameshow host opens
one of the other doors to show a donkey.
• You are offered the opportunity to change to the
other door.
• Should you?
5
![Page 6: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/6.jpg)
Proof of Monty Hall
• Like we said yesterday, yes you should.
• And here’s why
6
![Page 7: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/7.jpg)
Proof 1 - Simple
• Initially you had a 1/3 chance of being right.
• That means a 2/3 chance of being wrong.
• If you were wrong, you should pick a different door,
and you know which door to pick now.
7
![Page 8: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/8.jpg)
Proof 2 - Enumerate
• Car at 1, 2, or 3
• Player picks 1
8
Car at 1 Car at 2 Car at 3
Host Opens 2
Lose x Win (twice)
Host Opens 3
Lose Win (twice)
x
Switching has a 2/3 chance of winning
![Page 9: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/9.jpg)
Conditional Probability
• Bayes’ Theorem of Conditional Probability
• Hinges on the concept of dependent variables.
•What is the chance that X happens given that Y has
happened.
• If X and Y are unrelated, it’s just the probability of X
happening
9
![Page 10: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/10.jpg)
Example
•What is the likelihood of flipping a coin and getting
heads, if we have just flipped a coin and got heads.
•One thing can’t affect the other.
‣ Probability of X given Y = P(X) = 1/2
•What is the likelihood that the next train will be
late if the last train was late(Actually, although the events are related, this
one is more based on Queue Theory...)
10
![Page 11: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/11.jpg)
Bayes’ Theorem
• P(A|B) => Probability of A given B has happened.
• P(A|B) = ( P(B|A) * P(A) ) / P(B)
• In AI we make a lot of use of this theorem
‣ “Bayesian Classification”
‣What is the likelihood that this is thing given that we have
observed data
11
![Page 12: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/12.jpg)
Bayesian Monty Hall
•What is the probability that Door1 wins, given we
have seen that Door 2 does not?
• 3 variables Car, Selection, Host - drawn from {1,2,3}
• P(C = 1 | S = 1, H = 2)
12
![Page 13: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/13.jpg)
Proof
• See Wikipedia entry on “Monty Hall Problem” for recap
of maths shown in class
13
![Page 14: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/14.jpg)
Spam Filtering
• Spam detection can be done with Bayes’ Theorem
•What is the likelihood that this message is spam
given it has these characteristics?
• Characteristics are typically keywords, origin, header
info etc.
14
![Page 15: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/15.jpg)
Spam Filtering
• Variables Spam, Characteristics
• P( S | C ) = ( P( C | S ) P ( C ) ) / P ( S )
•We can learn all the values of the RHS of this from
“training data”.
• Bayes’ Theorem then allows us to generalise to
items that aren’t in the training data.(Note that actual spam filters are much
more sophisticated, but still use Bayes)
15
![Page 16: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/16.jpg)
Training Data
• Big data set
• Pre-classified (by hand)
• Statistical analysis builds up a picture of what spam
looks like
‣ E.g. Emails that include “viagra” are typically spam
• Future emails can be classified using the stats we
learnt from the training data
• Refine analysis by “Report Spam” and “Not Spam”16
![Page 17: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/17.jpg)
Using Bayesian Classifiers
•We’ll see next week how we can use Bayes’
Theorem in games to classify players into
“stereotypes”
• And we can use Utility Theory from last lecture to
exploit these stereotypes
17
![Page 18: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/18.jpg)
Expected Value
• Expected Value is another statistical measure.
• “How much do I expect to win on average”
• Yesterday we talked about an example
‣ Guaranteed £1 or even chance at £3
• P(X) = 1/2, Payout is 3
‣ E(X) = £1.50
18
![Page 19: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/19.jpg)
Using Expected Value
• Expected Value can be used to make informed
choices.
• If we get to play the £1/£3 game repeatedly, over
time we will do better picking £3.
• Note that if we play only once, we may win nothing.
‣Which explains the result in £1,000,000/£3,000,000 game
• Expected Value can be deceptive, but it can also be
helpful.19
![Page 20: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/20.jpg)
The St Petersburg Paradox
• You pay a fee to enter a game where a coin is
flipped repeatedly. The game ends when the first
tails is shown.
• The payout starts at £1 and doubles for every head
that is shown.
•When the game ends, you win whatever the payout
has reached.
20
![Page 21: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/21.jpg)
The St Petersburg Paradox
•What is a sensible entry fee?
•Would you pay £1 to play?
•Would you pay £10 to play?
21
![Page 22: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/22.jpg)
The St Petersburg Paradox
• See Wikipedia entry on St Petersburg Paradox for recap
of maths shown in class
22
![Page 23: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/23.jpg)
The St Petersburg Paradox
• The Expected Value of this game is infinite.
• Therefore it “makes sense” to pay any price to play.
• But of course it doesn’t.
‣ The high payout cases are infinitesimally unlikely.
•We’ll talk next week about how we can work
around this.
23
![Page 24: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/24.jpg)
Iterated Games
• If you repeatedly play a game we call it “Iterated”.
• Iterating opens up a whole host of other options.
• In games with equilibrium points, it doesn’t change
• But in games without equilibrium points, it makes a
massive difference.
• In the same way we saw with Expected Values, we
can “average out” equilibrium points for the game.
24
![Page 25: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/25.jpg)
Mixed Strategies
•When a player has a choice of A, B, C etc. these are
“Pure” strategies
•When we are playing the same game repeatedly, we
can also choose a “Mixed” strategy.
• This is a probability distribution across two or more
of the Pure strategies.
‣ E.g. P(A) = 2/3, P(C) = 1/3
25
![Page 26: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/26.jpg)
Games Without Equilibria
26
Odd Even
Odd -1 1
Even 1 -1
![Page 27: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/27.jpg)
Equilibria
• Remember the definition of an equilibrium point
• If Player 1 changes strategy, they can only do worse
(assuming Player 2 does not change)
• Likewise Player 2 cannot change their strategy
unilaterally and do any better either.
• For both players, this is the best they can hope to
achieve
27
![Page 28: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/28.jpg)
The Odds/Evens Game
• But this does not hold in Odds/Evens
• Player 1 chooses Odd and Player 2 chooses Even
‣ Player 2 would do better to unilaterally change to Odd.
• Player 1 chooses Even and Player 2 chooses Even
‣ Player 1 would do better to unilaterally change to Odd.
• This game has no equilibria!
28
![Page 29: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/29.jpg)
Pseudo-Equilibria
• Calculating appropriate mixed strategies is tough.
• It’s not important to know how to do it for this
course, just that it can be done.
• However an easy approach that sometimes works
‣Delete all dominated strategies (consider that a strategy
may be dominated by a mixed strategy...)
‣ Find a combination that will give the same average payoff
regardless of your opponent.
29
![Page 30: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/30.jpg)
Iterated Odd/Even
•We talked previously about how best to play the
Odd/Even game, and how to vary your strategy.
•What works best is not to think or reason or plot
or scheme.
• A simple mixed strategy works best
‣ P(Odd) = 0.5, P(Even) = 0.5
• Regardless of your opponent, you will get the value
of the game, which is 0.30
![Page 31: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/31.jpg)
Iteration For Communication
• In non-zero sum games, it may be to our advantage
to telegraph to the other player our intentions.
• But we have no way of communicating.
• In an iterated game, we can send our intentions
using the choice strategy.
‣Our previous plays become a transcript of the message
we are sending
31
![Page 32: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/32.jpg)
Optimal Prisoner’s Dilemma
• The best strategy for Iterated Prisoner’s Dilemma is
tit-for-tat.
• Signal initially to your opponent that you are willing
to cooperate.
• Subsequently, play the strategy that the opponent
played last time.
• Punishes betrayal, rewards cooperation.
32
![Page 33: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/33.jpg)
Iterated Prisoner’s Dilemma
•Why is this a good thing?
• Consider the Prisoner’s Dilemma
•We can signal to the other player that we are willing
to cooperate with them.
‣We gain the best mutual payout.
‣ Removes a lot of the risk.
33
![Page 34: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/34.jpg)
The Hangman Paradox
• The Hangman Paradox is something to be wary of.
• A prisoner has been sentenced to be executed.
• He has been told that it will take place next week.
• He has also been told that it will be a surprise.
34
![Page 35: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/35.jpg)
The Hangman Paradox
• It can’t happen on Friday
‣ As that’s the last day of the week, if it did it would not be
a surprise.
• And if it can’t happen on Friday, equally it can’t
happen on Thursday by the same logic.
• By induction, he can’t be executed!
35
![Page 36: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/36.jpg)
The Hangman Paradox
• Having realised that he can’t be executed, he now
feels safe.
•On Wednesday, the hangman arrives to execute him.
• He is, as predicted, very surprised.
36
![Page 37: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/37.jpg)
Hangman Paradox for Iterated Games
• It’s easy to fall into the same reasoning for iterated
games.
‣ In the final iteration, there is no consequence to betrayal
‣ By induction, the case for cooperating at all falls apart
• This might be true for a determinate number of
iterations.
•What about an indeterminate number?
37
![Page 38: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/38.jpg)
Summary
• Lots of Probability
‣ Bayesian Probability
‣ Expected Value
• Iterated Games
• Mixed Strategies
• Cooperation
38
![Page 39: Lecture 2 - Probability](https://reader034.vdocument.in/reader034/viewer/2022042606/54b5c0324a79596e0a8b4854/html5/thumbnails/39.jpg)
Next Week
• Covering Poker in detail
• Designing agents to play games
• Mathematical models of players
39