Lec 18 Nov 12

Probability – definitions and simulation

(Discrete) Sample space

Experiment: a physical act such as tossing a coin or rolling a die.

Sample space – set of outcomes. Coin toss Sample Space S = { head, tail} Rolling a die Sample space S = {1, 2, 3, 4, 5, 6}

Tossing a coin twice. Sample space S = {(h,h), (h,t), (t,h), (t,t)}

Events and probability

Event E is any subset of sample space S. You flip 2 coins

Sample space S = {(h,h), (h,t), (t,h), (t,t)}

Event: both tosses produce same result E = {(h,h), (t,t)}

Prob(E) = |E|/ |S| In the above example, p(E) = 2/4 = 0.5Question: what is the probability of getting at

least one six in three roles of a die?

Bernoulli trial

Bernoulli trials are experiments with two outcomes. (success with prob = p and failure with prob = 1 – p.)

Example: rolling an unloaded die. Success is defined as getting a role of 1.

p(success) = 1/6

Random Variable

Random variable (RV) is a function that maps the sample space to a number. E.g. the total number of heads X you get if you

flip 100 coins

Another example: Keep tossing a coin until you get a head. The RV

n is the number of tosses. Event = { H, TH, TTH, TTTH, … } RV n(H) = 1, n(TH) = 2, n(TTH) = 3, … etc.

Common Distributions

Uniform X: U[1, N] X takes values 1, 2, …, N

E.g. picking balls of different colors from a box

Binomial distribution X takes values 0, 1, …, n

N coin tosses. What is the prob. That there are exactly k tails?

P X 1i N

P X 1n iin

i p pi

Conditional Probability

P(A|B) is the probability of event A given that B has occurred. Suppose 6 coins are tossed. Given that

there is at least one head, what is the probability that the number of heads is 3?

Definition:

p(A|B) =

)(

)(

Bp

BAp

Baye’s Rule

If X and Y are events, then p(X|Y) = p(Y|X) p(X)/p(Y)

Useful in situation where p(X), p(Y) and p(Y|X) are easier to compute than p(X|Y).

Independent events

Definition: X and Y are independent if P X Y P X P Yx y x y

Monty Hall Problem

You're given the choice of three doors: Behind one door is a car; behind the others, goats. You want to pick the car.

You pick a door, say No. 1 The host, who knows what's behind the

doors, opens another door, say No. 3, which has a goat.

Do you want to pick door No. 2 instead?

                        

                        

Host mustreveal Goat B

                        

                        

Host mustreveal Goat A

                       

                           

  

Host revealsGoat A

orHost reveals

Goat B

           

             

Monty Hall Problem: Bayes Rule

: the car is behind door i, i = 1, 2, 3 : the host opens door j after you

pick door i

iC

ijH

1 3iP C

0

0

1 2

1 ,

ij k

i j

j kP H C

i k

i k j k

Monty Hall Problem: Bayes Rule continued

WLOG, i=1, j=3

13 1 11 13

13

P H C P CP C H

P H

13 1 11 1 1

2 3 6P H C P C

Monty Hall Problem: Bayes Rule continued

13 13 1 13 2 13 3

13 1 1 13 2 2

, , ,

1 11

6 31

2

P H P H C P H C P H C

P H C P C P H C P C

1 131 6 1

1 2 3P C H

Monty Hall Problem: Bayes Rule continued

1 131 6 1

1 2 3P C H

You should switch!

2 13 1 131 2

13 3

P C H P C H

Continuous Random Variables

What if X is continuous? Probability density function (pdf)

instead of probability mass function (pmf)

A pdf is any function that describes the probability density in terms of the input variable x.

f x

Probability Density Function

Properties of pdf

Actual probability can be obtained by taking the integral of pdf E.g. the probability of X being between

0 and 1 is

0,f x x

1f x

1

0P 0 1X f x dx

Cumulative Distribution Function

Discrete RVs

Continuous RVs

X P XF v v

X P Xi

ivF v v

X

vF v f x dx

X

dF x f x

dx

Common Distributions

Normal

E.g. the height of the entire population

2X ,N

22

1exp ,

2 2

xf x x

-5 -4 -3 -2 -1 0 1 2 3 4 50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x

f(x)

Moments

Mean (Expectation): Discrete RVs:

Continuous RVs: Variance:

Discrete RVs:

Continuous RVs:

XE

X P Xii iv

E v v XE xf x dx

2X XV E

2X P X

ii iv

V v v 2XV x f x dx

Properties of Moments

Mean If X and Y are independent,

Variance If X and Y are independent,

X Y X YE E E

X XE a aE

XY X YE E E

2X XV a b a V

X Y (X) (Y)V V V

Moments of Common Distributions

Uniform Mean ; variance

Binomial Mean ; variance

Normal Mean ; variance

X 1, ,U N 1 2N 2 1 12N

X ,Bin n p

np 2np

2X ,N

2

Simulating events by Matlab programs

Write a program in Matlab to distribute the 52 cards of a deck to 4 people, each getting 13 cards. All the choices must be equally likely.

One way to do this is as follows: map each card to a number 1, 2, …, 52. Generate a random permutation of the array a[1 2 … 52], then give the cards a[1:13] to first player, a[14:26] to second player etc.

Random permutation generation

We can use ceil(rand()*n) to generate a random number from the set {1, 2, …, n}.

Algorithm generate a random permutation:

1. Start with array a = [ 1 2 … n]

2. For j = n: -1: 1 randomly pick a number r in [1..j]. Switch a[r]

and a[j]

3. Output a.