1 algorithms csci 235, fall 2012 lecture 9 probability
TRANSCRIPT
1
Algorithms
CSCI 235, Fall 2012Lecture 9Probability
2
Permutations
How many ways can you take k items from a set of n items?Order matters (ab is counted separately from ba)No duplicates allowed (don't count aa)
Example: S = {a, b, c, d}, how many ways can we take 2 items?
abacad
babcbd
cacbcd
dadbdc
Total pairs = 124 ways to pick first3 ways to pick second
In general: The number of permutations of k items from a set of n is:
€
n(n −1)(n − 2)...(n − k +1) =n!
(n − k)!
3
Combinations
Combinations are like permutations, except order doesn't matter.ab and ba are considered the same.
abacad
bcbd cd
Combination of 4 things taken 2 at a time:
Each set of k items has k! possible permutations.number of combinations = number of permutations divided by k!
€
n
k
⎛
⎝ ⎜
⎞
⎠ ⎟=
n!
k!(n − k)!
4
k-tuples and k-selections
k-tuple (or k-string): Like a permutation, but can have duplicates.
aaabacad
babbbcbd
cacbcccd
dadbdcdd
n choices for 1st itemn choices for 2nd item, etc.
Total possible k-tuples for k items taken from a group of n items is nk
k-selection: Like combination, but can have duplicates.
aaabacad
bbbcbd
cccd dd
Formula:
€
(n −1+ k)!
k!(n −1)!
5
Sample Spaces
A sample space is the set of all possible outcomes for an experiment.
Experiment A: flip 3 coins
Sample space = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
# of possibilities: ?
Experiment B: Flip a coin until you get heads.
Sample space = {H, TH, TTH, TTTH, . . .}
23
6
Diagramming sample spaces
Experiment A:(flip 3 coins)
7
Diagramming sample spaces
Experiment A:(flip 3 coins)
H3 H1H2H3
T3 H1H2T3
H3 H1T2H3
T3 H1T2T3
H3 T1H2H3
T3 T1H2T3
H3 T1T2H3
T3 T1T2T3
T1
T2
H2
T2
H2
H1
Experiment B:(flip until heads)
Samplespace(the set of all possible outcomes)
8
Diagramming sample spaces
Experiment A:(flip 3 coins)
H3 H1H2H3
T3 H1H2T3
H3 H1T2H3
T3 H1T2T3
H3 T1H2H3
T3 T1H2T3
H3 T1T2H3
T3 T1T2T3
T1
T2
H2
T2
H2
H1
Experiment B:(flip until heads) H1 T1H2 T1T2H3 T1T2T3H4
H1 H2 H3 H4
T1 T2 T3 T4 . . .
Samplespace(the set of all possible outcomes)
...
9
Events
An event is a subset of the sample space:Experiment A (flip 3 coins):
Event A1: First flip is head = {HHH, HHT, HTH, HTT}Event A2: Second flip is tail = {HTH, HTT, TTH, TTT}Event A3: Exactly 2 tails = {HTT, THT, TTH}Event A4: Two consecutive flips are the same=
{HHH, HHT, HTT, THH, TTH, TTT}
Experiment B (flip until get heads):Event B1: First flip is head = {H}Event B2: First flip is tail = {TH, TTH, TTTH ...}Event B3: Even number of flips = {TH, TTTH, TTTTTH ...}
10
Definitions
If A and B are events in sample space S, then
"A and B" is translated
€
A∩ B
"A or B" is translated
€
A∪B
"not A" is translated
€
S − A
Two events are mutually exclusive if
€
A∩ B =∅
11
Examples
1. Of the four events in Experiment A, which pairs are mutually exclusive?
2. Of the three events in Experiment B, which pairs are mutually exclusive?
12
Recall events
An event is a subset of the sample space:Experiment A (flip 3 coins):
Event A1: First flip is head = {HHH, HHT, HTH, HTT}Event A2: Second flip is tail = {HTH, HTT, TTH, TTT}Event A3: Exactly 2 tails = {HTT, THT, TTH}Event A4: Two consecutive flips are the same=
{HHH, HHT, HTT, THH, TTH, TTT}
Experiment B (flip until get heads):Event B1: First flip is head = {H}Event B2: First flip is tail = {TH, TTH, TTTH ...}Event B3: Even number of flips = {TH, TTTH, TTTTTH}
13
Probability DistributionA probability distribution Pr{ } on sample space S is any mapping from events of S to [0...1] such that the following axioms hold:
1) Pr{A} >= 0 for any event A
2) Pr{S} = 1
€
3) If A∩ B =∅ then Pr{A∪B} = Pr{A} + Pr{B}
Example:Flip two coins: S= {HH, HT, TH, TT}Event A = {HT} Event B = {TH}
What is
€
Pr{A∪B}?
14
Some useful theorems
€
1) Pr{∅ } = 0
€
2) Pr{S − A} =1− Pr{A}
€
3) If A⊆B then Pr{A} ≤ Pr{B}
€
4) Pr{A∪B} = Pr{A} + Pr{B} - Pr{A∩ B}
Example: Flip two coins.Event A = {HH, HT, TH}Event B = {HT, TH, TT}
What is
€
Pr{A∪B}?
15
Working with probability treesIf events at distinct stages of a probability tree are independent, then the probability of a leaf is the product of the probabilities on the path to the leaf.Experiment A (flip 3 weighted coins): H1=1/3, T1 = 2/3; H2=1/4, T2=3/4; H3=1/5, T3 = 4/5
16
Working with probability treesIf events at distinct stages of a probability tree are independent, then the probability of a leaf is the product of the probabilities on the path to the leaf.
H3 H1H2H3
T3 H1H2T3
H3 H1T2H3
T3 H1T2T3
H3 T1H2H3
T3 T1H2T3
H3 T1T2H3
T3 T1T2T3
T1
T2
H2
T2
H2
H1
Experiment A (flip 3 weighted coins): H1=1/3, T1 = 2/3; H2=1/4, T2=3/4; H3=1/5, T3 = 4/5
(1/3)(1/4)(1/5) = 1/60
(1/3)(1/4)(4/5) = 4/60 = 1/15
(1/3)(3/4)(1/5) = 3/60 = 1/20
(1/3)(3/4)(4/5) = 12/60 = 1/5
(2/3)(1/4)(1/5) = 2/60 = 1/30
(2/3)(1/4)(4/5) = 8/60 = 2/15
(2/3)(3/4)(1/5) = 6/60 = 1/10
(2/3)(3/4)(4/5) = 24/60 = 2/5
17
Another example
Experiment B (flip a weighted coin until heads): Hi = 1/3, Ti = 2/3
H1 T1H2 T1T2H3 T1T2T3H4
H1 H2 H3 H4
T1 T2 T3 T4 . . .
1/3(2/3)(1/3)= 2/9
(2/3)(2/3)(1/3)= 4/27
(2/3)(2/3)(2/3)(1/3)= 8/81
Probability of S =
€
1
3
2
3
⎛ ⎝ ⎜
⎞ ⎠ ⎟k
=k= 0
∞
∑1
3
2
3
⎛ ⎝ ⎜
⎞ ⎠ ⎟k
=k= 0
∞
∑1
3
1
1− 2/3
⎛ ⎝ ⎜
⎞ ⎠ ⎟=
1
3
1
1/3
⎛ ⎝ ⎜
⎞ ⎠ ⎟=1
1/3 + 2/9 + 4/27 + 8/81 + ... =