cs 237: probability in computing and materials...example: flip three coins, and output head if there...
TRANSCRIPT
![Page 1: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/1.jpg)
CS 237: Probability in Computing
Wayne SnyderComputer Science Department
Boston University
Lecture 2: Conclusion on Probability Spaces; Finite Spaceso Probability Spaces and the Axiomatic Methodo Classification of Probability Spaces:
o Discrete (Finite, Countably Infinite)o Continuous (Uncountably Infinite)o Equiprobable vs Not-Equiprobable
o Finite Probability o Equiprobable Caseo Non-Equiprobable Case
![Page 2: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/2.jpg)
The Sample Points can be just about anything (numbers, letters, words, people, etc.) and the Sample Space (= any set of sample points) can be
FiniteExample: Flip three coins, and output head if there are at least two heads
showing, and tails otherwise (as if the coins "vote" for the outcome!)
S= { head, tail }
Countably InfiniteExample: Flip a coin until heads appears, and report the number of flips
S = { 1, 2, 3, 4, .... }
Uncountably InfiniteExample: Spin a pointer on a circle labelled with real numbers [0..1) and
report the number that the pointer stops on.
S = [0 .. 1)
Discrete
Continuous
Review: Random Experiments, Sample Spaces and Sample Points
![Page 3: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/3.jpg)
An Event is any subset of the Sample Space. An event A is said to have occurredif the outcome of the random experiment is a member of A. We will be mostly interested specifying a set by some characteristic, and then calculating the probability of that event occurring.
Example: Toss a die and output the number of dots showing. Let A = "there are an even number of dots showing" and B = "there are at least 5 dots showing."
S = { 1, 2, 3, 4, 5, 6 } A = { 2, 4, 6 } B = { 5, 6 }
Review: Sample Spaces, Sample Points, and Events
123456
4
The event A occurred, since
The event B did not occur:
o
7O
e
O
![Page 4: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/4.jpg)
We will be mostly interested in questions involving the probability of particular events occurring, so let us pay particular attention to the notion of an event.
Example: Toss a die and output the number of dots showing. Let A = "thereare an even number of dots showing."
S = { 1, 2, 3, 4, 5, 6 }
The set of possible events is the power-set of S, the set of all subsets,
So for this example we have 26 = 64 possible events, including
o The empty or "impossible event" . ("What is the probability of rolling a 9?")
o The "certain event" S. ("What is the probability of less than 10 dots?")
o All "elementary events" of one outcome: { 1 }, { 2 } , { 3 }, ..., { 6 }.
etc..... This gives you the most flexible way of discussing the results of an experiment....
Review: Sample Spaces, Sample Points, and Events
00
![Page 5: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/5.jpg)
To model a random experiment, we specify a Probability Space = a pair (S, P), where S is a sample space, and P is a probability function
which assigns a probability (a real number) to each possible event and such that P satisfies the following three Axioms of Probability:
P1: For any event A, we have P(A) ≥ ".
P2: For the certain event S, we have P(S) = %. ".
P3: For any two disjoint events A and B (A ∩ ( = ), we have
P(A ∪ ( ) = P(A) + P(B)
Also, we have an alternate version of the third axiom, for countable unions of events:
P'3: For any countably infinite sequence of events A1, A2, A3, ... which are pairwise disjoint (for any i and j, i ≠ j implies (Ai ∩ +" = )) we have
P( A1 ∪ +2 ∪ +3 ∪ … ) = P(A1) + P(A2) + P(A3) + ...
Probability Spaces and Probability Axioms
O
i
too
![Page 6: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/6.jpg)
These axioms make perfect sense if we consider Venn Diagrams where we use area as indicating probability, so the area of an event in the diagram = probability of that event.
Probability Spaces and Probability Axioms
P1: For any event A, we have P(A) ≥ ".
"The area of each event is non-negative."
P2: For the certain event S, we have P(S) = %. ".
"The area of the whole sample space is 1.0."
P3: For any two disjoint events A and B we haveP(A ∪ ( ) = P(A) + P(B)
"If two regions of S do not overlap, then thearea of the two regions combined is the sumof the area of each region."
S
A
B
o
parea.io
E
KA
PCBAREA
froBpc
![Page 7: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/7.jpg)
The axioms can be used to prove various results about probability.
Proof:
Probability Spaces and Probability Axioms
Theorem: P(∅) = 0.0
PC
Pfs 1.0 N
S S IT DISSENT SET THEM r
p s p Sudi DEF Fomci o pG told Psi a l O 1 PLI Pla l
![Page 8: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/8.jpg)
The axioms can be used to prove various results about probability.
Proof:
Probability Spaces and Probability Axioms
Theorem: For any event A, P(A) ∈ 0 . . 1 .
t
NII at cnI
PCs PIATRA as
1 a PIATRA Pa S A
Hadza Crl PLAS Pla PLACID
![Page 9: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/9.jpg)
So we measure the probability of events on a real-number scale from 0 to 1:
Probability Spaces and Probability Axioms
Impossible Certain
More probableLess probable
0.0 1.00.5
Equally probable
I PAI KAI PCD to
a 0A
PROB
![Page 10: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/10.jpg)
Recall that probability spaces can be characterized by the characteristics of their sample space: discrete (finite or countably infinite) or continueous (uncountable).
Furthermore, we may characterize a probability function as being:
Equiprobable: All sample points (= elementary events) have the same probability.
Not Equiprobable: All sample points do NOT have the same probability.
When the sample space is finite, it is easy to see how this might happen:
Finite and Equiprobable:Example: Flip a coin, report how many heads are showing.
S = { 0, 1 } P( 0 ) = 0.5 P( 1 ) = 0.5
Finite and NOT Equiprobable:Example: Flip two coins, report how many heads are showing.
S = { 0, 1, 2 } P( 0 ) = 0.25 P( 1 ) = 0.5 P( 2 ) = 0.25
Probability Functions: Equiprobable vs Not EquiprobablePROB
D
z
![Page 11: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/11.jpg)
Probability Functions: Equiprobable vs Not Equiprobableaaa
FEET 7H HH 2
JH Test HT 1
i
i
PdaB PCs 3 pgT TT
RA t PID I 1 I
![Page 12: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/12.jpg)
In order to specify a probability space for a particular problem, it suffices to giveo The Probability Space (a set S)o The Probability Function (a function P : -> [0..1] )
In order to check that you indeed have a correct probability space, it generally suffices to check axiom P2: P(S) = 1.0.
Example: Flip a coin, report how many heads are showing.S = { 0, 1 } P = { 0.5, 0.5 } <= Just give the probability of each sample point.
Check: 0.5 + 0.5 = 1.0
Example: Flip two coins, report how many heads are showing.S = { 0, 1, 2 } P = { 0.25, 0.5, 0.25 }
Check: 0.25 + 0.5 + 0.25 = 1.0
How to Specify a Probability SpaceSet of all events:
F I P Co B
5 5
![Page 13: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/13.jpg)
But when the sample space is countably infinite, the probability function can NOT be equiprobable!
Countably Infinite and Not Equiprobable:Example: Flip a coin until a heads appears, and return the number of flips.
S = { 1, 2, 3, ... } P = { 1/2, 1/4, 1/8, ... }
But suppose a space were countably infinite and equiprobable:S = { 1, 2, 3, .... } P = { p, p, p, ... } for some p > 0.
Then p + p + p + .... = not 1.0
Conclusion: When the sample space is countably infinite, the probability function can NOT be equiprobable.
Probability Functions: Equiprobable vs Not Equiprobable
Check: 1/2 + 1/4 + 1/8 + ... = 1.0
a IpGi Patc a 70 to t i
![Page 14: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/14.jpg)
Probability Functions: Equiprobable vs Not Equiprobable
ROCCA DIE UNTIL I APPEARScount ROLLS
5 91,2 3 4 a
Asp EE
II FEE
p l
K 2 2 2x x I 1 11 1g
I
![Page 15: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/15.jpg)
When the sample space is uncountable, say with the spinner, it is possible for the probability function to be equiprobable or non-equiprobable.
Uncountable and Equiprobable:Example: Spin the spinner and report the real number
showing.S = [0..1) Any point is equally likely
Uncountable and NOT Equiprobable:Example: Heights of Human Beings:
Probability Functions: Equiprobable vs Not Equiprobable
People are more likely to be close to the average height than at the extremes!
RANDOMC
![Page 16: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/16.jpg)
When the sample space is uncountable, as with our spinner, things can get a bit complicated.....
Question: Suppose you spin a spinner. What is the probability that the pointer lands EXACTLY on 0.141592... (the decimal part of 1 )?
Anomolies with Continuous Probability Spaces
Hint: There are two possibilities: o The probability is 0.o The probability is NOT 0.
Can you come up with an argument for or against either of these?
is ko Tt 3 1 1
If1415
I 1
E
is
![Page 17: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/17.jpg)
When the sample space is uncountable, as with our spinner, things can get a bit complicated.....
Question: Suppose you spin a spinner. What is the probability that the pointer lands EXACTLY on 0.5?
Answer: 0Why? Proof by contradiction: Suppose the probability is p > 0. Then this must also be true for ANY real number in the range [0..1). But then we have the same problem as with countable non-equiprobable spaces: p + p + p + .... = , , violating P2.
As a consequence, when we discuss events in continuous probability, it only makes sense to talk about countable numbers of unions and intersections of all possible intervals [a..b] , [a..b), (a..b], (a..b), etc.
We will explore this further in the next homework......
Anomalies with Continuous Probability Spaces
There is actually a whole field of study in mathematics called "Measure Theory" that deals with this problem!a
![Page 18: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/18.jpg)
For finite probability spaces, it is easy to calculate the probability of an event; we just have to apply axiom P3:
If event A = { a1, a2, ..., an }, then
P( A ) = P( { a1, a2, ..., an } ) = P( a1 ) + P( a2 ) + ... + P( an )
S A
Example: Toss a die and output the number of dots showing. Let A = "there are an even number of dots showing" and B = "there are at least 5 dots showing."
1
3
5
2
4
6 BEquiprobable: area of each elementary event is 1/6 = 0.16666...
Finite Probability Spaces
We can illustrate simple problems by using the "area" = "probability" analogy:
P(A) = P(2) + P(4) + P(6)= 1/6 + 1/6 + 1/6= ½
P(B) = P(5) + P(6)= 1/6 + 1/6= 1/3
![Page 19: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/19.jpg)
S
Example: Flip three fair coins and count the number of heads. Let A = "2 heads are showing" and B = "at most 2 heads are showing."
The equiprobable "pre-sample space" is
configuration: { TTT, TTH, THT, THH, HTT, HTH, HHT, HHH }# heads: 0 1 1 2 1 2 2 3
S = { 0, 1, 2, 3 }P = { 1/8, 3/8, 3/8, 1/8 } B
Finite Probability Spaces
P(A) = P(2) = 3/8
P(B) = P(0) + P(1) + P(2)= 1/8 + 3/8 + 3/8= 7/8
1 320
A
Not Equiprobable: area ofeach elementary event is different: 0.125 0.325 0.325 0.125
![Page 20: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/20.jpg)
Finite Equiprobable Probability SpacesFor finite and equiprobable probability spaces,
it is easy to calculate the probability:
Here, "area" = "number of elements."
Example: Flip a coin, report how many heads are showing? Let A = "the coin lands with tails showing"
S = { 0, 1 }P = { ½, ½ }
S
A
0 1
|A| = cardinality of set A = number of membersO
![Page 21: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/21.jpg)
Finite Equiprobable Probability SpacesFor finite and equiprobable probability spaces,
it is easy to calculate the probability:
Here, "area" = "number of elements."
Example: Roll a die, how many dots showing on the top face? Let A = "less than4 dots are showing."
S = { 1, 2, .... , 6 } P = { 1/6, 1/6, .... , 1/6 }
S
A 4
5
3
1
62
![Page 22: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/22.jpg)
Finite Equiprobable Probability Spaces
![Page 23: CS 237: Probability in Computing and Materials...Example: Flip three coins, and output head if there are at least two heads showing, and tails otherwise (as if the coins "vote" for](https://reader033.vdocument.in/reader033/viewer/2022060922/60ad817c77af92779431aed8/html5/thumbnails/23.jpg)
Finite Equiprobable Probability Spaces