probability theory random variables and distributions · 2014-03-10 · probability theory so...
TRANSCRIPT
![Page 1: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/1.jpg)
Probability Theory Random Variables and Distributions
Rob Nicholls
MRC LMB Statistics Course 2014
![Page 2: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/2.jpg)
Introduction
![Page 3: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/3.jpg)
Introduction
Significance magazine (December 2013) Royal Statistical Society
![Page 4: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/4.jpg)
Introduction
Significance magazine (December 2013) Royal Statistical Society
![Page 5: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/5.jpg)
Introduction
Significance magazine (December 2013) Royal Statistical Society
![Page 6: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/6.jpg)
Contents
• Set Notation • Intro to Probability Theory • Random Variables • Probability Mass Functions • Common Discrete Distributions
![Page 7: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/7.jpg)
Set Notation
A set is a collection of objects, written using curly brackets {}
If A is the set of all outcomes, then:
A set does not have to comprise the full number of outcomes E.g. if A is the set of dice outcomes no higher than three, then:
A = {heads, tails}
A = {one, two, three, four, five, six}
A = {one, two, three}
![Page 8: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/8.jpg)
Set Notation
If A and B are sets, then:
!A
A∪B
A∩B
A \ B
∅
Complement – everything but A
Union (or)
Intersection (and)
Not
Empty Set
![Page 9: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/9.jpg)
Set Notation
Venn Diagram:
A B
one
two, three six four five
![Page 10: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/10.jpg)
Set Notation
Venn Diagram:
A B
A = {two, three, four, five}
one
two, three six four five
![Page 11: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/11.jpg)
Set Notation
Venn Diagram:
A B
B = { four, five, six}
one
two, three six four five
![Page 12: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/12.jpg)
Set Notation
Venn Diagram:
A B
A∩B = { four, five}
one
two, three six four five
![Page 13: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/13.jpg)
Set Notation
Venn Diagram:
A B
A∪B = {two, three, four, five, six}
one
two, three six four five
![Page 14: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/14.jpg)
Set Notation
Venn Diagram:
A B
(A∪B ") = {one}
one
two, three six four five
![Page 15: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/15.jpg)
Set Notation
Venn Diagram:
A B
A \ B = {two, three}
one
two, three six four five
![Page 16: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/16.jpg)
Set Notation
Venn Diagram:
A B
(A \ B !) = {one, four, five, six}
one
two, three six four five
![Page 17: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/17.jpg)
Probability Theory
To consider Probabilities, we need:
1. Sample space: Ω
2. Event space:
3. Probability measure: P
![Page 18: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/18.jpg)
Probability Theory
To consider Probabilities, we need:
1. Sample space: Ω - the set of all possible outcomes
Ω = {heads, tails}
Ω = {one, two, three, four, five, six}
![Page 19: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/19.jpg)
Probability Theory
To consider Probabilities, we need:
2. Event space: - the set of all possible events
Ω = {heads, tails}Ω = {{heads, tails},{heads},{tails},∅}
![Page 20: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/20.jpg)
Probability Theory
To consider Probabilities, we need:
3. Probability measure: P P must satisfy two axioms:
P :F→ [0,1]
P(Ω) =1
P(iAi ) = P(Ai )
i∑
Probability of any outcome is 1 (100% chance) If and only if are disjoint A1,A2,…
![Page 21: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/21.jpg)
Probability Theory
To consider Probabilities, we need:
3. Probability measure: P P must satisfy two axioms:
P :F→ [0,1]
P(Ω) =1
P(iAi ) = P(Ai )
i∑
Probability of any outcome is 1 (100% chance) If and only if are disjoint A1,A2,…
P({one, two}) = P({one})+P({two})13=16+16
![Page 22: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/22.jpg)
Probability Theory
To consider Probabilities, we need:
1. Sample space: Ω 2. Event space: 3. Probability measure: P
As such, a Probability Space is the triple: (Ω, ,P )
![Page 23: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/23.jpg)
Probability Theory
To consider Probabilities, we need:
The triple: (Ω, ,P ) i.e. we need to know: 1. The set of potential outcomes; 2. The set of potential events that may occur; and 3. The probabilities associated with occurrence of those events.
![Page 24: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/24.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
![Page 25: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/25.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
A = {one, two}!A = {three, four, five, six}
P(A) =1/ 3P( !A ) = 2 / 3
![Page 26: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/26.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
![Page 27: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/27.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
A B
![Page 28: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/28.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
A B
![Page 29: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/29.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
A B
![Page 30: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/30.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
A = {one, two}B = {two, three}A∪B = {one, two, three}A∩B = {two}
P(A) =1/ 3P(B) =1/ 3P(A∪B) =1/ 2P(A∩B) =1/ 6
![Page 31: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/31.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
A ⊆ B P(A) ≤ P(B) P(B \ A) = P(B)−P(A)If then and
A B
![Page 32: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/32.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
A ⊆ B P(A) ≤ P(B) P(B \ A) = P(B)−P(A)If then and
A B
![Page 33: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/33.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
A ⊆ B P(A) ≤ P(B) P(B \ A) = P(B)−P(A)If then and
A B
![Page 34: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/34.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
A ⊆ B P(A) ≤ P(B) P(B \ A) = P(B)−P(A)If then and
A B
![Page 35: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/35.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(A∪B) = P(A)+P(B)−P(A∩B)
A ⊆ B P(A) ≤ P(B) P(B \ A) = P(B)−P(A)If then and
A = {one, two}B = {one, two, three}B \ A = {three}
P(A) =1/ 3P(B) =1/ 2P(B \ A) =1/ 6
![Page 36: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/36.jpg)
Probability Theory
Notable properties of a Probability Space (Ω, ,P ):
P( !A ) =1−P(A)
P(∅) = 0
P(A∪B) = P(A)+P(B)−P(A∩B)
A ⊆ B P(A) ≤ P(B) P(B \ A) = P(B)−P(A)If then and
![Page 37: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/37.jpg)
Probability Theory So where’s this all going? These examples are trivial!
![Page 38: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/38.jpg)
Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B1, B2 and B3, each of which contain a number of coloured balls: • B1 – 2 red and 4 white • B2 – 1 red and 2 white • B3 – 5 red and 4 white A ball is randomly removed from one the bags. The bags were selected with probability: • P(B1) = 1/3 • P(B2) = 5/12 • P(B3) = 1/4 What is the probability that the ball came from B1, given it is red?
![Page 39: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/39.jpg)
Probability Theory
Conditional probability: Partition Theorem: Bayes’ Theorem:
P(A | B) = P(A∩B)P(B)
P(A) = P(A∩Bi )i∑ If the partition
P(A | B) = P(B | A)P(A)P(B)
Bi A
![Page 40: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/40.jpg)
Random Variables
A Random Variable is an object whose value is determined by chance, i.e. random events Maps elements of Ω onto real numbers, with corresponding probabilities as specified by P
![Page 41: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/41.jpg)
Random Variables
A Random Variable is an object whose value is determined by chance, i.e. random events Maps elements of Ω onto real numbers, with corresponding probabilities as specified by P Formally, a Random Variable is a function:
X :Ω→R
![Page 42: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/42.jpg)
Random Variables
A Random Variable is an object whose value is determined by chance, i.e. random events Maps elements of Ω onto real numbers, with corresponding probabilities as specified by P Formally, a Random Variable is a function: Probability that the random variable X adopts a particular value x:
P({w ∈Ω : X(w) = x})
X :Ω→R
![Page 43: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/43.jpg)
Random Variables
A Random Variable is an object whose value is determined by chance, i.e. random events Maps elements of Ω onto real numbers, with corresponding probabilities as specified by P Formally, a Random Variable is a function: Probability that the random variable X adopts a particular value x:
P({w ∈Ω : X(w) = x})
P(X = x)Shorthand:
X :Ω→R
![Page 44: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/44.jpg)
Random Variables
Example:
Ω = {heads, tails}X :Ω→ {0,1}
P(X = x) =P({heads}) x =1
P({tails}) x = 0
!
"##
$##
If the result is heads then WIN - X takes the value 1 If the result is tails then LOSE - X takes the value 0
P(X = x) =1 2 x ∈ {0,1}
![Page 45: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/45.jpg)
Random Variables
Example: Win £20 on a six, nothing on four/five, lose £10 on one/two/three
Ω = {one, two, three, four, five, six}
X :Ω→ {−10,0, 20}
![Page 46: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/46.jpg)
Random Variables
Example: Win £20 on a six, nothing on four/five, lose £10 on one/two/three
Ω = {one, two, three, four, five, six}
X :Ω→ {−10,0, 20}
P(X = x) =
P({six}) =1 6 x = 20
P({ four, five})=1 3 x = 0
P({one, two, three}) =1 2 x = −10
"
#
$$$
%
$$$
Note – we are considering the probabilities of events in
![Page 47: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/47.jpg)
Probability Mass Functions
Given a random variable: The Probability Mass Function is defined as: Only for discrete random variables
X :Ω→ A
pX (x) = P(X = x)
![Page 48: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/48.jpg)
Probability Mass Functions Example:
Win £20 on a six, nothing on four/five, lose £10 on one/two/three
pX (x) =
P({six}) =1 6 x = 20
P({ four, five})=1 3 x = 0
P({one, two, three}) =1 2 x = −10
"
#
$$$
%
$$$
![Page 49: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/49.jpg)
Probability Mass Functions
Notable properties of Probability Mass Functions:
pX (x) ≥ 0
pX (x)x∈A∑ =1
![Page 50: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/50.jpg)
Probability Mass Functions
Notable properties of Probability Mass Functions:
Interesting note: If p() is some function that has the above two properties, then it is the mass function of some random variable…
pX (x)x∈A∑ =1
pX (x) ≥ 0
![Page 51: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/51.jpg)
Mean:
X :Ω→ AFor a random variable
E(X) = xpX (x)x∈A∑
Probability Mass Functions
![Page 52: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/52.jpg)
Probability Mass Functions
Mean:
X :Ω→ AFor a random variable
E(X) = xpX (x)x∈A∑
![Page 53: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/53.jpg)
Probability Mass Functions
Mean:
X :Ω→ AFor a random variable
E(X) = xpX (x)x∈A∑ 1
nxi
i=1
n
∑Compare with:
![Page 54: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/54.jpg)
Probability Mass Functions
Mean:
X :Ω→ AFor a random variable
E(X) = xpX (x)x∈A∑
Median: pX (x)x≤m∑ ≥1 2 pX (x)
x≥m∑ ≥1 2any m such that: and
![Page 55: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/55.jpg)
Probability Mass Functions
Mean:
X :Ω→ AFor a random variable
E(X) = xpX (x)x∈A∑
Median: any m such that: and
Mode: : most likely value argmaxx
pX (x)( )
pX (x)x≤m∑ ≥1 2 pX (x)
x≥m∑ ≥1 2
![Page 56: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/56.jpg)
Common Discrete Distributions
![Page 57: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/57.jpg)
Common Discrete Distributions
The Bernoulli Distribution:
p : success probability
X :Ω→ {0,1} pX (x) =p x =1
1− p x = 0
"
#$$
%$$
X ~ Bern(p)
![Page 58: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/58.jpg)
Common Discrete Distributions
The Bernoulli Distribution:
p : success probability
X :Ω→ {0,1}
X ~ Bern(p)
Example:
X : {heads, tails}→ {0,1}
pX (x) =1 2 x ∈ {0,1}
X ~ Bern(1 2)Therefore
pX (x) =p x =1
1− p x = 0
"
#$$
%$$
![Page 59: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/59.jpg)
Common Discrete Distributions
X ~ Bin(n, p)
pX (x) =nx!
"#$
%& px (1− p)n−xX :Ω→ {0,1,...,n}
The Binomial Distribution:
n : number of independent trials p : success probability
E(X) = np
![Page 60: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/60.jpg)
Common Discrete Distributions
X ~ Bin(n, p)
pX (x) =nx!
"#$
%& px (1− p)n−xX :Ω→ {0,1,...,n}
The Binomial Distribution:
n : number of independent trials p : success probability
pX (x) : probability of getting x successes out of n trials : probability of x successes : probability of (n-x) failures : number of ways to achieve x successes and (n-x) failures (Binomial coefficient)
px
(1− p)n−x
nx!
"#$
%&=
n!x!(n− x)!
E(X) = np
![Page 61: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/61.jpg)
Common Discrete Distributions
X ~ Bin(n, p)
pX (x) =nx!
"#$
%& px (1− p)n−xX :Ω→ {0,1,...,n}
The Binomial Distribution:
n : number of independent trials p : success probability
n =1 : pX (x) = px (1− p)1−x =p x =1
1− p x = 0
"#$
%$
X ~ Bin(1, p) ⇔ X ~ Bern(p)
E(X) = np
![Page 62: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/62.jpg)
Common Discrete Distributions
X ~ Bin(n, p)
pX (x) =nx!
"#$
%& px (1− p)n−xX :Ω→ {0,1,...,n}
The Binomial Distribution:
n : number of independent trials p : success probability
E(X) = np
n =10p = 0.5
![Page 63: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/63.jpg)
Common Discrete Distributions
X ~ Bin(n, p)
pX (x) =nx!
"#$
%& px (1− p)n−xX :Ω→ {0,1,...,n}
The Binomial Distribution:
n : number of independent trials p : success probability
E(X) = np
n =100p = 0.5
![Page 64: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/64.jpg)
Common Discrete Distributions
X ~ Bin(n, p)
pX (x) =nx!
"#$
%& px (1− p)n−xX :Ω→ {0,1,...,n}
The Binomial Distribution:
n : number of independent trials p : success probability
E(X) = np
n =100p = 0.8
![Page 65: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/65.jpg)
Common Discrete Distributions
Example:
Number of heads in n fair coin toss trials
Ω = {heads :heads,heads : tails, tails :heads, tails : tails}n = 2
Ω = 2nIn general:
X :Ω→ {0,1,...,n}
![Page 66: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/66.jpg)
Common Discrete Distributions
Example:
X ~ Bin(n,1 2)
Number of heads in n fair coin toss trials
Ω = {heads :heads,heads : tails, tails :heads, tails : tails}n = 2
Ω = 2n
pX (x) =nx!
"#$
%&0.5n
In general:
E(X) = n 2
Notice:
X :Ω→ {0,1,...,n}
![Page 67: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/67.jpg)
Common Discrete Distributions
X ~ Pois(λ)
pX (x) =λ xe−λ
x!X :Ω→ {0,1,...}
The Poisson Distribution: E(X) = λ
λ : average number of counts (controls rarity of events)
Used to model the number of occurrences of an event that occur within a particular interval of time and/or space
![Page 68: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/68.jpg)
Common Discrete Distributions
• Want to know the distribution of the number of occurrences of an event Binomial?
• However, don’t know how many trials are performed – could be infinite!
• But we do know the average rate of occurrence:
The Poisson Distribution:
⇒
E(X) = λ
X ~ Bin(n, p) ⇒ E(X) = np⇒ λ = np
⇒ p = λn
![Page 69: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/69.jpg)
Common Discrete Distributions
Binomial: pX (x) =n!
x!(n− x)!px (1− p)n−x
p = λn
⇒ pX (x) =n!
x!(n− x)!λn#
$%
&
'(x
1− λn
#
$%
&
'(n−x
![Page 70: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/70.jpg)
Common Discrete Distributions
Binomial: pX (x) =n!
x!(n− x)!px (1− p)n−x
p = λn
⇒ pX (x) =n!
x!(n− x)!λn#
$%
&
'(x
1− λn
#
$%
&
'(n−x
pX (x) =λ x
x!n!
nx (n− x)!
1− λn
#
$%
&
'(n
1− λn
#
$%
&
'(x
![Page 71: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/71.jpg)
pX (x) =n!
x!(n− x)!px (1− p)n−x
p = λn
⇒ pX (x) =n!
x!(n− x)!λn#
$%
&
'(x
1− λn
#
$%
&
'(n−x
pX (x) =λ x
x!n!
nx (n− x)!
1− λn
#
$%
&
'(n
1− λn
#
$%
&
'(x
Common Discrete Distributions
Binomial:
1
1 as n→∞
![Page 72: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/72.jpg)
pX (x) =n!
x!(n− x)!px (1− p)n−x
p = λn
⇒ pX (x) =n!
x!(n− x)!λn#
$%
&
'(x
1− λn
#
$%
&
'(n−x
pX (x) =λ x
x!n!
nx (n− x)!
1− λn
#
$%
&
'(n
1− λn
#
$%
&
'(x
Common Discrete Distributions
Binomial:
1
1 as n→∞
pX (x) = Limn→∞
λ x
x!1− λ
n$
%&
'
()n$
%&&
'
())
![Page 73: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/73.jpg)
pX (x) =n!
x!(n− x)!px (1− p)n−x
p = λn
⇒ pX (x) =n!
x!(n− x)!λn#
$%
&
'(x
1− λn
#
$%
&
'(n−x
pX (x) =λ x
x!n!
nx (n− x)!
1− λn
#
$%
&
'(n
1− λn
#
$%
&
'(x
Common Discrete Distributions
pX (x) = Limn→∞
λ x
x!1− λ
n$
%&
'
()n$
%&&
'
()) =
λ x
x!e−λ
Binomial:
1
1 as n→∞
☐
![Page 74: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/74.jpg)
This is referred to as the: • “Poisson Limit Theorem” • “Poisson Approximation to the Binomial” • “Law of Rare Events”
Common Discrete Distributions
The Poisson distribution is the Binomial distribution as If then If n is large and p is small then the Binomial distribution can be approximated using the Poisson distribution
n→∞
Xn ~ Bin(n, p) Xnd! →! Pois(np)
λ: n→∞ ⇒ p→ 0fixed
Poisson is often more computationally convenient than Binomial
![Page 75: Probability Theory Random Variables and Distributions · 2014-03-10 · Probability Theory So where’s this all going? These examples are trivial! Suppose there are three bags, B](https://reader034.vdocument.in/reader034/viewer/2022042411/5f2a2ceb0523be312b1debd1/html5/thumbnails/75.jpg)
References
Countless books + online resources! Probability theory and distributions: • Grimmett and Stirzker (2001) Probability and Random
Processes. Oxford University Press. General comprehensive introduction to (almost) everything mathematics (including a bit of probability theory): • Garrity (2002) All the mathematics you missed: but need to
know for graduate school. Cambridge University Press.