probability theory random variables and distributions
DESCRIPTION
Probability Theory Random Variables and Distributions. Rob Nicholls MRC LMB Statistics Course 2014. Introduction. Introduction. Significance magazine (December 2013) Royal Statistical Society. Introduction. Significance magazine (December 2013) Royal Statistical Society. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/1.jpg)
Probability TheoryRandom Variables and
DistributionsRob Nicholls
MRC LMB Statistics Course 2014
![Page 2: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/2.jpg)
Introduction
![Page 3: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/3.jpg)
Introduction
Significance magazine (December 2013) Royal Statistical Society
![Page 4: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/4.jpg)
Introduction
Significance magazine (December 2013) Royal Statistical Society
![Page 5: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/5.jpg)
Introduction
Significance magazine (December 2013) Royal Statistical Society
![Page 6: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/6.jpg)
Contents• Set Notation• Intro to Probability Theory• Random Variables• Probability Mass Functions• Common Discrete Distributions
![Page 7: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/7.jpg)
Set NotationA set is a collection of objects, written using curly brackets {}
If A is the set of all outcomes, then:
A set does not have to comprise the full number of outcomes
E.g. if A is the set of dice outcomes no higher than three, then:
![Page 8: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/8.jpg)
Set NotationIf A and B are sets, then:
Complement – everything but A
Union (or)
Intersection (and)
Not
Empty Set
![Page 9: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/9.jpg)
Set NotationVenn Diagram:
AB
one
two, three six four five
![Page 10: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/10.jpg)
Set NotationVenn Diagram:
AB
one
two, three six four five
![Page 11: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/11.jpg)
Set NotationVenn Diagram:
AB
one
two, three six four five
![Page 12: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/12.jpg)
Set NotationVenn Diagram:
AB
one
two, three six four five
![Page 13: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/13.jpg)
Set NotationVenn Diagram:
AB
one
two, three six four five
![Page 14: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/14.jpg)
Set NotationVenn Diagram:
AB
one
two, three six four five
![Page 15: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/15.jpg)
Set NotationVenn Diagram:
AB
one
two, three six four five
![Page 16: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/16.jpg)
Set NotationVenn Diagram:
AB
one
two, three six four five
![Page 17: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/17.jpg)
Probability TheoryTo consider Probabilities, we need:
1. Sample space: Ω
2. Event space:
3. Probability measure: P
![Page 18: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/18.jpg)
Probability TheoryTo consider Probabilities, we need:
1. Sample space: Ω - the set of all possible outcomes
![Page 19: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/19.jpg)
Probability TheoryTo consider Probabilities, we need:
2. Event space: - the set of all possible events
![Page 20: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/20.jpg)
Probability TheoryTo consider Probabilities, we need:
3. Probability measure: P
P must satisfy two axioms:
Probability of any outcome is 1 (100% chance)
If and only if are disjoint
![Page 21: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/21.jpg)
Probability TheoryTo consider Probabilities, we need:
3. Probability measure: P
P must satisfy two axioms:
Probability of any outcome is 1 (100% chance)
If and only if are disjoint
![Page 22: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/22.jpg)
Probability TheoryTo consider Probabilities, we need:
1. Sample space: Ω2. Event space: 3. Probability measure: P
As such, a Probability Space is the triple: (Ω, ,P )
![Page 23: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/23.jpg)
Probability TheoryTo consider Probabilities, we need:
The triple: (Ω, ,P )
i.e. we need to know:1. The set of potential outcomes;2. The set of potential events that may occur; and3. The probabilities associated with occurrence of those
events.
![Page 24: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/24.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
![Page 25: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/25.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
![Page 26: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/26.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
![Page 27: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/27.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
A B
![Page 28: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/28.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
A B
![Page 29: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/29.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
A B
![Page 30: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/30.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
![Page 31: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/31.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
If then and
A B
![Page 32: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/32.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
If then and
A B
![Page 33: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/33.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
If then and
A B
![Page 34: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/34.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
If then and
A B
![Page 35: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/35.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
If then and
![Page 36: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/36.jpg)
Probability TheoryNotable properties of a Probability Space (Ω, ,P ):
If then and
![Page 37: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/37.jpg)
Probability TheorySo where’s this all going? These examples are trivial!
![Page 38: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/38.jpg)
Probability TheorySo where’s this all going? These examples are trivial!
Suppose there are three bags, B1, B2 and B3, each of which contain a number of coloured balls:• B1 – 2 red and 4 white• B2 – 1 red and 2 white• B3 – 5 red and 4 white
A ball is randomly removed from one the bags.The bags were selected with probability:• P(B1) = 1/3• P(B2) = 5/12• P(B3) = 1/4
What is the probability that the ball came from B1, given it is red?
![Page 39: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/39.jpg)
Probability Theory
Conditional probability:
Partition Theorem:
Bayes’ Theorem:
If the partition
![Page 40: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/40.jpg)
Random VariablesA Random Variable is an object whose value is determined by chance, i.e. random events
Maps elements of Ω onto real numbers, with corresponding probabilities as specified by P
![Page 41: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/41.jpg)
Random VariablesA Random Variable is an object whose value is determined by chance, i.e. random events
Maps elements of Ω onto real numbers, with corresponding probabilities as specified by P
Formally, a Random Variable is a function:
![Page 42: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/42.jpg)
Random VariablesA Random Variable is an object whose value is determined by chance, i.e. random events
Maps elements of Ω onto real numbers, with corresponding probabilities as specified by P
Formally, a Random Variable is a function:
Probability that the random variable X adopts a particular value x:
![Page 43: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/43.jpg)
Random VariablesA Random Variable is an object whose value is determined by chance, i.e. random events
Maps elements of Ω onto real numbers, with corresponding probabilities as specified by P
Formally, a Random Variable is a function:
Probability that the random variable X adopts a particular value x:
Shorthand:
![Page 44: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/44.jpg)
Random VariablesExample:
If the result is heads then WIN - X takes the value 1If the result is tails then LOSE - X takes the value 0
![Page 45: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/45.jpg)
Random VariablesExample:
Win £20 on a six, nothing on four/five, lose £10 on one/two/three
![Page 46: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/46.jpg)
Random VariablesExample:
Win £20 on a six, nothing on four/five, lose £10 on one/two/three
Note – we are considering the probabilities of events in
![Page 47: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/47.jpg)
Probability Mass FunctionsGiven a random variable:
The Probability Mass Function is defined as:
Only for discrete random variables
![Page 48: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/48.jpg)
Probability Mass FunctionsExample:
Win £20 on a six, nothing on four/five, lose £10 on one/two/three
![Page 49: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/49.jpg)
Probability Mass FunctionsNotable properties of Probability Mass Functions:
![Page 50: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/50.jpg)
Probability Mass FunctionsNotable properties of Probability Mass Functions:
Interesting note:If p() is some function that has the above two properties, then it is the mass function of some random variable…
![Page 51: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/51.jpg)
Mean:
For a random variable
Probability Mass Functions
![Page 52: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/52.jpg)
Probability Mass Functions
Mean:
For a random variable
Mean
![Page 53: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/53.jpg)
Probability Mass Functions
Mean:
For a random variable
Compare with:
Mean
![Page 54: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/54.jpg)
Probability Mass Functions
Mean:
For a random variable
Median: any m such that: and
MeanMedian
![Page 55: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/55.jpg)
Probability Mass Functions
Mean:
For a random variable
Median: any m such that: and
Mode: : most likely value
MeanMedianMode
![Page 56: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/56.jpg)
Common Discrete Distributions
![Page 57: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/57.jpg)
Common Discrete DistributionsThe Bernoulli Distribution:
p : success probability
![Page 58: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/58.jpg)
Common Discrete DistributionsThe Bernoulli Distribution:
p : success probability
Example:
Therefore
![Page 59: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/59.jpg)
Common Discrete DistributionsThe Binomial Distribution:
n : number of independent trialsp : success probability
![Page 60: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/60.jpg)
Common Discrete DistributionsThe Binomial Distribution:
n : number of independent trialsp : success probability
: probability of getting x successes out of n trials
: probability of x successes
: probability of (n-x) failures
: number of ways to achieve x successes and (n-x) failures (Binomial coefficient)
![Page 61: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/61.jpg)
Common Discrete DistributionsThe Binomial Distribution:
n : number of independent trialsp : success probability
![Page 62: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/62.jpg)
Common Discrete DistributionsThe Binomial Distribution:
n : number of independent trialsp : success probability
![Page 63: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/63.jpg)
Common Discrete DistributionsThe Binomial Distribution:
n : number of independent trialsp : success probability
![Page 64: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/64.jpg)
Common Discrete DistributionsThe Binomial Distribution:
n : number of independent trialsp : success probability
![Page 65: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/65.jpg)
Common Discrete DistributionsExample:
Number of heads in n fair coin toss trials
In general:
![Page 66: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/66.jpg)
Common Discrete DistributionsExample:
Number of heads in n fair coin toss trials
In general:
Notice:
![Page 67: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/67.jpg)
Common Discrete DistributionsThe Poisson Distribution:
λ : average number of counts (controls rarity of events)
Used to model the number of occurrences of an event that occur within a particular interval of time and/or space
![Page 68: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/68.jpg)
Common Discrete Distributions
• Want to know the distribution of the number of occurrences of an event Binomial?
• However, don’t know how many trials are performed – could be infinite!
• But we do know the average rate of occurrence:
The Poisson Distribution:
![Page 69: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/69.jpg)
Common Discrete DistributionsBinomial:
![Page 70: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/70.jpg)
Common Discrete DistributionsBinomial:
![Page 71: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/71.jpg)
Common Discrete DistributionsBinomial:
1
1
![Page 72: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/72.jpg)
Common Discrete DistributionsBinomial:
1
1
![Page 73: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/73.jpg)
Common Discrete DistributionsBinomial:
1
1
☐
![Page 74: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/74.jpg)
This is referred to as the:• “Poisson Limit Theorem” • “Poisson Approximation to the Binomial”• “Law of Rare Events”
Common Discrete DistributionsThe Poisson distribution is the Binomial distribution as
If then
If n is large and p is small then the Binomial distribution can be approximated using the Poisson distribution
fixed
Poisson is often more computationally convenient than Binomial
![Page 75: Probability Theory Random Variables and Distributions](https://reader035.vdocument.in/reader035/viewer/2022062315/568161d7550346895dd1dcd3/html5/thumbnails/75.jpg)
ReferencesCountless books + online resources!
Probability theory and distributions:
• Grimmett and Stirzker (2001) Probability and Random Processes. Oxford University Press.
General comprehensive introduction to (almost) everything mathematics (including a bit of probability theory):
• Garrity (2002) All the mathematics you missed: but need to know for graduate school. Cambridge University Press.