10: the normal (gaussian) distributionweb.stanford.edu/class/cs109/lectures/10_normal_gaussian...did...
TRANSCRIPT
10: The Normal (Gaussian) DistributionLisa Yan and Jerry Cain
October 5, 2020
1
Lisa Yan and Jerry Cain, CS109, 2020
Quick slide reference
2
3 Normal RV 10a_normal
15 Normal RV: Properties 10b_normal_props
21 Normal RV: Computing probability 10c_normal_prob
30 Exercises LIVE
Normal RV
3
10a_normal
Lisa Yan and Jerry Cain, CS109, 2020
Todayβs the Big Day
4
Today
Lisa Yan and Jerry Cain, CS109, 2020
def An Normal random variable π is defined as follows:
Other names: Gaussian random variable
Normal Random Variable
5
π π₯ =1
π 2ππβ π₯βπ 2/2π2
π~π©(π, π2)Support: ββ,β
Variance
Expectation
πΈ π = π
Var π = π2
π~π©(π, π2)mean
variance
π₯
ππ₯
Lisa Yan and Jerry Cain, CS109, 2020
Carl Friedrich Gauss
Carl Friedrich Gauss (1777-1855) was a remarkably influentialGerman mathematician.
Did not invent Normal distribution but rather popularized it6
Lisa Yan and Jerry Cain, CS109, 2020
Why the Normal?
β’ Common for natural phenomena: height, weight, etc.
β’ Most noise in the world is Normal
β’ Often results from the sum of many random variables
β’ Sample means are distributed normally
7
Thatβs what they
want you to believeβ¦
Lisa Yan and Jerry Cain, CS109, 2020
Why the Normal?
β’ Common for natural phenomena: height, weight, etc.
β’ Most noise in the world is Normal
β’ Often results from the sum of many random variables
β’ Sample means are distributed normally
8
Actually log-normal
Just an assumption
Only if equally weighted
(okay this one is true, weβll see
this in 3 weeks)
Lisa Yan and Jerry Cain, CS109, 2020
0
0.05
0.1
0.15
0.2
0.25
0 β¦ 44 48 52 56 60 64 β¦ 900 β¦ 44 48 52 56 60 64 β¦ 90
Okay, so why the Normal?
Part of CS109 learning goals:
β’ Translate a problem statement into a random variable
In other words: model real life situations with probability distributions
9
value
How do you model student heights?
β’ Suppose you have data from one classroom.
Fits perfectly!
But what about in
another classroom?
Lisa Yan and Jerry Cain, CS109, 2020
A Gaussian maximizes entropy for a
given mean and variance.
Part of CS109 learning goals:
β’ Translate a problem statement into a random variable
In other words: model real life situations with probability distributions
0
0.05
0.1
0.15
0.2
0.25
0 β¦ 44 48 52 56 60 64 β¦ 900 β¦ 44 48 52 56 60 64 β¦ 90
Okay, so why the Normal?
10
Occamβs Razor:
βNon sunt multiplicanda
entia sine necessitate.β
Entities should not be multiplied
without necessity.
value
How do you model student heights?
β’ Suppose you have data from one classroom.
β’ Same mean/var
β’ Generalizes well
Lisa Yan and Jerry Cain, CS109, 2020
I encourage you to stay critical of how
to model real-world phenomena.
Why the Normal?
β’ Common for natural phenomena: height, weight, etc.
β’ Most noise in the world is Normal
β’ Often results from the sum of many random variables
β’ Sample means are distributed normally
11
Actually log-normal
Just an assumption
Only if equally weighted
(okay this one is true, weβll see
this in 3 weeks)
Lisa Yan and Jerry Cain, CS109, 2020
Anatomy of a beautiful equation
Let π~π© π, π2 .
The PDF of π is defined as:
12
π π₯ =1
π 2ππβ
π₯ β π 2
2π2
normalizing constantexponential
tail
symmetric
around π
variance π2
manages spread
π₯
ππ₯
Lisa Yan and Jerry Cain, CS109, 2020
Campus bikes
You spend some minutes, π, travelingbetween classes.
β’ Average time spent: π = 4 minutes
β’ Variance of time spent: π2 = 2 minutes2
Suppose π is normally distributed. What is the probability you spend β₯ 6 minutes traveling?
13
π~π©(π = 4, π2 = 2)
π π β₯ 6 = ΰΆ±6
β
π(π₯)ππ₯ = ΰΆ±6
β 1
π 2ππβ
π₯ β π 2
2π2 ππ₯
(call me if you analytically solve this)Loving, not scaryβ¦except this time
Lisa Yan and Jerry Cain, CS109, 2020
Computing probabilities with Normal RVs
For a Normal RV π~π© π, π2 , its CDF has no closed form.
π π β€ π₯ = πΉ π₯ = ΰΆ±ββ
π₯ 1
π 2ππβ
π¦ β π 2
2π2 ππ¦
However, we can solve for probabilities numerically using a function Ξ¦:
πΉ π₯ = Ξ¦π₯ β π
π
14
Cannot be
solved
analytically
β οΈ
CDF of
π~π© π, π2A function that has been
solved for numerically
To get here, weβll first
need to know some
properties of Normal RVs.
Normal RV: Properties
15
10b_normal_props
Lisa Yan and Jerry Cain, CS109, 2020
Properties of Normal RVs
Let π~π© π, π2 with CDF π π β€ π₯ = πΉ π₯ .
1. Linear transformations of Normal RVs are also Normal RVs.
If π = ππ + π, then π~π©(ππ + π, π2π2).
2. The PDF of a Normal RV is symmetric about the mean π.
πΉ π β π₯ = 1 β πΉ π + π₯
16
Lisa Yan and Jerry Cain, CS109, 2020
1. Linear transformations of Normal RVs
Let π~π© π, π2 with CDF π π β€ π₯ = πΉ π₯ .
Linear transformations of X are also Normal.
If π = ππ + π, then π~π© ππ + π, π2π2
Proof:
β’ πΈ π = πΈ ππ + π = ππΈ π + π = ππ + π
β’ Var π = Var ππ + π = π2Var π = π2π2
β’ π is also Normal
17
Proof in Ross,
10th ed (Section 5.4)
Linearity of Expectation
Var ππ + π = π2Var π
Lisa Yan and Jerry Cain, CS109, 2020
2. Symmetry of Normal RVs
Let π~π© π, π2 with CDF π π β€ π₯ = πΉ π₯ .
The PDF of a Normal RV is symmetric about the mean π.
πΉ π β π₯ = 1 β πΉ π + π₯
18
π(π₯)
π₯π
Lisa Yan and Jerry Cain, CS109, 2020
Using symmetry of the Normal RV
19
1. π π β€ π§
2. π π < π§
3. π π β₯ π§
4. π π β€ βπ§
5. π π β₯ βπ§
6. π(π¦ < π < π§) π€
A. πΉ π§
B. 1 β πΉ(π§)
C. πΉ π§ β πΉ(π¦)
= πΉ π§
π§
π(π§)
πΉ π β π₯ = 1 β πΉ π + π₯
π = 0
Let π~π© 0,1 with CDF π π β€ π§ = πΉ π§ .
Suppose we only knew numeric valuesfor πΉ π§ and πΉ π¦ , for some π§, π¦ β₯ 0.
How do we compute the following probabilities?
Lisa Yan and Jerry Cain, CS109, 2020
Using symmetry of the Normal RV
20
1. π π β€ π§
2. π π < π§
3. π π β₯ π§
4. π π β€ βπ§
5. π π β₯ βπ§
6. π(π¦ < π < π§)
A. πΉ π§
B. 1 β πΉ(π§)
C. πΉ π§ β πΉ(π¦)
= πΉ π§
= πΉ π§
= 1 β πΉ(π§)
= 1 β πΉ(π§)
= πΉ π§
= πΉ π§ β πΉ(π¦)
Symmetry is particularly useful when
computing probabilities of zero-mean
Normal RVs.
π§
π(π§)
πΉ π β π₯ = 1 β πΉ π + π₯
π = 0
Let π~π© 0,1 with CDF π π β€ π§ = πΉ π§ .
Suppose we only knew numeric valuesfor πΉ π§ and πΉ π¦ , for some π§, π¦ β₯ 0.
How do we compute the following probabilities?
Normal RV:Computing probability
21
10c_normal_probs
Lisa Yan and Jerry Cain, CS109, 2020
Computing probabilities with Normal RVs
Let π~π© π, π2 .
To compute the CDF, π π β€ π₯ = πΉ π₯ :
β’ We cannot analytically solve the integral (it has no closed form)
β’ β¦but we can solve numerically using a function Ξ¦:
πΉ π₯ = Ξ¦π₯ β π
π
22
CDF of the
Standard Normal, π
Lisa Yan and Jerry Cain, CS109, 2020
The Standard Normal random variable π is defined as follows:
Other names: Unit Normal
CDF of π defined as:
Standard Normal RV, π
23
π~π©(0, 1) Variance
Expectation πΈ π = π = 0
Var π = π2 = 1
π π β€ π§ = Ξ¦(π§)
Note: not a new distribution; just
a special case of the Normal
Lisa Yan and Jerry Cain, CS109, 2020
Ξ¦ has been numerically computed
24
π π β€ 1.31 = Ξ¦(1.31)
ππ§
π§
Ξ¦(π§)
Standard Normal Table only has
probabilities Ξ¦(π§) for π§ β₯ 0.
Lisa Yan and Jerry Cain, CS109, 2020
History fact: Standard Normal Table
25
The first Standard Normal Table was computed by Christian Kramp, French astronomer (1760β1826), in Analysedes RΓ©fractions Astronomiques et Terrestres, 1799
Used a Taylor series expansion to the third power
Lisa Yan and Jerry Cain, CS109, 2020
Probabilities for a general Normal RV
Let π~π© π, π2 . To compute the CDF π π β€ π₯ = πΉ π₯ ,we use Ξ¦, the CDF for the Standard Normal π~π©(0, 1):
πΉ π₯ = Ξ¦π₯ β π
πProof:
26
πΉ π₯ = π π β€ π₯
= π π β π β€ π₯ β π = ππ β π
πβ€π₯ β π
π
= π π β€π₯ β π
π
Algebra + π > 0
Definition of CDF
β’πβπ
π=
1
ππ β
π
πis a linear transform of π.
β’ This is distributed as π©1
ππ β
π
π,1
π2π2 =π© 0,1
β’ In other words, πβπ
π= π~π© 0,1 with CDF Ξ¦.= Ξ¦
π₯ β π
π
Lisa Yan and Jerry Cain, CS109, 2020
Probabilities for a general Normal RV
Let π~π© π, π2 . To compute the CDF π π β€ π₯ = πΉ π₯ ,we use Ξ¦, the CDF for the Standard Normal π~π©(0, 1):
πΉ π₯ = Ξ¦π₯ β π
πProof:
27
πΉ π₯ = π π β€ π₯
= π π β π β€ π₯ β π = ππ β π
πβ€π₯ β π
π
= π π β€π₯ β π
π
Algebra + π > 0
Definition of CDF
β’πβπ
π=
1
ππ β
π
πis a linear transform of π.
β’ This is distributed as π©1
ππ β
π
π,1
π2π2 =π© 0,1
β’ In other words, πβπ
π= π~π© 0,1 with CDF Ξ¦.= Ξ¦
π₯ β π
π
1. Compute π§ = π₯ β π /π.
2. Look up Ξ¦ π§ in Standard Normal table.
Lisa Yan and Jerry Cain, CS109, 2020
Campus bikes
You spend some minutes, π, traveling between classes.β’ Average time spent: π = 4 minutesβ’ Variance of time spent: π2 = 2 minutes2
Suppose π is normally distributed. What is the probability you spend β₯ 6 minutes traveling?
28
π~π©(π = 4, π2 = 2) π π β₯ 6 = ΰΆ±6
β
π(π₯)ππ₯ (no analytic solution)
1. Compute π§ =π₯βπ
π2. Look up Ξ¦(π§) in table
π π β₯ 6 = 1 β πΉπ₯(6)
= 1 β Ξ¦6 β 4
2
Γ
β 1 βΞ¦ 1.41
1 β Ξ¦ 1.41β 1 β 0.9207= 0.0793
Lisa Yan and Jerry Cain, CS109, 2020
Is there an easier way? (yes)
Let π~π© π, π2 . What is π π β€ π₯ = πΉ π₯ ?
β’ Use Python
β’ Use website tool
29
from scipy import statsX = stats.norm(mu, std)X.cdf(x)
SciPy reference:https://docs.scipy.org/doc/scipy/refere
nce/generated/scipy.stats.norm.html
Website tool: https://web.stanford.edu/class/cs109
/handouts/normalCDF.html