lecture 4a : probability theory review
DESCRIPTION
Advanced Artificial Intelligence. Lecture 4A : Probability Theory Review. Outline. Axioms of Probability Product and chain rules Bayes Theorem Random variables PDFs and CDFs Expected value and variance. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/1.jpg)
Lecture 4A: Probability Theory Review
Advanced Artificial Intelligence
![Page 2: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/2.jpg)
Outline
• Axioms of Probability• Product and chain rules • Bayes Theorem• Random variables• PDFs and CDFs• Expected value and variance
![Page 3: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/3.jpg)
Introduction
• Sample space - set of all possible outcomes of a random experiment– Dice roll: {1, 2, 3, 4, 5, 6}– Coin toss: {Tails, Heads}
• Event space - subsets of elements in a sample space– Dice roll: {1, 2, 3} or {2, 4, 6}– Coin toss: {Tails}
![Page 4: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/4.jpg)
• Axioms of Probability– for all
![Page 5: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/5.jpg)
examples
• Coin flip– P(H)– P(T)– P(H,H,H)– P(x1=x2=x3=x4)– P({x1,x2,x3,x4} contains more than 3 heads)
![Page 6: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/6.jpg)
Set operations
• Let and • and
• Properties:
• If then
![Page 7: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/7.jpg)
Conditional Probability
• A and B are independent if
• A and B are conditionally independent given C if
![Page 8: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/8.jpg)
Conditional Probability
• Joint probability:
• Product rule:
• Chain rule of probability:
![Page 9: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/9.jpg)
examples
• Coin flip– P(x1=H)=1/2– P(x2=H|x1=H)=0.9– P(x2=T|x1=T)=0.8– P(x2=H)=?
![Page 10: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/10.jpg)
Conditional Probability
• probability of A given B• Example:– probability that school is closed – probability that it snows– probability of school closing if it snows– probability of snowing if school is closed
![Page 11: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/11.jpg)
Conditional Probability
• probability of A given B• Example:– probability that school is closed – probability that it snows
P(A, B) 0.005
P(B) 0.02
P(A|B) 0.25
𝑃 ( 𝐴∨𝐵 )= 𝑃 ( 𝐴∩𝐵 )𝑃 (𝐵 )
![Page 12: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/12.jpg)
Quiz
• P(D1=sunny)=0.9• P(D2=sunny|D1=sunny)=0.8• P(D2=rainy|D1=sunny)=?• P(D2=sunny|D1=rainy)=0.6• P(D2=rainy|D1=rainy)=?• P(D2=sunny)=?• P(D3=sunny)=?
![Page 13: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/13.jpg)
Joint Probability
• Multiple events: cancer, test result
13
Has cancer? Test positive? P(C,TP)
yes yes 0.018
yes no 0.002
no yes 0.196
no no 0.784
![Page 14: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/14.jpg)
Joint Probability
• The problem with joint distributions
It takes 2D-1 numbers to specify them!
14
![Page 15: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/15.jpg)
Conditional Probability
• Describes the cancer test:
• Put this together with: Prior probability
15
![Page 16: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/16.jpg)
Has cancer? Test positive?
P(TP, C)
yes yes
yes no
no yes
no no
Has cancer? Test positive? P(TP, C)
yes yes 0.018
yes no 0.002
no yes 0.196
no no 0.784
Conditional Probability
• We have:
• We can now calculate joint probabilities
16
![Page 17: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/17.jpg)
Conditional Probability
• “Diagnostic” question: How likely do is cancer given a positive test?
17
Has cancer? Test positive? P(TP, C)
yes yes 0.018
yes no 0.002
no yes 0.196
no no 0.784
![Page 18: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/18.jpg)
Bayes Theorem
• We can relate P(A|B) and P(B|A) through Bayes’ rule:
![Page 19: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/19.jpg)
Bayes Theorem
• We can relate P(A|B) and P(B|A) through Bayes’ rule:
Posterior Probability Likelihood
Normalizing Constant
Prior Probability
![Page 20: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/20.jpg)
Bayes Theorem
• We can relate P(A|B) and P(B|A) through Bayes’ rule:
• can be eliminated with law of total probability:
=
![Page 21: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/21.jpg)
Random Variables
• Random variable X is a function s.t.
• Examples:– Russian roulette: X = 1 if gun fires and X = 0 otherwise and – X = # of heads in 10 coin tosses
![Page 22: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/22.jpg)
Cumulative Distribution Functions
• Defined as such that • Properties:
![Page 23: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/23.jpg)
Probability Density Functions
• For discrete random variables, defined as:
• Relates to CDFs:
• =
![Page 24: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/24.jpg)
Probability Density Functions
• For continuous random variables, defined as:
• Relates to CDFs:
![Page 25: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/25.jpg)
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle
![Page 26: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/26.jpg)
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle f(X)
1/2
2
X
![Page 27: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/27.jpg)
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle f(X)
1/2
2
X
![Page 28: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/28.jpg)
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle
• for
f(x)
1/2
2
x
F(x)
1
2
x
![Page 29: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/29.jpg)
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle
• for • =
f(x)
1/2
2
x
F(x)
1
2
x
![Page 30: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/30.jpg)
Expectation
• Given a discrete r.v. X and a function
• For a continuous r.v. X and a function
![Page 31: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/31.jpg)
Expectation
• Expectation of a r.v. X is known as the first moment, or the mean value, of that variable ->
• Properties:• for any constant • for any constant • +
![Page 32: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/32.jpg)
Variance
• For a random variable X, define:
• Another common form:
• Properties:• for any constant • for any constant
![Page 33: Lecture 4A : Probability Theory Review](https://reader030.vdocument.in/reader030/viewer/2022033021/568130ae550346895d96be0d/html5/thumbnails/33.jpg)
Gaussian Distributions
• Let , then
where represents the mean and the variance• Occurs naturally in many phenomena– Noise/error– Central limit theorem