lecture 4: maximum likelihood · lecture 4: maximum likelihood. review – common distributions...
TRANSCRIPT
![Page 1: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/1.jpg)
Lecture 4:Maximum Likelihood
![Page 2: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/2.jpg)
Review – Common Distributions
● Uniform● Normal● Lognormal● Beta● Exponental/Laplace● Gamma
● Binomial● Bernoulli● Poisson● Negative Binomial● Geometric
Continuous Discrete
![Page 3: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/3.jpg)
What are we trying to do?
● “Confronting models with data”● How is the data modeled?
– What type of data?
– What process generated this data?
– What distributions are an appropriate description of the data?
● How is the process modeled?● How are the parameters modeled?
![Page 4: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/4.jpg)
![Page 5: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/5.jpg)
Why are we trying to do this?
● Quantify states & relationships– What is Y?
– How is Y related to X?
● Test Hypotheses● Prediction● Decision making
![Page 6: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/6.jpg)
![Page 7: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/7.jpg)
How do we do this?
![Page 8: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/8.jpg)
Likelihood
● Probability of observing a given data point x conditional on parameter value
● Likelihood principle: a parameter value is more likely than another if it the one for which the data are more probably
L=P X=x∣=P data∣model
![Page 9: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/9.jpg)
Example – Mortality Rate
● Assume mortality rate is constant – – but is an UNKNOWN we want to estimate
● ai is a KNOWN time of death
Pr aaiaa = Pr die now given that
plant is still alive ⋅Pr plant is
still alive≈ a × e−a
= Expa∣a
![Page 10: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/10.jpg)
L=Pr a∣∝Exp a∣
![Page 11: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/11.jpg)
An Observation● A plant is observed to die on day 10
● From this observation, what is the best estimate for ?
![Page 12: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/12.jpg)
A few things to note
● A likelihood surface is NOT a PDF
● Pr(X | ) ≠ Pr( | X)
● Does not integrate to 1● No, you can't just normalize it● The model parameter is being varied, not the
random variable– i.e. the x-axis is fixed, not random
● Cannot interpret surface in terms of it's mean, variance, quantiles
![Page 13: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/13.jpg)
Maximum Likelihood
● Step 1: Write a likelihood function describing the likelihood of the observation
● Step 2: Find the value of the model parameter that maximized the likelihood
dLd
=0
![Page 14: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/14.jpg)
L = e−a
ln L = ln−a
∂ ln L∂
=1−a=0
ML =1a= 0.1day−1
![Page 15: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/15.jpg)
L = e−a
ln L = ln−a
∂ ln L∂
=1−a=0
ML =1a= 0.1day−1
ln e−a
ln ln e−a
ln −a
![Page 16: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/16.jpg)
L = e−a
ln L = ln−a
∂ ln L∂
=1−a=0
ML =1a= 0.1day−1
∂[ ln −a]∂
∂ ln ∂
−∂a∂
1−a
![Page 17: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/17.jpg)
L = e−a
ln L = ln−a
∂ ln L∂
=1−a=0
ML =1a= 0.1day−1
1=a
1=a1a=
a = 10
![Page 18: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/18.jpg)
![Page 19: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/19.jpg)
● Suppose a second plant dies at day 14● Step 1: Define the likelihood
A second data point
L = Pr a1,a2∣
= Pr a2∣a1,Pr a1∣
= Pr a2∣Pr a1∣
= Exp a2∣Exp a1∣
Assume measurements are independent
![Page 20: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/20.jpg)
● Step 2: Find the maximum
L = e−a1⋅e−a2
ln L = 2 ln−a1−a2
∂ ln L∂
=2−a1a2=0
ML =2
a1a2
= 0.0833day−1
![Page 21: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/21.jpg)
![Page 22: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/22.jpg)
![Page 23: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/23.jpg)
A whole data set
● Step 1: Define Likelihood
L = Pr a1,a2,⋯, an∣
= ∏i=1
n
Pr ai∣
= ∏i=1
n
Exp ai∣
Assume measurements are independent
![Page 24: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/24.jpg)
● Step 2: Find the maximum
L = ∏i=1
n
e−ai
ln L = ∑i=1
n
ln−ai
= n ln−∑i=1
n
ai
∂ ln L∂
=n−∑
i=1
n
ai=0
ML =n
∑i=1
n
ai
= 1/a
![Page 25: Lecture 4: Maximum Likelihood · Lecture 4: Maximum Likelihood. Review – Common Distributions Uniform Normal Lognormal Beta Exponental/Laplace Gamma Binomial Bernoulli Poisson](https://reader036.vdocument.in/reader036/viewer/2022070815/5f0f42257e708231d44344f6/html5/thumbnails/25.jpg)