Week6JeongminLee

ComputerScienceDepartmentUniversityofPittsburgh

CS1675IntrotoMachineLearning– Recitation

Homework3

• Averagescore:85.27(std:20.34)•Median:92.50• Theproblemthatmanystudenthadmistake• DeriveMLestimateforexponentialdistribution(3b)• Creating3dplotforGaussiandistribution

Homework3

• DeriveMLestimateforexponentialdistribution(3b)

DensityEstimation

• GivenasetofobservationsX,estimateaprobabilitydistributionthatgeneratedX• (Assumption:)ObservationXisgeneratedfromanunknownprobabilitydistributionp(X)

TwoMethods

• MaximumLikelihoodEstimation(ML)<- ourfocus• BayesianParameterEstimation• MaximumAPosterioriEstimation(MAP)

MaximumLikelihoodEstimation(ML)

• Goal:Maximizethelikelihoodofdata

• Logismonotonicallyincreasingfunction

Θ"# = 𝑎𝑟𝑔𝑚𝑎𝑥*+,*-𝑝(𝐷|Θ, 𝜁)

Θ"# = 𝑎𝑟𝑔𝑚𝑎𝑥*+,*-𝑝 𝐷 Θ, 𝜁

= 𝑎𝑟𝑔𝑚𝑎𝑥*+,*- log 𝑝 𝐷 Θ, 𝜁

MaximumLikelihoodEstimation(ML)

©Hauskrecht

(Binomialdistribution)

MaximumLikelihoodEstimation(ML)

©Hauskrecht

Now,youcanjustplugyourobservations(N1,andN2)intothisequation=Knowingtheparameter=Knowingtheprobabilitydistribution

DeriveMLestimateforExponentialDist.

• ExponentialDistribution:

• MaximumLikelihood(ML)estimateofb:

• Log-likelihood:

©RachelMisbin

DeriveMLestimateforExponentialDist.

• Let’ssimplifyit:

(logofprod=sumoflog)

©RachelMisbin

DeriveMLestimateforExponentialDist.

• Optimizelog-likelihoodbytakingpartialderivativew.r.t b

©RachelMisbin

𝛿log(𝑥)𝛿𝑥

=1𝑥

𝛿(1/𝑥)𝛿𝑥

=1𝑥<

DeriveMLestimateforExponentialDist.

• Setthepartialderivativetozero:

• Solveforb:

©RachelMisbin

Homework3

• Creating3dplotforGaussiandistribution

Creating3dplotforGaussiandistribution

• Meanandcovariancewehave:

mean = [3.6377, 7.8506];cov =[3.6414,1.0779;1.0779,3.7831];

Creating3dplotforGaussiandistribution

• CreateXandYcoordinategrid

x=-5:0.1:15;y =-5:0.1:15;[X,Y]=meshgrid(x,y);%meshgrid:replicatestheinputgridvectorstoasetofcoordinatestorectangulargrid[X,Y]

Creating3dplotforGaussiandistribution

• ComputeZaxisusingmvnpdf

Z=mvnpdf([X(:)Y(:)],mean,cov);

• ChangetheformofZinto2Dgrid

Z=reshape(Z,length(y),length(x));

Creating3dplotforGaussiandistribution

• Figureon3dsurfaceusingsurffunction:

figure;surf(x,y,Z);

Thanks!-

Questions?