a simple gibbs sampler - university of michigancsg.sph.umich.edu/abecasis/class/815.23.pdf ·...
TRANSCRIPT
![Page 1: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/1.jpg)
A Simple Gibbs Sampler
Biostatistics 615/815Lecture 23
![Page 2: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/2.jpg)
Scheduling
Practice Exam
Review session• Next Monday, December 6
Final Assessment• Monday, December 13
![Page 3: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/3.jpg)
Optimization Strategies
Single Variable• Golden Search• Quadratic Approximations
Multiple Variables• Simplex Method• E-M Algorithm• Simulated Annealing
![Page 4: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/4.jpg)
Simulated Annealing
Stochastic Method
Sometimes takes up-hill steps• Avoids local minima
Solution is gradually frozen• Values of parameters with largest impact on
function values are fixed earlier
![Page 5: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/5.jpg)
Gibbs Sampler
Another MCMC Method
Update a single parameter at a time
Sample from conditional distribution when other parameters are fixed
![Page 6: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/6.jpg)
Gibbs Sampler Algorithm
steps. previousrepeat and Increment
),...,,...,,|( from for valueSample b. say update, tocomponent Selecting a.
:by valuesparameter ofset next theDefine
valuesparameter of choice particular aConsider
1121)1(
)(
t
θθθθθxθpθi
kiiit
i
t
+−+
θ
![Page 7: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/7.jpg)
Alternative Algorithm
steps. previousrepeat and Increment
),...,,|( from for valueSample z. ...
),...,,|( from for valueSample c.
),...,,|( from for valueSample b. in turn ,1 component,each Updatea.
:by valuesparameter ofset next theDefine
valuesparameter of choice particular aConsider
131)1(
312)1(
2
321)1(
1
)(
t
θθθxθpθ
θθθxθpθ
θθθxθpθ .. k
kkt
k
kt
kt
t
−+
+
+
θ
![Page 8: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/8.jpg)
Key Property:Stationary Distribution
ondistributiposterior their fromvaluesparameter sample sampler to Gibbs expect the we,Eventually
)|(~)|(~
fact...In
)|,...,,()|,...,(),,...,|(
as ddistribute is ),...,,(Then
)|,...,,(~),...,,( that Suppose
)1()(
21221
)()(2
)1(1
21)()(
2)(
1
xpxp
xpxpxp
xp
tt
kkk
tk
tt
kt
ktt
θθθθ +
+
⇒
= θθθθθθθθ
θθθ
θθθθθθ
![Page 9: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/9.jpg)
Gibbs Sampling for Mixture Distributions
Sample each of the mixture parameters from conditional distribution• Dirichlet, Normal and Gamma distributions are
typical
Simple alternative is to sample the origin of each observation• Assign observation to specific component
![Page 10: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/10.jpg)
Sampling A Component
Calculate the probability that the observation originated from each component…
… use random number(s) to assign component membership. How?
∑==
lljl
ijijj xf
xf,xiZ
),|(),|(
),,|Pr(ηφπ
ηφπηφπ
![Page 11: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/11.jpg)
C Code: Sampling A Componentint sample_group(double x, int k,
double * probs, double * mean, double * sigma){int group; double p = Random();double lk = dmix(x, k, probs, mean, sigma);
for (group = 0; group < k - 1; group++){double pgroup = probs[group] *
dnorm(x, mean[group], sigma[group])/lk;
if (p < pgroup) return group;
p -= pgroup;}
return k - 1;}
![Page 12: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/12.jpg)
Calculating Mixture Parameters
iiZj
iiji
iZjiji
ii
iZji
nxnxs
nxxnnp
n
j
j
j
⎟⎟⎠
⎞⎜⎜⎝
⎛−=
=
=
=
∑
∑
∑
=
=
=
:
22
:
:1 Before sampling a
new origin for an observation…
… update mixture parameters given current assignments
Could be expensive!
![Page 13: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/13.jpg)
C Code:Updating Parametersvoid update_estimates(int k, int n,
double * prob, double * mean, double * sigma,double * counts, double * sum, double * sumsq)
{int i;
for (i = 0; i < k; i++){prob[i] = counts[i]/ n;mean[i] = sum[i] / counts[i];sigma[i] = sqrt((sumsq[i] - mean[i]*mean[i]*counts[i])
/ counts[i] + 1e-7);}
}
![Page 14: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/14.jpg)
C Code:Updating Mixture Parameters IIvoid remove_observation(double x, int group,
double * counts, double * sum, double * sumsq){counts[group] --;sum[group] -= x;sumsq[group] -= x * x;}
void add_observation(double x, int group,double * counts, double * sum, double * sumsq)
{counts[group] ++;sum[group] += x;sumsq[group] += x * x;}
![Page 15: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/15.jpg)
Selecting a Starting State
Must start with an assignment of observations to groupings
Many alternatives are possible, I chose to perform random assignments with equal probabilities…
![Page 16: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/16.jpg)
C Code:Starting Statevoid initial_state(int k, int * group,
double * counts, double * sum, double * sumsq){int i;
for (i = 0; i < k; i++)counts[i] = sum[i] = sumsq[i] = 0.0;
for (i = 0; i < n; i++){group[i] = Random() * k;
counts[group[i]] ++;sum[group[i]] += data[i];sumsq[group[i]] += data[i] * data[i];}
}
![Page 17: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/17.jpg)
The Gibbs Sampler
Select initial state
Repeat a large number of times:• Select an element• Update conditional on other elements
If appropriate, output summary for each run…
![Page 18: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/18.jpg)
C Code:Core of The Gibbs Samplerinitial_state(k, probs, mean, sigma, group, counts, sum, sumsq);for (i = 0; i < 10000000; i++)
{int id = rand() % n;
if (counts[group[id]] < MIN_GROUP) continue;
remove_observation(data[id], group[id], counts, sum, sumsq);update_estimates(k, n - 1, probs, mean, sigma,
counts, sum, sumsq);group[id] = sample_group(data[id], k, probs, mean, sigma);add_observation(data[id], group[id], counts, sum, sumsq);
if ((i > BURN_IN) && (i % THIN_INTERVAL == 0))/* Collect statistics */
}
![Page 19: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/19.jpg)
Gibbs Sampler:Memory Allocation and Freeingvoid gibbs(int k, double * probs, double * mean, double * sigma)
{int i, * group = (int *) malloc(sizeof(int) * n);double * sum = alloc_vector(k);double * sumsq = alloc_vector(k);double * counts = alloc_vector(k);
/* Core of the Gibbs Sampler goes here */
free_vector(sum, k);free_vector(sumsq, k);free_vector(counts, k);free(group);}
![Page 20: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/20.jpg)
Example ApplicationOld Faithful Eruptions (n = 272)
Old Faithful Eruptions
Duration (mins)
Freq
uenc
y
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0
05
1015
20
![Page 21: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/21.jpg)
Notes …
My first few runs found excellent solutions by fitting components accounting for very few observations but with variance near 0
Why? • Repeated values due to rounding• To avoid this, set MIN_GROUP to 13 (5%)
![Page 22: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/22.jpg)
Gibbs Sampler Burn-InLogLikelihood
-450
-400
-350
-300
-250
0 2500 5000 7500 10000
Iteration
Log-
Like
lihoo
d
![Page 23: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/23.jpg)
Gibbs Sampler Burn-InMixture Means
1
2
3
4
5
0 2500 5000 7500 10000
Iteration
Mea
n
![Page 24: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/24.jpg)
Gibbs Sampler After Burn-InLikelihood
0 20 40 60 80 100
-300
-290
-280
-270
-260
-250
Iteration (Millions)
Like
lihoo
d
![Page 25: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/25.jpg)
Gibbs Sampler After Burn-InMean for First Component
0 20 40 60 80 100
1.80
1.85
1.90
1.95
Iteration (Millions)
Com
pone
nt M
ean
![Page 26: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/26.jpg)
Gibbs Sampler After Burn-In
0 20 40 60 80 100
23
45
6
Iteration (Millions)
Com
pone
nt M
ean
![Page 27: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/27.jpg)
Notes on Gibbs Sampler
Previous optimizers settled on a minimum eventually
The Gibbs sampler continues wandering through the stationary distribution…
Forever!
![Page 28: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/28.jpg)
Drawing Inferences…
To draw inferences, summarize parameter values from stationary distribution
For example, might calculate the mean, median, etc.
![Page 29: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/29.jpg)
Component Means
Mean
Den
sity
1 2 3 4 5 6
02
46
810
Mean
Den
sity
1 2 3 4 5 6
02
46
810
Mean
Den
sity
1 2 3 4 5 60
24
68
![Page 30: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/30.jpg)
Component Probabilities
Probability
Den
sity
0.0 0.2 0.4 0.6 0.8 1.0
02
46
8
Probability
Den
sity
0.0 0.2 0.4 0.6 0.8 1.0
02
46
810
Probability
Den
sity
0.0 0.2 0.4 0.6 0.8 1.0
01
23
45
6
![Page 31: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/31.jpg)
Overall Parameter Estimates
The means of the posterior distributions for the three components were:• Frequencies of 0.073, 0.278 and 0.648• Means of 1.85, 2.08 and 4.28 • Variances of 0.001, 0.065 and 0.182
Our previous estimates were:• Components contributing .160, 0.195 and 0.644• Component means are 1.856, 2.182 and 4.289• Variances are 0.00766, 0.0709 and 0.172
![Page 32: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/32.jpg)
Joint Distributions
Gibbs Sampler provides other interesting information and insights
For example, we can evaluate joint distribution of two parameters…
![Page 33: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/33.jpg)
Component Probabilites
0.05 0.10 0.15 0.20 0.25 0.30
0.05
0.10
0.15
0.20
0.25
0.30
Group 1 Probability
Gro
up 3
Pro
babi
lity
![Page 34: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/34.jpg)
Today …
Introduction to Gibbs sampling
Generates posterior distributions of parameters conditional on data
Provides insight into joint distributions
![Page 35: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/35.jpg)
A little bit of theory
Highlight connection between Simulated Annealing and the Gibbs sampler …
Fill in some details of the Metropolis algorithm
![Page 36: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/36.jpg)
Both Methods Are Markov Chains
The probability of any state being chosen depends only on the previous state
States are updated according to transition matrix with elements pij. This matrix defines important properties, including periodicity and irreducibility.
)|Pr(),...,|Pr( 110011 −−−− ====== nnnnnnnn iSiSiSiSiS
![Page 37: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/37.jpg)
Metropolis-HastingsAcceptance Probability
πππ
qqππ
aqπqπ
a
ππ
iSjSqq
i
j
jiiji
jij
iji
jijij
ji
nnij
of valuesactual not the known, bemust
ratio Only the
if ,1minor ,1min
:isy probabilit acceptance Hastings-Metropolis The
stateeach of iesprobabilit relative thebe and Let
)| propose(Let 1
=⎟⎟⎠
⎞⎜⎜⎝
⎛=⎟
⎟⎠
⎞⎜⎜⎝
⎛=
=== +
![Page 38: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/38.jpg)
Metropolis-Hastings Equilibrium
e.communicat tostatesall allowmust density proposal thehappen, toFor this
Pr whereondistributi mequilibriuan reach it will Chain, Markov a
update toalgorithm Hastings-Metropolis theuse weIf
ii)(S π==
![Page 39: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/39.jpg)
Gibbs Sampler
1,1min e,consequenc a As
that ensuressampler Gibbs The
=⎟⎟⎠
⎞⎜⎜⎝
⎛=
=
iji
jijij
jijiji
a
ππ
ππ
![Page 40: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/40.jpg)
Simulated Annealing
statesy probabilithigh given to are ghtslarger wei res, temperatulowAt flattened ison distributiy probabilit theratures,high tempeAt
with replace
,parameter re temperatuaGiven
1
1
)(
∑=
jj
iiiπ
τ
ττ
π
ππ
τ
![Page 41: A Simple Gibbs Sampler - University of Michigancsg.sph.umich.edu/abecasis/class/815.23.pdf · Simulated Annealing zStochastic Method zSometimes takes up-hill steps • Avoids local](https://reader033.vdocument.in/reader033/viewer/2022051523/5a7a5c287f8b9aec2d8c60fa/html5/thumbnails/41.jpg)
Additional Reading
If you need a refresher on Gibbs sampling• Bayesian Methods for Mixture Distributions
M. Stephens (1997)http://www.stat.washington.edu/stephens/
Numerical Analysis for Statisticians• Kenneth Lange (1999)• Chapter 24