the markov chain monte carlo method isabelle stanton may 8, 2008 theory lunch
TRANSCRIPT
![Page 1: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/1.jpg)
The Markov Chain Monte Carlo Method
Isabelle StantonMay 8, 2008 Theory Lunch
![Page 2: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/2.jpg)
Monte Carlo vs Las Vegas
Las Vegas Algorithms are randomized and always give the correct results but gamble with computation time
Quicksort
Monte Carlo algorithms have fixed running time but may be wrong
Simulated Annealing Estimating volume
![Page 3: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/3.jpg)
Markov Chains
a memoryless stochastic process, eg, flipping a coin
56
4
32
1
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/6 1/6
1/6
1/6 1/6
1/6
![Page 4: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/4.jpg)
Other Examples of Markov Chains
Shuffling cards Flipping a coin PageRank Model Particle systems – focus of MCMC work
![Page 5: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/5.jpg)
General Idea
Model the system using a Markov Chain Use a Monte Carlo Algorithm to perform some
computation task
![Page 6: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/6.jpg)
Applications
Approximate Counting - # of solutions to 3-SAT or Knapsack
Statistical Physics – when do phase transitions occur?
Combinatorial optimization – simulated annealing type of algorithms
We'll focus on counting
![Page 7: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/7.jpg)
Monte Carlo Counting How do you estimate the volume
of a complex solid? Render with environment maps
efficiently? Estimate an integral numerically?
![Page 8: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/8.jpg)
(Picnic) Knapsack
Holds 20
weighs 4
weighs 10
weighs 4
weighs 2
weighs 5
What is a solution?How many solutions are there?
![Page 9: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/9.jpg)
Counting Knapsack Solutions
Item weights: a = (a0,...a
n)
Knapsack size: a real number b Estimate the number of {0,1} vectors, x, that
satisfy a*x ≤ b
Let N denote the number of solutions
![Page 10: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/10.jpg)
Naїve Solution
Randomly generate x Calculate a*x If a*x ≤ b return 2n
else return 0
This will return N in expectation: 0*(2n-N) + N*2n / 2n
![Page 11: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/11.jpg)
Is this fast?
Counterexample: a = (1, ... 1) and b = n/3 Any solution has less than n/3 1's There are (n choose n/3)*2n/3 solutions
![Page 12: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/12.jpg)
no
Pr(sample x, ||x|| ≤ n/3) < (n choose n/3)*2-2n/3
In expectation, need to generate 2n/3 x's before we get a single solution!
Any polynomial number of trials will grossly underestimate N
![Page 13: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/13.jpg)
Knapsack with MCMC
Let Mknap
be a markov chain withstate space Ω(b) = {x | a*x ≤ b}
This will allow us to sample a solution
![Page 14: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/14.jpg)
Various Mknap
000
001 010 100
011 101 110
111
a=(0,.5,.5) b = 1.5
a=(0,1,1) b = 1.5
001 010 100
110101011
000000
001 010 100
110101
![Page 15: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/15.jpg)
Mknap
Transitions
Transitions With probability 1/2, x transitions to x Otherwise, select an i u.a.r.
from 0 to n-1 and flip
the ith bit of x.
If x' is a
solution,
transition there.
000
001 010 100
011 101 110
111
001 010 100
110101
000000
001 010 100
110101
a=(0,1,1) b = 1.5
0.5
0.5
0.50.5
0.50.5
1/6 1/61/6
1/61/6
1/61/6
![Page 16: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/16.jpg)
Connected?
Is Mknap
connected?
Yes. To get from x to x' go through 0.
![Page 17: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/17.jpg)
Ergodicity
What is the stationary distribution of Knapsack? Sample each solution with prob 1/N
A MC is ergodic if the probability distribution over the states converges to the stationary distribution of the system, regardless of the starting configuration
Is Mknap
ergodic? Yes.
![Page 18: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/18.jpg)
Algorithm Idea
Start at 0 and simulate Mknap
for enough steps that the distribution over the states is close to uniform
Why does uniformity matter? Does this fix the problem yet?
![Page 19: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/19.jpg)
The trick
Assume that a0 ≤ a
1 ... ≤ a
n (0,1,2,…,n-1,n)
Let b0 = 0 and b
i = min{b, Σia
j}
|Ω(bi-1
)| ≤ |Ω(bi)| - why?
|Ω(bi)| ≤ (n+1)|Ω(b
i-1)| - why?
Change any element of Ω(bi) to one of Ω(bi-1) by switching
the rightmost 1 to a 0
![Page 20: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/20.jpg)
How does that help?
|Ω(b)| = |Ω(bn)| = |Ω(b
n)|/|Ω(b
n-1)| x
|Ω(bn-1
)|/|Ω(bn-2
)| x ... x |Ω(b1)|/Ω|(b
0)| x |Ω(b
0)|
We can estimate each of these ratios by doing a walk on Ω(b
i) and computing the fraction of
samples in Ω(bi-1
)
Good estimate since
|Ω(bi-1
)| ≤ |Ω(bi)| ≤ (n+1)|Ω(b
i-1)|
![Page 21: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/21.jpg)
Analysis
Ignoring bias, the expectation of each trial is |Ω(b
i-1)|/|Ω(b
i)|
We perform t = 17ε-2n2 steps Focus on Var(X)/E(X)^2 in analyzing efficiency
for MCMC methods
![Page 22: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/22.jpg)
Analysis
If Z is the product of the trials, E[Z] = П |Ω(b
i-1)|/|Ω(b
i)|
*Magic Statistics Steps* Var(Z)/(E[Z])2 ≤ ε2/16 By Chebyshev's:
Pr[(1-ε/2)|Ω(b)| ≤ Z ≤ (1+ε/2)|Ω(b)| ] ≥ 3/4
![Page 23: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/23.jpg)
Analysis
We used nt = 17ε-2n3 steps This is a FPRAS (Fully Polynomial Randomized
Approximation Scheme) Except... what assumption did I make?
![Page 24: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/24.jpg)
Mixing Time
Assumption: We are close to the uniform distribution in 17ε-2n2 steps
This is known as the mixing time It is unknown if this distribution mixes in
polynomial time
![Page 25: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/25.jpg)
Mixing Time
What does mix in polynomial time? Dice – 1 transition Shuffling cards – 7 shuffles ferromagnetic Ising model at high temperature –
O(nlog n) What doesn't?
ferromagnetic Ising model at low temperature – starts to form magnets
![Page 26: The Markov Chain Monte Carlo Method Isabelle Stanton May 8, 2008 Theory Lunch](https://reader036.vdocument.in/reader036/viewer/2022062409/5697bfef1a28abf838cba100/html5/thumbnails/26.jpg)
Wes Weimer Memorial Conclusion Slide
The markov chain monte carlo
method models the problem
as a Markov Chain and then
uses random walks Mixing time is important P# problems are hard Wes likes trespassing