uniform sampling of polytopes and concentration of...
Post on 11-Jul-2020
2 Views
Preview:
TRANSCRIPT
-
Uniform Sampling of Polytopesand Concentration of Measure
Sam Power
Cambridge Centre for AnalysisCantab Capital Institute for the Mathematics of Information
sp825@cam.ac.uk
June 7, 2018
Sam Power (CCA) Geometric MCMC June 7, 2018 1 / 25
-
Overview
1 Problem Statement
2 Basic Methods of Solution
3 Markov Chain Approaches
4 New Insights and Algorithms
5 Conclusion and Future Directions
Sam Power (CCA) Geometric MCMC June 7, 2018 2 / 25
-
Task
Uniform Sampling from Polytopes
K = {x ∈ Rd : 〈ai, x〉 6 bi for 1 6 i 6 f}
Sam Power (CCA) Geometric MCMC June 7, 2018 3 / 25
-
Halfspace
Figure: A halfspace in 2d
Sam Power (CCA) Geometric MCMC June 7, 2018 4 / 25
-
Polytope via Halfspaces
Figure: Building a polygon by intersecting halfspaces
Sam Power (CCA) Geometric MCMC June 7, 2018 5 / 25
-
Low-Dimensional Approach
Figure: Uniform points in a circle: generate points in a square (easy), throw awaythe points that miss the circle (even easier!)
Sam Power (CCA) Geometric MCMC June 7, 2018 6 / 25
-
Ball Walk (Lovasz, Simonovits, ...)
Figure: At your current location ...
Sam Power (CCA) Geometric MCMC June 7, 2018 7 / 25
-
Ball Walk (Lovasz, Simonovits, ...)
Figure: Draw a ball of radius r around you and move uniformly within that ball.
Sam Power (CCA) Geometric MCMC June 7, 2018 8 / 25
-
Barrier Walks (Chen, Dwivedi, et al.)
Figure: Propose from a locally-informed ellipsoid instead
Sam Power (CCA) Geometric MCMC June 7, 2018 9 / 25
-
Hit-and-Run (Smith, Belisle, Romeijn)
Figure: Pick a direction uniformly at random, move uniformly along that line.
Sam Power (CCA) Geometric MCMC June 7, 2018 10 / 25
-
Hit-and-Run
Sam Power (CCA) Geometric MCMC June 7, 2018 11 / 25
-
Generalising Hit-and-Run
No ‘parameters’ to tune
· · · still subtle freedom in the algorithm
How to ‘hit’
How to ‘run’
Some work on ‘hit’ proposals (Kaufman & Smith, Polyak & Gryazina)
Almost no work on ‘run’ proposals.
Sam Power (CCA) Geometric MCMC June 7, 2018 12 / 25
-
Generalising Hit-and-Run
No ‘parameters’ to tune
· · · still subtle freedom in the algorithm
How to ‘hit’
How to ‘run’
Some work on ‘hit’ proposals (Kaufman & Smith, Polyak & Gryazina)
Almost no work on ‘run’ proposals.
Sam Power (CCA) Geometric MCMC June 7, 2018 12 / 25
-
Generalising Hit-and-Run
No ‘parameters’ to tune
· · · still subtle freedom in the algorithm
How to ‘hit’
How to ‘run’
Some work on ‘hit’ proposals (Kaufman & Smith, Polyak & Gryazina)
Almost no work on ‘run’ proposals.
Sam Power (CCA) Geometric MCMC June 7, 2018 12 / 25
-
Generalising Hit-and-Run
No ‘parameters’ to tune
· · · still subtle freedom in the algorithm
How to ‘hit’
How to ‘run’
Some work on ‘hit’ proposals (Kaufman & Smith, Polyak & Gryazina)
Almost no work on ‘run’ proposals.
Sam Power (CCA) Geometric MCMC June 7, 2018 12 / 25
-
Generalising Hit-and-Run
No ‘parameters’ to tune
· · · still subtle freedom in the algorithm
How to ‘hit’
How to ‘run’
Some work on ‘hit’ proposals (Kaufman & Smith, Polyak & Gryazina)
Almost no work on ‘run’ proposals.
Sam Power (CCA) Geometric MCMC June 7, 2018 12 / 25
-
Generalising Hit-and-Run
No ‘parameters’ to tune
· · · still subtle freedom in the algorithm
How to ‘hit’
How to ‘run’
Some work on ‘hit’ proposals (Kaufman & Smith, Polyak & Gryazina)
Almost no work on ‘run’ proposals.
Sam Power (CCA) Geometric MCMC June 7, 2018 12 / 25
-
General Setup
If we are
At x,
Sample direction u ∼ Hx, and
Move a distance t ∼ Rx,u in that direction,
we have transition kernel
fH,R(x→ y) =H(u) ·R(t)‖y − x‖d−1
where y = x+ tu.
Sam Power (CCA) Geometric MCMC June 7, 2018 13 / 25
-
Perfect Hit-and-Run
Observation: we can make this kernel independent of y:
Hx(u) ∝ a(x, u)d (1)Rx,u(t) ∝ td−1 · I[0 6 t 6 a(x, u)] (2)
Intuition: convex geometry
; perfect sampling!
· · · but, there’s a hitch
Sam Power (CCA) Geometric MCMC June 7, 2018 14 / 25
-
Perfect Hit-and-Run
Observation: we can make this kernel independent of y:
Hx(u) ∝ a(x, u)d (1)Rx,u(t) ∝ td−1 · I[0 6 t 6 a(x, u)] (2)
Intuition: convex geometry
; perfect sampling!
· · · but, there’s a hitch
Sam Power (CCA) Geometric MCMC June 7, 2018 14 / 25
-
Perfect Hit-and-Run
Observation: we can make this kernel independent of y:
Hx(u) ∝ a(x, u)d (1)Rx,u(t) ∝ td−1 · I[0 6 t 6 a(x, u)] (2)
Intuition: convex geometry
; perfect sampling!
· · · but, there’s a hitch
Sam Power (CCA) Geometric MCMC June 7, 2018 14 / 25
-
Perfect Hit-and-Run
Observation: we can make this kernel independent of y:
Hx(u) ∝ a(x, u)d (1)Rx,u(t) ∝ td−1 · I[0 6 t 6 a(x, u)] (2)
Intuition: convex geometry
; perfect sampling!
· · · but, there’s a hitch
Sam Power (CCA) Geometric MCMC June 7, 2018 14 / 25
-
Seeing a polygon from the inside
Sam Power (CCA) Geometric MCMC June 7, 2018 15 / 25
-
Seeing a polygon from the inside
Sam Power (CCA) Geometric MCMC June 7, 2018 16 / 25
-
Seeing a polygon from the inside
Sam Power (CCA) Geometric MCMC June 7, 2018 17 / 25
-
Seeing a polygon from the inside
Sam Power (CCA) Geometric MCMC June 7, 2018 18 / 25
-
Making it practical
Bottleneck is sampling H
Could try focus Markov chain methods on H, then t is easy
· · · but think about what H looks like
More tractable: use a finite direction set
Sam Power (CCA) Geometric MCMC June 7, 2018 19 / 25
-
Making it practical
Bottleneck is sampling H
Could try focus Markov chain methods on H, then t is easy
· · · but think about what H looks like
More tractable: use a finite direction set
Sam Power (CCA) Geometric MCMC June 7, 2018 19 / 25
-
Making it practical
Bottleneck is sampling H
Could try focus Markov chain methods on H, then t is easy
· · · but think about what H looks like
More tractable: use a finite direction set
Sam Power (CCA) Geometric MCMC June 7, 2018 19 / 25
-
Preferential Gibbs Sampler
Sam Power (CCA) Geometric MCMC June 7, 2018 20 / 25
-
Overcomplete Preferential Gibbs Sampler
Sam Power (CCA) Geometric MCMC June 7, 2018 21 / 25
-
Making it stochastic
Key difficulty is in picking good directions
Want algorithm to be insensitive to conditioning
So · · ·
Pick a random collection of directions at each step.
Sam Power (CCA) Geometric MCMC June 7, 2018 22 / 25
-
Making it stochastic
Key difficulty is in picking good directions
Want algorithm to be insensitive to conditioning
So · · ·
Pick a random collection of directions at each step.
Sam Power (CCA) Geometric MCMC June 7, 2018 22 / 25
-
Making it stochastic
Key difficulty is in picking good directions
Want algorithm to be insensitive to conditioning
So · · ·
Pick a random collection of directions at each step.
Sam Power (CCA) Geometric MCMC June 7, 2018 22 / 25
-
Making it stochastic
Key difficulty is in picking good directions
Want algorithm to be insensitive to conditioning
So · · ·
Pick a random collection of directions at each step.
Sam Power (CCA) Geometric MCMC June 7, 2018 22 / 25
-
Dynamic Compass
Sam Power (CCA) Geometric MCMC June 7, 2018 23 / 25
-
Future Directions
Understand tradeoffs in # of directions
Getting away from ‘bad’ points
Go adaptive?
Affine-invariant version
Sam Power (CCA) Geometric MCMC June 7, 2018 24 / 25
-
Thank you!
Sam Power (CCA) Geometric MCMC June 7, 2018 25 / 25
Problem StatementBasic Methods of SolutionMarkov Chain ApproachesNew Insights and AlgorithmsConclusion and Future Directions
fd@rm@0: fd@rm@1: fd@rm@2: fd@rm@3: fd@rm@4: fd@rm@5: fd@rm@6: fd@rm@7:
top related