data-driven markov chain monte carlo presented by tomasz malisiewicztomasz malisiewicz for advanced...
Post on 22-Dec-2015
219 views
TRANSCRIPT
![Page 1: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/1.jpg)
Data-Driven Markov Chain Monte Carlo
Presented by Tomasz Malisiewiczfor Advanced Perception
3/1/2006
![Page 2: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/2.jpg)
Overview of Talk
• What is Image Segmentation?
• How to find a good segmentation?
• DDMCMC results
Image segmentation in a Bayesian statistical framework
Markov Chain Monte Carlo for exploring the space of all segmentations
Data-Driven methods for exploiting image data and speeding up MCMC
![Page 3: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/3.jpg)
DDMCMC Motivation
• Iterative approach: consider many different segmentations and keep the good ones
• Few tunable parameters, ex) # of segments encoded into prior
• DDMCMC vs Ncuts
![Page 4: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/4.jpg)
Berkeley Segmentation Database Image 326038
Berkeley Ncuts K=30 DDMCMC
![Page 5: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/5.jpg)
Why a rigorous formulation?
• Allows us to define what we want the segmentation algorithm to return
• Assigning a Score to a segmentation
![Page 6: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/6.jpg)
Formulation #1(and you thought you knew what image segmentation was)
• Image Lattice: • Image:• For any point either or
• Lattice partition into K disjoint regions:
• Region is discrete label map:• Region Boundary is Continuous:
An image partition intodisjoint regions is not
An image segmentation!Regions Contents Are Key!
![Page 7: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/7.jpg)
Formulation #2(and you thought you knew what image segmentation was)
• Each Image Region is a realization from a probabilistic model
• are parameters of model indexed by• A segmentation is denoted by a vector of hidden
variables W; K is number of regions
• Bayesian Framework:
Space of allsegmentations
PriorLikelihoodPosterior
![Page 8: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/8.jpg)
Prior over segmentations(do you like exponentials?)
•
~ uniform
# of modelparamsWant less regions
Want round-ish regions
Want small regions
Want less complex models
![Page 9: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/9.jpg)
Likelihood for Images
• Visual Patterns are independent stochastic processes
• is model-type index • is model parameter vector • is image appearance in i-th region
Grayscale
Color
![Page 10: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/10.jpg)
Four Gray-level Models
Uniform Clutter Texture Shading
• Gray-level model space:
Gaussian Intensity Histogram
FB ResponseHistogram
B-Spline
![Page 11: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/11.jpg)
Three Color Models (L*,u*,v*)
• Gaussian• Mixture of 2
Gaussians• Bezier Spline
• Color model space:
![Page 12: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/12.jpg)
Calibration
• Likelihoods are calibrated using empirical study• Calibration required to make likelihoods for
different models comparable (necessary for model competition)
Principled?or
Hack?
![Page 13: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/13.jpg)
What did we just do?
Def. of Segmentation:
Score (probability) of Segmentation:
Likelihood of Image = product of region likelihoods
Regions defined by k-partition:
![Page 14: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/14.jpg)
What do we do with scores?
Search
![Page 15: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/15.jpg)
Search through what? Anatomy of Solution Space
• Space of all k-partitions
• General partition space
• Space of all segmentations
Partitionspace
K Modelspaces
SceneSpace
or
![Page 16: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/16.jpg)
Searching through segmentations
Exhaustive Enumeration of all segmentations
Greedy Search (Gradient Ascent)
Stochastic Search
MCMC based exploration
Takes too long!
Local minima!
Takes too long
Described in the rest of this talk!
![Page 17: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/17.jpg)
Why MCMC
• What is it?
• What does it do?
-A clever way of searching through a high-dimensional space-A general purpose technique of generating samples from a probability
-Iteratively searches through space of all segmentations by constructinga Markov Chain which converges to stationary distribution
![Page 18: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/18.jpg)
![Page 19: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/19.jpg)
![Page 20: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/20.jpg)
Designing Markov Chains
• Three Markov Chain requirements
• Ergodic: from an initial segmentation W0, any other state W can be visited in finite time (no greedy algorithms); ensured by jump-diffusion dynamics
• Aperiodic: ensured by random dynamics
• Detailed Balance: every move is reversible
![Page 21: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/21.jpg)
5 Dynamics
1.) Boundary Diffusion
2.) Model Adaptation
3.) Split Region
4.) Merge Region
5.) Switch Region Model
At each iteration, we choose a dynamic with probability q(1),q(2),q(3),q(4),q(5)
![Page 22: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/22.jpg)
Dynamics 1: Boundary Diffusion
• Diffusion* within
Boundary Between
Regions i and j
Brownian MotionAlong
Curve Normal
TemperatureDecreases over
Time
*Movement within partition space
![Page 23: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/23.jpg)
Dynamics 2: Model Adaptation
• Fit the parameters* of a region by steepest ascent
*Movement within cue space
![Page 24: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/24.jpg)
Dynamics 3-4: Split and Merge
• Split one region into twoRemainingVariables
Areunchanged
Probability ofProposed Split
Conditional Probability of how likely chain proposes to move to W’ from W
Data-Driven Speedup
![Page 25: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/25.jpg)
Dynamics 3-4: Split and Merge
• Merge two RegionsRemainingVariables
Areunchanged
Probability ofProposed Merge
Data-Driven Speedup
![Page 26: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/26.jpg)
Dynamics 5: Model Switching
• Change models
• Proposal ProbabilitiesData-Driven Speedup
![Page 27: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/27.jpg)
Motivation of DD
• Region Splitting: How to decide where to split a region?
• Model Switching: Once we switch to a new model, what parameters do we jump to?
vs
Model Adaptation Required some initial parameter vector
![Page 28: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/28.jpg)
Data Driven Methods
• Focus on boundaries and model parameters derived from data: compute these before MCMC starts
• Cue Particles: Clustering in Model Space• K-partition Particles: Edge Detection
• Particles Encode Probabilities Parzen Window Style
![Page 29: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/29.jpg)
Cue Particles In ActionClustering in Color Space
![Page 30: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/30.jpg)
K-partition Particles in Action
• Edge detection gives us a good idea of where we expect a boundary to be located
![Page 31: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/31.jpg)
Particles or Parzen Window* Locations?
• What is this particle business about?
• A particle is just the position of a parzen-window which is used for density estimation
1D particles*Parzen Windowing also known as: Kernel Density Estimation, Non-parametric densityestimation
![Page 32: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/32.jpg)
Are you awake: What did we just do?
• Scores (Probability of Segmentation) Search
• 5 MCMC dynamics
• Data-Driven Speedup (key to making MCMC work in finite time)
So what type of answer does the Markov Chain return?
What can we do with this answer?How many answers to we want?
![Page 33: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/33.jpg)
Multiple Solutions
• MAP gives us one solution
• Output of MCMC sampling
How do we get multiple solutions?
Parzen Windows: Again Scene Particles
![Page 34: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/34.jpg)
Why multiple solutions?
• Segmentation is often not the final stage of computation
• A higher level task such as recognition can utilize a segmentation
• We don’t want to make any hard decision before recognition
• multiple segmentations = good idea
![Page 35: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/35.jpg)
K-adventurers
• We want to keep a fixed number K of segmentations but we don’t want to keep trivially different segmentations
• Goal: Keep the K segmentations that best preserve the posterior probability in KL-sense
• Greedy Algorithm:
- Add new particle, remove worst particle
![Page 36: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/36.jpg)
Results (Multiple Solutions)
![Page 37: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/37.jpg)
Results
![Page 38: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/38.jpg)
Results (Color Images)http://www.stat.ucla.edu/~ztu/DDMCMC/benchmark_color/benchmark_color.htm
![Page 39: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/39.jpg)
Conclusions
• DDMCMC: Combines Generative (top-down) and Discriminative (bottom-up) approaches
• Traverse the space of all segmentations via Markov Chains
• Does your head hurt? • Questions?
![Page 40: Data-Driven Markov Chain Monte Carlo Presented by Tomasz MalisiewiczTomasz Malisiewicz for Advanced PerceptionAdvanced Perception 3/1/2006](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d7a5503460f94a5e4e0/html5/thumbnails/40.jpg)
References
• DDMCMC Paper: http://www.cs.cmu.edu/~efros/courses/AP06/Papers/tu-pami-02.pdf
• DDMCMC Website: http://www.stat.ucla.edu/%7Eztu/DDMCMC/DDMCMC_segmentation.htm
• MCMC Tutorial by Authors: http://civs.stat.ucla.edu/MCMC/MCMC_tutorial.htm