![Page 1: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/1.jpg)
Mean-Field Theory and Its Applications In Computer Vision1
1
![Page 2: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/2.jpg)
Introduction
2
• Problem formulation
• Mean-field based inference method
• Strategy for incorporating different costs
![Page 3: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/3.jpg)
Labelling problem
3Stereo Object detection
Assign a label to each image pixel
Object segmentation
![Page 4: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/4.jpg)
Problem Formulation
Find a Labelling that maximize the conditional probability
4
![Page 5: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/5.jpg)
Inference
5
• T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001
• Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999
• Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-1999
• J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001
Message Passing• Besag. On the Statistical Analysis of Dirty
Pictures, JRSS, 1986• Boykov et al. Fast Approximate Energy
Minimization via Graph Cuts, PAMI 2001• Komodakis et al. Fast Approximate
Optimal Solutions for Single and Dynamic MRFs, CVPR, 2007
• Lempitsky et al. Fusion Moves for Markov Random Field Optimization, PAMI, 2010
Move-Making
• Chekuri et al. Approximation Algorithms for Metric Labelling, SODA, 2001
• M. Goemans et al. Improved Approximate Algorithms for Maximum-Cut, JACM, 1995
• M. Muramatsu et al. A New SOCP Relaxation for Max-Cut, JORJ, 2003
• RaviKumar et al. QP Relaxation for Metric Labelling, ICML 2006
Convex Relaxations• K. Alahari et.al. Dynamic Hybrid
Algorithms for MAP Inference, PAMI 2010
• P. Kohli et al. On Partial Optimality in Multilabel MRFs, ICML, 2008
• C. Rother et al. Optimizing Binary MRFs via Extended Roof Duality, CVPR, 2007
Other Algorithms
![Page 6: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/6.jpg)
Inference
6
• T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001
• Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999
• Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-99
• J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001
Message Passing
• Variational message passing algorithm• We focus on mean-field based inferenceWe focus on mean-field based inference
![Page 7: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/7.jpg)
Mean-field methods
7
• Intractable inference with distribution
P
• Approximate distribution from tractable family
• Mean-fields methods (Jordan et.al., 1999)
![Page 8: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/8.jpg)
Variational Inference
8
• Minimize the KL-divergence between Q and P
![Page 9: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/9.jpg)
Variational Inference
9
• Minimize the KL-divergence between Q and P
![Page 10: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/10.jpg)
Variational Inference
10
• Minimize the KL-divergence between Q and P
![Page 11: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/11.jpg)
Variational Inference
11
• Minimize the KL-divergence between Q and P
![Page 12: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/12.jpg)
Markov Random Field (MRF)
12
• Graph:
• A simple MRF
Product of potentials defined over cliques
![Page 13: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/13.jpg)
Markov Random Field (MRF)
13
• Graph:
• In general
Un-normalized part
![Page 14: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/14.jpg)
Energy minimization
14
• Potential and energy
![Page 15: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/15.jpg)
Variational Inference
15
Entropy of Q
Expectation of costunder Q distribution
![Page 16: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/16.jpg)
Naïve Mean Field
16
• Family : assume all variables are independent
![Page 17: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/17.jpg)
Variational Inference
17
• Shannon’s entropy decomposes
![Page 18: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/18.jpg)
Variational Inference
18
• Stationary point solution• Marginal update in mean-field
• Normalizing constant:
![Page 19: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/19.jpg)
Variational Inference
19
• Marginal for variable i taking label l
![Page 20: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/20.jpg)
Variational Inference
20
• Marginal for variable i taking label l
• An assignment of all variables in clique c
![Page 21: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/21.jpg)
Variational Inference
21
• Marginal for variable i taking label l
• An assignment of all variables in clique c
• An assignment of all variables apart from x_i
![Page 22: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/22.jpg)
Variational Inference
22
• Marginal for variable i taking label l
• An assignment of all variables in clique c
• An assignment of all variables apart from x_i
• Marginal distribution of all variables in c apart from x_i
![Page 23: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/23.jpg)
Variational Inference
23
• Marginal for variable i taking label l
• An assignment of all variables in clique c
• An assignment of all variables apart from x_i
• Marginal distribution of all variables in c apart from x_i
• Summation evaluates the expected value of cost over distribution Q given that x_i takes label l
![Page 24: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/24.jpg)
Simple Illustration
24
Naïve mean-field
approximation
![Page 25: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/25.jpg)
Mean-field algorithm
25
• Iterative algorithm• Iterate till convergence
• Update marginals of each variable in each iteration
![Page 26: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/26.jpg)
Q distribution
26
![Page 27: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/27.jpg)
Max posterior marginal (MPM)
27
• MPM with approximate distribution:
• Empirically achieves very high accuracy:
• MAP solution / most likely solution
![Page 28: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/28.jpg)
Structured Mean Field
28
• Naïve mean field can lead to poor solution• Structured (higher order) mean-field
![Page 29: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/29.jpg)
How to make a mean-field algorithm
29
• Pick a model• Unary, pairwise, higher order cliques
• Define a cost• Potts, linear truncated, robust PN
• Calculate the marginal • Calculate the expectation of cost defined
![Page 30: Mean-Field Theory and Its Applications In Computer Vision1 1](https://reader035.vdocument.in/reader035/viewer/2022062404/5515ea31550346d46f8b50ed/html5/thumbnails/30.jpg)
How to make a mean-field algorithm
30
• Use this plug-in strategy in many different models• Grid pairwise CRF• Dense pairwise CRF• Higher order model• Co-occurrence model • Latent variable model• Product label space