mean-field theory and its applications in computer vision2 1
TRANSCRIPT
![Page 1: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/1.jpg)
Mean-Field Theory and Its Applications In Computer Vision2
1
![Page 2: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/2.jpg)
Problem Formulation
2
Grid CRF
construction
Grid CRF leads to over smoothing around boundariesDense CRF is able to recover fine boundaries
Dense CRF construction
![Page 3: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/3.jpg)
Long Range Interaction
3
• Able to recover proper flow for objects• Teddy arms recovered using Global interaction
image Local interaction Global interaction Ground truth
Optical flow
![Page 4: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/4.jpg)
Marginal Update
4
• Marginal Update for large neighbourhood:
Very Expensive Step (O(n2))
![Page 5: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/5.jpg)
Inference in Dense CRF
5
• Time complexity increases• Neighbourhood size• MCMC takes 36 hours on 50K variables• Graph-cuts based algorithm takes hours
![Page 6: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/6.jpg)
Inference in Dense CRF
6
• Time complexity increases• Neighbourhood size• MCMC takes 36 hours on 50K variables• Graph-cuts based algorithm takes hours
• Not practical for vision applications
![Page 7: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/7.jpg)
Inference in Dense CRF
7
• Time complexity increases• Neighbourhood size• MCMC takes 36 hours on 50K variables• Graph-cuts based algorithm takes hours• Filter-based Mean-field Inference takes 0.2 secs
• Possibility of development of many exciting vision applications
![Page 8: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/8.jpg)
Efficient inference
8
• Assume Gaussian pairwise weight
Label compatibility function
![Page 9: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/9.jpg)
Efficient inference
9
• Assume Gaussian pairwise weight
Mixture of Gaussians
Bilateral Spatial
![Page 10: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/10.jpg)
Bilateral filter
10
pp
output input
pp
reproducedfrom [Durand 02]
S
IIIGGW
IBFq
qqpp
p qp ||||||1
][rs
output input
![Page 11: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/11.jpg)
Marginal update
11
• Assume Gaussian pairwise weight
![Page 12: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/12.jpg)
How does it work
12
Very Expensive Step (O(n2))
![Page 13: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/13.jpg)
Message passing from all Xj to all Xi
13
Accumulates weights from all other pixels except itself
![Page 14: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/14.jpg)
Message passing from all Xj to all Xi
14
Convert as Gaussian filtering step: Accumulate weights from all other pixels except itself
![Page 15: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/15.jpg)
Message passing from all Xj to all Xi
15
Convert as Gaussian filtering step: Accumulate weights from all other pixels except itself
![Page 16: Mean-Field Theory and Its Applications In Computer Vision2 1](https://reader036.vdocument.in/reader036/viewer/2022062417/5515eb0f550346d46f8b5147/html5/thumbnails/16.jpg)
Efficient filtering steps
16
Now discuss how to do efficient filtering step