![Page 1: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/1.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions
Vladimir Kolmogorov, Ramin Zabih
![Page 2: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/2.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
2/21
Evelyn Gutschier, Markus Sareika
Overview:
Traditional Stereo Methods Energy Minimization via Graph Cuts Stereo with Occlusions Voxel Labeling Algorithm Pixel Labeling Algorithm Results and Conclusions
![Page 3: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/3.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
3/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo Methods
pixel correspondences labeling (disparity)
Traditional Stereo Problem
![Page 4: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/4.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
4/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo Methods Disparity
depth
disparity
disparity ~ depthground truth disparity
![Page 5: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/5.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
5/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo MethodsBinocular Stereo
goal is to compute pixels correspondences traditional stereo problem pixel labeling problem advantage: can be solved by graph cuts problem is formulated as energy term new goal: find the minimizing labeling
![Page 6: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/6.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
6/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo MethodsEnergy Function
E fp P
D p f pp , q N
V p , q f p , f q
f f 1 , ... , f P
cost for assigning labels smoothness term
find labeling that minimizes
we assign the label to pixel p when p of image I corresponds to p + in I‘f p
f p
![Page 7: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/7.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
7/21
Evelyn Gutschier, Markus Sareika
Traditional Stereo MethodsEnergy Function
D p f p I p I ' p f p ²
V f p , f q T f p f q
other models:absolute distancequadratic
• data cost – gives penalty for different intensities
• smoothness term – gives penalty for discontinuities (Potts model)
V min K , f p f qV min K , f p f q ²
![Page 8: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/8.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
8/21
Evelyn Gutschier, Markus Sareika
Energy Minimization via Graph Cuts
Max-flow / Min-Cut(Ford and Fulkerson Algorithm, Push-Relabel Method)
![Page 9: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/9.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
9/21
Evelyn Gutschier, Markus Sareika
Energy Minimization via Graph Cuts convex V vs.
metric / semimetric α-β-swap move α-expansion move: assigning label α to an
arbitrary set of pixels
Initial Labeling α-expansionα-β-swap
V , V ,V , 0
V , V , V ,
![Page 10: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/10.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
10/21
Evelyn Gutschier, Markus Sareika
Stereo with Occlusions
![Page 11: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/11.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
11/21
Evelyn Gutschier, Markus Sareika
Stereo with Occlusions
treat input symmetrically scene elements only visible in single view
physically correct scenes geometric constraints occlusions physically possible labelings
introduce constraints in the problem formulation
graph cuts perform unconstrained energy minimization
![Page 12: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/12.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
12/21
Evelyn Gutschier, Markus Sareika
Voxel Labeling Algorithm
discrete scene of voxels
voxel v is active when visible from both cameras
uniqueness constraint – 1:1 correspondence of pixels
![Page 13: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/13.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
13/21
Evelyn Gutschier, Markus Sareika
Voxel Labeling AlgorithmEnergy Function
E g E data g E occ g E smooth g
C occ P occ gmatching penalty
(only active voxels)
occlusion penalty set of occluded pixels
smoothness term(Potts model)
![Page 14: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/14.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
14/21
Evelyn Gutschier, Markus Sareika
Pixel Labeling AlgorithmEnergy Function
E f E data f E smooth f
E datav p , q N
f p f q d v D vlike traditional stereo but for both images e.g. Potts modelactive ?
![Page 15: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/15.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
15/21
Evelyn Gutschier, Markus Sareika
Minimizing the Energy
convert constrained into unconstrained minimization problem write as sum over pairs form of energy function = standard stereo problem
minimization with α-expansion algorithm modified definition of α-expansion move for voxel labeling
E g E data g E smooth g E valid g(0=valid, else ∞)
uniqueness
![Page 16: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/16.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
16/21
Evelyn Gutschier, Markus Sareika
Results and Conclusions
ground truth
voxel labeling pixel labelingtraditional s.p.
Tsukuba ref. image
![Page 17: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/17.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
17/21
Evelyn Gutschier, Markus Sareika
Results and Conclusions
efficient energy minimization polynominal time instead of exponential time
traditional stereo algorithm is faster pixel labeling better than voxel labeling:
prohibits ‚holes‘ in the scene allows to use other effective smoothness terms
algorithms can be extended for multiple cameras
![Page 18: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/18.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
18/21
Evelyn Gutschier, Markus Sareika
Multi-view Stereo via Volumetric Multi-view Stereo via Volumetric Graph CutsGraph Cuts
![Page 19: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/19.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
19/21
Evelyn Gutschier, Markus Sareika
Recent Work
Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions, 2006 TUW
![Page 20: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/20.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
20/21
Evelyn Gutschier, Markus Sareika
Questions?
![Page 21: Graph Cut Algorithms for Binocular Stereo with Occlusions](https://reader033.vdocument.in/reader033/viewer/2022051517/568158c1550346895dc60be7/html5/thumbnails/21.jpg)
Graph Cut Algorithms for Binocular Stereo with Occlusions Math Basics for Vision and Graphis 2007
21/21
Evelyn Gutschier, Markus Sareika
References
M. Bleyer, M. Gelautz, „Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions“, 2007
Y. Boykov, O. Veksler, R. Zabih, „Fast Approximate Energy Minimization via Graph Cuts“, 2001
V. Kolmogorov, R. Zabih, „Graph Cut Algorithms for Binocular Stereo with Occlusions“, 2005
V. Kolmogorov, R. Zabih, „What energy functions can be minimized via graph cuts“, 2004
V. Kolmogorov, R. Zabih, „Generalized multi-camera scene reconstruction using graph cuts“, July 2003
V. Kolmogorov, R. Zabih, „Multi-camera Scene Reconstruction via Graph Cuts“, 2002 S. Seits, C. Dyer, „Photorealistic Scene Reconstruction by Voxel Coloring“, 1997 R.Szeliski, R. Zabih, „An Experimental Comparison of Stereo Algorithms“, 1999