m 15338 : depth map estimation software version 2
DESCRIPTION
M 15338 : Depth Map Estimation Software version 2. Olgierd Stankiewicz Krzysztof Wegner team supervisor: Marek Domański Chair of Multimedia Telecommunications and Microelectronics Poznan University of Technology, Poland. April, 27th 2008, Archamps. Outline. Depth map quality measurement - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/1.jpg)
1
M15338: Depth Map Estimation
Software version 2
April, 27th 2008, Archamps
Olgierd StankiewiczKrzysztof Wegner
team supervisor: Marek DomańskiChair of Multimedia Telecommunications and Microelectronics
Poznan University of Technology, Poland
![Page 2: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/2.jpg)
2
Outline Depth map quality measurement
Ground-truth map View resynthesis
View synthesis tool Depth map estimation tools
Belief Propagation based estimation Accuracy refinement by mid-level
hypothesis Summary
![Page 3: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/3.jpg)
3
Depth map quality
Commonly used: ‘Bad-Pixels’
Miss information about error magnitude and energy
Requires ground-truth disparity map
thresholdyxdyxG
thresholdyxdyxGyxpixelbad
),(),(0
),(),(1),(
![Page 4: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/4.jpg)
4
Depth map quality NBP-SAD (Normalized Bad Pixel SAD)
NBP-SSD (Normalized Bad Pixel SSD)
Still, requires ground-truth disparity map
pixelsbadyx
yxdyxGpixelsbadofcount
SADNBP,
),(),(1
pixelsbadyx
yxdyxGpixelsbadofcount
SSDNBP,
2),(),(
1
![Page 5: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/5.jpg)
5
Depth map quality measurement by view resynthesis
End-user never sees depth-map Resynthesis
No standarized method Tool employs straight-forward method
PSNR (Peak Signal-to-Noise Ratio)of resynthesized view as quality measure
![Page 6: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/6.jpg)
6
Bad-Pixels vs PSNRBad-pixels vs PSNR of resynthesis
HSABM+OF [1]ThreeViewBP [9]
AdaptingBP [4]
SubPixDoubleBP [6]
AdaptOvrSegBP [7]
PlaneFitBP [8]
Our proposal BP
Double BP[5]
SSD+MF [3]
29
30
31
32
33
34
35
36
0,00% 1,00% 2,00% 3,00% 4,00% 5,00% 6,00% 7,00% 8,00% 9,00%
Bad-pixels [%]
PS
NR
[d
B]
![Page 7: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/7.jpg)
7
View synthesis tool
Simple and straight-forward For linearly positioned stereo pairs
only Two disparity maps and
corresponding reference views Weighting of pixels from side-views,
translated according to their disparity
![Page 8: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/8.jpg)
8
View synthesis tool
![Page 9: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/9.jpg)
9
Belief Propagation based depth estimation tool
Alternative for Hierarchical-Shape Adaptive Block Matching
Employs message passing for optimization of disparity map
hierarchical processing in layers
Pixel differences (1-point SAD) used as observations
![Page 10: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/10.jpg)
10
Message passing in Belief Propagation
mt s→d – message passed in t-th iteration from node s to node d, V(fp,fq) – cost of belief change from disparity fp to disparity fq.
![Page 11: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/11.jpg)
11
Message in Belief Propagation
Single message contains information about all possible disparities
![Page 12: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/12.jpg)
12
Hierarchical processing in Belief Propagation
Higher resolution
Lower resolution
from the lowest resolution to the full resolution in
coarse-to-fine manner
![Page 13: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/13.jpg)
13
Belief propagation
Vpq(xp,xq) – transition cost in node q between disparity xp and xq
insisted by nodeł p
Vp(xp) – observation in node p about disparity xp (SAD value)mpq(xq) – message from node p to q about disparity xq
![Page 14: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/14.jpg)
14
Belief propagation
Pot model Simpleand computationally efficient .
Stable beliefs are prefered
![Page 15: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/15.jpg)
15
Belief propagation results
1th iteration 20 iterations 300 iterations
Middlebury test results – 1,65% of bad-pixelsBest Middlebury algorithm – 0,88% of bad-pixels
![Page 16: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/16.jpg)
16
Bad-Pixels vs PSNRBad-pixels vs PSNR of resynthesis
HSABM+OF [1]ThreeViewBP [9]
AdaptingBP [4]
SubPixDoubleBP [6]
AdaptOvrSegBP [7]
PlaneFitBP [8]
Our proposal BP
Double BP[5]
SSD+MF [3]
29
30
31
32
33
34
35
36
0,00% 1,00% 2,00% 3,00% 4,00% 5,00% 6,00% 7,00% 8,00% 9,00%
Bad-pixels [%]
PS
NR
[d
B]
![Page 17: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/17.jpg)
17
Accuracy refinement by mid-level hypothesis
Low computational cost Improves accuracy of disparity
map (number of disparity levels) Spatial resolution unchanged Focuses on unit-step edges in
disparity map
![Page 18: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/18.jpg)
18
Unit-step edges
![Page 19: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/19.jpg)
19
Mid-level hypothesis
Hypothesis spread along unit-step edges
![Page 20: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/20.jpg)
20
Refinement by mid-level hypothesis
Pixel accurate disparity (1x) After refinement (8x)
![Page 21: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/21.jpg)
21
Works over untextured regions
![Page 22: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/22.jpg)
22
ResultsChange of bad-pixels relative to (x1) during iterative refinement
0,00%
20,00%
40,00%
60,00%
80,00%
100,00%
120,00%
x1 x2 x4 x8 x16
Bad
-pix
els
chan
ge
![Page 23: M 15338 : Depth Map Estimation Software version 2](https://reader035.vdocument.in/reader035/viewer/2022062517/56813adb550346895da32586/html5/thumbnails/23.jpg)
23
Summary
New version of experimental Depth Estimation software
Quality measurement problem with respect to multi-view applications
Simple view resynthesis tool Belief Propagation depth estimation
tool Novel technique for accuracy
refinement