cross-based local multipoint filtering jiangbo lu 1, keyang shi 2, dongbo min 1, liang lin 2, and...
TRANSCRIPT
Cross-Based Local Multipoint Filtering
Jiangbo Lu1, Keyang Shi2, Dongbo Min1,
Liang Lin2, and Minh N. Do3
1Advanced Digital Sciences Center, 2Sun Yat-Sen University,
3Univ. of Illinois at Urbana-Champaign
Computer Vision and Pattern Recognition(CVPR), 2012.1
Outline
• Introduction
• Related Work
• Proposed Algorithm
• Experimental Results
• Conclusion
2
Introduction
3
Background
• Edge-preserving smoothing filtering:
• A key component for many computer vision applications
• Goal :
• remove noise or fine details• the structure/edge should be well preserved
• Bilateral filter(BF), Guided filter(GF)
4
Objective
• Present a cross-based framework of performing local multipoint filtering efficiently.
• Two main steps:• 1) multipoint estimation• 2) aggregation
• CLMF-0、 CLMF-1
5
Guided Filter (GF)
fixed-sized square window
Cross-Based Local Multipoint Filtering(CLMF)
adaptive window size
Related work
6
Cross-based local support decision[19]
7
[19] K. Zhang, J. Lu, and G. Lafruit. Cross-based local stereo matching using orthogonal integral images. IEEE Trans. CSVT, 19(7):1073–1079, July 2009.
Bilateral Filter[15]
8
[15] C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In Proc. of ICCV, 1998.
Guided Filter[6]
9
[6] K. He, J. Sun, and X. Tang. Guided image filtering. In Proc. of ECCV, 2010.
Guided Filter[6]
10
Guided Filter[6]
•
11
ProposedAlgorithm
12
Definition
• Z : Filter input• I : Guidance image• Y : Filter output
• : estimation point
• : observation point(support pixel)
• Ωp : local support region of
• Wp : square window of a radius r
• {hp, hp, hp, hp } : cross skeleton13
0 1 2 3
Y
Z
Yi = Zi - ni
Yi = aIi + b
[19] :
If , =1
Otherwise, =0
Adaptive Scale Selection
• Decide for each direction an appropriate arm length
• Cross-based method[19]
• Running average of the intensity of all the pixels covered by the current span h
• More robust against the measurement noise
14p h span h (right arm)
Adaptive Scale Selection
• Gradient reversal artifact
15
Generalization of Local Multipoint Filtering
• Zero-order (order m = 0 ) or first-order polynomial(m=1) model:
•
•
• The model should be biased toward low-order polynomials to avoid over-fitting and gradient increase.
16
Generalization of Local Multipoint Filtering
• Zero-order (order m = 0 ) or first-order polynomial(m=1) model:
• Use “least squares” to fit the data (Similar with GF) :
• ϵ is a regularization parameter to discourage the choices of large (i≥1)17
Generalization of Local Multipoint Filtering
• Zero-order (order m = 0 ) or first-order polynomial(m=1) model:
• Solutions:
18
m=0
m=1
: the number of pixels in
: mean of I in
: variance of I in 2
• Guided Filter(GF) : multipoint estimates are averaged together
• CLMF : weighted averaged
Generalization of Local Multipoint Filtering
19
Summary & Comparison
• O(1) time linear regression and aggregation (independent of the window radius r)
20
ExperimentalResults
21
Implementation• Raw matching cost[9]:
• Winner-Take-All / Occlusion detection and filling[14]
• r = 17, R = 3, τ = 20, and τs = 20
22[9] X. Mei, X. Sun, M. Zhou, S. Jiao, H. Wang, and X. Zhang. On building an accurate stereo matching system on graphics hardware. In Proc. of GPUCV, 2011.[14] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz. Fast cost-volume filtering for visual correspondence and beyond. In Proc. of CVPR, 2011.
1.Scanline filling : the lowest disparity of the spatially closest nonoccluded pixel2.Median filter :
23
CLMF-1 CLMF-0Ground Truth
Experimental Results• Middlebury evaluation
24
Tsukuba
Rank:23
Conclusion
25
Conclusion
• Propose a generic framework of performing cross-based local multipoint filtering efficiently
• CLMF-0 and CLMF-1 find very competitive applications into many computer vision
• More generalized than GF
• Cross-based technique is very friendly for GPUs[20]
• Plan to map the filters onto GPUs for speedup
26
[20] K. Zhang, J. Lu, Q. Yang, G. Lafruit, R. Lauwereins, and L. V. Gool. Real-time and accurate stereo: A scalable approach with bitwise fast voting on CUDA. IEEE Trans. CSVT, 21(7):867–878, July 2011.
Full-Image Guided Filtering for Fast Stereo Matching
Qingqing Yang, Dongxiao Li, Member, IEEE,
Lianghao Wang, and Ming Zhang
IEEE SIGNAL PROCESSING LETTERS, VOL. 20, NO. 3, MARCH 201327
Outline
• Objective
• Proposed Algorithm
• Experimental Results
• Conclusion
28
Objective
• Propose a novel full-image guided filtering method
• A novel scheme called weight propagation is proposed to compute support weights.
• Edge-preserving
• Low-complexity
29
ProposedAlgorithm
30
Filter Modeling• C : filter input
• C’i : filter output at pixel i
• Wi.j : weight of pixel pair (i,j)
• Ni : normalizing constant
31
(p.q)Pi,j : adjacent nodes on the path Pi,j Tp.q(I) : propagation function
Ω : smoothness parameterBest path : minimum propagation weight → high complexity
Choose horizontal first policy
Implementation
• Two pass model• 1)Horizontal direction in separate rows• 2)the same way in separate columns
32Pr
Implementation
• Horizontal:
• For an element r in a row, the intermediate sum of weighted value :
33
Pr
u- : the left neighbor of uu+ : the right neighbor of u
Can be further accelerated by using the two-pass scan paradigm[15]The intermediate results are stored in two temporary arrays.
Implementation
• Horizontal:
• The scan process is a sequential computation of weighted cumulative sum:
• Simply computed by:
34
Pr
horizontal path → vertical pathreduce the complexity : 4 multiplication and 8 additions (each element)
AL : the weighted cumulative sums calculated from the left to right
Implementation
• Cost Volume C:
• CBT : BT measure[18]
• CGD : absolute difference of gradient
• Winner-Take-All:
• Post-processing:• Cross checking : occlusions / mismatch pixels are filled by the
lowest disparity value of the nearest non-occluded pixel• Weighted median filter
35
[18] S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Trans. Patt. Anal. Mach. Intell., vol. 20, no. 4, pp. 401–406, 1998.
Comparison• Employ as many related pixels as possible
• Important for cost filtering in large textureless regions
36
Bilateral filter
Proposed
Comparison
37
Bilateral filter
Proposed
ExperimentalResults
38
Experimental Results
• Core Duo 3.16 GHz CPU
• 2 GB 800MHz RAM
• No parallelism technique is utilized.
• The average runtime for cost-volume filtering : 68 ms (on the Middlebury benchmark data sets)
• 27 faster than the approach [13] using guided image filtering (1850 ms).
39
[13] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz. Fast cost-volume filtering for visual correspondence and beyond. In Proc. of CVPR, 2011.
40
Experimental Results
41
Experimental Results
42
Conclusion
43
Conclusion
• The novel weight propagation method ensures support elements are assigned.
• All elements in the input signal contribute to the filtering approach.
• Outperforms all local methods on the Middlebury benchmark in terms of both speed and accuracy
44