separable bilateral filtering for fast video preprocessing
DESCRIPTION
T.Q. Pham and L.J. van Vliet, ICMEIEEE Int. Conf. on Multimedia & ExpoThe Netherlands, 2005.TRANSCRIPT
1
Tuan Q. Pham & Lucas J. van Vliet
Delft University of TechnologyThe Netherlands
Quantitative Imaging Group
Faculty of Applied Sciences
Separable bilateral filteringfor fast video preprocessing
ICME 2005IEEE International Conference on Multimedia & ExpoJuly 6-8, 2005, Amsterdam, The Netherlands
Contents
1. Bilateral filtering:
Edge-preserving filter
High computational complexity
2. Separable implementation:
Good approximation of the original filter
Linear computational complexity
3. Application to video preprocessing: Noise reduction Better compressed video
Gaussian filtering revisited
Gaussian filtering: weights depend on distance to the center pixel
Noisy step edge Gaussian weights Gaussian filtered result
* =
Noisy step edge
. =
Adaptive filtering: avoid filtering across edges
Edge-preserving weights Edge-presered filtered result
Gaussian vs bilateral filtering
Gaussian filtering: weights depend on distance to the center pixel
Noisy step edge Gaussian weights Gaussian filtered result
* =
Noisy step edge Edge-preserving weights Edge-presered filtered result
. =
Bilateral filtering: weights depend on both spatial closeness and photometric similarity
.
How bilateral filtering works?
The weight is product of two Gaussian weights:
Spatial proximity:
Tonal similarity:
Local modesafter
bilateral filtering
Every sample is a weighted average of its neighbors:
0 0
1( ) ( , ) . ( )
s
O s w s s I sK
202
( ) ( )exp
2
tt
I s I sw
. s tw w w2
02
( )exp
2
ss
s sw
Example: Gaussian filtering
Noisy input: PSNR = 39.1 dB Gaussian filtered: PSNR = 67.9 dB
Example: Bilateral filtering
Noisy input: PSNR = 39.1 dB Bilateral filtered: PSNR = 41.6 dB
Computational complexity
Bilateral filtering kernel is space-variant → complexity is:
where N: number of pixels in the image
m: size of filtering kernel (m ≈ 7 is good enough)
d: image dimensionality
Previous attempt for fast bilateral filtering:
Piecewise-linear: approximate bilateral filtering with M Gaussian filtering - Durand & Dorsey (SIGGRAPH 2002) → complexity is:
Our approach: separable bilateral filtering
dO Nm
log( ) .O N N M M ≈ 17 for 8-bit images
O Nmd
Is bilateral filter separable?
Gaussian filter is space-invariant and separable:
=*
x
g(x)
y
g(y)
Bilateral filter is NOT separable: Space-variant kernel due to local
intensity dependency
I(s0)
I(s)
Kernel center
0
00
( ) . ( )( )
( )
s
s
ss
w s s I sO s
w s s
( ) gauss( ) . gauss( )sw s x y
However, even a highly non-linear filter like median filter is approximately separable (Narendra – PAMI 1981)
Separable bilateral filtering result
noisy Erika σnoise = 10 bilateral filtered in x-dimension followed by y-dimension filtering
Separable bilateral filtering is a good approximation of full kernel filtering:
Separable bilateral filtering result
noisy Erika σnoise = 10 bilateral filtered in x-dimension followed by y-dimension filtering
Separable bilateral filtering is a good approximation of full kernel filtering:
Image size Brute-force Durrand 2002 Separable Aniso. diffusion
256x256 4.46 0.37 0.21 2.76
512x512 17.88 1.59 0.89 26.02
61x61x61 5.29 4.20 0.45 5.25
256x256x212 56 min Out of memory 50.3 Out of memory
Pros: extremely fast (fixed spatial weight + LUT for tonal weight) Cons: effective filtering kernel is a slightly distorted
How separable bilateral filtering works?
Performance of separable bilateral filtering
Very good approximation of the full-kernel filter
Almost linear execution time per pixel
Better RMSE is achieved with separable bilateral filtering compared to full-kernel bilateral filtering with the same computation requirement
MPEG-1 Foreman with bilateral preprocessing
0 400 800 120013
14
15
16
bit-rate (K bits/s)
RM
SE
without preprocessingwith 3x3x3 full-kernelwith 9x9x5 separable
Less artifact with Bilateral preprocessing
Conclusions
Separable implementation of bilateral filtering:
Very good approximation of the original filter
Much faster than the original or other approximations
Applications in video preprocessing:
Improved quality of compressed video
Reduced processing time → real-time possibility
Literature
C. Tomasi and R. Manduchi, Bilateral fitering for gray and color images, Proc. of ICCV, USA, 1998, 839-846.
T.Q. Pham and L.J. van Vliet, Separable bilateral filtering for fast video preprocessing, Proc. of ICME’05.
F. Durrand and J. Dorsey, Fast bilateral filtering for the display of high dynamic range images, Proc. of SIGGRAPH’02, 2002, 844-847.
P. Perona and J. Malik, Scale-space filtering and edge detection using anisotropic diffusion, PAMI, vol. 12, no. 7, 1990, 629-639.
R. v.d. Boomgaard and J. v.d. Weijer. On the equivalence of local-mode finding, robust estimation and mean-shift analysis as used in early vision tasks. In Proc. of ICPR, pages 927-930, 2002.