Download - Image Enhancement [DVT final project]
DISP Lab, Graduate Institute of Communication Engineering, NTU
1
Image Enhancement [DVT final project]
Speaker: Yu-Hsiang WangAdvisor: Prof. Jian-Jung Ding
Digital Image and Signal Processing LabGraduate Institute of Communication Engineering
National Taiwan University
2
OutlineTargetPossible enhancement methodsInterpolation
◦Adaptive osculatory rational interpolationEnhancement
◦Bilateral Enhancers◦Non Local Means
ConclusionReference
3
TargetScale image from SD (Standard-
definition) to HD (High-definition).Given: 720x480, YCbCr, 4:2:2, 8
bits per channel.Target: 1920x1080, YCbCr, 4:4:4,
10 bits per channel.
4
Possible enhancement methodsSharpnessNoise reductionEdge smoothnessSkin-tone enhancementTexture enhancementSuper resolution (SR)
Multi-Image SR [1]
Single-Image Multi-Patch SR [1]
5
Interpolation: Bilinear1. Interpolate R1:2. Interpolate R2:3. Interpolate P:
2 11 11 21
2 1 2 1
x x x xf R f Q f Qx x x x
2 12 12 22
2 1 2 1
x x x xf R f Q f Qx x x x
2 11 2
2 1 2 1
y y y yf P f R f Ry y y y
[2]
6
Interpolation: Adaptive osculatory rational interpolationThe interpolation kernel function
of adaptive osculatory rational interpolation (AORI) is more accurately approximate to the ideal interpolation not only in space domain but also in frequency domain.
Apply 4 points to interpolate 1 point in one direction.
7
Interpolation: Adaptive osculatory rational interpolationThe interpolation function
◦where r(x) is the interpolated value, g(xk) are the sample values, RI(x) is the interpolation
kernel.
3
0
,k kk
r x g x RI x x
8
Interpolation: Adaptive osculatory rational interpolationThe interpolation kernel
[M. Hu and J. Q. Tan, ”Adaptive osculatory rational interpolation for image processing,” Journal of Computational and Applied Mathematics, vol. 195, pp. 46-53, 2006.]
2
2
2
2
0.168 0.9129 1.08081,
0.8319 1.0808
0.1953 0.5858 0.39051 2,
2.4402 1.7676
0 2 .
x xx
x x
x xRI x x
x x
x
9
Interpolation: Adaptive osculatory rational interpolationThe original 1920x1080 image:
10
Interpolation: Adaptive osculatory rational interpolationThe interpolated image by AORI:
11
Interpolation: Adaptive osculatory rational interpolationThe original 1920x1080 image:
12
Interpolation: Adaptive osculatory rational interpolationThe interpolated image by AORI:
13
Enhancement: Bilateral EnhancersExtended from the bilateral filter
(BF).BF:
◦A nonlinear filter adopts a low-pass Gaussian filter for both the domain filter and the range filter.
◦Smooth the noise while preserving edges.
14
Enhancement: Bilateral EnhancersBilateral filter (BF)
◦with the normalization factor
where r is the input image, h is the output image, Ω(x) is a subset of the input image r with
x and ξ are the pixels coordinates. (row, column)
[C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. ICCV, pp. 839–846, 1998.]
1
( ), , ,c s d
x
h x k x r x r r x
( )
, , ,c s d
xk x x r r x
5 5
15
Enhancement: Bilateral EnhancersBilateral filter (BF)
◦with the normalization factor
The function c operates on the spatial domain designed as
The s operates on the range domain designed as
1
( ), , ,c s d
x
h x k x r x r r x
( )
, , ,c s d
xk x x r r x
2, exp2 c
c
xx
2
2, exp2 s
s
r x rr r x
16
Enhancement: Bilateral EnhancersProblem of Bilateral filter:
◦Only edge-preserving and de-noising.
◦Do not enhance the sharpness of an image.
17
Enhancement: Bilateral EnhancersBilateral enhancers (sharpness +
smoothness)
◦The c function is set as
the same as in BF. (Gaussian function on spatial domain)
[C. Gatta and P. Radeva, “Bilateral enhancers,” IEEE ICIP, pp. 3161-3164, 2009.]
( )
, , ,c p d
xj x r x x r x r
1
( ), , ,c s d
x
h x k x r x r r xvs
2
1, exp22 cc
c
xx
18
Enhancement: Bilateral EnhancersBilateral enhancers (sharpness +
smoothness)
◦The p composes of two parts (p = ps + pe): the edge-preserving smoothing (ps)
the selective sharpening (pe)
( )
, , ,c p d
xj x r x x r x r
1
( ), , ,c s d
x
h x k x r x r r xvs
2
2, ,2s s
s
p exp
r x rr r x
2
2, 1 ,2e e
s
p exp
r x rr x r
19
Enhancement: Bilateral EnhancersThe edge-preserving smoothing
(ps)
◦ηs: adjusts the intensity of the blurring.
◦σs: controls how strong should be an edge to be preserved from the blurring.
If the intensity difference is small, Gaussian smoothing is performed.
2
2, ,2s s
s
p exp
r x rr r x
20
Enhancement: Bilateral EnhancersThe selective sharpening (pe)
◦ηe and σe have similar meaning as for the edge-preserving smoothing.
If the intensity difference is small, no enhancement is performed.
2
2, 1 ,2e e
s
p exp
r x rr x r
21
Enhancement: Bilateral EnhancersThe interpolated image by AORI:
22
Enhancement: Bilateral EnhancersEnhanced the previous page’s
image by BE.
23
Enhancement: Bilateral EnhancersThe original 1920x1080 image:
24
Enhancement: Bilateral EnhancersThe interpolated image by AORI:
25
Enhancement: Bilateral EnhancersEnhanced the previous page’s
image by BE.
26
Framework of System
Adaptive osculatory rational interpolation
Input 720 480 image I
Determine property of I by
DCT
Non local means for denoising
Bilateral enhancers for sharpness
1920 1080 image r
Restored 1920 1080 image
uniformnon-uniform
27
Enhancement: Non Local MeansPurpose: Noise reduction.Given an interpolated imageThe estimated value
◦W is a search window of fixed size (we choose 5x5 here) centered at pixel i.
◦The weights depend on the similarity between the pixel i and j.
[A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” IEEE CVPR, Jun. 2005.]
.r r i i I
, ,j W
NL r i i j r j
,j
i j
28
Enhancement: Non Local MeansThe weighted Euclidean distance
◦Nk denotes a square neighborhood of fixed size (we choose 3x3) and centered at a pixel k.
◦α is the standard deviation of the Gaussian kernel.
◦G(N) is the Gaussian kernel of the same size as Nk.
22
*i j i jr N r N r N r N G N
29
Enhancement: Non Local MeansThe weights are defined as
◦Z(i) is the normalizing constant
◦h denotes a degree of filtering. (It controls the decay of the weights function of the Euclidean distances.) (h = 2.5)
2
2
1, exp ,i jr N r N
i jZ i h
2
2expi j
j
r N r NZ i
h
30
Enhancement: Non Local MeansScheme of NL-means strategy.
[6]
31
Enhancement: Non Local MeansAdvantage:
◦NL-means compares the gray level of pixels.
◦Compare the geometrical configuration in a whole neighborhood.
Disadvantage:◦Do not perform sharpness.◦Blur some edges.
32
Enhancement: Non Local MeansThe interpolated image by AORI:
33
Enhancement: Non Local MeansDe-noise the previous image by NL-
means.
34
Enhancement: Non Local MeansThe interpolated image by AORI:
35
Enhancement: Non Local MeansDe-noise the previous image by NL-
means.
36
Enhancement: Non Local MeansEnhanced the previous page’s
image by BE.
37
ConclusionAdaptive osculatory rational
interpolation is more accurately approximate to the ideal interpolation.
Bilateral enhancers performs well at sharpness and smoothness.
Non local means is mainly used for noise reduction.
38
Reference [1] D. Glasner, S. Bagon, and M. Irani, “Super-resolution
from a single image,” ICCV, pp. 349-356, Sep. 2009. [2]http://en.wikipedia.org/wiki/Bilinear_interpolation [3] M. Hu and J. Q. Tan, ”Adaptive osculatory rational
interpolation for image processing,” Journal of Computational and Applied Mathematics, vol. 195, pp. 46-53, 2006.
[4] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. ICCV, pp. 839–846, 1998.
[5] C. Gatta and P. Radeva, “Bilateral enhancers,” IEEE ICIP, pp. 3161-3164, 2009.
[6] A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image denoising,” IEEE CVPR, Jun. 2005.