image blur estimation based on the average cone of ratio in ...sanja/papers/blurestimation/...select...
Post on 27-Mar-2021
0 Views
Preview:
TRANSCRIPT
Ghent University-TELIN/IPI-IBBT Image Processing and Interpretation Group
Image Blur Estimation Based on the Average Cone of Ratio in the Wavelet Domain
L. Platiša, N. Lukid, A. Pižurica, E. Vansteenkiste, W. Philips
7/10/2011 2 Blur Identification
• Introduction
• The proposed blur metric
• Experimental results
• Real-time implementation
• Conclusions
• Future research
Contents
7/10/2011 3 Blur Identification
• Blur is usually observed as a loss of image sharpness which corresponds to smoothening of the edges.
• Why blur evaluation?
– assessment of image quality
– image restoration
• Standard observation model
Introduction
),(),(*),(),( yxnyxfyxhyxg
g(x,y) - degraded image h(x,y) - blurring function (PSF)
f(x,y) - ideal image n(x,y) - noise
7/10/2011 4 Blur Identification
... characterize the regularity of the signals
Using wavelet transform to estimate local Lipschitz exponent, α
• [Jaffard, 1991]
• [Mallat and Zhong, 1992]
• [Malfait and Roose, 1997]
• [Hsung et al., 1999]
• [Pizurica et al., 2002]
Lipschitz exponents
Pointwise Lipschitz exponents
[courtesy of A. Pizurica]
Pointwise smoothness, two-microlocalization and wavelet coefficients, Jaffard, 1991 Characterization of signals from multiscale edges, Mallat and Zhong, 1992 Wavelet-based image denoising using a markov random field a priori model, Malfait and Roose, 1997 Denoising by singularity detection, Hsung et al., 1999 A joint inter- and intrascale statistical model for wavelet based Bayesian image denoising, Pizurica et al., 2002
7/10/2011 5 Blur Identification
Average cone ratio (ACR)
describes the “collective” evolution of the wavelet coefficients inside a cone of influence centred at the spatial position l.
• Properties of ACR
– good estimate of the local Lipschitz exponent α – significant robustness to noise
),(
,,
1
,
,1
2, ,1
logljCm
mjlj
k
nj lj
lj
lkn II
I
nk
Average cone ratio, ACR [Pizurica et al. 2002]
Average point ratio (APR) describes the evolution of the individual wavelet coefficients at the spatial position l.
7/10/2011 6 Blur Identification
• ACR is able to better separate the noise from the useful edges
Average Point Ratio vs. Average Cone Ratio
Conditional densities of APR and ACR, computed from scales 21 − 24.
The standard deviation of added noise is σ = 25. [courtesy of A. Pizurica]
ACR APR
noisy edges
noise noisy edges
noise
7/10/2011 7 Blur Identification
ACR metric. Probability density function
-1 0 1 2 3 40
0.2
0.4
0.6
0.8
1.0
ACR 2-4
PD
F
Degradation free
Noisy (var=25)
Blurred (r=3)
Blurred (r=3) &Noisy (var=25)
(σ2=25)
(σ2=25)
7/10/2011 8 Blur Identification
0.5 1 1.5 20.5
0.6
0.7
0.8
0.9
ACR 2-4P
DF
BL3
BL2
BL1
Ref.
0.5 1 1.5 20.5
0.6
0.7
0.8
0.9
ACR 2-4
PD
F
Ref.
BL3
BL1
BL2
0.5 1 1.5 20.5
0.6
0.7
0.8
0.9
1
1.1
ACR 2-4
PD
F
BL1
Ref.
BL3
BL2
Low sensitivity to noise
Noisy image (σ2=10)
Noise-free image
Gaussian blur Motion blur Circular blur
7/10/2011 9 Blur Identification
New blur metric: CogACR
Edge positions
CogACR
Reference
image
Degraded
image
0 2 40
0.2
0.4
0.6
0.8
ACR 2-4
PD
F
the center of gravity
of the ACR histogram
Wavelet transform
. . . . . . Scenario 1.
Reference image available
Scenario 2.
Ref. image NOT available
Image blur estimation based on the average cone of
ratio in the wavelet domain, Platisa et al., 2009
7/10/2011 10 Blur Identification
Blur model. Circular blur (pillbox)
Example test image
.
7/10/2011 11 Blur Identification
Example test images [130 real images]
7/10/2011 12 Blur Identification
Space of values for CogACR
0 2 4 6 8 100.5
0.75
1
1.25
1.5
1.75
2
2.25C
og
AC
R24
Level of blur
The results on 130 test images
Consistent behaviour for the whole range of tested blur
7/10/2011 13 Blur Identification
Extracting valid edges for ACR analysis
a) THRESHOLDING
WAVELET COEFFICIENTS
b) THRESHOLDING
INTERSCALE PRODUCTS
V H
Noisy σ2=10
Noise-free
Noisy σ2=25
Edge detection
based on:
V H
7/10/2011 14 Blur Identification
Tp=0.10
Selecting the threshold
V H
Tp=0.01
Tp=0.05
Tp=0.10
Tp=0.15
Tp=0.20
Tp=0.10
0 1 2 3 4 5 6 7 8 9 10
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
Level of blur
Cog
AC
R2
4
Tp=0.01
Tp=0.05
Tp=0.10
Tp=0.15
Tp=0.20
Tp - threshold percent coefficient
7/10/2011 15 Blur Identification
Selecting the threshold
H V Tp=0.10 Tp=0.01
Tp=0.05
Tp=0.10
Tp=0.15
Tp=0.20
Tp=0.10
0 1 2 3 4 5 6 7 8 9 10
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
Level of blur
Cog
AC
R2
4
Tp=0.01
Tp=0.05
Tp=0.10
Tp=0.15
Tp=0.20
7/10/2011 16 Blur Identification
Selecting the threshold
V H Tp=0.01 Tp=0.01
Tp=0.05
Tp=0.10
Tp=0.15
Tp=0.20
Tp=0.01
0 1 2 3 4 5 6 7 8 9 10
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Level of blur
Cog
AC
R2
4
Tp=0.01
Tp=0.05
Tp=0.10
Tp=0.15
Tp=0.20
7/10/2011 17 Blur Identification
Texture of the content
♦ Image class A
x Image class B
7/10/2011 18 Blur Identification
Texture of the content
r=0 r=1 r=2 r=3 r=4 r=6 r=8
7/10/2011 19 Blur Identification
0 2 4 6 8 100.5
0.75
1
1.25
1.5
1.75
2
Level of blur
Cog
AC
R24
Reference image available
Reference image NOT available
No-Reference metric
◊ Ref. image available
x Ref. image NOT available
7/10/2011 20 Blur Identification
0 2 4 6 8 100.5
0.75
1
1.25
1.5
1.75
2
Level of blur
Cog
AC
R24
Reference image available
Reference image NOT available
No-Reference metric
◊ Ref. image available
x Ref. image NOT available
7/10/2011 21 Blur Identification
• based on ACR histogram similarity, using histogram intersection metric
• preliminary results
No-Reference image classification
0 2 4 6 8 100.5
0.75
1
1.25
1.5
1.75
2
Level of blur
Cog
AC
R2
4
7/10/2011 23 Blur Identification
• based on ACR histogram similarity, using histogram intersection metric
step1. compute the intersected section of the input image with the template histograms
step2. select the best matching template histogram
step3. calculate CogACR of the input image
step4. based on CogACR curve of the selected image (template histogram), estimate the level of blur in the input image
No-Reference image classification
7/10/2011 24 Blur Identification
No-Reference image classification
0 2 4 6 8 100.5
0.75
1
1.25
1.5
1.75
2
Level of blur
Cog
AC
R2
4
selected template
input image
Input image
7/10/2011 25 Blur Identification
No-Reference image classification
0 2 4 6 8 100.5
0.75
1
1.25
1.5
1.75
2
Level of blur
Cog
AC
R2
4
selected template
input image
Input image
7/10/2011 26 Blur Identification
No-Reference image classification
0 2 4 6 8 100.5
0.75
1
1.25
1.5
1.75
2
Level of blur
Cog
AC
R2
4
selected template
input image
Input image
7/10/2011 27 Blur Identification
0 1 2 3 4 5 6 7 8 9 10
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
2.8
3.0
Level of blur
Metr
ic o
f blu
r
Noisy images. Kurtosis vs CogACR
Kurtosis
CogACR
--- noise free
--- noisy, var=10
--- noisy, var=25
Li et al. Blur identification based on kurtosis minimization. ICIP 2005.
7/10/2011 28 Blur Identification
CogACR compared to other blur metrics [Image 1]
[1] Hu and Haan. Low Cost Robust Blur Estimator. ICIP 2006.
[2] Tong et al. Blur detection for digital images using wavelet transform. ICME 2004.
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Radius of Gaussian blur
Est
imat
ed
lev
el o
f blu
r
Hu`s method
CogACR
Tong`s method
noise var=0
noise var=25
7/10/2011 29 Blur Identification
CogACR compared to other blur metrics [Image 2]
[1] Hu and Haan. Low Cost Robust Blur Estimator. ICIP 2006.
[2] Tong et al. Blur detection for digital images using wavelet transform. ICME 2004.
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Radius of Gaussian blur
Est
imat
ed
lev
el o
f blu
r
CogACR
Tong`s method
Hu`s method
noise var=0
noise var=25
7/10/2011 30 Blur Identification
CogACR. Pros & Cons
[1] Hu and Haan. Low Cost Robust Blur Estimator. ICIP 2006.
[2] Tong et al. Blur detection for digital images using wavelet transform. ICME 2004.
Hu and Haan 2006 [1] Tong et al. 2004 [2] CogACR
Range of distinguishable levels of Gaussian blur
R~(0..1.5) R~(0..2) R~(0..3.5)
Sensitivity to noise
High High Low
Computational complexity
Low High High
Image content dependency Yes Yes Yes
7/10/2011 31 Blur Identification
Real-time implementation
Cell Broadband Engine Architecture (CBEA): microprocessors dedicated to distributed data processing
• Computationaly most expensive part is wavelet transform (~38% proc. time)
• Real-time performance requires multiple SPE cores (3 out of 8)
PPE
Processor
SPE Processor
SPE Program
SPE Processor
SPE Program
SPE Processor
SPE Program
SPE Program
Referent
image
Blurred
image
EdgesWavelet t.
ACR CogACR
7/10/2011 32 Blur Identification
CBEA performance measurements
0
20
40
60
80
100
120
140
160
Frame rate on single SPE core
TongHuCogACR
(a)
Tong
Hu
CogACRSingle SPE core
Two SPE cores
Three SPE cores
0
20
40
60
80
100
120
140
160
Frame rate of CogACR on multiple SPE cores
Single SPE core
Two SPE cores
Three SPE cores
(b) Frame rate of CogACR on multiple
SPE coresFrame rate on single SPE core
7/10/2011 33 Blur Identification
New metric of blur, CogACR
• Sensitive to a wide range of blur levels
• Nearly insensitive to noise
• Valid in reference- and no-reference scenario
• Real-time performance could be achieved
• Current investigation
– automated content based image classification
– automated selection of Tp
Conclusions
7/10/2011 34 Blur Identification
Consider using orientation sensitive wavelet-like transforms (shearlets, curvelets, ...)
Directions for further investigation
-150
-100
-50
0
50
100
150
Work with blocks Use only the strongest edge in the image
Work with the region of interest as seen by humans
Does where you gaze on an image affect your
perception of quality? Applying visual attention to
image quality metric, Ninassi et al., 2007
7/10/2011 35 Blur Identification
Contact
LJILJANA PLATISA
Ghent University - TELIN - IPI - IBBT
Ljiljana.Platisa@Telin.UGent.be
http://telin.UGent.be/
top related