1 jayanta mukhopadhyay department of computer science & engineering indian institute of...
TRANSCRIPT
1
Jayanta MukhopadhyayDepartment of Computer Science & Engineering
Indian Institute of Technology, Kharagpur, 721302, India
Sanjit K. MitraMing Hsieh Dept. of Electrical Engineering
University of Southern California
Los Angeles, CA 90089, USA
COLOR ENHANCEMENT IN THE COMPRESSED DOMAIN
2
Image Enhancement: An Example
(a) Original (b) Enhanced
3
Enhancement Factors
• Dynamic Range of Intensity Values. Limited number of bit-planes.
• Varying illumination over the image space.
• Brightness
• Contrast
• Color
4
Enhancement Techniques
• Spatial Domain Techniques.
Work with the pixels values. Histogram Stretching, Pixel Mapping, Image Sharpening
• Compressed Domain Techniques. Works with DCT coefficients
5
Let x (m,n), 0 m,n N-1 be a 2-D image of size NxN. Its N-point DCT C(l,k), 0 l,k N-1 is defined as:
DCT: Definition
1 1
0 0
2 (2 1) (2 1)( , ) ( ) ( ) ( , ) cos cos
2 2
N N
m n
n l n kC l k l k x m n
N N N
otherwise 1
0for 2
1)(
pp
6
Motivations
Computation with reduced storage. Avoid overhead of entropy decoding
and encoding. Exploit spectral factorization for
improving the quality of result and speed of computation.
7
Existing Approaches
• Alpha Rooting Aghaglzadeh and Ersoy (1992), Opt.Engg• Multi Contrast Enhancement Tang, Peli and Acton (2003), IEEE SPL
A spectral band is defined with Yijs such that i+j=n; 0<n<15
Contrast measure (Hn)= Cumulative Energy Sum till nth band in original image ----------------------------------------------------------------------- Cumulative Energy Sum till nth band in enhanced image Each coefficient in a band is scaled by : Yij = λ Hn. Yij, , i+j=n
8
Existing Approaches
• Multi-Contrast Enhancement with Dynamic Range Compression (S. Lee (2007), IEEE CSVT)
Modification of DC coefficients and AC coefficients (following similar strategy of multi-contrast enhancement).
Normalized DC coefficients (x) are modified as follows:
9
Proposed Approach
• Adjust background illumination. Use DC coefficients of the Y component.
• Preserve Local Contrast. Scale AC coefficients of the Y component appropriately.
• Preserve Colors. Preserve Color Vectors in the DCT domain. DCT coefficients of Cb and Cr components.
10
Let μ and σ denote the mean and standard deviation of
an image. Contrast ζ of an image is defined here as: .
Contrast : Definition
Weber Law:
where is the difference in luminance between a stimulus and its surround, and L is the luminance of the surround
L
L
L
11
Theorem on Contrast Preservation in the DCT Domain
Let d be the scale factor for the DC coefficient and a a be the scale factor for the AC coefficients of a DCT block Y. The processed DCT block Ye is given by:
( , ), 0( , ),( , ) { d
a
Y i j i je Y i j otherwiseY i j
The contrast of the processed image then becomes a / d times of the contrast of the original image.
In this algorithm d = a = for preservation of the contrast.
12
Preservation of Colours in the DCT Domain
Let U and V be the DCT coefficients of the Cb and Cr components, respectively. If the luminance component Y of an image is uniformly scaled by a factor , the colors of the processed image with Ye , Ue and Ve are preserved by the following operations:
( , )( ( 128)) 128, 0
( , ),( , ) {U i j
N i jN
e U i j otherwiseU i j
( , )( ( 128)) 128, 0
( , ),( , ) {V i j
N i jN
e V i j otherwiseV i j
13
Enhancement by Scaling Coefficients
• Find the scale factor by mapping the DC coefficient with a monotonically increasing function.
• Apply scaling to all other coefficients in all the
components.
• For blocks having greater details, apply block
decomposition and re-composition strategy.
14
Mapping functions for adjusting the local background illumination
( (0,0))
(0,0)
f Y
Y
(TW)
Mitra and Yu , CVGIP’87
(DRC)
Lee, CSVT’07
(SF)
De, TENCON’89
15
Monotonic Mapping Functions
16
Scaling only DC coefficients
17
Scaling both DC and AC coefficients
18
Preservation of Contrast and Color
original
19
Block Decompos.
Smaller DCT blocks
8x8 block
Apply CES on smaller
blocks
Block Composition
Enhanced Block
Enhancement of Blocks with more details
20
Removal of Blocking Artifacts
original
21
Some Results
original AR MCE
MCEDRC TW-CES-BLK MSR
22
Enhancement near Edges
AR MCE MCEDRC
TW-CES-BLK DRC-CES-BLK SF-CES-BLK
23
Some Results
original AR MCE
MCEDRC TW-CES-BLK MSR
24
Enhancement near edges
AR MCE MCEDRC
TW-CES-BLK DRC-CES-BLK SF-CES-BLK
25
Some Results
original AR MCE
MCEDRC TW-CES-BLK MSR
26
Enhancement near edges
AR MCE MCEDRC
TW-CES-BLK DRC-CES-BLK SF-CES-BLK
27
Metrics for Comparison
2
2 2
4 xy
x y
x yQM
x y
Wang and Bovic (SPL, 2002)
JPEG Quality Metric (JPQM) Wang and Bovic (ICIP,2002)
2 2 2 20.3CM Susstrunk and Winkler (SPIE, 2004)
R G
2
R GB
28
Other Approaches
• Alpha Rooting (AR) : Aghaglzadeh and Ersoy (1992), Opt.Engg.
• Multi-Contrast Enhancement (MCE): Tang, Peli and Acton (2003), IEEE SPL
• Multi-Contrast Enhancement with Dynamic Range Compression (MCEDRC):
S. Lee (2007), IEEE CSVT
• Contrast Enhancement by Scaling (CES): Proposed work
• Multi-Scale Retinex (MSR) (a reference spatial domain technique):
Jobson, Rahman and Woodell (1997), IEEE IP
29
Average Performance Measures
Techniques JPQM CEF Y-QM
Cb-QM
Cr-QM
AR 8.58 0.97 0.80 0.67 0.67
MCE 7.00 0.94 0.76 0.67 0.67
MCEDRC 7.92 0.97 0.86 0.67 0.67
TW-CES-BLK
7.79 1.50 0.90 0.82 0.81
DRC-CES-BLK
8.16 1.18 0.86 0.76 0.76
SF-CES-BLK
8.13 1.25 0.89 0.78 0.77
30
Computational Complexities
Techniques
Per Pixel Operations
AR 1E + 1M
MCE 2.19M+1.97A
MCEDRC 0.03E+3.97M+2A
TW-CES 0.02E+4.02M+1.05A
DRC-CES 0.05E+4M+1.08A
SF-CES 0.03E+4.02M+1.06A
MSR 18E+1866378M+8156703A
aE+bM+cA implies a Exponentiation, b Multiplication and c Addition operations.
31
Iterative Enhancement
original
Iteration no.=1
Iteration no.=2
Iteration no.=3
Iteration no.=4
32
Conclusion
• A novel algorithm for enhancement of color images in the block DCT domain.
• It takes care of adjustment of back-ground illumination, local contrast, as well as preservation of colours.
• Comparative study shows the superiority of the proposed scheme compared to some of the existing schemes.
33
References
• [1] J. Mukherjee and S.K.Mitra, Enhancement of Color Images by
Scaling the DCT coefficients, IEEE trans. on Image Processing, vol. 17, no. 10, Oct., pp. 1783-1794, 2008. • [2] J. Mukherjee and S.K. Mitra, Color Enhancement in the Compressed Domain, Int. Conf. on Image Proc. (ICIP-2008), San Diego, USA, Oct. 12 -15, pp. 3144-3147, 2008 .
34
Thanks!