11 coding wavelets - purdue university
TRANSCRIPT
ECE 634: Digital Video Systems Wavelets: 2/21/17
Professor Amy Reibman MSEE 356
hJp://engineering.purdue.edu/~reibman/ece634/index.html
A short break to discuss wavelets
• Wavelet compression • Wavelets are another way to combine pixels to create coefficients with improved entropy and/or correlaQon
• Wavelets are used for compression (of images); denoising; many other applicaQons
Reading
• R. Gonzalez, “Digital Image Processing,” Chapter 7 (Wavelets)
• A. Skodras, C. Christopoulos, T. Ebrahimi, The JPEG2000 SQll Image Compression Standard, IEEE Signal Processing Magazine, vol. 18, pp. 36-‐58, Sept. 2001.
• B.E. Usevitch, “A tutorial on modern lossy wavelet image compression: FoundaQons of JPEG 2000,” IEEE Signal Processing Mag., vol. 18, pp.22-‐35, Sept. 2001.
Wavelets
• A small wave whose energy is concentrated in Qme • Comes from seismography, based on a small wave that results from a short-‐Qme impact
• The tool of wavelets developed to analyze seismic “wavelets”
• ConQnuous Time Wavelets, Discrete Wavelets
Many ways to talk about wavelets
• MathemaQcal elegance • MulQ-‐resoluQon pyramid analysis tool • Tree-‐structured Subband decomposiQon
• All have their uses; a complete overview of wavelets should incorporate all of them
MulQ-‐resoluQon pyramid analysis
1. Take image, decompose into a. low-‐resoluQon (coarse) approximaQon b. detail info that was lost due to the
approximaQon
2. Take (1a), decompose into a. low-‐resoluQon (coarse) approximaQon b. detail info that was lost due to the
approximaQon
MulQ-‐resoluQon image analysis
Gaussian and Laplacian Pyramids
9
Subband decomposiQon
h0(n)
h1(n)
g0(n)
g1(n)
+ Analysis Synthesis
1( )y n
0 ( )y n
( )x n ˆ( )x n
H0 (ω) 1( )H ω
π / 2
Low band High band
0 π ω
2 ↓
2 ↓
2 ↑
2 ↑
MulQ-‐rate analysis
h0(n)
h1(n)
g0(n)
g1(n)
+ Analysis Synthesis
1( )y n
0 ( )y n
( )x n ˆ( )x n
2 ↓
2 ↓
2 ↑
2 ↑
[ ][ ]
10 0 1 12
10 0 1 12
ˆ ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( )
X z H z G z H z G z X z
H z G z H z G z X z
= +
+ − + − −
filter X(z)Ho(z)
decimate Y0 (z) =12X(z1/2 )Ho(z
1/2 )+ X(−z1/2 )Ho(−z1/2 )"# $%
upsample Yo(z2 )
filter G0 (z)Yo(z2 )
Combining:
11
Subband decomposiQon
h0(n)
h1(n)
g0(n)
g1(n)
+ Analysis Synthesis
1( )y n
0 ( )y n
( )x n ˆ( )x n
H0 (ω) 1( )H ω
π / 2
Low band High band
0 π ω
2 ↓
2 ↓
2 ↑
2 ↑
0 0 1 1
0 0 1 1
( ) ( ) ( ) ( ) 0( ) ( ) ( ) ( ) 2
H z G z H z G zH z G z H z G z
− + − =
+ =For finite impulse response (FIR) filters and ignoring the delay
0 11
1 0
( ) ( 1) ( )
( ) ( 1) ( )
n
n
g n h ng n h n+
= −
= −
For error-‐free reconstrucQon
12
Tree-‐structured subband decomposiQon
h0(n)
h1(n)
Analysis
1( )y n
0 ( )y n
( )x n
2 ↓
2 ↓
h0(n)
h1(n)
2 ↓
2 ↓
h0(n)
h1(n)
2 ↓
2 ↓
Subband Example
LH: Low-‐pass VerQcal, high-‐pass horizontal
HL: high-‐pass VerQcal, low-‐pass horizontal
Comments
• Maximally decimated wavelet decomposiQon is a tree-‐structured subband decomposiQon
• The hierarchical decomposiQon AND the tree structure are powerful tools for many applicaQons
• “Maximally decimated” is not required for all applicaQons, and may cause problems
15
Wavelet Transform vs. Fourier Transform
Time-‐frequency distribuQon for (a) sampled data, (b) FFT, and (c) FWT basis
Fourier transform: Basis funcQon cover the enQre signal range Wavelet transform: Basis funcQons vary in frequency/scale and spaQal extent
High-‐frequency basis covers a smaller area Low-‐frequency basis covers a larger area
2D wavelet transform for images
H0
H1
↓2
↓2
H0 ↓2
H1 ↓2
H0 ↓2
H1 ↓2
Input Image (NxN)
Rows LL
LH
HL
HH
D. Gibbon and Z. Liu, 2013
LL: low pass for both horizontal and vertical directions LH: low pass for horizontal direction, high pass in vertical direction HL: low pass for vertical direction, high pass in horizontal direction HH: high pass in both horizontal and vertical directions
LL
LH
HL
HH
Columns
Wavelets for images
2D wavelet transform is accomplished by applying the 1D decomposiQon along rows of an image first, and then columns.
From [Usevitch01]
Wavelets for images
LL: low pass for both horizontal and vertical directions LH: low pass for horizontal direction, high pass in vertical direction HL: low pass for vertical direction, high pass in horizontal direction HH: high pass in both horizontal and vertical directions
LL
LH
HL
HH
From [Usevitch01]
significant horizontal energy
Constraints on Filters (1)
[ ][ ]
10 0 1 12
10 0 1 12
ˆ ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( )
X z H z G z H z G z X z
H z G z H z G z X z
= +
+ − + − −
( )ˆ ( ) ( )X z X z=
0 0 1 1
0 0 1 1
( ) ( ) ( ) ( ) 0( ) ( ) ( ) ( ) 2
H z G z H z G zH z G z H z G z
− + − =
+ =
For finite impulse response (FIR) filters and ignoring the delay
0 11
1 0
( ) ( 1) ( )
( ) ( 1) ( )
n
n
g n h ng n h n+
= −
= −
For perfect reconstrucQon
Z-‐transform :
23
Constraints on Filters (2)
• Biorthogonal
(2 ), ( ) ( ) ( ), , {0,1}i jh n k g k i j n i jδ δ− = − =
0 0 0 1
1 1 1 0
( ), (2 ) ( ), ( ), (2 ) 0
( ), (2 ) ( ), ( ), (2 ) 0
g k h n k n g k h n k
g k h n k n g k h n k
δ
δ
− = − =
− = − =
The analysis and synthesis filter impulse responses of all two-‐band, real-‐coefficient, perfect reconstrucQon filter banks are subject to the biorthogonality constraint
24
Common Wavelet Filters
• Haar: simple, orthogonal, low effecQveness • Daubechies 9/7: bi-‐orthogonal most commonly used if numerical reconstrucQon errors are acceptable
• LeGall 5/3: bi-‐orthogonal, integer operaQon, can be implemented with integer operaQons only, used for lossless image coding
Fig.2. Examples of parent-‐offspring dependencies in the spaQal-‐orientaQon tree.
★
Wavelets
• Allows for intra-‐scale predicQon (like many other compression methods) – equivalently the wavelet transform is a decorrela5ng transform just like the DCT as used by JPEG
• Allows for inter-‐scale (coarse-‐fine scale) predicQon
A reason wavelet compression is effecQve
• Coefficient entropies
Entropy
Original image
7.22
1-level Haar wavelet
5.96
1-level linear spline wavelet
5.53
2-level Haar wavelet
5.02
2-level linear spline wavelet
4.57
Interscale hierarchies are powerful
28
X )(XP
from Mike Spann
Parent of X
Interscale hierarchies
• Define sets S (small) and L (large) wavelet coefficients
• Without inter-‐scale dependencies
• Measured dependencies in Lena
2
#))(|(NSSXPSXP =∈∈ 2
#))(|(NLLXPLXP =∈∈
0.886 0.529 0.781 0.219
))(|( SXPSXP ∈∈ 2
#NS))(|( LXPLXP ∈∈
2
#NL
from Mike Spann
Intra-‐scale dependencies
X ∑=
=8
1
)(81n
nXcc
X1
X8 { }{ } TcLX
TcSX
n
n
>∈
≤∈
if
if
0.912 0.623 0.781 0.219
{ } )|( SXSXP n ∈∈ 2
#NS{ } )|( LXLXP n ∈∈
2
#NL
from Mike Spann
ApplicaQons
• JPEG 2000 image compression • (also used for digital cinema) • Image blending • Image denoising • (many others)
Pyramid Blending
Application: Denoising
How to characterize the difference between the images?
How do we use the differences to clean up the image?
hJp://www.cns.nyu.edu/pub/lcv/simoncelli96c.pdf
Application: Denoising
Original Noise-corrupted
Wiener filter Steerable pyramid coring
hJp://www.cns.nyu.edu/pub/lcv/simoncelli96c.pdf
JPEG • Pros
– Low complexity – Memory efficient – Reasonable coding efficiency
• Cons: – Single resoluQon – Single quality – No target bit-‐rate – Blocking arQfacts at low rate – Poor error resilience – Low flexibility, for example, no ROI
Yao Wang, 2016
JPEG2000 Features
• Improved coding efficiency • Full quality scalability
– From lossless to lossy at different bit rates • SpaQal scalability • Improved error resilience • Tiling • Region of interests • More demanding in memory and computaQon Qme
Yao Wang, 2016
JPEG 2000: Core Processing • Wavelet transform
– Daubechies 9-‐tap/7-‐tap filter for irreversible transform – Le Gall 5-‐tap/3-‐tap filter for reversible transformaQon
• QuanQzaQon – Separate quanQzaQon step-‐sizes for each subband
• Entropy encoding – Bit plane coding -‐ the most significant bit plane is coded first. – ArithmeQc coding, relaQve to an adapQve probability model
Spring 2013 ELEN E4830 Digital Image Processing
JPEG2000 vs. JPEG: Coding Efficiency
J2K R: Using reversible wavelet filters; J2K NR: Using non-reversible filter; VTC: Visual texture coding for MPEG-4 video
From [skodras01]
PSNR: Peak signal-‐to-‐noise raQo PSNR = 10 log10(MAX2/MSE)
Example Image
From [skodras01]
Another Example
From [skodras01]
How J2K Achieves Scalability
• Core: Wavelet transform – Yields a mulQ-‐resoluQon representaQon of an original image
– Wavelet coefficients are coded bit plane by bit plane
• SpaQal scalability can be achieved by reconstrucQng from only low resoluQon wavelet coefficients
• Quality scalability can be achieved by decoding only parQal bit planes
Quality Scalability of JPEG2000
From [skodras01]
SpaQal Scalability of JPEG2000
From [skodras01]
Wavelets have other uses
• Compression – Maximally decimated wavelets are most appropriate
• Signal analysis – Overcomplete wavelets may be more effecQve – Example: Steerable pyramids
Steerable pyramids: other wavelets
• Filters can measure local orientation direction and strength and phase at any orientation.
G2 H2