1
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 1
Multiresolution coding and wavelets
Predictive (closed-loop) pyramidsOpen-loop (“Laplacian”) pyramidsDiscrete Wavelet Transform (DWT)Quadrature mirror filters and conjugate quadrature filtersLifting and reversible wavelet transformWavelet theoryEmbedded zero-tree wavelet (EZW) coding
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 2
Interpolation error coding, I
Interpolator
InterpolatorSubsampling
• • •
• • •
Input picture
Reconstructed picture
Q
Q
+
+
-
- +
+
+
+
InterpolatorSubsampling
• • • •
Coder includes
Decoder
Sample encoded in current stage
Previously coded sample
2
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 3
Interpolation error coding, II
signals to be encoded
original image
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 4
Predictive pyramid, I
Interpolator
Subsampling
Q
Q
+
+-
- +
+
+
+
InterpolatorInterpolator
Filtering
Filtering
Subsampling
Input picture
Reconstructed picture
• • •
• • •• • • •
Coder includes
Decoder
Sample encoded in current stage
3
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 5
Predictive pyramid, II
Number of samples to be encoded =
1+1N
+1
N 2 + ...⎛ ⎝
⎞ ⎠ =
NN −1
x number of original image samples
subsampling factor
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 6
Predictive pyramid, III
original image
signals to be encoded
4
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 7
Comparison:interpolation error coding vs. pyramid
Resolution layer #0, interpolated to original size for display
Interpolation Error Coding Pyramid
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 8
Comparison:interpolation error coding vs. pyramid
Resolution layer #1, interpolated to original size for display
Interpolation Error Coding Pyramid
5
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 9
Comparison:interpolation error coding vs. pyramid
Resolution layer #2, interpolated to original size for display
Interpolation Error Coding Pyramid
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 10
Comparison:interpolation error coding vs. pyramid
Resolution layer #3
=(original)
Interpolation Error Coding Pyramid
6
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 11
Open-loop pyramid (Laplacian pyramid)
Interpolator
Interpolator
Subsampling
Q+
Q+ -
-
Filtering
Filtering
Interpolator
InterpolatorSubsampling
Input picture
Reconstructed picture
• • • • • • • •
Receiver Transmitter
[Burt, Adelson, 1983]
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 12
When multiresolution coding was a new idea . . .
This manuscript is okay if compared to some of the weaker papers.[. . .] however, I doubt that anyone will ever use this algorithm again.
Anonymous reviewer of Burt and Adelson‘s original paper, ca. 1982
7
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 13
Cascaded analysis / synthesis filterbanks
0h0g
1g
1h
0g
1g
0h
1h
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 14
Discrete Wavelet Transform
Recursive application of a two-band filter bank to the lowpass band of the previous stage yields octave band splitting:
Same concept can be derived from wavelet theory: Discrete Wavelet Transform (DWT)
frequency
8
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 15
2-d Discrete Wavelet Transform
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
...etc
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 16
2-d Discrete Wavelet Transform example
9
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 17
2-d Discrete Wavelet Transform example
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 18
2-d Discrete Wavelet Transform example
10
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 19
2-d Discrete Wavelet Transform example
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 20
2-d Discrete Wavelet Transform example
11
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 21
Two-channel filterbank
Aliasing cancellation if :0 1
1 0
( ) ( )( ) ( )
g z h zg z h z
= −− = −
[ ]
[ ] ( )
0 0 1 1
0 0 1 1
1ˆ( ) ( ) ( ) ( ) ( ) ( )21 ( ) ( ) ( ) ( )2
x z h z g z h z g z x z
h z g z h z g z x z
= +
+ − + − −Aliasing
2 2
2 2
0h
1h
0g
1g
( )x z ( )x̂ z
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 22
Example: two-channel filter bank with perfect reconstruction
Impulse responses, analysis filters:Lowpass Highpass
Impulse responses, synthesis filtersLowpass Highpass
Mandatory in JPEG2000Frequency responses:
|g |0
|h |0
1|h |
1|g |
π2
π
1
2
00
Frequency
Freq
uenc
y re
spon
se
1 1 3 1 1, , , ,4 2 2 2 4− −⎛ ⎞
⎜ ⎟⎝ ⎠
1 1 1, ,4 2 4
−⎛ ⎞⎜ ⎟⎝ ⎠
1 1 3 1 1, , , ,4 2 2 2 4
−⎛ ⎞⎜ ⎟⎝ ⎠
1 1 1, ,4 2 4
⎛ ⎞⎜ ⎟⎝ ⎠
“Biorthogonal 5/3 filters”“LeGall filters”
12
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 23
frequency ω
Classical quadrature mirror filters (QMF)
QMFs achieve aliasing cancellation by choosing
Highpass band is the mirror image of the lowpass band in the frequency domainNeed to design only one prototype filter
1 0
1 0
( ) ( )( ) ( )
h z h zg z g z
= −= − = −
Example:16-tap QMF filterbank
[Croisier, Esteban, Galand, 1976]
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 24
Conjugate quadrature filters
Achieve aliasing cancelation by
Impulse responses
Orthonormal subband transform!Perfect reconstruction: find power complementary prototype filter
( )( ) ( )
10 0
1 11 1
( ) ( )
( )
h z g z f z
h z g z zf z
−
− −
= ≡
= = − [Smith, Barnwell, 1986]
Prototype filter
[ ] [ ] [ ][ ] [ ] ( ) ( )
0 0
11 1 1 1k
h k g k f k
h k g k f k+
= − =
= − = − − +⎡ ⎤⎣ ⎦
( ) ( )2 22F Fω ω π+ ± =
13
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 25
Lifting
Analysis filters
L “lifting steps”First step can be interpreted as prediction of odd samples from the even samples
K0
1λ 2λ 1Lλ − LλΣ Σ
Σ Σ
K1
[ ]even samples 2x n
[ ]odd samples
2 1x n +
0low band y
1high band y
[Sweldens 1996]
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 26
Lifting (cont.)
Synthesis filters
Perfect reconstruction (biorthogonality) is directly built into lifting structurePowerful for both implementation and filter/wavelet design
1λ 2λ 1Lλ − LλΣ Σ
Σ Σ[ ]even samples 2x n
[ ]odd samples
2 1x n +
0low band y
1high band y
10K −
11K −
-
-
-
-
14
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 27
Example: lifting implementation of 5/3 filters
[ ]even samples 2x n
( )12
z− + 114z −+
Σ
Σ
1/2[ ]
odd samples 2 1x n +
0low band y
1high band y
Verify by considering response to unit impulse in even and odd input channel.
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 28
Reversible subband transform
Observation: lifting operators can be nonlinear.Incorporate the necessary rounding into lifting operator:
Used in JPEG2000 as part of 5/3 biorthogonal wavelet transform
K0
1λ 2λ 1Lλ − LλΣ Σ
Σ Σ
K1
[ ]even samples 2x n
[ ]odd samples
2 1x n +
0low band y
1high band y
15
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 29
Wavelet bases
( ) ( )
( )( ) ( ) ( )
( )
2
2
2
Consider Hilbert space of finite-energy functions .
Wavelet basis for : family of linearly independent functions
2 2
that span . Hence any signal
m m mn
x t
t t nψ ψ− −
=
= −
∈
x
x
L
L
L ( )( ) [ ] ( )
2 can be written as
m mn
m n
y n ψ∞ ∞
=−∞ =−∞
= ∑ ∑x
L
“mother wavelet”
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 30
Multi-resolution analysis
( ) ( ) ( ) ( ) ( ) ( )
( ) ( )( ) { }
( ) ( )
2 1 0 1 2 2
2
0
Upward compl
Nested subspaces
eteness
Downward complete
ness
Self-similari t fy i
m
m
m
m
V V V V V
V
V
x t V
− −
∈
∈
⊂ ⊂ ⊂ ⊂ ⊂ ⊂ ⊂
=
=
∈
0
… …
∪∩
Z
Z
L
L
( ) ( )
( ) ( ) ( ) ( )
( ) ( ) ( ){ } ( )
0 0
0
f 2
iff for all
There exists a "scaling function" with integer translates -
such that forms an orthonormal basis f
Translation i
or
nvaria
nce
mm
n
n n
x t V
x t V x t n V n
t t t n
V
ϕ ϕ ϕ
ϕ
−
∈
∈
∈ − ∈ ∈
=
Z
16
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 31
Multiresolution Fourier analysis
ω
( ){ } ( )span pn
p
nVϕ =
( ){ } ( )11span p pn n
Vϕ − −=
( ){ } ( )22span p pn n
Vϕ − −=
( ){ } ( )33span p pn n
Vϕ − −=
( ){ } ( )span pn
p
nWψ =
( ){ } ( )11span p pn n
Wψ − −=
( ){ } ( )22span p pn n
Wψ − −=
( ){ } ( )33span p pn n
Wψ − −=
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 32
Relation to subband filters( ) ( )
( ) [ ] ( ) ( )
( )
[ ] ( )
[ ] ( ) ( )
1
0 1
10 0
0 00
linear combinationof scaling functions in
Since , recursive definition of scaling function
2 2
Orthonormality
n ,
nn n
n
V
V V
t g n t g n t nϕ ϕ ϕ
δ ϕ ϕ
−
−
∞ ∞−
=−∞ =−∞
⊂
= = −
=
∑ ∑
[ ] ( ) ( ) [ ] ( ) ( )
[ ] [ ] ( ) ( ) [ ] [ ][ ]0
1 10 0 2
1 10 0 0 0
,
k unit norm and orthogonalto its 2-translates: correspondsto synthesis lowpass filter oforthonormal subband tra
2 , 2
i j ni j
i ji j i
g
g i t g j t dt
g i g j n g i g i n
ϕ ϕ
ϕ ϕ
∞− −
+−∞
− −
⎛ ⎞= ⎜ ⎟
⎝ ⎠
= − = −
∑ ∑∫
∑ ∑
nsform
17
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 33
Wavelets from scaling functions( ) ( ) ( )
( ) ( ) ( ) ( ) ( )
( ){ } ( ) ( )
( ) [ ] ( ) ( )
1
1
0 0 1
11
linear combinationof scaling functions
is orthogonal complement of in
and
Orthonormal wavelet basis for
p p p
p p p p p
n
nn
W V V
W V W V V
W V
t g n t
ψ
ψ ϕ
−
−
−
∞−
=−∞
⊥ ∪ =
⊂
= ∑
( )
[ ] ( )
[ ] ( ) ( )( ){ } ( ){ } ( )
1
1
11 0
0 0 1
in
2 2
Using conjugate quadrature high-pass synthesis filter
The mutually orthonormal
1 1
Easy
functions, and , together span .
to e
nn
n
n
n n n
V
g n t n
g n g n
Vϕ
ϕ
ψ
−
−
∞
∞
∈
+
∈
=−
= −
= − − −⎡ ⎤⎣ ⎦
∑
Z Z
( )( ){ } ( )2
,
xtend to dilated versions of to construct orthonormal wavelet basis
for .
mn n m
tψ
ψ∈Z
L
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 34
Calculating wavelet coefficients for a continuous signal
Signal synthesis by discrete filter bank
Signal analysis by analysis filters h0[k], h1[k]Discrete wavelet transform
( ) ( ) ( ) [ ] ( ) ( ) [ ] ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) [ ] ( )
( ) ( ) ( )
( ) [ ] ( )
( ) ( ) ( )1 1 1 1
0 0 0 0 00 0
1 1 1 1
0 1 1 1 10 1
Suppose continuous signal
Write as superposition of and
nn n
n ni j
x t V w t W
x t y n t n y n V
x t V w t W
x t y i y j
ϕ ϕ
ϕ ψ
∈ ∈
∈ ∈
∈ ∈
= − = ∈
∈ ∈
= +
∑ ∑
∑ ∑
Z Z
Z Z
( ) ( ) [ ] [ ] ( ) [ ] [ ]( ) [ ]00
0 1 10 0 1 1 2 2n
n i j
y n
y n g n i y j g n iϕ∈ ∈ ∈
⎛ ⎞= − + −⎜ ⎟
⎝ ⎠∑ ∑ ∑
Z Z Z
18
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 35
0h0g
1g
1h
0g
1g
0h
1h
Discrete Wavelet Transform
( ) [ ]00y n
( ) [ ]10y n
( ) [ ]11y n
( ) [ ]20y n
( ) [ ]21y n
( ) [ ]10y n
( ) [ ]00y n
Sampling( )x t
( )Interpolation
tϕ
( )x t
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 36
Different wavelets
Haar2/2 coeffs.
Daubechies8/8
Symlets8/8
Cohen-Daubechies-Feauveau17/11
[Gonzalez, Woods, 2001]
19
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 37
Daubechies orthonormal 8-tap filters
[Gonzalez, Woods, 2001]
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 38
8-tap Symlets
[Gonzalez, Woods, 2001]
20
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 39
Biorthogonal Cohen-Daubechies-Feauveau 17/11 wavelets
[Gonzalez, Woods, 2001]
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 40
Wavelet compression results
Original512x512
8bpp
0.074 bpp 0.048 bpp
Errorimages
enlarged
[Gonzalez, Woods, 2001]
21
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 41
Embedded zero-tree wavelet algorithmX X X XX X X XX X X XX X X X
X X X X
X
X
X X X X
X X X XX X X XX X X XX X X X
X
X X X X
X X X XX X X XX X X XX X X X
Idea: Conditional coding of all descendants (incl. children)Coefficient magnitude > threshold: significant coefficientsFour cases
ZTR: zero-tree, coefficient and all descendants are not significantIZ: coefficient is not significant, but some descendants are significantPOS: positive significantNEG: negative significant
„Parent“
„Children“
„Descendants“
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 42
Embedded zero-tree wavelet algorithm (cont.)
For the highest bands, ZTR and IZ symbols are merged into one symbol ZSuccessive approximation quantization and encoding
Initial „dominant“ pass• Set initial threshold T, determine significant coefficients• Arithmetic coding of symbols ZTR, IZ, POS, NEG
Subordinate pass• Refine magnitude of all coefficients found significant so far by one bit
(subdivide magnitude bin by two)• Arithmetic coding of sequence of zeros and ones.
Repeat dominant pass• Omit previously found significant coefficients• Decrease threshold by factor of 2, determine new significant
coefficients• Arithmetic coding of symbols ZTR, IZ, POS, NEG
Repeat subordinate and dominate passes, until bit budget is exhausted.
22
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 43
Embedded zero-tree wavelet algorithm (cont.)
Decoding: bitstream can be truncated to yield a coarser approximation: „embedded“ representationFurther details: J. M. Shapiro, „Embedded image coding using zerotrees of wavelet coefficients,“ IEEE Transactions on Signal Processing, vol. 41, no. 12, pp. 3445-3462, December 1993.Enhancement SPIHT coder: A. Said, A., W. A. Pearlman, „A new, fast, and efficient image codec based on set partitioning in hierarchical trees,“ IEEE Transactions on Circuits and Systems for Video Technology, vol. 63 , pp. 243-250, June 1996.
Bernd Girod: EE398A Image Communication I Multiresolution & Wavelets no. 44
Summary:multiresolution and subband coding
Resolution pyramids with subsampling 2:1 horizontally and verticallyPredictive pyramids: quantization error feedback („closed loop“)Transform pyramids: no quantization error feedback („open loop“)Pyramids: overcomplete representation of the imageCritically sampled subband decomposition: number of samples not increasedDiscrete Wavelet Transform = cascaded dyadic subband splitsQuadrature mirror filters and conjugate quadrature filters: aliasing cancellationLifting: powerful for implementation and wavelet constructionLifting allows reversible wavelet transform Zero-trees: exploit statistical dependencies across subbands