multiresolution image processing - university of california, berkeley
Post on 03-Feb-2022
2 Views
Preview:
TRANSCRIPT
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 1
Multiresolution image processing
Laplacian pyramidsSome applications of Laplacian pyramidsDiscrete Wavelet Transform (DWT)Wavelet theoryWavelet image compression
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 2
Image pyramids
[Burt, Adelson, 1983]
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 3
Image pyramid example
original image
Gaussian pyramid Laplacian pyramid
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 4
Overcomplete representation
2
Number of samples in Laplacian or Gaussian pyramid =1 1 1 41 ... x number of original image samples4 4 4 3P
⎛ ⎞+ + + + ≤⎜ ⎟⎝ ⎠
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 5
LoG vs. DoG
Laplacian of Gaussian Difference of Gaussians
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 6
Image processing with Laplacian pyramid
Analysis Synthesis
Interpolator
Interpolator
Subsampling
+
+ -
-
Filtering
Filtering
Interpolator
InterpolatorSubsampling
Inputpicture
Processedpicture
• • • • • • • • • • • •
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 7
Expanded Laplacian pyramid
Analysis Synthesis
+
+ -
-
Filtering
Filtering
Inputpicture
Processedpicture
• • • • • • • • • • • •
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 8
Mosaicing in the image domain
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 9
Mosaicing by blending Laplacian pyramids
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 10
Expanded Laplacian pyramids
Original: begoniaOriginal: begonia
Original: dahliaOriginal: dahlia
Gaussian level 2Gaussian level 2
Gaussian level 2Gaussian level 2
Laplacian level 2Laplacian level 2
Laplacian level 2Laplacian level 2
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 11
Blending Laplacian pyramidsLevel 0Level 0 Level 2Level 2
Level 4Level 4 Level 6Level 6
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 12
Image analysis with Laplacian pyramid
Analysis
Interpolator
Interpolator
Subsampling
+
+ -
-
Filtering
Filtering
Subsampling
Inputpicture
•
Recognition/detection/segmentationresult
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 13
Multiscale edge detection
Zero-crossings of Laplacian images of different scales
Spurious edges removed
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 14
Multiscale face detection
InputNetwork Outputsu
bsam
plin
g
Preprocessing Neural network
pixels20 by 20
Extracted windowInput image pyramid(20 by 20 pixels)
Correct lighting Histogram equalization Receptive fieldsHidden units
[Rowley, Baluja, Kanade, 1995]
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 15
Example: multiresolution noise reduction
Original Noisy Noise reduction:3-level Haar transform
no subsamplingw/ soft coring
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 16
1-d Discrete Wavelet Transform
Recursive application of a two-band filter bank to the lowpass band of the previous stage yields octave band splitting:
frequency
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 17
Cascaded analysis / synthesis filterbanks
0h0g
1g
1h
0g
1g
0h
1h
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 18
2-d Discrete Wavelet Transform
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
ωx
ωy
...etc
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 19
2-d Discrete Wavelet Transform example
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 20
2-d Discrete Wavelet Transform example
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 21
2-d Discrete Wavelet Transform example
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 22
2-d Discrete Wavelet Transform example
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 23
2-d Discrete Wavelet Transform example
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 24
Review: Z-transform and subsampling
Generalization of the discrete-time Fourier transform
Fourier transform on unit circle: substituteDownsampling and upsampling by factor 2
( ) [ ] ; ; n
n
x z x n z z r z r∞
− − +
=−∞
= ∈ < <∑ C
jz e ω=
2 2( )x z ( ) ( )12
x z x z+ −⎡ ⎤⎣ ⎦
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 25
Two-channel filterbank
Aliasing cancellation if :0 1
1 0
( ) ( )( ) ( )
g z h zg z h z
= −− = −
[ ] [ ]
[ ] [ ] ( )
0 0 0 1 1 1
0 0 1 1 0 0 1 1
1 1ˆ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )2 21 1( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )2 2
x z g z h z x z h z x z g z h z x z h z x z
h z g z h z g z x z h z g z h z g z x z
= + − − + + − −
= + + − + − −
Aliasing
2 2
2 2
0h
1h
0g
1g
( )x z ( )x̂ z
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 26
Example: two-channel filter bank with perfect reconstruction
Impulse responses, analysis filters:Lowpass highpass
Impulse responses, synthesis filtersLowpass highpass
Mandatory in JPEG2000Frequency responses:
|g |0
|h |0
1|h |
1|g |
π2
π
1
2
00
FrequencyFr
eque
ncy
resp
onse
1 1 3 1 1, , , ,4 2 2 2 4− −⎛ ⎞
⎜ ⎟⎝ ⎠
1 1 1, ,4 2 4
−⎛ ⎞⎜ ⎟⎝ ⎠
1 1 3 1 1, , , ,4 2 2 2 4
−⎛ ⎞⎜ ⎟⎝ ⎠
1 1 1, ,4 2 4
⎛ ⎞⎜ ⎟⎝ ⎠
“Biorthogonal 5/3 filters”“LeGall filters”
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 27
Lifting
Analysis filters
L “lifting steps”First step can be interpreted as prediction of odd samples from the even samples
K0
1λ 2λ 1Lλ − LλΣ Σ
Σ Σ
K1
[ ]even samples 2x n
[ ]odd samples
2 1x n +
0low band y
1high band y
[Sweldens 1996]
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 28
Lifting (cont.)
Synthesis filters
Perfect reconstruction (biorthogonality) is directly build into lifting structurePowerful for both implementation and filter/wavelet design
1λ 2λ 1Lλ − LλΣ Σ
Σ Σ[ ]even samples 2x n
[ ]odd samples
2 1x n +
0low band y
1high band y
10K −
11K −
-
-
-
-
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 29
Example: lifting implementation of 5/3 filters
[ ]even samples 2x n
( )12
z− + 114z −+
Σ
Σ
1/2[ ]
odd samples 2 1x n +
0low band y
1high band y
Verify by considering response to unit impulse in even and odd input channel.
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 30
Conjugate quadrature filters
Achieve aliasing cancelation by
Impulse responses
With perfect reconstruction: orthonormal subband transform!Perfect reconstruction: find power complementary prototype filter
( )( ) ( )
10 0
1 11 1
( ) ( )
( )
h z g z f z
h z g z zf z
−
− −
= ≡
= = − [Smith, Barnwell, 1986]
Prototype filter
[ ] [ ] [ ][ ] [ ] ( ) ( )
0 0
11 1 1 1k
h k g k f k
h k g k f k+
= − =
= − = − − +⎡ ⎤⎣ ⎦
( ) ( )( ) 222jjF e F e ω πω ±+ =
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 31
Wavelet bases
( ) ( )
( )( ) ( ) ( )
( )
2
2
2
Consider Hilbert space of finite-energy functions .
Wavelet basis for : family of linearly independent functions
2 2
that span . Hence any signal
m m mn
x t
t t nψ ψ− −
=
= −
∈
x
x
L
L
L ( )( ) [ ] ( )
2 can be written as
m mn
m ny n ψ
∞ ∞
=−∞ =−∞
= ∑ ∑x
L
“mother wavelet”
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 32
Multi-resolution analysis
( ) ( ) ( ) ( ) ( ) ( )
( ) ( )( ) { }
( ) ( )
2 1 0 1 2 2
2
0
Upward compl
Nested subspaces
eteness
Downward complete
ness
Self-similari t fy i
m
m
m
m
V V V V V
V
V
x t V
− −
∈
∈
⊂ ⊂ ⊂ ⊂ ⊂ ⊂ ⊂
=
=
∈
0
… …
∪∩
Z
Z
L
L
( ) ( )
( ) ( ) ( ) ( )
( ) ( ) ( ){ } ( )
0 0
0
f 2
iff for all
There exists a "scaling function" with integer translates -
such that forms an orthonormal basis f
Translation i
or
nvaria
nce
mm
n
n n
x t V
x t V x t n V n
t t t n
V
ϕ ϕ ϕ
ϕ
−
∈
∈
∈ − ∈ ∈
=
Z
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 33
Multiresolution Fourier analysis
ω
( ){ } ( )span pn
p
nVϕ =
( ){ } ( )11span p pn n
Vϕ − −=
( ){ } ( )22span p pn n
Vϕ − −=
( ){ } ( )33span p pn n
Vϕ − −=
( ){ } ( )span pn
p
nWψ =
( ){ } ( )11span p pn n
Wψ − −=
( ){ } ( )22span p pn n
Wψ − −=
( ){ } ( )33span p pn n
Wψ − −=
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 34
Relation to subband filters( ) ( )
( ) [ ] ( ) ( )
( )
[ ] ( )
[ ] ( ) ( )
1
0 1
10 0
0 00
linear combinationof scaling functions in
Since , recursive definition of scaling function
2 2
Orthonormality
n ,
nn n
n
V
V V
t g n t g n t nϕ ϕ ϕ
δ ϕ ϕ
−
−
∞ ∞−
=−∞ =−∞
⊂
= = −
=
∑ ∑
[ ] ( ) ( ) [ ] ( ) ( )
[ ] [ ] ( ) ( ) [ ] [ ][ ]0
1 10 0 2
1 10 0 0 0
,
k unit norm and orthogonalto its 2-translates: correspondsto synthesis lowpass filter oforthonormal subband tra
2 , 2
i j ni j
i ji j i
g
g i t g j t dt
g i g j n g i g i n
ϕ ϕ
ϕ ϕ
∞− −
+−∞
− −
⎛ ⎞= ⎜ ⎟
⎝ ⎠
= − = −
∑ ∑∫
∑ ∑
nsform
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 35
Wavelets from scaling functions( ) ( ) ( )
( ) ( ) ( ) ( ) ( )
( ){ } ( ) ( )
( ) [ ] ( ) ( )
1
1
0 0 1
11
linear combinationof scaling functions
is orthogonal complement of in
and
Orthonormal wavelet basis for
p p p
p p p p p
n
nn
W V V
W V W V V
W V
t g n t
ψ
ψ ϕ
−
−
−
∞−
=−∞
⊥ ∪ =
⊂
= ∑
( )
[ ] ( )
[ ] ( ) ( )( ){ } ( ){ } ( )
1
1
11 0
0 0 1
in
2 2
Using conjugate quadrature high-pass synthesis filter
The mutually orthonormal
1 1
Easy
functions, and , together span .
to e
nn
n
n
n n n
V
g n t n
g n g n
Vϕ
ϕ
ψ
−
−
∞
∞
∈
+
∈
=−
= −
= − − −⎡ ⎤⎣ ⎦
∑
Z Z
( )( ){ } ( )2
,
xtend to dilated versions of to construct orthonormal wavelet basis
for .
mn n m
tψ
ψ∈Z
L
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 36
Calculating wavelet coefficients for a continuous signal
Signal synthesis by discrete filter bank
Signal analysis by analysis filters h0[k], h1[k]Discrete wavelet transform
( ) ( ) ( ) [ ] ( ) ( ) [ ] ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) [ ] ( )
( ) ( ) ( )
( ) [ ] ( )
( ) ( ) ( )1 1 1 1
0 0 0 0 00 0
1 1 1 1
0 1 1 1 10 1
Suppose continuous signal
Write as superposition of and
nn n
n ni j
x t V w t W
x t y n t n y n V
x t V w t W
x t y i y j
ϕ ϕ
ϕ ψ
∈ ∈
∈ ∈
∈ ∈
= − = ∈
∈ ∈
= +
∑ ∑
∑ ∑
Z Z
Z Z
( ) ( ) [ ] [ ] ( ) [ ] [ ]( ) [ ]00
0 1 10 0 1 1 2 2n
n i j
y n
y n g n i y j g n iϕ∈ ∈ ∈
⎛ ⎞= − + −⎜ ⎟
⎝ ⎠∑ ∑ ∑
Z Z Z
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 37
0h0g
1g
1h
0g
1g
0h
1h
1-d Discrete Wavelet Transform
( ) [ ]00y n
( ) [ ]10y n
( ) [ ]11y n
( ) [ ]20y n
( ) [ ]21y n
( ) [ ]10y n
( ) [ ]00y n
Sampling( )x t
( )Interpolation
tϕ
( )x t
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 38
Example: Daubechies wavelet, order 2
ϕ ψ
0h
0g
1h
1g
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 39
Example: Daubechies wavelet, order 9
ϕ ψ
0h
0g
1h
1g
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 40
Comparison JPEG vs. JPEG2000
Lenna, 256x256 RGBBaseline JPEG: 4572 bytes
Lenna, 256x256 RGBJPEG-2000: 4572 bytes
Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 41
Comparison JPEG vs. JPEG2000
JPEG with optimized Huffman tables8268 bytes
JPEG20008192 bytes
top related