lecture1 2012 2013sgn-3057: digital image processing ii texts • lecture material (see webpage)...

74
SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/ SGN-3057: General Information Lectures given by Karén Egiazarian (e-mail: firstname.lastname<at>tut.fi) Webpage: www.cs.tut.fi/courses/SGN-3057/ Schedule lectures: 8 weeks, 4 hours/week, Time and place: Mondays 12:15 – 14:00, TB219 period III Wednesdays, 14:15 – 16:00, TB 222 Goals In-dept view of selected topics of image processing Practical tasks in image processing laboratory

Upload: others

Post on 13-Oct-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

SGN-3057: General Information

Lectures given by Karén Egiazarian (e-mail: firstname.lastname<at>tut.fi)

•  Webpage: www.cs.tut.fi/courses/SGN-3057/ •  Schedule lectures: 8 weeks, 4 hours/week,

•  Time and place: Mondays 12:15 – 14:00, TB219 period III Wednesdays, 14:15 – 16:00, TB 222

Goals •  In-dept view of selected topics of image processing •  Practical tasks in image processing laboratory

Page 2: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

General Information (cont.)

Pre-requisites and position •  Basic or introductory signal processing •  Digital Image Processing I or Digitaalinen kuvankäsittely I

Grade formation •  No exam. •  Final mark is computed based on four units as follows: •  Classroom exercises, 40% of the mark •  First laboratory work, 20% of the mark (IV period) •  Second laboratory work, 20% of the mark (IV period) •  Third laboratory work, 20% of the mark (IV period) •  All four units must be passed otherwise final mark is not given

Page 3: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

•  A total of 7 exercises.. •  40% mandatory to pass the course. •  Two exercise groups • -Group A: Tuesdays,

12:15-13:45, TB215 • - Group B: Tuesdays,

16:15-17:45, TB 215

• Each class can take approx. 25-30 people – equal division into groups. Students prepare the answers at home and present their solution at the whiteboard (with comments) if asked.

Exercises Pavlo Molchanov firstname.lastname<at>tut.fi

Page 4: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Texts •  Lecture material (see webpage) •  Textbook

[1] R. Gonzalez and R. Woods, Digital Image Processing, 2nd ed., Prentice-Hall, 2002.

•  Additional books

[2] John Russ, The Image Processing Handbook, Fourth Edition, CRC Press, 2002 [3] Y. Shi, H. Sun, Image and Video Compression for Multimedia Engineering: Fundamentals, Algorithms, and Standards, CRC Press, 1999 [4] K. Rao and P. Yip, The Transform and Data Compression Handbook, CRC Press, 2001 [5] V.Katkovnik, K.Egiazarian, and J.Astola, Local Approximations in Signal and Image Processing, SPIE Press, vol. PM157, 2006.

Page 5: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelets and Multiresolution Processing

•  Preview – Good old Fourier transform

•  A transform where the basis functions are sinusoids, hence localized in frequency but not localized in time; in transform domain the temporal information is lost

– Short-time Fourier transform – Wavelets

•  Varying frequency and limited duration; an attempt to reveal both the frequency and temporal information in the transform domain

– Multiresolution analysis •  Signal (image) representation at multiple resolutions

Page 6: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Background •  Images

–  Connected regions of similar texture combined to form different objects

•  Small sized or low in contrast objects: examined at high resolution •  Large sized or high in contrast: examined at a coarse view •  Several resolutions needed to distinguished between different

objects

–  Mathematically images are 2-D arrays of intensity values of locally varying statistics (resulting from edges, homogenous regions, etc. – see Fig. 7.1)

Page 7: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Histograms computed in local squares

Page 8: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Basic ideas of linear transformation

•  To change the coordinate system in which a signal (image/video) is represented in order to make it much better suited for processing (e.g.,compression).

•  To represent all the useful signal features and important phenomena in as compact manner as possible.

•  Important to compact the bulk of the signal energy into the fewest number of transform coefficients.

Page 9: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Four properties of a Good Transform:

Provide good energy compaction

Desirable to be orthogonal

Decorrelate the image pixels

There is no universal good transform. Among block transforms – KLT is statistically optimal (first 3 properties fulfiled, but 4th – no). Instead, in image processing, fixed ”suboptimal” transforms are used, such as DFT, DCT, lapped transforms, wavelets, etc.

efficient computational algorithm should exist

Page 10: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Introduction: Fourier, Short-time Fourier

transforms and modulated filter banks Classical Fourier transform:

•  ‘uncertainty principle’ u[k] has a narrow support (localized), then U(.) has a wide support (non-localized), and vice versa

•  Notion of ‘frequency that varies with time’ not accommodated

πωωω 20 , ].[)( ≤≤= ∑+∞

−∞=

k

kjj ekueU ∫=π

ωω ωπ

2

0

).(21][ deeUku kjj

Page 11: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Fourier transform acts as an optical prism

Fourier transform (Fourier spectrum)

Page 12: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Short-time Fourier (STFT) transform

+∞<<∞−≤≤−= ∑+∞

−∞=

− nenkwkuneUk

kjjSTFT , 20 ].[].[),( πωωω

w[k] is a window function (Gabor transform w[k] is Gaussian)

Page 13: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

STFT can be re-writen as

∑+∞

−∞=

−−− −=k

nkjnjjSTFT enkwkueneU )(].[].[.),( ωωω

This is a convolution with a filter njenw ω].[−

Computing it for discrete frequencies 1,...,0 , 2. −== NkN

kkπ

ω

make a set of filters ][][ , ].[][ 0/.2

0 nwnhenhnh Nnkjk −== π

This leads to a DFT-modulated filter bank with a prototype filter – window function

Page 14: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Image Pyramids

•  Creation by iterated approximations and interpolations (predictions)

•  The approximation output is taken as the input for the next resolution level

•  For a number of levels P, two pyramids: approximation (Gaussian) and prediction residual (Laplacian), are created

•  approximation

filters could be: •  averaging

•  low-pass (Gaussian)

Page 15: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Gaussian and Laplacian Pyramids

J

J – 1

J – 2 J – 3

Starting image is 512×512; convolution kernel (approximation kernel) is 5×5 Gaussian

Different resolutions are appropriate for different image objects (window, vase, flower, etc.)

Page 16: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Subband Coding

•  The image is decomposed into a set of band-limited components (subbands).

•  Two-channel, perfect reconstruction system; two sets of half-band filters

•  Analysis: h0 – low-pass and h1 – high-pass and Synthesis: g0 – low-pass and g1 – high-pass

Page 17: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Brief reminder of z-transform properties

[ ] [ ]

[ ] )()(,....5,3,10,...4,2,0)2/(

)()(2

)()(21)( )2()()(2

2

2/12/1

zXzXnnnx

xxnx

zXzXzXnxnxnx

upup

downdown

=⇔⎩⎨⎧

=

===↑

−+=⇔==↓

Downsampled and upsampled by a factor of two sequences in z-domain

[ ][ ] [ ]

( ) )()1()(

)()(21)(ˆ )(22)(ˆ

1 nxzXZ

zXzXzXnxnx

n−=−

−+=⇔↓↑=

Downsampling followed by upsampling

Page 18: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

2-channel filter bank: Analysis bank

G’

H’ 2

2

x s

d

lowpass channel

highpass channel

•  H’ is the lowpass filter and G’ is the highpass filter. •  2 is the downsampling operator: (1 3 4 6 5) (1 4 5).

Page 19: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

2-channel filter bank: Synthesis bank

•  H is the lowpass filter and G is the highpass filter. •  2 is the upsampling operator: (1 4 5) (1 0 4 0 5).

G

H 2

2

s

d

lowpass channel

highpass channel

y

Page 20: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

A biorthogonal filter bank

G’

H’ 2

2 x

s

d

G

H 2

2

s

d y

Biorthogonal (or perfect) filter bank: if y=x for all inputs x .

Page 21: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

An orthogonal filter bank

G’

H’ 2

2 x

s

d

Orthogonal filter bank: if it is biorthogonal, and both analysis filters H’ and G’ are the time reversals of the synthesis filters H & G: H=(1, 2, 3) H’=(3, 2, 1).

G

H 2

2

s

d y

=H

Page 22: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Subband Coding

[ ][ ]

[ ]

[ ]

[ ]

[ ]

[ ]

⎥⎦

⎤⎢⎣

−−

−==⎥

⎤⎢⎣

⎥⎦

⎤⎢⎣

−==⎥

⎤⎢⎣

=−+−

=+

=

−−+−+

++=

−−++

+−−+=

−−+=

−−+=

⇔∗

)()(

(z))det(2

)()(

)()()()(

)( where0], [2)()()()(

)( )(

0)()()()( )22)()()()( )1

:conditions PR Two

)()(ˆ :PR) -tion reconstruc(perfect free-errorFor

)()()()()(21

)()()()()(21)(ˆ : termsRearanged

)()()()()(21

)()()()()(21)(ˆ :branch Synthesis

)()()()(21)(

)()()()(21)(

:branch Analysis

)()()()(

0

1

1

0

11

00

11

0010

1100

1100

1100

1100

111

000

2/12/11

2/12/111

2/12/10

2/12/100

00

zHzH

zGzG

zHzHzHzH

zzHzHzHzH

zGzG

zGzHzGzHzGzHzGzH

zXzX

zXzGzHzGzH

zXzGzHzGzHzX

zXzHzXzHzG

zXzHzXzHzGzX

zXzHzXzHzY

zXzHzXzHzY

zXzHnxnh

m

m

H

H

Page 23: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Subband Coding

}1,0{, ),()()2(),( :conditions erestrictiv More

}1,0{, ),()()2(),( :conditions

0)2(),( ;0)2(),( );()2(),( :shown that also becan It

)()2(),()2()( :cancel termsindexed Odd

)(2)()()1()()(:transform-z inverseBy

2)()()()( : totransformscondition PRfirst The

)()()((z))det(

2)()( :(-z))det((z))det( Since

)()((z))det(

2)()()( filters synthesis and analysis pas-low theofproduct theTake :alityBiorthogon

)()1()()()1()(

)()(

)()(

: 2for or )()1()(

)()1()()(

)()()(

of trasform-z inverse Take

2let and delay term omit theLet

(z))det( i.e. delay, pure a is (z))det( filters FIRFor

011011

0000

0000

0000

101!

1000

01

11

0

0

1

1

0

01

1

10

0

1

1

0

)12(

)12(

=−=+

=−=−

=−=−=−

=−=−

=−−+−

=−−+

−=−−

=−=

−==

−=

−=⇔⎥

⎤⎢⎣

−−

−−=⎥

⎤⎢⎣

⎡=

−=

−=⇔⎥

⎤⎢⎣

−−

−=⎥

⎤⎢⎣

=

=

∑∑∑

+

+

+−

+−

jimijkmgkg

jinijknhkg

knhkgknhkgnknhkg

nknhkgknhkg

nknhkgknhkgzGzHzGzH

zPzHzHzGzH

zHzHzGzHzP

nhngnhng

zHzH

zGzG

-anhngnhng

zHzH

zGzG

azaz

ij

ij

k

kn

k

mmm

m

n

n

n

n

k

kmm

δδ

δδ

δ

δ

δ

orthogonal

alBiorthogon

HHH

H

HH

Page 24: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Families of half-band PR filters

•  For each family of filters, a prototype filter is designed and the other filters are computed from the prototype.

•  QMF – quadrature-mirror filters

•  CQF – conjugate-quadrature filters

•  2K is the length (number of coefficients, filter taps)

}1,0{ ),12()()12()1()( 01

=−−=

−−−=

inKgnhnKgng

ii

n

Page 25: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Separable filtering in 2-D (images)

•  Vertical followed by horizontal filtering

•  Four output subbands:

• approximation, and vertical, horizontal, and diagonal detail subbands

Page 26: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Example: 8-tap orthonormal filter designed by Daubechies

Page 27: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Example: four band subband filtering of an image

Aliasing is presented in the vertical and horizontal detail subbands. It is due to the down-sampling and will be canceled during the reconstruction stage.

Page 28: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

•  Wavelet transform decomposes a signal into a set of basis functions.

•  These basis functions are called wavelets

•  Transform a continuous function into a highly redundant function of two variables – translation (t) and scale (s).

Continuous Wavelet transform

ωω

ω

τψ

τ

τψψ

ψτ

ψ

τψ

ψ

τ

τψ

dC

dsdsx

sWC

xf

sx

sx

xxfsW

s

s

s

∫ ∫

∞−

∞ ∞

∞−

∞−

Ψ=

=

−=

=

2

0 2,

,

,

)( where

)(),(1)(

)(1)( where

)()(),(

Page 29: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Comparison between Fourier Transform and Continuous WT

Page 30: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

•  The standard Fourier Transform (FT) decomposes the signal into individual frequency components.

•  The Fourier basis functions are infinite in extent. •  FT can never tell when or where a frequency occurs. •  Any abrupt changes in time in the input signal f(t) are spread

out over the whole frequency axis in the transform output F(ω) and vice versa.

•  WT uses short window at high frequencies and long window at low frequencies (recall a and b in previous formula). It can localize abrupt changes in both time and frequency domains.

Wavelet transforms vs. Fourier transform

Page 31: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Discrete Wavelet Transform (DWT)

∑ ∑∑

=+=

=

=

00

0

)(),(1)(),(1)(

)()(1),(

)()(1),(

,,0

,

,0

jj k kjk kj

x kj

x kj

xkjWM

xkjWM

xf

xxfM

kjW

xxfM

kjW

ψϕ

ψ

ϕ

ψϕ

ψ

ϕ

Transform pair (forward and inverse transform) applicable for discrete signals (or sampled signals)

Here x is a discrete variable: x = 0,1,2,…M - 1 Normally, M = 2J; j0 = 0; j = 0,1,2,…J-1 and k = 0,1,2,..2 j-1 Wϕ and Wψ are referred as approximation and detail coefficients respectively.

Page 32: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

•  The Haar “aperture” function is

•  Haar’s theorem (1905): All Haar wavelets , together with the constant function 1, consist into an orthonormal basis for the Hilbert space of all square integrable functions on [0, 1].

).(1)(1)( 12/12/10harr xxx xx <≤<≤ −=ψ

1

-1

11/2

haar,kjψ

Haar Transform (oldest and simplest wavelets)

Page 33: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Let’s consider a 1D 4-pixel Image [ 9 7 3 5]

[ 9 7 3 5]

(9 + 7)/2 (3 + 5)/2

[ 8 4 ] (9 - 7)/2 (3 - 5)/2

[ 1 -1 ]

(8 + 4)/2

6 (8 – 4)/2

2

[ 6 2 1 -1 ]

WT

Haar Transform (example)

Page 34: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Haar Transform (cont.)

2

24

2/

2/

000

of rows in the are filters analysis QMF ingcorrespond of tscoefficien The bank.filter FIR tap- twodefine functions basisHaar The

1111

21 ;

220000221111

1111

41

./)1(,...,/2,/1,/0for )( of elements thecontainsmatrix Haar theof rowth -i The

]1,0[ otherwize2/2/)5.0(

2/)5.0(2/)1(

02

21)()(

]1,0[ ,1)()(

:0for 21 and 0for 1or 0 ,10 where,12such that integer thedefine we, generate To

.2 where,1,...1,0for ]1,0[ interval closed ,continuous over the defined areThey .)( functions basisHaar thecontains matrix. tscoefficien transformresulting theis and matrix, transforman is matrix; image an is where

H

HH

HH

THFHFHT

⎥⎦

⎤⎢⎣

−=

⎥⎥⎥⎥

⎢⎢⎢⎢

−−=

−=

<≤−

−<≤−

⎪⎩

⎪⎨

−==

∈==

≠≤≤==−≤≤−+=

=−=∈

××=

NNNNNzzh

zqzqqzq

Nzhzh

zN

zhzh

pqpqnpqkkNNkzzh

NNNN

i

pp

pp

p

p

pqk

pp

nk

Page 35: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Haar Functions in a Discrete Wavelet Transform

Page 36: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelet transform of images

Wavelets work for decomposing signals (such as images) into hierarchy of increasing resolutions: as we consider more layers, we get more and more detailed look at the image.

Page 37: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

From http://www.maths.bris.ac.uk/~wavethresh/

Page 38: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Another Example: The Shannon wavelets

•  The Shannon’s “aperture” function is:

•  Theorem:

is an orthonormal basis of L2(R). },:)({Shannon

Zkjxj,k ∈ψ

).( sinc )2( sinc 2)(Shannon

xxx −=ψ

Page 39: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Shannon wavelets (cont’d)

•  How to visualize the orthonormal basis ? Answer: go to the Fourier domain !

•  According to Shannon:

–  All signals bandlimited to (-π, π) can be represented by sinc(x-n)… –  those bandlimited to (-2π, π ) U (π, 2π), by –  those bandlimited to (-4π, 2π ) U (2π, 4π), by –  ...

).( sinc )2( sinc 2)(shannon

xxx −=ψ

pi-pi

Fourier of 2sinc(2x)Fourier of sinc(x)

-2pi 2pi

).( nx −ψ).2(2,1 nxn −= ψψ

Page 40: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

•  According to Shannon: –  All signals bandlimited to (- , ) can be represented by sinc(x-n)… –  those bandlimited to (-2 , - ) U ( , 2 ), by –  those bandlimited to (-4 ,-2 )U(2 , 4 ), by –  ...

. . . . . .

π π

π π

π π ππ

ππ ).( nx −ψ).2(2,1 nxn −= ψψ

.s' )(by drepresente nx −ψ

.s' )2(2by drepresente n1, nx −= ψψ

.s' )2/(2/1by drepresente n1,- nx −= ψψ

π π4π22π

ξ

Shannon wavelets (cont’d)

Page 41: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Partition of the time-frequency plane

•  Heisenberg’s uncertainty principle requires that each TF atom must have:

•  Thus, for an optimal localization, the “life time” of an atom must influence its scale or frequency content.

frequency

time

sinc(x-n)

sinc(2x-m)

sinc(4x-k)

.2π≥Δ⋅Δ xt

Page 42: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Page 43: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Multiresolution Expansions

In Multiresolution Analysis (MRA) a scaling function is used to create a series of approximations of a function (image), each differing by a factor of two. Additional functions, wavelets, are used to encode the difference between adjacent approximations.

•  Series Expansions •  Linear combination of expansion functions:

•  Basis functions and function space:

•  Dual functions:

•  Cases: orthonormal basis, biorthogonal basis, overcomplete expansions

∑= k kk xxf )()( ϕα

{ })(span xV kk

ϕ=

{ } dxxfxxfxx kkk ∫∗== )()(~)(),(~such that )(~ ϕϕαϕ

Page 44: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Multiresolution Expansions

•  Scaling Functions •  Set of functions {ϕ j,k(x)} formed by integer translations and binary scaling of a prototype function ϕ (x): •  translation (position) parameter k; scale parameter j •  scaling functions can be made to span the space of all measurable, square-integrable functions, denoted by L2(R) •  by fixing the scale to j=j0, the resulting set spans a subspace of L2(R)

•  if f (x)∈Vj0 then

)2(2)( 2/, kxx jjkj −= ϕϕ

{ })(span ,00xV kj

kj ϕ=

∑= k kjk xxf )()( ,0ϕα

Page 45: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

The Haar Scaling Function

)(21)(

21)( 12,12,1,0 xxx kkk ++= ϕϕϕ

Any V0 expansion function can be represented by V1 expansion functions, i.e. V0 ⊂ V1.

Page 46: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Basis for Vector space Vj

•  Basis functions for Vj are called scaling functions and are denoted by φ.

v A simple basis for Vj is given by the set of scaled and translated box functions:

φij(x) : = φ (2j x - i) i = 0,…, 2j – 1 where φ(x) := 1, for 0 ≤ x <1

0, otherwise

v Example basis for V2:

Page 47: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

MRA Requirements

MRA requirements 1:

The scaling function is orthogonal to its integer translates

MRA requirement 2: The subspaces spanned by the scaling function at low scales are nested within those spanned at higher scales ∞−∞− ⊂⊂⊂⊂⊂ VVVVVV ...... 2101

MRA requirement 3: The only functions that is common to all Vj is

f (x) = 0

MRA requirement 4: Any function can be represented with arbitrary precision

Page 48: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Dilatation Equation

)2(2)()(

equation dilatation called-so obtain the we0 to and both setting

)2(2)()(

)()(12/)1(

,

,1,

nxnhx

kj

nxnhx

xx

n

jn

jkj

njn nkj

−=

−=

=

∑∑

++

+

ϕϕ

ϕϕ

ϕαϕ

ϕ

ϕ

Under the MRA requirements, the expansion functions of subspace Vj can be expressed as a weighted sum of the expansion functions of subspace Vj+1.

Page 49: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelet Functions

Define a wavelet function ψ (x) that, together with its integer translates and binary scalings, spans the difference between any two adjacent scaling subspaces

{ }

jjj

k kjkj

kjk

j

jjkj

WVV

xfWxf

xWkxx

⊕=

=⇒∈

=

−=

+

∑1

,

,

2/,

)()( if

)(span

)2(2)(

ψα

ψ

ψψ

We call Wj orthogonal complement of Vj in Vj+1. 0)(),( ,, =xx ljkj ψϕ

Page 50: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelet Functions

The space of measurable, square-integrable functions can be expressed as

......)(

...)(

...)(

210122

2112

1002

⊕⊕⊕⊕⊕=

⊕⊕⊕=

⊕⊕⊕=

−− WWWWWLWWVLWWVL

RRR

Any wavelet function can be expressed as a weighted sum of shifted, double-resolution scaling functions

)1()1()( where

)2(2)()(

nhnh

nxnhxn

n

−−=

−=∑ϕψ

ψ ϕψ

Page 51: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Definition of Wavelets

•  Define Wj as the orthogonal complement of Vj in Vj+1, i.e. Wj ⊕ Vj=Vj+1

•  A collection of linearly independent functions ψij(x) spanning Wj are called wavelets: – The basis functions ψij(x) and φij(x) form a basis

in Vj+1

– For each j ψij(x) orthogonal to φij(x) – Wavelets are orthogonal to each other

Page 52: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Haar Wavelets revisited

•  The wavelets corresponding to the box basis are known as Haar Wavelets:

ψij(x) : = ψ (2j x - i) i = 0,…, 2j – 1 where 1, for 0 ≤ x <1/2

ψ(x):= -1, for 1/2 ≤ x <1 0, otherwise

v Example: Haar Wavelets for W1

Page 53: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Haar Wavelets example revisited

Page 54: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Haar Wavelet Function Coefficients

)(82)(

42)(

)(82)(

423)(

)()()(

elsewhere15.05.00

01

1)( :functionet Haar wavel

2/1)1(;2/1)0( :filteret Haar wavel

2/1)1()0( :filter scalingHaar

2,00,0

2,00,0

xxxf

xxxf

xfxfxf

xx

x

hh

hh

d

a

da

ψψ

ϕϕ

ψ

ψψ

ϕϕ

−−

=

−=

+=

⎪⎩

⎪⎨

<≤

<≤

−=

−==

==

Page 55: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelet Series Expansions

)(),()(

)(),()(

)()()()()(

,

,

,,

00

000

xxfkd

xxfkc

xkdxkcxf

kjj

kjj

jj k kjjk kjj

ψ

ϕ

ψϕ

=

=

+= ∑ ∑∑∞

=

Consider the orthogonal case

c(k) are referred as approximation coefficients and d(k) are referred as detail coefficients. Example:

⎩⎨⎧ <≤

=otherwise

10

0)(

2 xxxf

Page 56: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Fast Wavelet Transform (FWT) In brief

Analysis filter bank Synthesis filter bank

Page 57: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

A Two-stage (Two-scale) FWT: Analysis

Page 58: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

A Two-stage (Two-scale) FWT: Synthesis

Page 59: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Example: Two-scale FWT by Haar

Page 60: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Example: Two-scale FWT by Haar

Page 61: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelet Packets

The Wavelet Transform decomposes a function into a series of logarithmically related frequency bands. Low frequencies are grouped into narrow bands, while the high frequencies are grouped into wider bands. Wavelet Packets are generalization that allows greater control over the time-frequency plane partitioning. Consider the two-scale filter bank as a binary tree. The wavelet coefficients are at the nodes of the tree. The root node contains the highest-scale approximation coefficients (i.e. the sampled signal itself). Each node contains coefficients representing different subspaces – subspace analysis tree.

Page 62: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

WT: Filter Bank, Analysis Tree, and Spectrum Splitting

Page 63: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelet Packets

Wavelet packets are conventional wavelet transforms in which the details are also iteratively filtered.

Subscripts in the figure show the scale and a string of A’s and D’s encoding the path from the parent to the node.

The packet tree almost triples the number of decompositions.

Page 64: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelet Packets: Filter Structure and Spectrum Splitting

Page 65: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelet Packets

The frequency selectivity can be increased toward high frequencies

Page 66: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

2-D Wavelet Packets

In 2-D, separable implementation of wavelet packets leads to a quad-tree structure. The frequency spectrum is divided into four areas. The low-frequency area is in the centre.

Page 67: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Portion of 2-D Wavelet Packet Tree

Page 68: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Wavelet Packet Decomposition

Page 69: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Optimal Wavelet Packet Basis Search

Entropy-based criteria for searching the optimal (the best) wavelet packet decomposition

An additive cost function for comparing different decompositions

Page 70: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Optimal Wavelet Packet Basis

Page 71: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Some applications •  FBI fingerprints. •  JPEG2000. •  Image indexing and image search engines •  Image modeling •  Image denoising, deblurring and restorations. •  Texture analysis •  Direct processing tools on the wavelets domain •  Time series analysis, etc.

Page 72: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Some extensions

•  Random Wavelets Expansion (RWE) by Mumford-Gidas [2001], to model the scale-invariance of general images.

•  Geometric Wavelets:

–  D. Donoho (Stanford): ridgelets, wedgelets, curvelets. –  S. Mallat (Ecole Polytechnique,France)[2001]: beamlets. –  T. Chan & H.-M. Zhou (UCLA) [2000], A. Cohen [2002]:

integrate computational PDE techniques such as the ENO scheme into wavelet transforms, to better capture shocks (discontinuities).

Extra material for reading is in the webpage of the course

Page 73: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

Let’s repeat what we have learned. Watch the WaveletMovie

(http://www.loni.ucla.edu/~dinov/WAIR.dir/WaveletMovie_caption.html)

Page 74: lecture1 2012 2013SGN-3057: Digital Image Processing II  Texts • Lecture material (see webpage) • Textbook [1] R. Gonzalez and R. Woods, Digital

SGN-3057: Digital Image Processing II www.cs.tut.fi/courses/SGN-3057/

QUESTIONS?

See you next week