series expansion with wavelets - tu graz

Advanced Signal Processing 2 - 2007Reinisch Bernhard

Teichtmeister Georg

Series Expansion with Wavelets

2/60ASP2 2007Teichtmeister G. Reinisch B.

Wavelet construction

Introduction

● Series expansion● Fourier Series: Either periodic or bandlimited

signals● Timedomain: No frequency information● Fourierdomain: No time information● Is there something between?



Contents

● Basics of signal representation● Wavelets

– Haar wavelet– Multiresolution analysis– Construction of the Sinc - Wavelet

● Wavelets derived from iterated filter banks– Haar case, Sinc case, general construction

● Wavelet series and its properties● Practical outlook (image processing)



Recap of Series expansion

• Signals are points in a Vectorspace• Time-Domain: Basis functions are infinite short

impulses• Signals can be projected onto other basis

functions

∫

∑∞

∞−

∞

−∞=

=

=

duufuufu

tufutf

kk

kkk

)()()(),(

)()(),()(

*ϕϕ

ϕϕ



Possible Basis Functions

• Fourierseries– periodic

• Fouriertransform– bandlimited

• STFT– Infinite set of Fourier Transforms

• Piecewise Fourier Series• Wavelets

∑∞

−∞=

=k

TktjekFtf /)2(][)( π

∫−

=π

π

ωω ωπ

deeXnx njj )(21][



Short Time Fourier Transform

● Window Signal● Compute the Fourier Transform● Shift window and repeat⇒ Spectrogram, Periodogram



Time and Frequency Resolution

• Window has Energydistribution in both: Frequency (σω) and Time (σn).

• Uncertainty principle:

• Optimality is only reached by Gaussian window21≥nσσω



STFT T/F-Resolution

● Constant over Time and Frequency



Piecewise Fourierseries

● Fourier Series with non-overlapping rectangular windows in time and periodic expansion

● Why?– Overlapping windows are redundant information– Good Time Resolution– Representation of arbitrary functions

● Bad Frequency Resolution● Errors at boundaries



Desired Features of Basis Functions

● Simple characterization● Localization Properties in Time and

Frequency● Invariance under certain operations● Smoothness properties● Moment properties



Haar - Expansion

• Simplest Wavelet Expansion

• Scaled and shifted Wavelets:

• m … Scale• n … Timeshift

)2(2)( 2/, ntt mmnm −= −− φφ



Dyadic Tiling

• Resolution depends on Frequency now• )2(2)( 2/

, ntt mmnm −= −− φφ



Orthonormal Basis for L2?

● Two wavelets on the same Scale have no common support

● Shorter wavelet always averages to zero● Shifting so that jump matches, is not possible



Proof: Definitions

• Consider functions which are constant on

• and have finite support on

• Can approximate L2 arbitrarily well• We call it

]2)1(,2[ 00 mm nn −− +

]2,2[ 11 mm−

)()( 0 tf m−



Proof: Scaling function

• The scaling function

• Approximating the piecewise constant function⎪⎩

⎪⎨⎧

+<≤=−−

−

otherwisentnt

mmm

nm0

2)1(22)( 000

0

2,ϕ

)2(2

2

)(

0020

0

10

0

00

)()(

1

,)()(

mmmn

mm

N

Nnnm

mn

m

nff

N

ftf

m−−−

+

−

−=−

−−

−

=

=

= ∑ ϕ



Proof: Illustration



Proof: Keystep

• Examination of two adjacent Intervals

• Now can be expressed as

• For m0=0, n=1, this means

[ ) [ )0000 2)22(,2)12( and 2)12(,22 mmmm nnnn −−−− +++

)(3)(2)()( [3,4) and )3,2[

3,02,03,0)0(

32,0)0(

2 tttftf ϕϕϕϕ +=+

)()( 0 tf m−

)()( 12,)(

2,)(

0

0

120

0

2tftf nm

mnm

mnn +−−

−−

++ ϕϕ



Proof: Average and Difference

The function can also be expressed as the average

and the difference

over two intervals

)()( 0 tf m−

)(22 ,1

)(12

)(2

0

00

tffnm

mn

mn

+−

−+

− + ϕ

)(22 ,1

)(12

)(2

0

00

tffnm

mn

mn

+−

−+

− − φ



Proof: Coefficients

• With

• We get

( )

( ))(12

)(2

)1(

)(12

)(2

)1(

000

000

212

1

mn

mn

mn

mn

mn

mn

ffd

fff

−+

−+−

−+

−+−

−=

+=

)()(

)()(

,1)1(

,1)1(

12,)(

2,)(

0

0

0

0

0

0

120

0

2

tdtf

tftf

nmm

nmm

nmm

nmm

nn

nn

+−+−

+−+−

+−−

−−

+

=++

φϕ

ϕϕ



Proof: Finalization

• Applying all the things we can write

• Repeating the Average/Difference scheme for higher scales leads to

∑∑−

−=+−

+−−

−=+−

+−

+−+−−

+=

=+=12/

2/,1

)1(12/

2/,1

)1(

)1()1()(

)()(

)()()(

0

0

0

0

000

N

Nnnm

mn

N

Nnnm

mn

mmm

tdtf

tdtftf

φϕ

M

Mm

mm nnm

mn

m

mm nnm

mn

mm

mm

mm

mm

mm

td

tdtftf

εφ

φ

+=

=+=

∑ ∑

∑ ∑+

+−=

−

−=

+−=

−

−=

−

−

−

−

−

1

0

1

1

1

0

1

1

0

1

12

2,

)(

1

12

2,

)()1()(

)(

)()()(



Proof: Illustration



Multiresolution

● Successive approximation● Coarse approximation + added details● Coarse and detail subspace are orthogonal● Leads to self-similar Wavelets in Scale● Useful for applications



Axiomatic Definition (1)

• Sequence of embedded closed subspaces

• Upward Completeness

• Downward Completeness

KK 21012 −− ⊂⊂⊂⊂ VVVVV

)(2 RLVmZm

=∈U

}0{=∈

mZmVI



Axiomatic Definition (2)

• Scale Invariance

• Shift Invariance

• Existance of a orthonormal Basis– Non-orthogonal Basis can be orthogonalized

0)2()( VtfVtf mm ∈⇔∈

ZnVntfVtf n ∈∀∈−⇒∈ )()( 0



Orthogonal Complements

• Vm is a subspace of Vm-1

• We define Wm the orthogonal subset of Vm in Vm-1•• Vm is the space of the scaling functions• Wm the space of the wavelets• By repeating we get

mmm WVV ⊕=−1

MZmWRL

∈⊕=)(2



Constructing the Sinc Wavelet

• Now the scaling functions will be the space of bandlimited functions

• V0 is bandlimited between [-π, π], V-1 between[-2π, 2π]

• W0 the functions bandlimited to [-2π, -π] combined with [π, 2π]

•001 WVV ⊕=−



Scaling function

• The scaling function is given by

ttt

ππϕ sin)( =



Representation of φ

• V0 belongs to V-1

• φ(t) can be represented by basis functions of V-1

• Without proof

)(),2(2][ ;1][

)2(][2)(

00

0

tntngng

ntngtn

ϕϕ

ϕϕ

−==

−= ∑∞

−∞=

∑∈

−=

+−−=

Zn

n

ntngt

ngng

)2(][2)(

]1[)1(][

1

01

ϕφ



Construction Kernel

• g0 is given by

• And finally the wavelet⎩⎨⎧ ≤≤−−

=

=

−

otherwiseeeG

nnng

jj

02)(

2/)2/sin(

21][

220

0

ππωω ω

ππ

)2/3cos(2/

)2/sin()( tt

tt πππφ =



Sinc Wavelet: Illustration



Iterated filter banks

● Until now we constructed wavelets by scalingand shifting of orthonormal function families– Based on multiresolution analysis

● Different approach by filter banks– Iteration leads to a wavelet– Key proberties

● regularity● degree of regularity



Haar case

• Low- and Highpass

• Iterate the filter bank on the lowpass channel

• Multirate signal processing results

⎥⎦⎤

⎢⎣⎡ −=⎥⎦

⎤⎢⎣⎡=

21,

21;

21,

21

10 gg



Haar case

● size-8 discrete Haar transform

● Number of coefficients growth exponentially● Continuous time function

● Length bounded, piecewise constant



Sinc case (1)

( ) ]1[1][;2/

)2/sin(2

1][ 010 +−−== ngngn

nng n

ππ

● Impulse responses (low and highpass filter)

● Fourier transform

● Now consider the iterated filter bank– Upsampling filter impulse– Emulate the Haar construction with g0[n], g1[n]– And define a scaling function



Sinc case (2)

• For further analysis: important part is in the brackets• This product is 2i2π periodic -> in the end it‘s only a

perfect lowpass (sinc scaling function)

1

12/

10

(i)

00

20

200

)(0

1

12/2/)(

02/(i)

)(

2/)2/sin(

2)(

:rewritecan we

)(2

1)(M

:short)()...()()(

:where2/

)2/sin()(2)(

)(oftransformFourier

1

1

1

+

+−

=

+

+−−−

+

−

+

⎥⎦

⎤⎢⎣

⎡⎟⎠⎞

⎜⎝⎛=Φ

=

=

=Φ

∏ i

ij

i

kk

j

jjjji

i

ijjii

i

i

i

ii

eM

eG

eGeGeGeG

eeG

t

ωωωω

ω

ωωω

ϕ

ω

ω

ωωωω

ωω



Sinc case (3)

● Cumbersome way ● But we have gained a more general

construction● The key is the infinite product

– Does this product converge and to what– Converge to what kind of scaling function



Iterated filter banks cont. (1)

● General construction– Two channel orthogonal filter bank– g0[n], g1[n] are low- and highpass filter– Iterate on the branch of the lowpass filter and

process this to infinity– Express the two filters after i-steps– Multirate conclusions

● „Filtering with Gi(z) followed by upsampling by 2 isequivalent to upsampling by 2 followed by filtering wihGi(z2)„




● Discrete time iteratedfilters combined with thecontinuous time functions

● Normalization and rescaling

● Graphical function– piecewise constant– halving the intervall




● Fourier domain act as above● In the iteration scheme we are interesting in

convergence

● This will lead us to regularity discussion



Regularity

● The existence of the limit are criticalconditions– Limits exist if g0[n] are regular– Regular filter leads through iteration to a scaling

function with some degree of smoothness(regularity)

– But not only convergence is sufficient we needalso L2 convergence to build orthonormal bases

– A lot of sufficient conditions, different approaches



Wavelet series and properties

• Enumeration of some general properties of basisfunctions

• Wavelet– Linearity, Shift, Dyadic sampling and time frequency

tiling, Scaling, Localization, decay properties

∫

∑∞+

∞−

∈

==

=

dttfttftnmF

tnmFtf

nmnm

Znmnm

)(),()(),(],[

)(],[)(

,,

,,

ψψ

ψ



Linearity

● The wavelet series is linear. The prooffollows from the linearity of the inner product

))(())(()]()([R ba,any for then

)(),(n]F[m,T[f(t)]

Toperator suppose

nm,

tgbTtfaTtgbtfaT

tft

+=+∈

== ψ



Shift

• For Fourier transform– pair: f(t), F(ω) … f(t-tau), e-jωtau F(ω)

• Now for the wavelet series

[ ] ( ) ( )

[ ] ( ) ( )

[ ] mmknmFktfZkkZ

dttfntnmF

dttftnmF

mmm

mm

mmm

nm

<−↔−

∈=∈

+−=

−=

−

−

−−∞+

∞−

−

+∞

∞−

∫

∫

',2,)2(,2or 2

222,

,

''

2/'

,'

ττ

τψ

τψ



Scaling

• For Fourier transform– pair: f(t), F(ω) … f(at), 1/a*F(ω/a)

[ ] ( ) ( )

[ ] ( )

[ ]nkmFtfZka

dttfna

tanmF

dtatftnmF

kk

k

mm

nm

,2)2(,2

22/1,

,

2/'

,'

−↔

∈=

⎟⎟⎠

⎞⎜⎜⎝

⎛−=

=

−

−

−∞+

∞−

−

+∞

∞−

∫

∫

ψ

ψ



Dyadic sampling and time frequency tiling

• It is important to locate the basis functions in thetime-frequency plane

• sampling in time, at scale m, with period 2m

• The frequency is the inverse of scale, we find ifthe wavelet ist centered around ω0 then:

• This leads to dyadic sampling of time frequencyplane

( ) 2)( 0,, ntt mmnm −=ψψ

mnm 2/ around centered is )( 0, ωωΨ



Dyadic sampling

● The dots indicate the center of the wavelets● The scale axis is logarithmic



Time localization (1)

● Suppose we are interested in the signalaround t=t0

● Which values of F[m,n] carry informationabout signal f(t) at t0 => f(t0)

● Suppose wavelet ψ(t) is supported on theinterval [-n1, n2]

● Ψm,0(t) is supported on [-n12m, n22m]● Ψm,n(t) is supported on [(-n1+n)2m, (n2+n)2m]



Time localization (2)

• At scale m, wavelet coefficients with index n satisfy( ) ( )

10m-

20m-

201

22rewritten becan

22

ntnnt

nntnn mm

−≤≤−

+≤≤+−



Frequency localization (1)

• Suppose now in localization, but now in frequencydomain

( ) ( )

( )( ) ( )

( ) ( ) ωωωπ

ψ

ω

ψψ

ω

ω

deFnmF

dttftnmF

e

ntt

njmm

nm

njm

mmnm

m

m

∫

∫∞+

∞−

∞+

∞−

−

−−

Ψ=

=

Ψ

−=

22/

,

2m/2

2/,

2221],[

],[

22

is ansformFourier tr the22



Frequency localization (2)

• Suppose that the wavelet vanishes in the Fourierdomain outside the region [ωmin,ωmax]

• At specific scale m, the support of Ψm,n(ω) will be[ωmin/2m,ωmax/2m]

• Therefore, a frequency component ω0 incluencesat scale m

⎟⎟⎠

⎞⎜⎜⎝

⎛≤≤⎟⎟

⎠

⎞⎜⎜⎝

⎛

≤≤

0

max2

0

min2

max0

min

loglog

rewrite22

ωω

ωω

ωωω

m

mm



Decay properties

● Fourier series can be used to characterizethe regularity of a signal (decay of thetransfrom coefficients)– Global regularity

● The wavelet transform can be used in a similiar way – Local regularity



Multidimensional wavelets

● A way to build multidimensional wavelets isto use tensor products of their onedimensional counterparts– This will lead to different „mother“ wavelets– Scale changes are now represented in matrices– offers diagonal scaling but is also more restricted



Practical aspects

● Wavelets in matlab

● Images– Image compression– Edge detection– De-noising



Matlab

● Matlab wavelet toolbox– Command line

● Help wavelet– Gui tool (wavemenu)

● 1D wavelets analysis● 2D wavelets analysis● De-noising● Image Fusion● Compression



Example 1D wavelet transform

● Discrete/continuous wavelet transform● coefs = cwt(S, SCALES, ‚wname‘)



Compression (1)

● Wavelet calculation● Find small coefficients and discard them● Store only remaining coefficients● Lossy compression

● Good compression with a fast convergencespeed of the wavelet and good decay of thecoefficients



Compression (2)



Edge detection

original decomposition

approximationis set to zero

reconstruction



References

● [1] Wavelets and Subband Coding, Martin Vetterli, JelenaKovacevic, ISBN 0-13-097080-8

● [2] Wavelets – praktische Aspekte, Markus Grabner, VO2006

● [3] AK Computergrafik Bildverarbeitung und Mustererkennung WS 2006/07



Advanced Signal Processing 2 2007Teichtmeister GeorgReinisch Bernhard

Series Expansion with Wavelets

Thank you for you attention!

series expansion with wavelets - tu graz

Documents