siganl extension & dwt
DESCRIPTION
its good bookTRANSCRIPT
An Introduction to Signal Expansions and the
Discrete Wavelet Transform
James E. Fowler
Department of Electrical & Computer EngineeringGeosystems Research Institute
Mississippi State University
1 / 38
Hilbert Spaces
Vectors
C: set of complex numbers
R: set of real numbers
Z: set of integers
A vector x is a N-tuple
x = {x1, x2, . . . , xN} (1)
contained in CN (or RN)
2 / 38
Hilbert Spaces
Vector Spaces
A vector space is any subset E ⊆ CN , coupled with addition and
multiplication operations, which satisfies:
commutation: x + y = y + x
association: (x + y) + z = x + (y + z); (αβ) x = α (βx)
distribution: α (x + y) = αx + αy; (α+ β) x = αx + βx
additive identity: there exists 0 ∈ E such that x + 0 = x
additive inverse: there exists −x ∈ E such that x + (−x) = 0
multiplicative identity: 1 · x = x
3 / 38
Hilbert Spaces
Subspaces
A subspace is any set M ⊆ E closed under addition and
multiplication:
x, y ∈ M → x + y ∈ M (2)
x ∈ M, α ∈ C → αx ∈ M (3)
For S ⊂ E, the span of S, span (S), is a subset of E consisting of all
linear combinations of vectors in S:
span (S) =
{
∑
i
αixi | xi ∈ S, αi ∈ C}
(4)
4 / 38
Hilbert Spaces
Bases
The vectors S = {x1, x2, . . . } are linearly independent if
∑
i
αixi = 0 (5)
is true only when
αi = 0, ∀i (6)
The vectors S = {x1, x2, . . . } ⊂ E form a basis of vector space E if◮ span (S) = E◮ the vectors of S are linearly independent
5 / 38
Hilbert Spaces
Basis Expansion
A basis allows the representation, or expansion, of any vector in a
vector space using the basis vectors.
If S = {x1, x2, . . . } is a basis of E, and y ∈ E, then there exist
constants αi ∈ C such that
y =∑
i
αixi. (7)
The set of expansion coefficients αi is unique given y.
6 / 38
Hilbert Spaces
Inner Product
An inner product on vector space E is a function mappingE × E → C such that
◮ 〈x + y, z〉 = 〈x, z〉+ 〈y, z〉◮ 〈x, αy〉 = α〈x, y〉◮ 〈x, y〉∗ = 〈y, x〉◮ 〈x, x〉 ≥ 0 with equality only if x = 0
A vector space equipped with an inner product is an inner-product
space
A complete inner-product space is a Hilbert space (completeness:
all Cauchy sequences converge to a vector in the vector space)
The norm of x is ‖x‖ =√
〈x, x〉Vector x and y are orthogonal iff 〈x, y〉 = 0
7 / 38
Hilbert Spaces
Finite-Dimensional Spaces
N is finite: x = {x1, x2, . . . , xN} ∈ CN
The conventional inner product:
〈x, y〉 =N∑
i=1
x∗i yi (8)
The norm is then
‖x‖ =
√
√
√
√
N∑
i=1
|xi|2 (9)
Holds for RN also (R3 is 3D Euclidean space)
8 / 38
Hilbert Spaces
Square-Summable Sequences
Let x[n] be a discrete-time signal (sequence):
n ∈ Z (10)
x[n] ∈ C, ∀n (11)
Hilbert space ℓ2(Z) is the set of all sequences x[n] ∈ C∞ such that
‖x‖ <∞ (12)
where
‖x‖ =√
〈x, x〉 (13)
〈x, y〉 =∞∑
n=−∞x∗[n] · y[n] (14)
9 / 38
Hilbert Spaces
Square-Integrable Functions
Let f (t) be a continuous-time signal (function):
t ∈ R (15)
f (t) ∈ C, ∀t (16)
Hilbert space L2(R) is the set of all functions f (t) such that
‖f‖ <∞ (17)
where
‖f‖ =√
〈f, f〉 (18)
〈f, g〉 =∫ ∞
−∞f ∗(t)g(t) dt (19)
10 / 38
Hilbert Spaces
Orthonormal Bases
A vector x is normalized when
‖x‖ = 1 (20)
An orthonormal set S = {xi} satisfies:
〈xi, xj〉 ={
1, i = j,
0, i 6= j(21)
An orthonormal set of vectors that is also a basis is an
orthonormal basis
11 / 38
Hilbert Spaces
Orthonormal Basis Expansion
If S = {xi} is an orthonormal basis of Hilbert space E, then, for all
y ∈ E,
y =∑
i
αixi (22)
where the expansion coefficients are
αi = 〈xi, y〉 (23)
If S is merely an arbitrary basis, the expansion coefficients αi may
be difficult to calculate for a given vector y. The inner product
provides the coefficients for an orthonormal basis.
12 / 38
Time and Frequency Expansions
Finite-Length Sequences
Consider the Hilbert space CN of length-N sequences—we willlook at two common bases for CN :
◮ time basis◮ frequency basis
An important signal: the Kronecker delta
δ[n] =
{
1, n = 0,
0, n 6= 0(24)
13 / 38
Time and Frequency Expansions
The Time Basis
For 0 ≤ m ≤ N − 1, define vectors ψm
ψm[n] = δ[n− m] =
{
1, n = m,
0, n 6= m(25)
with discrete Fourier transform (DFT) Ψm
Ψm[k] = F[
δ[n− m]]
=
N−1∑
n=0
δ[n− m]e−jω0nk = e−jω0mk (26)
where ω0 = 2πN
14 / 38
Time and Frequency Expansions
The Time Basis
The set S = {ψm,m = 0, . . . ,N − 1} forms an orthonormal basis of
length-N sequences:
〈ψm1,ψm2
〉 =N−1∑
n=0
δ[n− m1]δ[n− m2] =
{
1, m1 = m2,
0, m1 6= m2
(27)
For any length-N sequence x,
〈ψm, x〉 =N−1∑
n=0
δ[n− m]x[n] = x[m] (28)
x[n] =N−1∑
m=0
x[m]δ[n− m] =N−1∑
m=0
〈ψm, x〉ψm[n] (29)
15 / 38
Time and Frequency Expansions
Orthonormal Time Basis for a Length-N Sequence
ψ0 = δ[n]
0 n1 2
× 〈ψ0, x〉︸ ︷︷ ︸
x[0]
=
〈ψ0, x〉ψ0
0 n1 2
Σ =
x[n]
0 n1 2
expanded as:
ψ1 = δ[n− 1]
0 n1 2
× 〈ψ1, x〉︸ ︷︷ ︸
x[1]
=
〈ψ1, x〉ψ1
0 n1 2
x[n]
0 n1 2
ψ2 = δ[n− 2]
0 n1 2
× 〈ψ2, x〉︸ ︷︷ ︸
x[2]
=
〈ψ2, x〉ψ2
0 n1 2
N−1∑ 16 / 38
Time and Frequency Expansions
The Time Basis
In the time domain:
x[n] =N−1∑
m=0
〈ψm, x〉ψm[n] (30)
where ψm[n] = δ[n− m]← support: one value of n
In the frequency domain (take DFT of both sides):
X[k] =N−1∑
m=0
〈ψm, x〉Ψm[k] (31)
where Ψm[k] = F[
ψm[n]]
= e−jω0mk ← support: all values of k
17 / 38
Time and Frequency Expansions
The Frequency Basis
Let φl[n] =1√N
ejω0nl where ω0 = 2πN
The set S′ = {φl, l = 0, . . . ,N − 1} forms an orthonormal basis of
length-N sequences
The DFT of length-N sequence x[n] is
X[l] = F[
x[n]]
=N−1∑
n=0
x[n]e−jω0nl =√
N
N−1∑
n=0
φ∗l [n]x[n] =√
N〈φl, x〉
(32)
The inverse DFT is
x[n] = F−1[
X[l]]
=1
N
N−1∑
l=0
X[l]ejω0nl =N−1∑
l=0
〈φl, x〉φl[n] (33)
18 / 38
Time and Frequency Expansions
The Frequency Basis
In the time domain:
x[n] =N−1∑
l=0
〈φl, x〉φl[n] (34)
where φl[n] =1√N
ejω0nl ← support: all values of n
In the frequency domain (take DFT of both sides):
X[k] =N−1∑
l=0
〈φl, x〉Φl[k] (35)
where Φl[k] = F[
φl[n]]
=√
Nδ[k − l]← support: one value of k
19 / 38
Time and Frequency Expansions
Representation of a SequenceExample for sequence of length N = 4
Time-basis Representation Frequency-basis Representation
ψm[n] = δ[n−m], Ψm[k] = e−jω0mk φl[n] =1
√
Nejω0nl, Φl[k] =
√
Nδ[k − l]
(corresponds to the “natural” time-domain representation) (corresponds to the frequency-domain representation given by DFT)
n
time
k
freq
uen
cy
0 1 2 3
0
1
2
3
ψ0 ψ2 ψ3ψ1
n
time
k
freq
uen
cy
φ0
φ3
φ1
φ2
0 1 2 3
0
1
2
3
20 / 38
Time and Frequency Resolution
Resolution
Resolution: the ability to distinguish between two closely located
signal features
Time resolution: ∆n = time separation between bases
rt =1
∆n(36)
Frequency resolution: ∆k = frequency separation between bases
rf =1
∆k(37)
21 / 38
Time and Frequency Resolution
Time-basis Representation Frequency-basis Representation
ψm[n] = δ[n−m], Ψm[k] = e−jω0mk φl[n] =1
√
Nejω0nl, Φl[k] =
√
Nδ[k − l]
n
time
k
freq
uen
cy
0 1 2 3
0
1
2
3
ψ0 ψ2 ψ3ψ1
∆n = 1, ∆k = N
rt = 1, rf = 1/N
n
time
k
freq
uen
cy0 1 2 3
0
1
2
3
φ0
φ3
φ1
φ2
∆n = N, ∆k = 1
rt = 1/N, rf = 1
22 / 38
Time and Frequency Resolution
Continuous-Time Time/Frequency Bases
Time basis: Dirac delta, δ(t)
∆t = 0 rt =1∆t
=∞⇒
∆ω =∞ rf =1
∆ω = 0
(38)
Frequency basis: continuous-time Fourier transform
∆t =∞ rt =1∆t
= 0
⇒∆ω = 0 rf =
1∆ω =∞
(39)
23 / 38
General Time-Frequency Tilings
Continuous-Time Time/Frequency Bases
Time and frequency (Fourier) bases are “all or nothing” (resolution
is 0 or∞)
We would like bases with 0 < rt, rf <∞Define “spread” (∆t or ∆ω) as second moment:
(∆t)2 =∫
t2|ψ(t)|2 dt∫|ψ(t)|2 dt
(∆ω)2 =∫ω2|Ψ(ω)|2 dω∫|Ψ(ω)|2 dω
time domain frequency domain(40)
24 / 38
General Time-Frequency Tilings
The Uncertainty Principle
For any ψ(t) with Fourier transform Ψ(ω),
∆t∆ω ≥ 12
or rtrf ≤ 2 (41)
We can have arbitrarily large resolution (time or frequency) only by
giving up resolution of the other quantity
Example:◮ To get ∆t = 0, ∆ω must be∞ (which is time basis)◮ For ∆ω <∞, we must use ∆t > 0
25 / 38
The Short-Time Fourier Transform
The Continuous-Time Fourier Transform
F(ω) = F [f (t)] is the set of coefficients of an expansion of
continuous-time function f (t) ∈ L2(R) using basis functions
ψω(t) = ejωt (42)
which provides a basis function for each ω ∈ RResolution:
∆ω = 0 (43)
∆t =∞ (44)
The short-time Fourier transform (STFT) attempts to improve the
time resolution by windowing each ψω(t) to reduce its time extent
26 / 38
The Short-Time Fourier Transform
The STFT
The STFT:
f (t) =1
2π
∫
R
∫
RS(τ, ω)ψτ,ω(t) dτ dω (45)
Basis functions—Fourier basis coupled with window function g(t):
ψτ,ω(t) = g(t − τ)ejωt (46)
Coefficients:
S(τ, ω) =⟨
ψτ,ω, f⟩
(47)
=
∫
Rψ∗τ,ω(t)f (t) dt (48)
=
∫ ∞
−∞f (t)g(t − τ)e−jωt dt (49)
27 / 38
The Short-Time Fourier Transform
The STFT Basis Functions
Basis function ψτ,ω(t) is indexed by two indices:◮ Time: τ◮ Frequency: ω
τ and ω determine location in time-frequency plane of the basis
function’s corresponding tile:
∆ω
∆ω
∆t
∆t
ω2
ω1
τ1 τ2t
ω
|ψτ2,ω2(t)|2|ψτ1,ω1
(t)|2
|Ψτ1,ω1(t)|2
|Ψτ2,ω2(t)|2
28 / 38
The Short-Time Fourier Transform
The STFT Window
The window function, g(t), is chosen to give the desired resolution
tradeoff, subject to ∆t∆ω ≥ 12
Wide window (∆t large)→ good frequency resolution (∆ω small)
Narrow window (∆t small)→ poor frequency resolution (∆ω large)
f
t
f
t
Narrow window Wide window
29 / 38
The Short-Time Fourier Transform
The Balian-Low Theorem
If the STFT basis functions ψτ,ω(t) form an orthogonal basis, then
either ∆t or ∆ω must be∞To have good resolution in time and frequency simultaneously in
STFT, one must use a non-orthogonal basis.
30 / 38
The Short-Time Fourier Transform
Advantages of Orthonormal Bases
If f ∈ E is a vector in Hilbert space E, and S = {ψi} is a basis of E, then
f =∑
i
αiψi (50)
If S is an orthonormal basis:
Easy calculation of expansion coefficients:
αi = 〈ψi, f〉 (51)
Parseval’s theorem:
‖f‖2 =∑
i
|αi|2 (52)
That is, total energy of f is partitioned among the αi coefficients.
31 / 38
Wavelet-Series Expansion
Wavelet-Series Expansion
Commonly called the “discrete wavelet transform” (DWT)
Expands a continuous-time function into coefficients which are
discrete in time and frequency:
f (t) =∑
k
∑
j
aj,kψj,k(t) (53)
time: k ∈ Zfrequency: j ∈ ZSimilar to STFT in that decomposition is in terms of both time and
frequency—but, orthonormal wavelet bases exist with good time
and frequency resolution simultaneously
32 / 38
Wavelet-Series Expansion
The Mother Wavelet
Wavelet-series expansion systems are generated from one
function, ψ(t), called the mother wavelet
The basis functions are created by scaling and translating the
mother wavelet:
ψj,k(t) = 2j/2ψ(2jt − k) for j, k ∈ Z (54)
The 2j/2 factor maintains normalization:
∥
∥ψj,k
∥
∥
2= ‖ψ‖2 = 1 (55)
33 / 38
Wavelet-Series Expansion
Scaling and Translation
ψ(t)
j = 0 j = 1 j = 2
34 / 38
Wavelet-Series Expansion
Scaling and Translation
k controls displacement of ψj,k(t) in time
j is the scale—increasing j yields:◮ “compaction” of ψj,k(t) in time◮ ψj,k(t) functions spaced closer together in time◮ “taller” ψj,k(t) functions
When j is large:◮ ψj,k(t) functions have small ∆t; thus, ∆ω is large◮ good time resolution
When j is small:◮ ψj,k(t) functions have large ∆t; thus, ∆ω is small◮ good frequency resolution
35 / 38
Wavelet Time-Frequency Tiling
time
frequency
36 / 38
Wavelet Time-Frequency Tiling
Why is this a good tiling?
Natural signals are usually lowpass—wavelet tiling concentrates
frequency resolution where majority of signal energy resides
Discontinuities (edges) involve high-frequency energy that
accounts for a small portion of total energy—location in time is
more important than frequency composition—wavelet tiling has
good time resolution for high-frequency content
37 / 38
Summary
Wavelets provide an expansion in the form:
f (t) =∑
k
∑
j
⟨
ψj,k, f⟩
ψj,k(t) (56)
The expansion can be orthonormal—coefficients can be
calculated with inner products
The expansion is multiresolution—time and frequency resolutions
vary across time-frequency plane
The time-frequency tiling matches characteristics of natural
(lowpass) signals with salient discontinuities
38 / 38