wavelets and subband coding solutions manual
TRANSCRIPT
Solutions Manualto accompany Wavelets and Subband Codingby Martin Vetterli & Jelena Kovačević
Grace Chang, Michael Goodwin, Vivek K Goyal & Ton Kalker
Originally published 1995 by Prentice Hall PTR, Englewood Cliffs, New Jersey.
Reissued by the authors 2007.
This work is licensed under the Creative Commons Attribution-Noncommercial-
No Derivative Works 3.0 License. To view a copy of this license, visit
http://creativecommons.org/licenses/by-nc-nd/3.0/ or send a letter to
Creative Commons, 171 Second Street, Suite 300, San Francisco, CA 94105 USA.
Solution Manualfor
Wavelets and Subband Codingby
Martin Vetterli and Jelena Kovacevic
Solutions Prepared By:
Grace Chang
Michael Goodwin
Vivek Goyal
Ton Kalker
Please send questions or comments about the solutions [email protected]
1
Chapter 2
2.1 Legendre polynomialsWe assume the inner product definition preceding (2.2.1). Let xk = tk−1 for k ∈ Z+. We initiatethe Gram-Schmidt procedure with
y1 =x1
‖x1‖=
1
(∫ 1−1 1 dt)
12
=1√2.
Continuing the Gram-Schmidt orthogonalization as on page 22,
v2 = 〈y1, x2〉y1 =(∫ 1
−1
1√2t dt
)1√2
= 0
y2 =x2 − v2‖x2 − v2‖
=t
(∫ 1−1 t dt)
12
=
√32t
v3 = 〈y1, x3〉y1 + 〈y2, x3〉y2 =(∫ 1
−1
1√2t · t dt
)1√2
+
(∫ 1
−1
√32t · t2 dt
)√32
=13
y3 =x3 − v3‖x3 − v3‖
=t2 − 1
3
(∫ 1−1(t
2 − 13)2 dt)
12
=3√
52√
2
(t2 − 1
3
)
This process can be continued to find an orthonormal set of arbitrary size. It can be proven byinduction that
yn+1 =
√2n+ 1
2· (−1)n
2nn!· d
n
dtn[(1− t2)n
].
2.2 Proof of Theorem 2.4Let S = {x1, x2, . . . xK} be an orthonormal system in E = RN or CN . (For S to be an orthonormalbasis will require K = N , but there is no reason to assume this a priori.) The proof will proceedas (a) ⇒ (e) ⇒ (d) ⇒ (b) ⇒ (a).
(a) ⇒ (e): See Problem 2.8(a).
(e) ⇒ (d): (d) is a special case of (e) with y1 = y2 = y.
(d) ⇒ (b): Let y ∈ E. Suppose 〈xi, y〉 = 0 for i = 1, 2, . . . ,K. Then by (d), ‖y‖2 = 0. ThusDefinition 2.2(d) implies y = 0.
(b) ⇒ (a): Let y ∈ E. We wish to show that y =∑K
i=1〈xi, y〉xi equals y. Define e by e = y − y.Then for k ∈ {1, 2, . . . ,K},
〈xk, e〉 =
⟨xk, y −
K∑i=1
〈xi, y〉xi
⟩
= 〈xk, y〉 −K∑i=1
〈xk, 〈xi, y〉xi〉
= 〈xk, y〉 −K∑i=1
〈xi, y〉〈xk, xi〉
2
= 〈xk, y〉 −K∑i=1
〈xi, y〉δki by the orthonormality of S
= 〈xk, y〉 − 〈xk, y〉 = 0.
Now (b) implies that e = 0. Thus S satisfies the requirements for an orthonormal basis for E.
2.3 Orthogonal transforms and l∞ norm(a) The orthogonal transforms T2 correspond to rotations. We can thus find the bounds a2 andb2 by considering rotations of vectors on the unit circle. By the definition of the l∞ norm, theupper bound b2 is clearly 1 since there is no vector on the unit circle whose maximum element isgreater than 1. The lower bound is achieved when both components of the vector are equal; thisis true because any rotation of such a vector increases one of the components, which increases thel∞ norm. So the result is:
1√2≤ l∞[T2v] ≤ 1
(b) The arguments from (a) can be easily extended to n dimensions. For vectors on an n-dimensionalsphere, the minimum l∞ norm occurs when all of the elements are equal:
for v = [v1v2 . . . vn]T with ‖v‖2 = 1 =⇒ v21 + v2
2 + . . .+ v2n = 1
vi = vj ∀ i, j =⇒ nv21 = 1 =⇒ v1 =
1√n
= an
The maximum l∞ norm occurs when all but one of the elements are zero, so bn = 1.Combining the results gives:
1√n≤ l∞[Tnv] ≤ 1
2.4 Norms of operatorsRecall that the definition of the operator norm is given in Appendix 2.A. Also recall that the normfor l2(Z) is given on page 19.
(a) Let x ∈ l2(Z) be a unit norm sequence.
‖Ax‖2 =∑n
|m[n] · x[n]|2 =∑n
|m[n]|2|x[n]|2 =∑n
|x[n]|2 = ‖x‖2 = 1,
where the third equality is because |m[n]| = 1 for all n. Thus ‖A‖ = 1.
(b) Let x ∈ l2(Z) and let y = Ax. Notice that[y[2n]
y[2n + 1]
]=[
1 11 −1
] [x[2n]
x[2n + 1]
].
Thus we can write
|y[2n]|2 + |y[2n+ 1]|2 = |x[2n] + x[2n+ 1]|2 + |x[2n]− x[2n + 1]|2 = 2|x[2n]|2 + 2|x[2n + 1]|2.
From this it is clear that ‖y‖2 = 2‖x‖2. Therefore ‖A‖ =√
2.
3
2.5 Least-squares approximation in an orthonormal representationOur problem is to minimize ‖y − y‖, where y =
∑Ni=1 αixi and y =
∑Ki=1 βixi. Subtracting gives
y − y =K∑i=1
(αi − βi)xi +N∑
i=K+1
αixi.
Thus,
〈xm, y − y〉 =
⟨xm,
K∑i=1
(αi − βi)xi +N∑
i=K+1
αixi
⟩={αm − βm m = 1, . . . ,Kαm m = K + 1, . . . , N
,
where we have used the linearity of the inner product and the orthonormality of {xm}Nm=1. NowParseval’s equality (2.2.5) gives
‖y − y‖2 =N∑m=1
|〈xm, y − y〉|2 =K∑m=1
|αm − βm|2 +N∑
m=K+1
|αm|2.
This is minimized by choosing βm = αm for m = 1, . . . ,K, which is the desired result.
2.6 Least-squares solutionLet A ∈ Rm×n with m > n. Consider the problem of “solving” y = Ax in the sense of finding x tominimize ‖y −Ax‖. In Section 2.3.2, geometric reasons are given to justify choosing
x =(ATA
)−1AT y. (1)
We wish give a calculus justification.As preparation, we manipulate (1) in the opposite direction than in the text to yield
AT (y −Ax) = 0. (2)
Considering (2) elementwise gives
aTi (y −Ax) = 0 for i = 1, 2, . . . , n, (3)
where ai is the ith column of A.Now let E = ‖y −Ax‖2 = (y −Ax)T (y −Ax).
∂E
∂xi= aTi (y −Ax) + (y −Ax)Tai
= 2aTi (y −Ax)
By (3), if x is chosen according to (1), then each partial derivative ∂E/∂xi is zero. Therefore (1)gives a local extremum of E. Since E is a quadratic function with positive leading term, this isguaranteed to be a global minimum.
2.7 Least-squares solution to a linear system of equations(a) If y belongs to the range (or column space) of A, then there exists x such that Ax = y. Hence,
y = A(ATA)�1ATy
= A(ATA)�1(ATA)x
= Ax
= y .
4
(b) If y is orthogonal to the column space of A, then y ∈ N (AT ), the nullspace of AT . That is,ATy =0, and so y = A(ATA)�1ATy =0.
2.8 Parseval’s formulae(a) Let {xi} be an orthonormal basis for E. Let y ∈ E. Then y =
∑k αkxk, where αk = 〈xk, y〉.
‖y‖2 = 〈y, y〉 =
⟨∑k
αkxk,∑�
α�x�
⟩
=∑k
α∗k
⟨xk,
∑�
α�x�
⟩
=∑k
α∗k
(∑�
α�〈xk, x�〉)
=∑k
∑�
α∗kα�δk� by the orthonormality of the xi’s
=∑k
α∗kαk
=∑k
|〈xk, y〉|2
This establishes (2.2.5).Let y1, y2 ∈ E. Then y1 =
∑k αkxk, where αk = 〈xk, y1〉, and y2 =
∑k βkxk, where βk =
〈xk, y2〉.
〈y1, y2〉 =
⟨∑k
αkxk,∑�
β�x�
⟩
=∑k
α∗k
⟨xk,
∑�
β�x�
⟩
=∑k
α∗k
(∑�
β�〈xk, x�〉)
=∑k
∑�
α∗kβ�δk� by the orthonormality of the xi’s
=∑k
α∗kβk
=∑k
〈xk, y1〉∗〈xk, y2〉
This establishes (2.2.6).
(b) The manipulations are very similar to the orthogonal basis case. Let {xi, xi} constitute apair of biorthonormal bases for E. Let y1, y2 ∈ E. Then we can write y1 =
∑k〈xk, y1〉xk and
y2 =∑
k〈xk, y1〉xk.
〈y1, y2〉 =
⟨∑k
〈xk, y1〉xk,∑�
〈x�, y2〉x�
⟩
5
=∑k
〈xk, y1〉∗⟨xk,
∑�
〈x�, y2〉x�
⟩
=∑k
〈xk, y1〉∗(∑
�
〈x�, y2〉〈xk, x�〉)
=∑k
∑�
〈xk, y1〉∗〈x�, y2〉δk� by (2.2.?)
=∑k
〈xk, y1〉∗〈xk, y2〉
This establishes (2.2.12). By expanding y1 in terms of xk’s and y2 in terms xk’s, (2.2.13) can besimilarly proven. (2.2.11) is a special case of (2.2.12) with y1 = y2 = y.
2.9 Symmetric and antisymmetric functions(a) Let
fs(x) =12(f(x) + f(−x))
andfa(x) =
12(f(x)− f(−x)) ,
then it is straightforward to verify that fs(x) ∈ S, fa(x) ∈ A, and that any f(x) ∈ L2([−π, π]) canbe written as
f(x) = fs(x) + fa(x) .
(b) An orthonormal basis for S is {1√2π,cosnx√
2π
}, n = 1, 2, . . .
(which are symmetric functions). Similarly,{sinnx√
2π
}, n = 1, 2, . . .
are asymmetric functions, and they form an orthonormal basis for A.
(c) From part (a), since any f ∈ L2([−π, π]) can be written as f(x) = fs(x) + fa(x), this impliesthat L2([−π, π]) ⊆ S ⊕ A. Conversely, from the decomposition in part (b) and the fact thatthese functions form an orthonormal basis for L2([−π, π]), it follows that the sum of any functionsfs ∈ S and fa ∈ A is equal to some f ∈ L2([−π, π]), implying L2([−π, π]) ⊇ S ⊕ A. Hence,L2([−π, π]) = S ⊕A.
2.10 Downsampling by NConsider x[n] and y[n] to be obtained from sampling xc(t) with sampling periods T and NT ,respectively.
Y (ejω) = XNT
( ω
NT
)=
1NT
∑k
Xc
(ω
NT− k 2π
NT
)by (2.4.35)
6
=1NT
N−1∑�=0
∞∑n=−∞
Xc
(ω
NT− (nN + �)
2πNT
)by defining n and � through k = nN + �
=1N
N−1∑�=0
[1T
∞∑n=−∞
Xc
(ω − 2π�NT
− n2πT
)]
=1N
N−1∑�=0
X(ej(ω−2π�)/N ) by (2.4.36)
2.11 Downsampling and aliasingThe problem statement is incorrect. Suppose x[n] is downsampled by a factor of N . The appearanceof aliasing is equivalent to the existence of (open intervals of) ω such that more than one term of(2.5.13) is non-zero. The easiest way to check for aliasing is to sketch the stretched and shiftedversions of X(ejω) and see if they have any overlap.
(a) H ′(ejω/2) and H ′(ej(ω−π)/2) are identical (and thus overlap). Therefore filtering by H ′(ejω)prior to downsampling by 2 will not avoid aliasing.
(b) & (c) Similarly incorrect.
2.12 Basis patterns(a) Let’s consider the problem in matrix notation. Let x = [x[0], x[1], . . . , x[N − 1] ]T and φk =[φk[0], φk[1], . . . , φk[N − 1] ]T , k = 0, 1, . . . , N − 1. Since x ∈ R
N , it can be written as a linearcombination of the φk’s only if {φk, k = 0, 1, . . . , N − 1} form a basis of RN . Let Φ be thematrix whose columns are {φk, k = 0, 1, . . . , N − 1}, i.e. Φ = [φ0 φ1 . . . φN�1]. Then Φis a circulant matrix and it can be diagonalized by the Fourier matrix as Φ = F�1ΛF , whereΛ = diag{P [0], P [1], . . . , P [N − 1]}, and P [k], k = 0, 1, . . . , N − 1, are the DFT coefficients of p[n].Thus, Φ is full rank (implying that the vectors {φk, k = 0, 1, . . . , N − 1} form a basis) if and onlyif P [k] �= 0, k = 0, 1, . . . , N − 1.
(b) Assuming that {φk, k = 0, 1, . . . , N − 1} form a basis of RN , we wish to expand x as
x = Φα ,
where α = [α0, α1, . . . , αN−1]T . Since Φ is full rank, it is invertible. Thus,
α = Φ�1x = F�1Λ�1F x .
2.13 Periodically time-varying systemsA linear, time-varying system is characterized by a two-variable impulse response h[k, n], which isthe response to input x[n] = δ[n − k]. Since the input can be written as x[n] =
∑k x[k]δ[n − k],
the output is given by y[n] =∑
k x[k]h[k, n].We are interested in the case where h[k, n] is periodic in k with period N . As on page 71, define
the polyphase components byxi[n] = x[nN + i]
for i = 0, 1, . . . , N − 1. Denote the upsampled version of xi by xi, so
xi[n] ={xi[n/N ], n/N ∈ Z
0, otherwise={x[n+ i], n/N ∈ Z
0, otherwise
7
The input x[n] can be decomposed to give
x[n] =N−1∑i=0
xi[n− i].
By superposition, the output is given by
y[n] =∑k
h[k, n]N−1∑i=0
xi[n− i]
=N−1∑i=0
∑k
h[k, n]xi[n− i]
=N−1∑i=0
∑k: k−i
N∈Z
h[k, n]xi[n − i] because the excluded terms are zero
=N−1∑i=0
∑k: k−i
N∈Z
h[i, n]xi[n− i] by the periodicity of h
=N−1∑i=0
∑k
h[i, n]xi[n− i] because the added terms are zero.
The final expression shows the output as a sum of N terms. The ith term is the ith polyphasecomponent, upsampled and filtered by an i-sample delayed version of h[i, n].
2.14 Interpolation of oversampled signals(a) Since the bandlimited signal is oversampled, there are many possible interpolation filters, aslong as they select only the base copy of the spectrum. Let the sampling period T = 2π
ωs= 2
3 . H2(ω)must satisfy
H2(ω) ={T if ω ∈ [−π, π]0 if ω ≥ 2π
The most straightforward H2(ω) is obtained from convolving in the frequency domain two rectan-gular windows:
ω
T
3π/2−3π/2
ωG ( )1
ω
1
π/2−π/2
ωG ( )2
H ( )2 ω
ω
T
π−π 2π−2π=*
Let ω1 and ω2 be the cutoff frequencies of G1(ω) and G2(ω), respectively. They can be found bysolving the equations
ω1 + ω2 = 2π ,
ω1 − ω2 = π .
8
This yields ω1 = 3π2 and ω2 = π
2 .In the time-domain, h2(t) is the product of two sinc functions (up to a scaling constant), which
is clearly a function of 1t2
:
h2(t) =sin( 3π
2T t)3π2T t
sin( π2T t)π2 t
.
(b) For ωs = 4π, T = 2πωs
, H3(ω) must satisfy
H3(ω) ={T if ω ∈ [−π, π]0 if ω ≥ 3π
H3(ω) can be obtained by convolving (in the frequency domain) one “big” rectangular window,G1(ω), with two identical “small” rectangular windows, G2(ω):
ω
T
ωG ( )1
ω
1
π/2−π/2
ωG ( )2
=*2π−2πω
1
π/2−π/2
ωG ( )2
*
H ( )ω
ω
T
π−π 3π−3π
3
Let ω1 and ω2 be the cutoff frequencies of G1(ω) and G2(ω), respectively. They can be found bysolving the equations
ω1 + 2ω2 = 3π ,
ω1 − 2ω2 = π .
This yields ω1 = 2π and ω2 = π2 .
In the time domain, h3(t) is the product of 3 sinc functions, so it decays as 1t3 :
h3(t) =sin(2π
T t)2πT t
(sin( π
2T t)π2 t
)2
To show that H3(ω) has a continuous derivative, it is easier to consider it in the time-domainand the transform it back to the frequency domain. Recall that the Fourier transform pair of thefrequency domain derivative is
tn h(t) FT←→ jndnH(ω)dωn
.
The scaling constant j does not affect the continuity of the problem so we will ignore it.
t h3(t) =sin(2π
T t)2πT
(sin( π
2T t)π2 t
)2
Hence, t h3(t) is a product of 2 identical sinc functions and a sine function (and a multiplicative con-stant), which corresponds to convolving a triangle window with a pair of impulses in the frequencydomain. This is just adding together two triangle windows (which are continuous function), hencethe derivative is continuous. Note that the second derivative is not continuous, since t2 h3(t) is a
9
product of a sinc function and 2 sine functions, which translates into adding together 4 rectangularwindows in the frequency domain, and rectangular windows are not continuous.
(c) Generalizing to ωs = (i+ 1)π, Hi(ω) must satisfy
Hi(ω) ={T if ω ∈ [−π, π]0 if ω ≥ iπ
Hi(ω) is obtained from convolving a “big” window with cutoff ω1 with (i − 1) “small” windowswith cutoff ω2, where ω1 and ω2 can be solved from the equations
ω1 + (i− 1)ω2 = iπ ,
ω1 − (i− 1)ω2 = π .
This yields ω1 = iπ2 and ω2 = π
2 . The time-domain sequence is a product of i sinc functions andthus is a function of 1
ti:
hi(t) =sin( iπ2T t)
iπ2T t
(sin( π
2T t)π2 t
)(i−1)
.
Now consider the n-th derivative of Hi(ω), where 0 ≤ n ≤ (i−2). In the time domain, this becomes
tn hi(t) =sin( iπ2T t) (sin( π
2T t))(i−1)
iπ2T
π2 t
(i−n),
which is a product of i−n sinc functions and n sine functions. In the frequency domain, this becomesa convolution of i−n rectangular functions with n pairs of impulses (from the sine function). Notingthat convolving 2 or more rectangle windows yield continuous functions, we conclude that the n-thderivative of Hi(ω) is continuous for 0 ≤ n ≤ (i− 2).
2.15 Uncertainty relation(a) By definition,
∆2t =
∫ +∞
−∞t2 |f(t)|2 dt and ∆2
ω =∫ +∞
−∞ω2 |F (ω)|2 dω
where f(t) and F (ω) are a Fourier transform pair. Scaling by a yields the function fs(t) =√af(at)
and its Fourier transform Fs(ω) =1√aF(ωa
).
This scaling preserves the L2 norm:
L2(fs) =∫ +∞
−∞
∣∣√af(at)∣∣2 dt =
∫ +∞
−∞a |f(at)|2 dt =
∫ +∞
−∞
∣∣f(t′)∣∣2 dt′ = L2(f)
For the scaled function, the uncertainty product is
∆2ω∆2
t =∫ +∞
−∞dt
∫ +∞
−∞dω t2
∣∣√af(at)∣∣2 ω2
∣∣∣F (ωa
)∣∣∣2Substituting t′ = at and ω′ = ω
a yields:
∆2ω∆2
t =∫ +∞
−∞
1adt′∫ +∞
−∞a dω′ 1
a2(t′)2
∣∣f (t′)∣∣2 a2(ω′)2∣∣F (ω′)∣∣2
=∫ +∞
−∞dt′ (t′)2
∣∣f(t′)∣∣2 ∫ +∞
−∞dω′ (ω′)2
∣∣F (ω′)∣∣210
which completes the proof. Note that the operations of time-shifting and modulation do changethe uncertainty product ∆2
ω∆2t .
(b) The Fourier transform of the rectangular pulse p(t) ={
1 −12 ≤ t ≤
12
0 otherwiseis given by
P (ω) =∫ +∞
−∞p(t) e−jωtdt =
∫ + 12
− 12
e−jωtdt
= − 1jω
e−jωt∣∣∣∣+
12
− 12
=2ω
sin(ω
2
)Then,
∆2t =
∫ + 12
− 12
t2dt =t3
3
∣∣∣∣+12
− 12
=112
and ∆2ω = 4
∫ +∞
−∞sin2
(ω2
)dω −→∞
so the uncertainty product ∆2ω∆2
t is unbounded for a rectangular pulse.
(c) Consider the triangular pulse given by q(t) = p(t) ∗ p(t) where p(t) is the rectangular pulse ofpart (b):
q(t) =
{ √32(1− |t|) −1 ≤ t ≤ 1
0 otherwise
where the scale factor√
32 is included to make q(t) a unit-energy function. (The uncertainty relation
only holds for unit-energy functions.) Then,
∆2t =
∫ 1
−1
32t2(1− |t|)2dt = 3
∫ 1
0t2(1− t)2dt
= 3(t3
3− t4
2+t5
5
)∣∣∣∣10
=110
The Fourier transform of q(t) is given by Q(ω) =
√32
4ω2
sin2(ω
2
), so
∆2ω =
∫ ∞
−∞ω2
(32
)16ω4
sin2(ω
2
)dω
= 48∫ ∞
0
1ω2
sin4(ω
2
)dω = 24
∫ ∞
0
sin4 x
x2dx
= 24∫ ∞
0
sin2 x
x2− sin2(2x)
4x2dx = 24
(π2− π
4
)= 6π
So the uncertainty relation holds: ∆2t∆
2ω =
(110
)(6π) = 0.6π ≥ π
2
(d) The central limit theorem can be applied to the sequence of convolved pulses:
fn(t) = p1(t) ∗ p2(t) ∗ · · · ∗ pn(t)
limn→∞
fn(t) =1
σ√
2πe−(t−η)2/2σ2
11
where the parameters η and σ depend on the functions pi(t). Since equality holds in the uncertaintyrelation for a Gaussian, the time-bandwidth product of a convolved sequence of pulses will approachπ/2.
2.16 Allpass filters(a) For H(z) to have real coefficients, the zeros as well as the poles must appear in complex-conjugate pairs, so H(z) is of the form:
H(z) =∏i
(bi + z−1
1 + biz−1
) ∏i
(a∗i + z−1
1 + aiz−1
) (ai + z−1
1 + a∗i z−1
)
where the ai are complex and the bi are real. The zeros are at locations {−1/bi,−1/ai,−1/a∗i }while the poles are at {−bi,−a∗i ,−ai}. The poles and zeros are at mirror locations with respect tothe unit circle. We can rewrite H(z) as
H(z) =
(∏i
bi + z−1
z−1(bi + z)
) ∏i
(a∗i + z−1
) (ai + z−1
)z−2 (ai + z) (a∗i + z)
=⇒ H(z) =Q(z−1)z−NQ(z)
where Q(z−1) is the N -th order numerator polynomial Q(z−1) = α0 + α1z−1 + . . . + αNz
−N , sothe allpass filter H(z) can be written as
H(z) =α0 + α1z
−1 + . . . + αNz−N
αN + αN−1z−1 + . . .+ α0z−N
which shows that the numerator and denominator are mirror polynomials.
(b) The autocorrelation a[k] is given by a[k] =∑k
h[k]∗h[k − n] = h[−n] ∗ h[n]∗. Taking the
DTFT yields A(ejω) = H(e−jω)H(e−jω)∗ = 1 by definition. The time-domain autocorrelation isgiven by the inverse transform of A(ejω), so
a[n] =12π
∫ π
−πejωndω =
1πn
sin(πn) = δ[n]
We know that {h[n − k]} is an orthonormal set because 〈h[n − k], h[n − l]〉 = a[k − l] = δ[k − l].To prove that it is an orthonormal basis for l2(Z) we can show (according to theorem 2.4b) that if〈x[n], h[n − k]〉 = 0 then x[n] = 0. The proof is straightforward:
〈x[n], h[n − k]〉 = 0 {k ∈ Z} =⇒∑n
x[n]∗h[n− k] = 0 =⇒ x[k]∗ ∗ h[−k] = 0
=⇒ X(e−jω)H(ejω)∗ = 0 =⇒ x(ejω) = 0 since |H(ejω)| = 1 ∀ω =⇒ x[n] = 0
We can thus conclude that {h[n − k]} is an orthonormal basis for l2(Z).
(c) Since 〈h[n], h[n − k]〉 = δ[k], we know that 〈h[n], h[n − 2k]〉 = δ[2k] = δ[k], so we can concludethat {h[n− 2k]} is an orthonormal set. Theorem 2.4b says that if we can find a non-zero x[n] suchthat 〈x[n], h[n − 2k]〉 = 0, then {h[n− 2k]} is not a basis. Taking x[n] = h[n − 1], we have
〈h[n− 1], h[n − 2k]〉 =∑n
h[n − 1]∗h[n− 2k] = δ[2k − 1] = 0
12
We can therefore conclude that {h[n − 2k]} is not a basis for l2(Z).
2.17 Parseval’s relation for non-orthogonal bases(a) First, we show that the sets {αi} and {βi} are linearly independent.
Supposen−1∑i=0
ciαi = 0. Then
⟨n−1∑i=0
ciαi, βj
⟩= 〈0, βj〉 = 0 and
⟨n−1∑i=0
ciαi, βj
⟩=
n−1∑i=0
ci 〈αi, βj〉 =
n−1∑i=0
ciδ[i− j] = cj so cj = 0, which proves that {αi} is a linearly independent set. By an analogous
argument, {βi} is also a linearly independent set. Since {αi} is a set of n linearly independentvectors in Rn, {αi} is a basis for Rn. So we can write any v ∈ Rn as a linear combination of the
basis vectors: v =n−1∑i=0
ciαi. We can then write
〈v, βj〉 =
⟨n−1∑i=0
ciαi, βj
⟩=
n−1∑i=0
ci 〈αi, βj〉 = cj
which verifies that v =n−1∑i=0
〈βi, v〉αi. Since {βi} is also a basis for Rn, we can similarly write
v =n−1∑i=0
diβi where 〈v, αj〉 =
⟨n−1∑i=0
diβi, αj
⟩= dj which likewise verifies that v =
n−1∑i=0
〈αi, v〉 βi.
(b) Consider v ∈ Rn as a column vector v = [v0 v1 . . . vn−1]T . Then, vα = Av where A is amatrix whose i-th row is αTi . The norm satisfies the relation ‖vα‖2 = vTαvα = vTATAv. We can
then write‖vα‖2‖v‖2 = uTATAu where u is the unit norm vector
v
‖v‖ . Therefore
√λA‖v‖ ≤ ‖vα‖ ≤
√ΛA‖v‖ where
{ΛA = sup‖u‖=1(uTATAu)λA = inf‖u‖=1(uTATAu)
From matrix algebra we know that ΛA and λA are the largest and smallest eigenvalue of ATA,respectively. Similarly √
λB‖v‖ ≤ ‖vβ‖ ≤√
ΛB‖v‖
where ΛB and λB are the largest and smallest eigenvalue of BTB, where B is a matrix whose i-th
row is βTi . But since BTB = (ATA)−1 we have λB =1
ΛAand ΛB =
1λA
so the bound for ‖vβ‖can be written as
1√ΛA‖v‖ ≤ ‖vβ‖ ≤
1√λA‖v‖
(c) For any v ∈ Rn
v =n−1∑i=0
〈βi, v〉αi =n−1∑i=0
〈αi, v〉 βi
13
Using these equivalences, we have
〈v, v〉 =
⟨n−1∑i=0
〈αi, v〉 βi〉, v⟩
=n−1∑i=0
〈αi, v〉 〈βi, v〉 = 〈vα, vβ〉
〈v, g〉 =
⟨n−1∑i=0
〈αi, v〉 βi〉, g⟩
=n−1∑i=0
〈αi, v〉 〈βi, g〉 = 〈vα, gβ〉
2.18 Circulant matrices(a) Consider circularly convolving 2 N -point sequences c[n] and g[n]. Since circular convolutionin time domain corresponds to multiplication in the DFT domain, the resulting sequence can bewritten as
c ∗p g = Cg = F�1ΛF g
where F = [W ijN ]i,j=0,...,N−1 is the Fourier matrix, Λ = diag{C[0], C[1], . . . , C[N−1]}, and C[k], k =
0, 1, . . . , N − 1 are the DFT coefficients of c[n]. Hence for any circulant matrix C, it can bediagonalized as
C = F�1ΛF .
This formulation implies that the columns of F�1 are the eigenvector of C, with C[k] being theassociated eigenvalues. So the matrix C is nonsingular if and only if none of the DFT coefficientsC[k] is zero.
(b) From the diagonalization of C, and using the fact that the determinant of a matrix is theproduct of its eigenvalues,
det(C) =N−1∏k=0
C[k] .
(c) SinceC−1 = F−1Λ−1F ,
it has the same form as C in Part (a), and thus it is circulant as well.
(d) Writing C1 and C2 as C1 = F−1Λ1F and C2 = F−1Λ2F , we see that
C1C2 = (F−1Λ1F )(F−1Λ2F )= F−1Λ1Λ2F
= F−1ΛF ,
where Λ = diag{C1[0]C2[0], C1[1]C2[1], . . . , C1[N − 1]C2[N − 1]}. So C1C2 is a circulant matrixwith eigenvalues C1[k]C2[k], k = 0, 1, . . . , N − 1.
Since Λ1Λ2 = Λ2Λ1, we haveC1C2 = C2C1.
2.19 Walsh basis(a)
W2 =[
W1 W1
W1 −W1
]=
⎡⎢⎢⎣
1 1 1 11 −1 1 −11 1 −1 −11 −1 −1 1
⎤⎥⎥⎦
14
W3 =[
W2 W2
W2 −W2
]=
⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣
1 1 1 1 1 1 1 11 −1 1 −1 1 −1 1 −11 1 −1 −1 1 1 −1 −11 −1 −1 1 1 −1 −1 11 1 1 1 −1 −1 −1 −11 −1 1 −1 −1 1 −1 11 1 −1 −1 −1 −1 1 11 −1 −1 1 −1 1 1 −1
⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦
W4 =[
W3 W3
W3 −W3
]
=
⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 −1 1 −1 1 −1 1 −1 1 −1 1 −1 1 −1 1 −11 1 −1 −1 1 1 −1 −1 1 1 −1 −1 1 1 −1 −11 −1 −1 1 1 −1 −1 1 1 −1 −1 1 1 −1 −1 11 1 1 1 −1 −1 −1 −1 1 1 1 1 −1 −1 −1 −11 −1 1 −1 −1 1 −1 1 1 −1 1 −1 −1 1 −1 11 1 −1 −1 −1 −1 1 1 1 1 −1 −1 −1 −1 1 11 −1 −1 1 −1 1 1 −1 1 −1 −1 1 −1 1 1 −11 1 1 1 1 1 1 1 −1 −1 −1 −1 −1 −1 −1 −11 −1 1 −1 1 −1 1 −1 −1 1 −1 1 −1 1 −1 11 1 −1 −1 1 1 −1 −1 −1 −1 1 1 −1 −1 1 11 −1 −1 1 1 −1 −1 1 −1 1 1 −1 −1 1 1 −11 1 1 1 −1 −1 −1 −1 −1 −1 −1 −1 1 1 1 11 −1 1 −1 −1 1 −1 1 −1 1 −1 1 1 −1 1 −11 1 −1 −1 −1 −1 1 1 −1 −1 1 1 1 1 −1 −11 −1 −1 1 −1 1 1 −1 −1 1 1 −1 1 −1 −1 1
⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦
(b) Let In denote the identity matrix of dimension n, and also note that Wk = W T
k.
W T
0 W0 = 1 = I1
W T
1 W1 =[
1 11 −1
] [1 11 −1
]= 2 I2
W T
2 W2 =[
W1 W1
W1 −W1
] [W1 W1
W1 −W1
]= 4 I4
So we see that
W T
kWk =
[Wk�1 Wk�1
Wk�1 −Wk�1
] [Wk�1 Wk�1
Wk�1 −Wk�1
]= 2k I2k .
Hence, 1√2k
Wk is an 2k × 2k orthonormal matrix, for k = 0, 1, 2, . . . .
15
(c)
T =
⎡⎢⎢⎢⎢⎢⎣
W01√2W1
12W2
123/2 W3
. . .
⎤⎥⎥⎥⎥⎥⎦
=
⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣
1 0 0 0 0 0 0 00 1√
21√2
0 0 0 0 00 1√
2− 1√
20 0 0 0 0
0 0 0 12
12
12
12 0
0 0 0 12 −1
212 −1
2 00 0 0 1
212 −1
2 −12 0 · · ·
0 0 0 12 −1
2 −12
12 0
...
⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦
Note that T T = T .
T T T =
⎡⎢⎢⎢⎢⎢⎣
W T
0 W012W T
1 W1122 W T
2 W2123 W T
3 W3
. . .
⎤⎥⎥⎥⎥⎥⎦ =
⎡⎢⎢⎢⎢⎢⎣
I1
I2
I4
I8
. . .
⎤⎥⎥⎥⎥⎥⎦
So T is an unitary matrix.
(d) The tiling of the time-frequency plane is approximately as follows (Ti denotes the i-th row ofT ):
. . .
ω
n0 1 3 7 15
T3
T2
T5
T7
T6
T4
T1
π
0
Figure P2.19(d)
16
Chapter 3
3.1 Orthogonality and completeness of the sinc basis(a) First, let’s prove 〈ϕ2k[n], ϕ2l+1[n]〉 = 0. Since ϕ2k[n] = ϕ0[n − 2k], it suffices to show that〈ϕ0[n], ϕ2l+1[n]〉 = 0, or more explicitly, 〈g0[n], g1[n− 2l]〉 = 0. By Parseval’s theorem,
〈g0[n], g1[n− 2l]〉 =12π
∫ π
−πG0(ejω)G1(ejω)e−jω2ldω .
Given g1[n] = (−1)ng0[−n+ 1], the Fourier transform of g1[n] is
G1(ejω) = −G0(e−j(ω−π))e−jω .
Since G0(ejω) is an ideal lowpass filter with cutoff at π/2, G1(ejω) is a highpass filter at cutoff π/2,and they do not overlap. Therefore, the integral is zero and the relation follows.
Next, let’s prove 〈ϕ2k+1[n], ϕ2l+1[n]〉 = δ[k − l] . Because ϕ2k+1[n] = ϕ1[n − 2k], it suffices toshow that 〈ϕ1[n], ϕ2l+1[n]〉 = δ[l], or 〈g1[n], g1[n− 2l]〉 = δ[l].
〈g1[n], g1[n− 2l]〉 = 〈(−1)ng0[−n+ 1], (−1)n−2lg0[−(n− 2l) + 1]〉= 〈g0[−n+ 1], g0[−n+ 2l + 1]〉= 〈g0[n], g0[n+ 2l]〉= δ[l]
where the last equality follows from the fact that g0[n] is orthonormal to its even shifts (i.e.〈g0[n], g0[n− 2l]〉 = δ[l]).
(b) Assume that the set {ϕk} is not complete. That is, suppose there exists a sequence x[n] with‖x‖ > 0 such that ‖〈ϕk, x〉‖ = 0 for all k.
‖〈ϕk, x〉‖ = 0 ⇐⇒ ‖〈ϕk, x〉‖2 = 0
⇐⇒∑k∈Z|〈ϕk[n], x[n]〉|2 = 0
⇐⇒ X[k] = 〈ϕk[n], x[n]〉 = 0, ∀k
Let’s look at the Fourier transform of the even and odd terms separately. The terms X[2k] =〈g0[n − 2k], x[n]〉 can be interpreted as x[n] convolved with g0[−n], followed by sampling at every2k index, where k ∈ Z. Noting that FT {g0[−n]} = G0(e−jω), we have (also see Equation (2.5.19))
X[2k] = 〈g0[n− 2k], x[n]〉 = 0 FT←→ 12[G0(e−jω)X(ejω) +G0(−e−jω)X(−ejω)
]= 0.
By Parseval’s equality,
0 =∞∑
k=−∞|X[2k]|2 =
12π
∫ π
−π
∣∣∣∣12[G0(e−jω)X(ejω) +G0(−e−jω)X(−ejω)]∣∣∣∣2 dω
=14
12π
∫ π
−π
(|G0(e−jω)|2|X(ejω)|2 + 2Re{G0(e−jω)X(ejω)G0(−e−jω)X(−ejω)}+
|G0(−e−jω)|2|X(−ejω)|2)dω
17
Note that G0(e−jω) = G0(ejω) because it is symmetric in ω. Also G0(−e−jω) is G0(ejω) modulatedby π, so they do not overlap (recall G0(ejω) is an ideal lowpass with a cutoff of π/2). Thus thecross term is 0. The above expression then becomes
14
12π
(∫ π2
−π2
|X(ejω)|2dω +∫
[−π,−π2]∪[ π
2,π]|X(−ejω)|2dω
)= 0,
which impliesX(ejω) = 0 ∀ω ∈ [−π
2,π
2].
Similarly, working with the odd terms X[2k + 1] = 〈g1[n− 2k], x[n]〉 will conclude that
X(ejω) = 0 ∀ω ∈ [−π,−π2
] ∪ [π
2, π].
Combining these two results shows that X[k] = 0 for all k implies X(ejω) = 0 for all ω ∈ [−π, π].
X(ejω) = 0 ∀ ω ⇐⇒ |X(ejω)|2 = 0 ∀ ω ⇐⇒ ‖x‖2 = 0 (by Parseval) .
This contradicts the assumption that ‖x‖ > 0, and thus the set is complete.
3.2 Necessity of shift by one in orthogonal basesWe assume the inner product definition preceding (2.2.1). As in (3.1.20), define
ϕ2k[n] = g0[n− 2k] =1√2
sin π2n
π2n
, and ϕ2k+1[n] = g1[n− 2k] = (−1)n1√2
sin π2n
π2n
.
We wish to show that the ϕk’s do not form an orthogonal basis for �2(Z). To show that the ϕk’sare not complete, we will construct x[n] ∈ �2(Z) with ‖x‖ = 1 that has zero expansion. Intuitively,the problem with the ϕk’s is that g0 and g1 are both even, so all of the putative basis functions areeven about an even point. Hence they cannot represent odd functions.
Let y[n] = sin π2n. Then for any k ∈ Z,
〈ϕ2k, y〉 =∑n
ϕ∗2k[n]y[n]
=∑n
g∗0 [n− 2k]y[n]
=∑m
g∗0 [m]y[m+ 2k] by substituting m = n− 2k
=∑m
1√2
sin π2m
π2m
sinπ
2(m+ 2k)
=∑m
1√2
sin π2m
π2m
(−1)k sinπ
2m
= 0 since the summand is an odd function of m.
Similarly, 〈ϕ2k+1, y〉 = 0 for all k ∈ Z. This does not work as a counterexample to completenessbecause y �∈ �2(Z). However, this is useful in constructing a counterexample.
For k ∈ Z+, let
xk[n] =
{1√
2k+2sin π
2n, −(2k + 1) ≤ n ≤ 2k + 10, otherwise
.
18
Each xk has unit norm, so xk ∈ �2(Z) for all k ∈ Z+. Let x = limk→∞ xk. Because �2(Z) is a
complete metric space, x ∈ �2(Z) and ‖x‖ = 1. It is easy to check that 〈ϕk, x〉 = 0 for all k, so theϕk’s do not form an orthogonal basis.
3.3 Nonorthogonal time-invariant expansionsA time-invariant expansion satisfies the relation: x[n] ←→ X[k] ⇐⇒ x[n − m] ←→ X[k − m].In terms of the basis vectors φk[n], this is equivalent to 〈φk[n], x[n −m]〉 = 〈φk−m[n], x[n]〉. Thechange of variable n → n + m on the left side gives 〈φk[n + m], x[n]〉 = 〈φk−m[n], x[n]〉, and asecond change of variable k → k + m gives 〈φk+m[n + m], x[n]〉 = 〈φk[n], x[n]〉, which impliesthat φk+m[n + m] = φk[n] for a time-invariant expansion, as derived in Proposition 3.3. For anonorthogonal time-invariant expansion, we have
〈φk[n], φk+m[n]〉 = 〈φk[n], φk[n −m]〉 = a[m] �= δ[m]
Taking the Fourier transform yields:
A(ejω) =∑m
(∑n
φ∗k[n]φk[n−m]
)e−jωm
=∑n
∑m
(φ∗k[n]e−jωn
) (φk[n−m]e−jω(m−n)
)=
∑n
φk[n]∗e−jωn∑
m′=m−nφk[−m′]e−jωm
′
= Φ∗(e−jω)Φ(e−jω) =∣∣Φ(e−jω)
∣∣2 =∣∣Φ(ejω)
∣∣2 = A(ejω) �= 1
Because it is not orthogonal, this expansion is frequency-selective. We conclude by noting that if weare given a function A(ejω) that is nonnegative on the unit circle, we can find Φ(ejω) within a phasefactor, and from that we can determine the frequency-selective nonorthogonal basis functions, inthis case denoted by ck[n]:
Ak(ejω) = Bk(ejω)B∗k(e
jω) = Ck(ω)ejθk(ω)Ck(ω)e−jθk(ω)
ck[n] =12π
∫ π
−πCk(ω)ejθk(ω)ejωndω
ck+m[n+m] =12π
∫ π
−πCk+m(ω)ejθk+m(ω)ejω(n+m)dω
The time-invariance requirement ck[n] = ck+m[n+m] implies that
Ck(ω)ejθk(ω) = Ck+m(ω)ejθk+m(ω)ejωm
which is satisfied by
Ck+m(ω) = Ck(ω) and θk(ω) = θk+m(ω) + ωm
3.4 Proof of Theorem 3.7(a) ⇔ (c): This equivalence follows from carefully lining up the rows and columns of Ta andTs. In (3.2.1), let us number the rows such that y0[0] is in the zeroth row of y and x[0] is in the
19
zeroth row of x. (It follows that yi[k] is in the (2k + i)-th row, k ∈ Z, i ∈ {0, 1}.) Recall thedefinitions of Ta following (3.2.1) and Ts following (3.2.6). With this row numbering, the impliedcolumn numbering has h0[0] in the (0,0) position of Ta and g0[0] in the (0,0) position of Ts. Nowthe (2k + i)-th row of Ta is hi[2k − n] and the (2� + j)-th row of Ts
T is gj [n − 2�]. Thus the(2k + i, 2�+ j) element of TaTs is 〈hi[2k − n], gj [n− 2�]〉 and it follows that (a) ⇔ (c).
(b) ⇔ (d): Recall the definitions of Gm(z) following (3.2.29) and of Hm(z) in (3.2.14). By simplysubstituting the definitions,
Gm(z)Hm(z) =[
G0(z)H0(z) +G1(z)H1(z) G0(z)H0(−z) +G1(z)H1(−z)G0(−z)H0(z) +G1(−z)H1(z) G0(−z)H0(−z) +G1(−z)H1(−z)
].
It follows that Gm(z)Hm(z) = 2I ⇔ (b). Since scalar rational functions commute and Gm(z) andHm(z) are square, Gm(z)Hm(z) = 2I ⇔ Hm(z)Gm(z) = 2I.
(d) ⇔ (e): Using (3.2.28) and (3.2.29), we can write
Hp(z2) =12Hm(z)
[1 11 −1
] [1 00 z−1
]and
Gp(z2) =12
[1 00 z
] [1 11 −1
]Gm(z).
Thus
Hp(z2)Gp(z2) =(
12
)2
Hm(z)[
1 11 −1
] [1 00 z−1
] [1 00 z
] [1 11 −1
]Gm(z)
=(
12
)2
Hm(z)2IGm(z)
=12Hm(z)Gm(z).
It follows immediately that Hp(z)Gp(z) = I ⇔ Hm(z)Gm(z) = 2I. Noting that all the matricesinvolved are square, (d) ⇔ (e).
(a) ⇔ (d): We will start with Hm(z)Gm(z) = 2I and show that this is equivalent to (a). (Sincethe matrices are square, this is equivalent to (d).) Looking at the (i, j)th element of the productHm(z)Gm(z) gives
Hi(z)Gj(z) +Hi(−z)Gj(−z) = 2δ[i − j].Using the modulation property, we can write the time-domain equivalents∑
n∈Zhi[n]gj [k − n] + (−1)k
∑n∈Z
hi[n]gj [k − n] = 2δ[i − j]δ[k],
or (1 + (−1)k
)∑n∈Z
hi[n]gj [k − n] = 2δ[i − j]δ[k], ∀k ∈ Z. (4)
To restrict our attention to even k, replace k by 2m. Noting that 1+(−1)2m = 2 and δ[2m] = δ[m],this gives ∑
n∈Zhi[n]gj [2m− n] = δ[i− j]δ[m], ∀m ∈ Z. (5)
20
Writing (5) as an inner product and replacing n and m by −n and −m, respectively, gives (a). Forodd k, δ[k] = 0 and 1 + (−1)k = 0. Thus (4) is automatically satisfied for odd k and (4) followsfrom (5) when the steps are reversed.
3.5 Constraint on the length of orthogonal FIR filtersWithout loss of generality, let the support of g0[n] be n = 0, 1, ..., L− 1, and specifically, that g0[0]and g0[L− 1] be non-zero. Since P (Z) = G0(z)G0(z−1), p[n] = g0[n] ∗ g0[−n], the support of p[n]is {−(L− 1),−(L− 2), ..., L − 2, L− 1}. In the time-domain, the autocorrelation satisfies
p[n] + (−1)np[n] = 2δ[n] .
In other words, p[2n] = 0 for n = ±1,±2, ..., and p[0] = 1. Evaluating at n = L− 1, and assumingL is odd and greater than 1, we have
p[L− 1] + (−1)L−1p[L− 1] = 2p[L− 1] = 0 .
Since p[L− 1] = g0[0]g0[L− 1], this implies that either g0[0] or g0[L− 1] is zero, contradicting theassumption. Hence, L cannot be odd, and thus must be even. (Note that the only case when Lcould be odd is for the trivial case of L = 1.)
3.6 Factorization
For the factors A(z) = (1 + z)3(1 + z−1
)3 and B(z) =1
256(3z2 − 18z + 38 − 18z−1 + 3z−2
)we have to show that P (z) = A(z) B(z) satisfies P (z) + P (−z) = 2. Factoring out the 256 inB(z) to simplify the algebra, we have:
256 [A(z)B(z) +A(−z)B(−z)] =(3z2 + 3z−2
)(A(z) +A(−z))
+(18z + 18z−1
)(−A(z) +A(−z)) + 38 (A(z) +A(−z))
Since A(z) = z3 + 6z2 + 15z + 20 + 15z−1 + 6z−2 + z−3
we have A(z) +A(−z) = 12z2 + 40 + 12z−2 and −A(z) +A(−z) = −2z3 − 30z − 30z−1 − 2z−3.Calculating the coefficients of each power of z yields:
256 [A(z)B(z) +A(−z)B(−z)] = z4(36− 36) + z2(120 − 540 + 456 − 36) + z3(0) + z(0)+ (36− 540 + 1520 − 540 + 36) + z−1(0) + z−3(0)+ z−2(120 − 540 + 456− 36) + z−4(36− 36)= 512
which confirms that P (z) + P (−z) = 2.Factorization yields B(z) = 4.14 × 10−4
(1− αz−1
) (1− α∗z−1
)(1− αz) (1− α∗z)
with α = −2.71 + 1.44j and where α∗ denotes the complex conjugate of α.For the orthogonal filter bank, we can take
H0(z) =√
4.14 × 10−4(1 + z−1
)3 (1− αz−1) (
1− α∗z−1)
so thath0[n] = 0.0205[1, 8.426, 28.720, 45.608, 33.757, 9.444]h1[n] = 0.0205[−9.444, 33.757,−45.608, 28.720,−8.426, 1]
21
3.7 Proof of Theorem 3.8Theorem 3.8 is essentially a specialization of Theorem 3.7 to the orthonormal, FIR case. Most ofthe proof of Theorem 3.7 (Problem 3.4) can be repeated with hi[n] = gi[−n], Hi(z) = Gi(z−1),Ta = Ts
T , Hm(z) = GmT (z−1), and Hp(z) = Gp
T (z−1).All that remains to be shown is G1(z) = −z−2K+1G0(−z−1) for some K ∈ Z. The first column
of GmT (z−1)Gm(z) = 2I is
GmT (z−1)
(G0(z)G1(z)
)=(
20
).
Explicitly computing[Gm
T (z−1)]−1 gives(
G0(z)G1(z)
)=
[Gm
T (z−1)]−1
(20
)
=1
G0(z−1)G1(−z−1)−G0(−z−1)G1(z−1)
(G1(−z−1) −G1(z−1)−G0(−z−1) G0(z−1)
)(20
)
=2
G0(z−1)G1(−z−1)−G0(−z−1)G1(z−1)
(G1(−z−1)−G0(−z−1)
).
Thus
G0(z) = A(z)G1(−z−1) and (6)G1(z) = −A(z)G0(−z−1), (7)
whereA(z) =
2G0(z−1)G1(−z−1)−G0(−z−1)G1(z−1)
.
Substituting −z−1 for z in (7) gives G1(−z−1) = −A(−z−1)G0(z). Substituting this in (6), givesG0(z) = −A(z)A(−z−1)G0(z), so
A(z)A(−z−1) = −1. (8)
All FIR solutions of (8) are of the form A(z) = z−2K+1. Combining this with (7) completes theproof.
3.8 Linear phase solutions(This proof is taken from reference [319]; in which further details can be found.) First, note thatsince H0(z) and H1(−z) both have linear phase, so does P (z) = H0(z)H1(−z). If P (z) is of oddlength 2N +1, then the analysis filters must either both have odd length or both have even length.We have
P (z)− P (−z) =2N∑i=0
piz−i[1− (−1)i] =
N−1∑i=0
2p2i+1z−(2i+1)
Since P (z) is of odd length, both P (z) and P (−z) are symmetric or anti-symmetric about thecenter point i = N , which implies that P (z) − P (−z) is symmetric about i = N and the singlenonzero coefficient is the central one pN . Thus, P (z) is symmetric and not antisymmetric, whichimplies that H0(z) and H1(−z) are either both symmetric or both anti-symmetric for P (z) of oddlength.
(a) L0 and L1 both odd. The length of P (z) is L0 + L1 + 1, which is also odd. The center ofsymmetry at (L0 + L1)/2 − 1 samples from the endpoints must be odd because the only nonzero
22
elements in P (z) − P (−z) are at odd powers of z (the center of symmetry is at a non-zero point).Thus, (L0 +L1)/2 = L0 +(L1−L0)/2 has to be even, which means that (L1−L0)/2 has to be oddbecause L0 is odd. Thus, the length difference L1−L0 is an odd multiple of 2. Note that there are nosame-length solutions. It was stated in the opening that H0(z) and H1(−z) can be both symmetricor both antisymmetric. The latter case is ruled out because the polyphase components are alsoantisymmetric when the length is odd, and therefore are not coprime, so perfect reconstructionwould not be possible.
(b) L0 and L1 both even. Again the center point (L0+L1)/2−1 has to be odd. Then L0+(L1−L0)/2again has to be even; since L0 is even, the length difference L1 − L0 is an even multiple of 2. Inthis case, same-length solutions do exist. If H0(z) is symmetric, so is H1(−z) (as shown in theopening). Then, H1(z) is antisymmetric because it has even length. Likewise, H1(z) is symmetricif H0(z) is antisymmetric.
(c) If P (z) has even length, one filter has odd length, and one even. Then
P (z)− P (−z) =2N−1∑i=0
piz−i[1− (−1)i] =
N−1∑i=0
2p2i+1z−(2i+1)
From 3.2.66, P (z) has only a single nonzero odd-indexed coefficient. Note however that pi =±p2N−1−i for i = 0, . . . , N − 1, so odd-indexed coefficients are paired with even-indexed ones. Itfollows that P (z) also has only one nonzero even-indexed coefficient, and is of the form P (z) =pjz
−j(1±z2N−1−2j). Clearly, all of the zeros of P (z) are on the unit circle at the 2(N − j)−1 rootsof ±1; then, all of the zeros of H0(z) and H1(−z) are on the unit circle. If H0(z) and H1(z) areboth to be antisymmetric, both must have zeros at z = 1. This means that H1(−z) has a zero atz = 1 or z = −1 depending on whether it is of odd or even length, respectively. This implies thatP (z) has either a double zero at z = 1 or a pair at z = 1 and z = −1. Since P (z) contains only the2(N − j) − 1 roots of ±1 both possibilities are ruled out. Thus, either both filters are symmetric,or one is antisymmetric and the other is symmetric.
3.9 Test conditions for linear phaseFrom condition (b) of Proposition 3.11, for both H0(z) and H1(z) to have the same lengths and belinear phase, they must both be of even lengths and one filter is symmetric while the other filteris antisymmetric. Let L be the length of the filters, and, without loss of generality, let H0(z) besymmetric and H1(z) be antisymmetric. Because the filters are linear phase, we can write
H0(z) = z−L+1H0(z−1) ,
H1(z) = −z−L+1H1(z−1) .
Writing H0(z) in its polyphase components, we have
H0(z) = H00(z2) + z−1H01(z2)= z−L+1(H00(z−2) + zH01(z−2))= z−L+2H01(z−2) + z−1(z−L+2H00(z−2)) .
Similarly,H1(z) = −z−L+2H11(z−2)− z−1(z−L+2H10(z−2)) .
23
Hp(z) =[H00(z) H01(z)H10(z) H11(z)
]
=
[z−
L2+1H01(z−1) z−
L2+1H00(z−1)
−z−L2+1H11(z−1) −z−L
2+1H10(z−1)
]
= z−L2+1
[H01(z−1) H00(z−1)−H11(z−1) −H10(z−1)
]
= z−L2+1
[1 00 −1
] [H01(z−1) H00(z−1)H11(z−1) H10(z−1)
]
= z−L2+1
[1 00 −1
] [H00(z−1) H01(z−1)H10(z−1) H11(z−1)
] [0 11 0
]
Hence, Hp(z) satisfies the linear phase testing condition with k = L/2 − 1.
3.10 Complex linear phase FIR solutions(a) First, we rederive the orthonormality relations of theorem 3.8 for the case of complex filters.We have
〈gi[n], g∗j [n+ 2m]〉 = δ[i− j]δ[m]
Using the Z-transform properties x[−n]↔ X(1/z) and x∗[n]↔ X∗(z∗) as well as the modulationproperty (−1)nx[n]↔ X(−z), we can rewrite the inner product relation above in the Z domain as
Gi(z)G∗j (1/z
∗) + Gi(−z)G∗j (−1/z∗) = 2δ[i − j]
For real filters, the Z transform is conjugate symmetric, so in the real case we can replace G∗j (1/z
∗)by Gj(z−1) to arrive at the relations in parts (b) and (d) of theorem 3.8. In the complex case, themodulation domain relations become
Gm(z)GHm(1/z∗) = 2I Hm(z) = GH
m(1/z∗)
In a complex orthogonal filterbank, we can choose
G1(z) = −zG∗0(−1/z∗)
H0(z) = G∗0(1/z
∗)H1(z) = G∗
1(1/z∗)
It is straightforward to verify (by direct substitution) that these filters satisfy the various orthogo-nality relations, some of which are:
G0(z)G∗1(1/z
∗) + G0(−z)G∗1(−1/z∗) = 0
H0(z)H∗0 (1/z∗) + H0(−z)H∗
0 (−1/z∗) = 2H0(z)H∗
1 (1/z∗) + H0(−z)H∗1 (−1/z∗) = 0
Now, we consider the complex case of proposition 3.12. Orthonormality implies
Hp(z)HHp (1/z∗) = I
from which we haveH00(z)H∗
00(1/z∗) + H01(z)H∗
01(1/z∗) = 1
24
The equations in 3.2.68 hold for complex filters as well, so we can substitute for H01(z) as inproposition 3.12. The resulting equation is
H00(z)H∗00(1/z
∗) + H00(z−1)H∗00(z
∗) = 1
As a check, note that conjugate symmetry for real filters then leads to the relation in the book:H00(z)H00(z−1) = 1/2. The Z domain equation for H00(z) implies the time-domain relation
Real {〈h00[m], h00[m− k]〉} =12
where the inner product implies a conjugation of the second term. An equivalent result can be de-rived for the other polyphase component. These requirements are non-restrictive; in other words,we have verified the existence of linear phase orthogonal filterbanks with complex coefficients.
(b) The factorization of P (z) for the D3 filter is derived in the solution to problem 3.6. It consistsof six zeros at −1 and a set of four conjugate reciprocal zeros:
P (z) = (1 + z)3(1 + z−1)3(1− αz−1)(1 − α∗z−1)(1− αz)(1 − α∗z)
The scale factor is omitted for simplicity. In problem 3.6, we showed that this autocorrelationsatisfies P (z) + P (−z) = 2 and indicated a factorization that yielded an orthogonal filterbankwith real coefficients; this resulted from interpreting P (z) as the autocorrelation G0(z)G0(z−1) andgrouping the quadruple of zeros (those not at z = −1) in conjugate pairs in the factorization. Forthe complex case, the orthogonality constraint derived in (a) shows that P (z) should be factored asG0(z)G∗
0(1/z∗). By grouping the quadruple of zeros in reciprocal pairs, we get a filter G0(z) with
complex coefficients that has linear phase and satisfies the validity constraint (ı.e. orthogonality).The linear phase synthesis filters are given by
G0(z) = (1 + z−1)3(1− αz−1)(1− αz)G1(z) = −z(1 + z)3(1 + α∗z)(1 + α∗z−1)
and the analysis filters are related as indicated in part (a).
3.11 Spectral factorization method for two-channel filter banks
(a) P (ejω) =−12
cos(3ω) + cos(ω) + 1 is not positive on the unit circle.
The Smith-Barnwell correction is applied by finding δ = minP (ejω) = −0.076 and taking P ′(z) =P (z)− δ
1− δ . P ′(z) satisfies the requirements P ′(z) + P ′(−z) = 2 and P (ejω) ≥ 0. We have P ′(z) =
H0(z)H0(z−1) where H0(z) = 0.689(1−2.108z−1)(1+1.291z−1 +z−2) and H1(z) = −z−3H0(−z−1).These constitute an orthogonal filter bank.
(b) P (z) = (z + 1 + z−1)(−1
4z2 +
14z +
12
+14z−1 − 1
4z−2
)P ′(z) = z−1P (z) satisfies P ′(z)− P ′(−z) = 2z−2k+1 (with k = 0 in this case) which gives rise to abiorthogonal filter bank with P ′(z) = H0(z)H1(−z) and the filters
H0(z) = z + 1 + z−1 G0(z) =2
detHm(z)H1(−z)
H1(z) =14z +
14− 1
2z−1 +
14z−2 +
14z−3 G1(z) =
−2detHm(z)
H0(−z)
25
whereHm(z) is the modulation matrix (see Section 3.2.4 on designing linear phase FIR filter banks).The result is
G0(z) = −14z2 +
14z +
12
+14z−1 − 1
4z−2 G1(z) = z2 − z + 1
(c) The Parks-McClellan algorithm yields a P (z) with an equiripple stopband (and passband).Suppose the minimum value of P (ejω) is δ; P (ejω) takes exactly this value at each of its minimain the stopband (except possibly at ω = π). Let there be L zeros, then there may be either �L/2�or �L/2� minima. Applying the correction yields P ′(z) =
P (z) − δ1− δ . The minimum of P ′(ejω)
is then zero and P ′(ejω) takes on this zero value at frequencies in the stopband where P (ejω) hasits minima. Thus the original L zeros of P (ejω) has been transformed to double zeros in P ′(ejω)(double zeros because the poles come in (α, 1/α∗) pairs, which are the same points on the unitcircle).
For more detail on the Parks-McClellan algorithm, consult the book Discrete-Time Signal Pro-cessing by Oppenheim and Shafer, Prentice Hall.
3.12 Complementary filtersWe prove that the filter H0(z) = (1 + z−1)N always has a complementary filter in three steps:
(1) Note that this H0(z) does not have any pairs of zeros at (−α,α); all of its zeros are at z = −1.
(2) Prove that H0(z) does not have any pairs of zeros at locations (−α,α) is equivalent to thecoprimeness of its polyphase components.
(3) Use Proposition 3.13 to conclude that H0(z) has a complementary filter because its polyphasecomponents are coprime, as indicated by (1) and (2).
The proof of (2) is as follows. Consider a function U(z) with polyphase components U0(z) andU1(z) so that U(z) = U0(z2) + z−1U1(z2). Construct a new function V (z) by adding two zeros at{α1, α2} to U(z) and determine its polyphase components:
V (z) = (1− α1z−1)(1− α2z
−1)[U0(z2) + z−1U1(z2)
]= (1− (α1 + α2)z−1 + α1α2z
−2)(U0(z2) + z−1U1(z2))=
[(1 + α1α2z
−2)U0(z2)− (α1 + α2)z−2U1(z2)]+ z−1
[−(α1 + α2)U0(z2) + (1 + α1α2z
−2)U1(z2)]
= V0(z2) + z−1V1(z2)
If α2 = −α1, then V0(z) = (1− α21z
−1)U0(z) and V1(z) = (1−α21z
−1)U1(z) and so V0(z) and V1(z)are not coprime. Thus we have proved that if V (z) has a zero pair (α,−α), then V0(z) and V1(z)are not coprime.
Conversely, if V0(z) and V1(z) are not coprime, they can be written as
V0(z) = (1− β2z−1)U0(z)V1(z) = (1− β2z−1)U1(z)
with the common zero β2 = α. Then
V (z) = (1− αz−1)(1 + αz−1)(U0(z2) + z−1U1(z2)),
26
thus V (z) has the zero pair (α,−α).Using these results, we can conclude that since all of the zeros of H0(z) are at z = −1, its
polyphase components are coprime; according to Proposition 3.13, it thus has a complementaryfilter.
3.13 Orthogonal lattice angle constraintsFor orthogonal filter banks (see equation 3.2.61 in example 3.3),
Hp(z) =[H00(z) H01(z)H10(z) H11(z)
]=
[cosα1 sinα1
− sinα1 cosα1
] [1 00 z−1
] [cosα0 sinα0
− sinα0 cosα0
]
From this factorization, we have
H0(z) = cosα0 cosα1 + sinα0 cosα1z−1 − sinα0 sinα1z
−2 + cosα0 sinα1z−3
H0(ejω)∣∣ω=π
= cosα0 cosα1 − sinα0 cosα1 − sinα0 sinα1 − cosα0 sinα1
= cos(α0 + α1)− sin(α0 + α1) = 0
We can thus conclude for the orthogonal lattice that α0 + α1 =π
4or
5π4
.
3.14 Interpolation followed by decimation(a) The z-transform of the input and output are related by:
X(z) =12X(z)
[H(z
12
)G(z
12
)+H
(−z 1
2
)G(−z 1
2
)]=
12X(z)
[P(z
12
)+ P
(−z 1
2
)]For x[n] = x[n− k], we have X(z) = z−kX(z), so the product filter P (z) must satisfy
12
[P(z
12
)+ P
(−z 1
2
)]= z−k =⇒ P (z) + P (−z) = 2z−2k
Thus P (z) must consist of a single even power of z.
(b) H(z) and G(z) can be expressed in terms of their polyphase components as
H(z) = H0(z2) + z−1H1(z2) and G(z) = G0(z2) + zG1(z2)
Combining this with the condition from (a) yields
P (z) + P (−z) = 2[H0(z2)G0(z2) + H1(z2)G1(z2)] = 2z−2k
which reduces toH0(z)G0(z) +H1(z)G1(z) = z−k
Using Bezout’s identity, we conclude that G(z) can be found if and only if the polyphase componentsof H(z) have no zeros in common (except for zeros at ∞), that is, they are coprime.
(c) The polyphase components of H ′(z) are equivalent (H ′0(z) = H ′
1(z) = 1+ z−1) and thus not co-prime, so there is no G′(z) such that perfect reconstruction is achieved. The polyphase componentsof H ′′(z) are
H ′′0 (z) = 1 + z−1 + z−2 and H ′′
1 (z) = 1 + z−1
27
UsingG′′
0(z) = 1− z−1 and G′′1(z) = 1− z−1 + z−2
givesH ′′
0 (z)G′′0(z) + H ′′
1 (z)G′′1(z) = 2
so a perfect reconstruction synthesis filter for H ′′(z) is given by
G′′(z) = G′′0(z
2) + zG′′1(z
2) = z + 1− z−1 − z−2 + z−3
If a causal filter is desired, z−2kG′′(z) can be used.
3.15 Orthogonality relations for an octave-band filter bank(a) First, let’s prove equation (3.3.16)
〈g(j)1 [n− 2jk], g(i)
1 [n − 2il]〉 = δ[i− j]δ[k − l] .
Let Qij(z) denote the z-transform of the correlation between g(i)1 [n] and g(j)
1 [n], so
Qij(z) = G(i)1 (z)G(j)
1 (z−1) .
Equation (3.3.16) is satisfied if and only if all the terms z(2il−2jk), for all i �= j and k �= l , in Qij(z)have zero coefficients. We proceed the prove by breaking the problem down into 2 cases: i = j andj < i (j > i is similar). Recall that
G(j)1 (z) = G1(z2j−1
)G0(z2j−2) · · ·G0(z2)G0(z) .
Case 1: i = jQij(z) can then be written as
Qjj(z) =[G1(z2j−1
)G0(z2j−2) · · ·G0(z2)G0(z)
] [G1(z−2j−1
)G0(z−2j−2) · · ·G0(z−2)G0(z−1)
]= G1(z2j−1
)G1(z−2j−1)︸ ︷︷ ︸ G0(z2j−2
)G0(z−2j−2) · · ·G0(z2)G0(z−2)G0(z)G0(z−1)︸ ︷︷ ︸
P (j−1)(z)
where P (j)(z) = G(j)0 (z)G(j)
0 (z−1) is the z-transform of the autocorrelation of g(j)0 [n] (see the proof
of Proposition 3.15). To prove (3.3.16) for i = j, it is equivalent to show that Qjj(z) has zerocoefficients for z2j(k−l) for all k �= l and k, l ∈ Z; i.e. z2jk terms for any k ∈ Z have zero coefficients.
From the orthogonality of g1[n] with respect to its even shifts, we have
〈g1[n− 2k], g1[n− 2l]〉 = δ[k − l]⇐⇒ R(z)= G1(z)G1(z−1) = 1 + zR1(z2)
where R(z) is the z-transform of the autocorrelation function of g1[n] and R1(z) is the 1st polyphasecomponent of R(z).
From the proof of Proposition 3.15, we can write P (j−1)(z) in its polyphase form as
P (j−1)(z) = 1 +2j−1−1∑n=1
znP (j−1)n (z2j−1
) .
28
Now we can write Qjj(z) as
Qjj(z) = R(z2j−1)P (j−1)(z)
=[1 + z2j−1
R1(z2j)]⎡⎣1 +
2j−1−1∑n=1
znP (j−1)n (z2j−1
)
⎤⎦
= 1 + z2j−1R1(z2j
) +2j−1−1∑n=1
znP (j−1)n (z2j−1
)
︸ ︷︷ ︸A
+ z2j−1R1(z2j
)2j−1−1∑n=1
znP (j−1)n (z2j−1
)
︸ ︷︷ ︸B
.
It is clear that the first two terms do not have powers of z2j. The third expression labelled as A
contains z-terms which have exponents of the form
2j−1−1∑n=1
(n+mn2j−1) = 2j−12j−1−1∑n=1
mn︸ ︷︷ ︸multiples of 2j−1
+2j−1−1∑n=1
n
where mn ∈ Z, n = 1, . . . , 2j−1 − 1. The latter summation can be found (by the trapezoidal rule)to be
2j−1−1∑n=1
n = (2j−1 − 1 + 1)(2j−1 − 1)/2 = 22j−3 − 22j−2 .
Hence, the expression A does not contain powers of z2j.
The expression B contains terms which have exponents of the form
2j−1 +m02j +2j−1−1∑n=1
(n+mn2j−1) = m02j + 2j−1(1 +2j−1−1∑n=1
mn)︸ ︷︷ ︸multiples of 2j−1
+2j−1−1∑n=1
n .
It is easy to verify that this expression does not contain powers of z2j.
Case 2: Without loss of generality, assume j < i.For j < i, we need to show that all coefficients of the terms z2jk−2il = z2j(k−l2i−j), for all
k, l ∈ Z, are zero. That is, we need to show that all coefficients of powers of z2jare zero.
The term Qij(z) can be written (after rearranging) as
Qij(z) =[G0(z2j−1
)G1(z−2j−1)] [G1(z2i−1
)G0(z2i−2) · · ·G0(z2j
)]P (j−1)(z) .
From the orthogonality of g0[n] and g1[n] with respect to even shifts, we have the following relation:
〈g0[n − 2k], g1[n− 2l]〉 = 0 ∀k, l ∈ Z⇐⇒ S(z)= G0(z)G1(z−1) = zS1(z2)
29
where S1(z) is the first polyphase component of S(z).Now Qij(z) can be expanded as
Qij(z) =[z2j−1
S1(z2j)] ⎡⎣1 +
2j−1−1∑n=1
znP (j−1)n (z2j−1
)
⎤⎦ [G1(z2i−1
)G0(z2i−2) · · ·G0(z2j
)].
After expanding Qij(z), the first term contains z terms whose exponents are of the form
2j−1 +m02j +m12i−1 +m22i−2 + · · ·+mi−j2j
= 2j−1 + 2j(m0 +m12i−j−1 +m22i−j−2 + · · · +mi−j) , ∀m0,m1, . . . ,mi−j ∈ Z ,
which cannot be multiples of 2j .Similarly, the second term contains exponents of the form
2j−1 +m02j +m12i−1 +m22i−2 + · · ·+mi−j2j +∑2j−1−1
k=1 (k + 2j−1nk)= 2j−1(1 +
∑2j−1−1k=1 nk) +
∑2j−1−1k=1 k + 2j(m0 +m12i−j−1 +m22i−j−2 + · · ·+mi−j) ,
∀m0,m1, . . . ,mi−j, n1, . . . , n2j−1−1 ∈ Z ,
which cannot be multiples of 2j .
(b) To show equation (3.3.17)
〈g(J)0 [n− 2Jk], g(j)
1 [n− 2j l]〉 = 0 , ∀j = 1, . . . , J,
it is equivalent to show that G(J)0 (z)G(j)
1 (z−1) has zero coefficients for the terms z2Jk−2j l for anyk, l ∈ Z (or equivalently, any powers of z2j
must have zero coefficients since we can write 2Jk−2j l =2j(2J−jk − l)).
By rearranging terms, we get
G(J)0 (z)G(j)
1 (z−1) =
⎧⎪⎪⎨⎪⎪⎩
P (j−1)(z)[G0(z2j−1
)G1(z−2j−1)] [G0(z2j
) · · ·G0(z2J−1)]
if j < J
P (j−1)(z)[G0(z2j−1
)G1(z−2j−1)]
if j = J .
By substituting the expressions for P (j−1)(z) and G0(z2j−1)G1(z−2j−1
) from part (a), and ex-panding the terms, it is straightforward to verify that G(J)
0 (z)G(j)1 (z−1) contains no powers of z2j
.
3.16 Tree-structured orthogonal filter banks(a) The z-transform of the equivalent filters are
G(2)0 (z) = G0(z)G0(z2) , G
(2)1 (z) = G0(z)G1(z2) ,
G(2)2 (z) = G1(z)G0(z2) , G
(2)3 (z) = G1(z)G1(z2) .
For G0(ejω) being ideal lowpass filter, and G1(ejω) being ideal highpass filter, the filters G(2)i (ejω)
are bandpass filters:
G(2)0 (ejω) =
{1 |ω| ≤ π
40 otherwise
30
G(2)1 (ejω) =
{1 π
4 ≤ |ω| ≤π2
0 otherwise
G(2)2 (ejω) =
{1 3π
4 ≤ |ω| ≤ π0 otherwise
G(2)3 (ejω) =
{1 π
2 ≤ |ω| ≤3π4
0 otherwise.
(b) Now let’s take the Haar filters
G0(z) =1√2(1 + z−1)
G1(z) =1√2(1− z−1) .
Multiplying out the expressions for G(2)i (z), i = 0, 1, 2, 3 and transform to time-domain yields
g(2)0 [n] =
12(δ[n] + δ[n − 1] + δ[n − 2] + δ[n− 3])
g(2)1 [n] =
12(δ[n] + δ[n − 1]− δ[n − 2]− δ[n− 3])
g(2)2 [n] =
12(δ[n]− δ[n − 1] + δ[n − 2]− δ[n− 3])
g(2)3 [n] =
12(δ[n]− δ[n − 1]− δ[n − 2] + δ[n− 3]) .
(c) Given that {g0[n], g1[n]} is an orthogonal pair implies that
〈gi[n], gi[n− 2k]〉 = δ[n − k] , i = 0, 1〈g0[n], g1[n− 2k]〉 = 0 ∀k ∈ Z
In z-domain, this implies we can write
Pi(z)= Gi(z)Gi(z−1) = 1 + zPi1(z2) , i = 0, 1
R(z)= G0(z)G1(z−1) = zR1(z2) ,
where Pi1(z) is the 1st polyphase component of Pi(z) and R1(z) is the 1st polyphase componentof R(z). The first equation says that the z-transform of the autocorrelation of gi[n] has only 1 inits 0-th polyphase component (because gi[n] is orthogonal to its even shifts). The second equationsays that the z-transform of the cross-correlation of g0[n] and g1[n] has only nonzero coefficients inodd powers of z (because g0[n] is orthogonal to all even shifts of g1[n]).
We need to show that for any i, j = 0, 1, 2, 3
〈g(2)i [n], g(2)
j [n− 4k]〉 = δ[i− j] ,∀k ∈ Z .
For i �= j, this is equivalent to showing that G(2)i (z)G(2)
j (z−1) (the z-transform of the cross-correlation) has zero coefficients for all powers of z4. There are 6 cases to be looked at, we will
31
only show the orthogonality between g(2)0 [n] and g
(2)1 [n], and the other cases are similar. The
cross-correlation can be written in the z-domain as
G(2)0 (z)G(2)
1 (z−1) = G0(z)G0(z2)G0(z−1)G1(z−2)= (1 + zP01(z2))(z2R1(z4)) .
It can be easily verified that this expression does not contribute to any powers of z4. Hence〈g(2)
0 [n], g(2)1 [n− 4k]〉 = 0 ∀k ∈ Z.
3.17 Define indices for full binary treeBecause there is a downsampling operation following the filters, the high frequency component gets“flipped”, corresponding to a bit reversal in the numbering scheme. Hence, the numbering schemecorresponds to these steps (assuming the upper branch is always highpass, and the lower branch islowpass):
1. At the root of the tree, number the upper branch 1, and the lower branch 0.
2. If the current node at level j is on a lower branch, then number its two downward branchesof level j + 1 the same way as the branches at level j.
3. If the current node at level j is on an upper branch, then number its two downward branchesof level j + 1 in the reverse way as the branches at level j.
4. If not at the end of the tree yet, then go back to step 2. If it is the end of the tree, thenlabel each leaf with the corresponding binary sequence (with the most significant bit beingthe branch at at the top of the tree). Convert this binary sequence to integer and we havethe desired indices.
The following figure illustrates the numbering scheme for J = 3:
1
0
0
0
1
1
0
1
0
1
G
G
G
G
G
G
G
G0
1
3
2
5
4
6
71
0
0
1
3.18 Iterated filters
In an iterated filter bank, filters at depth i have the form Hi(z) =i−1∏l=0
Hk(l)
(z2l)
where k(l) = 0
or 1. For linear phase H0(z) and H1(z) we have Hk(l)(ejω) = Ak(l)(ω)ejαk(l)ω where Ak(l)(ω) is a
32
real-valued function. Then,
Hk(l)
(z2l)
= Ak(l)
(2lω
)ejαk(l)2
lω
=⇒ Hi(ejω) =i−1∏l=0
Ak(l)
(2lω
)ejαk(l)2
lω
= A(ω)ejαω
where
A(ω) =i−1∏l=0
Ak(l)
(2lω
)and α =
i−1∑l=0
αk(l)2l
Therefore all of the Hi(z) are linear phase filters.
For even-length filters h0[n] and h1[n] that are respectively symmetric and anti-symmetric, theiterated filter lengths and symmetries can be derived as follows. First, we note that if Hk(z)is even-length, so is Hk(z2). Now, since the product of two even-length filters has odd length(consider the convolution), we know that all of the depth-two filters are of odd length since h0[n]and h1[n] are both even. Next we note that Hk(z2) has the same symmetry properties as Hk(z).Then, Ha(z) = H0(z)H0(z2) is the product of two symmetric filters, so Ha(z) is symmetric.Hb(z) = H0(z)H1(z2) is the product of a symmetric and an anti-symmetric filter, so Hb(z) isanti-symmetric. Hc(z) = H1(z)H0(z2) is the product of an anti-symmetric and a symmetric filter,so Hc(z) is anti-symmetric. Finally, Hd(z) = H1(z)H1(z2) is the product of two anti-symmetricfilters, so Hd(z) is symmetric.
3.19 Biorthogonality in a full treeFirst, we review some material and elaborate on the discussion in the text. In a 2-channel filterbank, the biorthogonality of the basis vectors is equivalent to a perfect reconstruction conditionon the analysis-synthesis modulation matrices (see theorem 3.7 and the subsection of 3.2.1 onmodulation domain analysis of 2-channel filterbanks):
Gm(z)Hm(z) = 2I ⇐⇒ Hm(z)Gm(z) = 2I
where
Gm(z) =[
G0(z) G1(z)G0(−z) G1(−z)
]and Hm(z) =
[H0(z) H0(−z)H1(z) H1(−z)
]Note the transposition of the modulates that is implicit in the definition of Gm and Hm. We derivethe biorthogonality conditions using the inverse relationship of the analysis and synthesis modula-tion matrices. To be completely explicit, the matrix equations above represent these relations:
G0(z)H0(z) + G1(z)H1(z) = 2 (1)G0(z)H0(−z) + G1(z)H1(−z) = 0 (2)G0(−z)H0(z) + G1(−z)H1(z) = 0 (3)
G0(−z)H0(−z) + G1(−z)H1(−z) = 2 (4)G0(z)H0(z) + G0(−z)H0(−z) = 2 (5)G1(z)H0(z) + G1(−z)H0(−z) = 0 (6)G0(z)H1(z) + G0(−z)H1(−z) = 0 (7)G1(z)H1(z) + G1(−z)H1(−z) = 2 (8)
33
where equations (1) to (4) are from Gm(z)Hm(z) = 2I and (5) to (8) are from Hm(z)Gm(z) = 2I.Note that there is some obvious redundancy in that (1) and (4) are modulates, as are (2) and(3); also, note that equations (5) through (8) can be derived from (1) through (4) as in done insection 3.2.1, which explicitly derives (5) from (1) and (2). The text also shows how applying themodulation property of the Z-transform to (5) through (8) yields the time-domain biorthogonalityconditions on the filter impulse responses, or equivalently the biorthogonal expansion basis vectors.(There is a time-reversal for the synthesis filter impulse responses, though.)
A full tree of depth J corresponds to an 2J -channel filterbank; for this reason we recommendproblem 3.21 in conjunction with this problem. For a tree of depth 2, the filters are given by:
Ha(z) = H0(z)H0(z2) Ga(z) = G0(z)G0(z2)Hb(z) = H0(z)H1(z2) Gb(z) = G0(z)G1(z2)Hc(z) = H1(z)H0(z2) Gc(z) = G1(z)G0(z2)Hd(z) = H1(z)H1(z2) Gd(z) = G1(z)G1(z2)
The modulation matrices are
Hm(z) =
⎡⎢⎢⎣H0(z)H0(z2) H0(Wz)H0(−z2) H0(−z)H0(z2) H0(−Wz)H0(−z2)H0(z)H1(z2) H0(Wz)H1(−z2) H0(−z)H1(z2) H0(−Wz)H1(−z2)H1(z)H0(z2) H1(Wz)H0(−z2) H1(−z)H0(z2) H1(−Wz)H0(−z2)H1(z)H1(z2) H1(Wz)H1(−z2) H1(−z)H1(z2) H1(−Wz)H1(−z2)
⎤⎥⎥⎦
Gm(z) =
⎡⎢⎢⎣
G0(z)G0(z2) G0(z)G1(z2) G1(z)G0(z2) G1(z)G1(z2)G0(Wz)G0(−z2) G0(Wz)G1(−z2) G1(Wz)G0(−z2) G1(Wz)G1(−z2)G0(−z)G0(z2) G0(−z)G1(z2) G1(−z)G0(z2) G1(−z)G1(z2)
G0(−Wz)G0(−z2) G0(−Wz)G1(−z2) G1(−Wz)G0(−z2) G1(−Wz)G1(−z2)
⎤⎥⎥⎦
The biorthogonality relations can be derived from the matrix equation HmGm = 4I; the derivationis explained for the N -channel case in the solution to problem 3.21, and is thus not repeated here.The relations are given by
〈gp[k], hq[4n − k]〉 = δ[p − q]δ[n]
where p ∈ {a, b, c, d} and q ∈ {a, b, c, d}. The condition in simply that the analysis filter and thetime-reversed synthesis filter in the same channel must be orthogonal with respect to shifts by 4;across channels, the analysis filters and time-reversed synthesis filters must be uncorrelated for allshifts by 4. The proof that biorthogonality holds for the tree-structured filterbank can be carriedout by extracting a single equation from the 4× 4 matrix equation HmGm = 4I and then showingthat it follows from relations (1) through (8). For instance, consider the specific relation
〈ga[k], ha[4n− k]〉 = δ[n]
that results from multiplying the first row of Hm and the first column of Gm. In the Z-domain,this corresponds to
H0(z)H0(z2)G0(z)G0(z2) + H0(Wz)H0(−z2)G0(Wz)G0(−z2) +H0(−z)H0(z2)G0(−z)G0(z2) + H0(Wz)H0(−z2)G0(−Wz)G0(−z2) = 4
⇒ G0(z2)H0(z2) [G0(z)H0(z) + G0(−z)H0(−z)] +G0(−z2)H0(−z2) [G0(Wz)H0(Wz) + G0(−Wz)H0(−Wz)] = 4
The bracketed expressions can be simplified using equation (5), resulting in
G0(z2)H0(z2) + G0(−z2)H0(−z2) = 2
34
which can be seen to be true by replacing z by z2 in equation (5). The other biorthogonalityrelations can be proven in a similar manner; the algebraic details are omitted in favor of outlininga more elegant proof.
Consider the matrix
T =
⎡⎢⎢⎣
1 0 0 00 0 1 00 1 0 00 0 0 1
⎤⎥⎥⎦
Left multiplication by T interchanges the second and third rows; right multiplication by T inter-changes the second and third columns. Note that T2 = I (and also that T = T−1 = TT ). To provebiorthogonality for the full tree, we just need to prove that HmGm = 4I as explained earlier. Thisis equivalent to proving that (THmT)(TGmT) = 4I. We can write the matrices in parenthesesin block form as
Hm′ = THmT =[H0(z2)Hm2(z) H0(−z2)Hm2(Wz)H1(z2)Hm2(z) H1(−z2)Hm2(Wz)
]
Gm′ = TGmT =[
G0(z2)Gm2(z) G1(z2)Gm2(z)G0(−z2)Gm2(Wz) G1(−z2)Gm2(Wz)
]where Gm2 and Hm2 denote the 2× 2 modulation matrices. Then,
Hm′Gm′ =[G0(z2)H0(z2)Gm2(z)Hm2(z) + G0(−z2)H0(−z2)Gm2(Wz)Hm2(Wz)G0(z2)H1(z2)Gm2(z)Hm2(z) + G0(−z2)H1(−z2)Gm2(Wz)Hm2(Wz)
G1(z2)H0(z2)Gm2(z)Hm2(z) + G1(−z2)H0(−z2)Gm2(Wz)Hm2(Wz)G1(z2)H1(z2)Gm2(z)Hm2(z) + G1(−z2)H1(−z2)Gm2(Wz)Hm2(Wz)
]
=[
2{G0(z2)H0(z2) +G0(−z2)H0(−z2)
}I2 2
{G1(z2)H0(z2) +G1(−z2)H0(−z2)
}I2
2{G0(z2)H1(z2) +G0(−z2)H1(−z2)
}I2 2
{G1(z2)H1(z2) +G1(−z2)H1(−z2)
}I2
]
=[
4I 00 4I
]= 4I
where the various identity matrices are of the appropriate size. This completes the proof thatbiorthogonality holds in a full tree of depth 2 when the base filters are biorthogonal. This argumentcan of course be iterated to arbitrary depths.
3.20 Closed-form formula for number of wavelet packetsLet MJ−1 be the number of orthonormal bases in a binary tree of depth J − 1. To generate abinary tree of depth J , we take a binary filter bank (of depth 1) to be the root, and attach to itstwo leaves any of the MJ−1 possible orthonormal bases (from the tree of depth J − 1). There areM2J−1 possible combinations. With the addition of the nil-tree tree (i.e. with no split at all), the
number of orthonormal bases in a binary tree of depth J is
MJ = M2J−1 + 1 .
To start the recursion, M1 = 2 (corresponding to the 1-level binary tree and the identity tree).
35
3.21 Biorthogonality in N-channel perfect reconstruction filter banksIn the modulation domain, the operation of an N-channel analysis-synthesis filterbank can beexpressed as ⎡
⎢⎢⎢⎣X(z)X(Wz)
...X(WN−1z)
⎤⎥⎥⎥⎦ =
1N
GmHm
⎡⎢⎢⎢⎣
X(z)X(Wz)
...X(WN−1z)
⎤⎥⎥⎥⎦
where W = ej2π/N and the matrices Gm and Hm are given by
Gm =
⎡⎢⎢⎢⎣
G0(z) G1(z) G2(z) · · · GN−1(z)G0(Wz) G1(Wz) G2(Wz) · · · GN−1(Wz)
......
......
...G0(WN−1z) G1(WN−1z) G2(WN−1z) · · · GN−1(WN−1z)
⎤⎥⎥⎥⎦
Hm =
⎡⎢⎢⎢⎣
H0(z) H1(z) H2(z) · · · HN−1(z)H0(Wz) H1(Wz) H2(Wz) · · · HN−1(Wz)
......
......
...H0(WN−1z) H1(WN−1z) H2(WN−1z) · · · HN−1(WN−1z)
⎤⎥⎥⎥⎦T
Note the transpose in this definition of Hm. This is an N -channel formulation of equation 3.2.14that has been extended to include the modulated versions of X(z). The perfect reconstructioncondition is
GmHm = NI ⇐⇒ HmGm = NI
where I is the N × N identity matrix. The order of the matrices can be reversed in this waybecause the left inverse and the right inverse are equivalent for square matrices (Gm and Hm areboth N ×N). Consider the inner product of the a-th row of Hm and the b-th column of Gm, wherethe indices a and b range from 0 to N − 1:
Ha(z)Gb(z) + Ha(Wz)Gb(Wz) + . . . + Ha(WN−1z)Gb(WN−1z) = NI(a, b)
where I(a, b) denotes the element in row a and column b of the identity matrix, namely δ[a − b].This can be rewritten as a time-domain convolution using the modulation property:∑
k
gb[k]ha[n− k](1 +W n +W 2n + · · ·+W (N−1)n) = Nδ[a− b]δ[n]
=⇒∑k
gb[k]ha[Nn− k] = δ[a − b]δ[n] =⇒ 〈gb[k], ha[Nn− k]〉 = δ[a− b]δ[n]
In terms of basis vectors, this is equivalent to the biorthogonality conditions
〈ϕb[k], ϕNn+a[k]〉 = δ[a − b]δ[n]
which is simply an N -channel generalization of the conditions given at the end of the 2-channelmodulation domain analysis section in the text (a subsection of 3.2.1).
36
3.22 Relationship between polyphase and modulation matricesThe expression relating Gp(z) and Gm(z) is similar to equations (3.4.9) and (3.2.29):
Gp(zN ) =
⎡⎢⎢⎢⎣
G00(zN ) G10(zN ) · · · G(N−1)0(zN )G01(zN ) G11(zN ) · · · G(N−1)1(zN )
......
. . ....
G0(N−1)(zN ) G1(N−1)(zN ) · · · G(N−1)(N−1)(zN )
⎤⎥⎥⎥⎦
=1N
⎡⎢⎢⎢⎣
1 0 · · · 00 z · · · 0...
.... . .
...0 0 · · · zN−1
⎤⎥⎥⎥⎦F
⎡⎢⎢⎢⎣
G0(z) G1(z) · · · GN−1(z)G0(WNz) G1(WNz) · · · GN−1(WNz)
......
. . ....
G0(WN−1N z) G1(WN−1
N z) · · · GN−1(WN−1N z)
⎤⎥⎥⎥⎦
︸ ︷︷ ︸Gm(z)
The expression relating Hp(z) and Hm(z) is similar to equation (3.2.28):
Hp(zN ) =
⎡⎢⎢⎢⎣
H00(zN ) H01(zN ) · · · H0(N−1)(zN )H10(zN ) H11(zN ) · · · H1(N−1)(zN )
......
. . ....
H(N−1)0(zN ) H(N−1)1(zN ) · · · H(N−1)(N−1)(zN )
⎤⎥⎥⎥⎦
=1N
⎡⎢⎢⎢⎣
H0(z) H0(WNz) · · · H0(WN−1N z)
H1(z) H1(WNz) · · · H1(WN−1N z)
......
. . ....
HN−1(z) HN−1(WNz) · · · HN−1(WN−1N z)
⎤⎥⎥⎥⎦
︸ ︷︷ ︸Hm(z)
F
⎡⎢⎢⎢⎣
1 0 · · · 00 z−1 · · · 0...
.... . .
...0 0 · · · z−(N−1)
⎤⎥⎥⎥⎦
In both equations, F is the N ×N Fourier transform matrix, with Fkl = W klN = e−j(2π/N)kl.
3.23 Modulation matrices(a) The modulation matrix Hm(z) is given by
Hm(z) =
⎡⎣ H0(z) H0(Wz) H0(W 2z)H1(z) H1(Wz) H1(W 2z)H2(z) H2(Wz) H2(W 2z)
⎤⎦ =
⎡⎣ H(z) H(Wz) H(W 2z)H(Wz) H(W 2z) H(z)H(W 2z) H(z) H(Wz)
⎤⎦
where W = e−j2π3 and Hi(z) = H(W iz). From Theorem 3.17, we have
Hp(z) = F
⎡⎣ Hp0(z) 0 0
0 Hp1(z) 00 0 Hp3(z)
⎤⎦ where F =
⎡⎣ 1 1 1
1 W W 2
1 W 2 W 4
⎤⎦
and from equation 3.4.9 and problem 3.22,
Hm(z) = Hp(z3) diag(1, z, z2) F
Substituting for Hp(z3) gives
Hm(z) = F diag(Hp0(z3), zHp1(z3), z2Hp2(z3)
)F
37
which yields the desired diagonalization
diag(Hp0(z3), zHp1(z3), z2Hp2(z3)
)= F−1 Hm(z) F−1
(b) Using the factorization from (a),
det (Hm(z)) = det(F diag
(Hp0(z3), zHp1(z3), z2Hp2(z3)
)F)
= [det(F )]2 det[diag
(Hp0(z3), zHp1(z3), z2Hp2(z3)
)]Computing det(F ) and simplifying gives
det (Hm(z)) = −27 z3 Hp0(z3) Hp1(z3) Hp2(z3)
(c) Sincedet (Hp(z)) = det(F ) Hp0(z) Hp1(z) Hp2(z)
we can use the result of (b) to write
det (Hp(z)) = z3 det(F ) det(Hp(z3)
)Note: the special form of Hp(z) from Theorem 3.17 was used in (a) to derive the diagonalizationof Hm(z).
3.24 Cosine modulated filter banks
(a) The filters are given by hk[n] =1√N
cos(
2k + 14N
(2n−N + 1)π)
Note that the window
function hpr[n] is rectangular and thus A0 = B0 and A1 = B1. The matrices A0 and A1 are givenby
A0 =
⎡⎢⎣ h0[2N − 1] · · · h0[N ]
......
hN−1[2N − 1] · · · hN−1[N ]
⎤⎥⎦ and A1 =
⎡⎢⎣ h0[N − 1] · · · h0[0]
......
hN−1[N − 1] · · · hN−1[0]
⎤⎥⎦
We first prove the second part of 3.4.6, namely A0AT1 = A1A
T0 = 0. The right tail (A1) is
symmetric:
hk[N − 1− n] =1√N
cos(
2k + 14N
(N − 2n− 1)π)
= hk[n]
The left tail (A0) is anti-symmetric:
hk[2N − 1− n] =1√N
cos(
2k + 14N
(−2n+ 3N − 1)π)
hk[N + n] =1√N
cos(
2k + 14N
(2n +N + 1)π)
because2k + 1
4N(−2n+ 3N − 1)π +
2k + 14N
(2n +N + 1)π = (2k + 1)π
and for any two angles φ1, φ2 that sum to an odd multiple of π, we have
φ1 + φ2 = (2k + 1)π =⇒ φ1 = (2k + 1)π − φ2 =⇒ cosφ1 = cos [(2k + 1)π − φ2] = − cosφ2
38
Because the tails are symmetric/antisymmetric, their inner product is 0; hence we have A0AT1 =
A1AT0 = 0. We now prove the first part of 3.4.6, namely that AT0A1 = AT1A0 = 0. The inner
product of the n-th row of AT0 and the m-th column of A1 is
1N
N−1∑k=0
cos(
2k + 14N
(2n−N + 1)π)
cos(
2k + 14N
(2m−N + 1)π)
Using the identity cosX cos Y =12
cos(X + Y ) +12
cos(X − Y ) the inner product can be rewrittenas
12N
N−1∑k=0
cos(
2k + 12N
(n+m−N + 1)π)
+ cos(
2k + 12N
(n−m)π)
We can simplify this using the following derivation
N−1∑k=0
cos(kx
π
N+ x
π
2N
)
=12
N−1∑k=0
ejkxπN ejx
π2N + e−jkx
πN e−jx
π2N
=12ejx
π2N
(1− ejxπ
1− ejx πN
)+
12e−jx
π2N
(1− e−jxπ
1− e−jx πN
)
=(
sin xπ2
sin xπ2N
)cos
(xπ2
)=
sin(xπ)2 sin xπ
2N
We can thus write the inner product as
12N
{sin((n−m)π)
2 sin (n−m)π2N
+sin((n+m−N + 1)π)
2 sin (n+m−N+1)π2N
}
In the inner product, n ∈ [N, 2N − 1] and m ∈ [0, N − 1], so the respective arguments of the sincfunctions are in the ranges:
n−m ∈ [1, 2N − 1] n+m−N + 1 ∈ [1, 2N − 1]
For these argument values, the sinc is always zero, so we have proven that AT0A1 = AT1A0 = 0. Wenow prove the second part of 3.4.5, that A0A
T0 + A1A
T1 = I. Note that the [k, j]-th constituent
equation of this matrix formula corresponds to the inner product of the length 2N filters hk[n] andhj [n]; proving the formula is equivalent to proving that 〈hk[n], hj [n]〉 = δ[k− j]. The inner productis given by
1N
2N−1∑k=0
cos[2k + 1
4N(2n −N + 1)π
]cos
[2j + 14N
(2n −N + 1)π]
=1
2N
2N−1∑k=0
cos[(k + j + 1)
(n− N − 1
2
)π
N
]+ cos
[(k − j)
(n− N − 1
2
)π
N
]= δ[k − j]
39
where the final step holds because the sum is taken over an integral number of periods of thecosines, so we have shown that A0A
T0 + A1A
T1 = I. We prove the first part of 3.4.5 in (b) since it
follows naturally from the results derived there.
(b) Consider BT0 B0 = AT0A0; the derivation used to find AT0 A1 can be applied here. Specifically,
we have
AT0 A0[n,m] =1
2N
{sin((n−m)π)
2 sin (n−m)π2N
+sin((n+m−N + 1)π)
2 sin (n+m−N+1)π2N
}
with the arguments now in the ranges n ∈ [N, 2N − 1] and likewise m ∈ [N, 2N − 1] so
n−m ∈ [0, N − 1] n+m−N + 1 ∈ [N + 1, 3N − 1]
For n−m = 0, the first sinc takes on the value N by L’Hopital’s rule; otherwise it is zero-valued.For n+m−N + 1 = 2N , the second sinc takes on the value −2N ; for other values it is zero. Sincen−m = 0 corresponds to the matrix diagonal and n+m = 3N−1 corresponds to the anti-diagonal,
we have AT0 A0 =12(I − J). For the product AT1A1, the arguments are in the ranges n ∈ [0, N − 1]
and m ∈ [0, N − 1], so
n−m ∈ [0, N − 1] n+m−N + 1 ∈ [−N + 1, N − 1]
The sinc functions in the sum are respectively non-zero for n − m = 0 and n + m − N + 1 = 0,which correspond to the diagonal n = m and the anti-diagonal n + m = N − 1. In both cases
the sinc functions take on the value 2N , so the result of the matrix product is AT1A1 =12(I + J).
Combining these results gives the first part of 3.4.5 for problem (a), namely
AT0 A0 +AT1A1 =12(I − J + I + J) = I
3.25 Orthogonal pyramid(a) For this problem we refer to figure 3.17; we denote the “original signal” (space V0) as x[n],the “coarse version” as y0[n], the “difference signal” (space W1) as y1[n], and the signal in spaceV1 as y′0[n]. In synthesis, the coarse signal y0[n] is upsampled and filtered with H0 to yield thesignal y′′0 [n]. The synthesis output y[n] is the sum of y′′0 [n] and y1[n]. In time-domain, perfectreconstruction is indicated by:
y1[n] = x[n]− y′0[n]y[n] = y′′0 [n] + y1[n]y′′0 [n] = y′0[n]
⎫⎬⎭ =⇒ y[n] = x[n]
Analogously, in the Z-transform domain perfect reconstruction is given by:
Y1(z) = X(z) − Y ′0(z)
Y (z) = Y ′′0 (z) + Y1(z)
Y ′′0 (z) = Y ′
0(z)
⎫⎬⎭ =⇒ Y (z) = X(z)
40
(b) Let H0 and H1 denote the filter matrices
H0 =1√2
⎡⎢⎢⎢⎢⎢⎢⎣
. . .1 1 0
1 10 1 1
. . .
⎤⎥⎥⎥⎥⎥⎥⎦ and H1 =
1√2
⎡⎢⎢⎢⎢⎢⎢⎣
. . .1 −1 0
1 −10 1 −1
. . .
⎤⎥⎥⎥⎥⎥⎥⎦
and let h0[n] =1√2[1 1] and h1[n] =
1√2[1 −1]. The following matrix relations hold:
H1HT0 = 0 and HT
0 H0 +HT1 H1 = I. Also,
y0 = H0xy′0 = HT
0 H0xy1 = x− y′0
⎫⎬⎭ =⇒ y1 = (I −HT
0 H0)x
=⇒ y′1 = H1y1 = (H1 −H1HT0 H0)x =⇒ y′1 = H1x
Reconstruction is achieved by
y = HT0 y0 +HT
1 y′1 = HT
0 H0x+HT1 H1x = x
(c) We showed in (b) that y0 = H0x and y′1 = H1x, which directly indicates the equivalence to atwo-channel filter bank with filters h0[n] and h1[n] (with synthesis filters being the time-reversedversion of h0[n] and h1[n]).
(d) If we define H and G to be the infinite matrices
H =
⎡⎣ h(L− 1) h(L− 2) h(L− 3) h(L− 4) · · · h(0)
h(L− 1) h(L− 2) · · · h(2) h(1) h(0)h(L− 1) · · · · · · h(2) h(1) h(0)
⎤⎦
and
G =
⎡⎣ g(L− 1) g(L − 2) g(L− 3) g(L− 4) · · · g(0)
g(L− 1) g(L− 2) · · · g(2) g(1) g(0)g(L− 1) · · · · · · g(2) g(1) g(0)
⎤⎦
By orthogonality relations, we have
HHT = I GGT = I GHT = 0 HGT = 0
We thus have y0 = Hx and
y′1 = G(I −HTH)x =⇒ y′1 = Gx − GHTHx =⇒ y′1 = Gx
Analysis in the Z-transform domain yields
Y0(z) =12
[H(z1/2)X(z1/2) +H(−z1/2)X(−z1/2)
]Y ′
1(z) =12
[G(z1/2)X(z1/2) +G(−z1/2)X(−z1/2)
]41
where G(z) = −z−(L−1)H(−z−1). Upsampling and filtering gives the reconstruction
Xr(z) =12
[Hr(z)H(z) − z−(L−1)Gr(z)H(−z−1)
]X(z)
+12
[Hr(z)H(−z) + z−(L−1)Gr(z)H(z−1)
]X(−z)
where Gr(z) and Hr(z) are the synthesis filters. If we choose Gr(z) = −H(−z)and Hr(z) = z−(L−1)H(z−1) we obtain
Xr(z) = z−(L−1)[H(z)H(z−1) +H(−z)H(−z−1)
]X(z)
Since H(z)H(z−1) corresponds to the autocorrelation of a filter which is orthogonal to its eventranslates (and has unit norm), it contains only odd powers of z except for a 0-th order term ofvalue 1. (Consider, for example, problem 2.16c.) We therefore have perfect reconstruction:
H(z)H(z−1) +H(−z)H(−z−1) = 2 =⇒ Xr(z) = 2z−(L−1)X(z)
3.26 Parseval’s formula for tight framesIn the case of tight frame, H0(z) = G0(z−1) and H1(z) = G1(z−1), and thus the analysis filterssatisfy
H0(z)H0(z−1) +H1(z)H1(z−1) = 2 .
‖y0‖2 + ‖y1‖2 =12π
∫ π
−π|H0(ejω)X(ejω)|2dω +
12π
∫ π
−π|H1(ejω)X(ejω)|2dω
=12π
∫ π
−π(|H0(ejω)|2 + |H1(ejω)|2)︸ ︷︷ ︸
2
|X(ejω)|2dω
= 2(12π
∫ π
−π|X(ejω)|2dω)
= 2 ‖x‖2
3.27 Quincunx orthonormal bases for l2(Z2)
To prove that the impulse responses with shifts on a quincunx lattice form an orthonormal basis,we need to show that they form an orthonormal system and that the system is complete. We dothese in turn. Equation 3.6.4 states
H0(z1, z2)H0(z−11 , z−1
2 ) + H0(−z1,−z2)H0(−z−11 ,−z−1
2 ) = 2
We have the following relations for two-dimensional Z-transforms:
H0(z1, z2) ⇐⇒ h[n1, n2]H0(−z1,−z2) ⇐⇒ (−1)n1(−1)n2h[n1, n2]H0(z−1
1 , z−12 ) ⇐⇒ h[−n1,−n2]
H0(−z−11 ,−z−1
2 ) ⇐⇒ (−1)n1(−1)n2h[−n1,−n2]
We can rewrite 3.6.4 in the time-domain as∑m1
∑m2
h0[m1,m2]h0[m1 − n1,m2 − n2] [1 + (−1)n1(−1)n2 ] = 2δ[n1, n2]
42
∑m1
∑m2
h0[m1,m2]h0[m1 − n1,m2 − (2k − n1)] = δ[n1, (2k − n1)]
=⇒ 〈h0[m1,m2], h0[m1 − n1,m2 − (2k − n1)]〉 = δ[n1, (2k − n1)]
Equation 3.6.5 statesH1(z1, z2) = −z−1
1 H0(−z−11 ,−z−1
2 )
from which we can easily derive the relations
H0(z1, z2) = z−11 H1(−z−1
1 ,−z−12 )
H0(z−11 , z−1
2 ) = z1H1(−z1,−z2)H0(−z1,−z2) = −z−1
1 H1(z−11 , z−1
2 )H0(−z−1
1 ,−z−12 ) = −z1H1(z1, z2)
Substituting into 3.6.4 yields
H1(z1, z2)H1(z−11 , z−1
2 ) + H1(−z1,−z2)H1(−z−11 ,−z−1
2 ) = 2
which verifies that
〈h1[m1,m2], h1[m1 − n1,m2 − (2k − n1)]〉 = δ[n1, (2k − n1)]
So we have proven that h0[n1, n2] and h1[n1, n2] are orthogonal with respect to their own shifts ona quincunx lattice; to complete the proof that these constitute an orthonormal system, we requirethat
〈h0[m1,m2], h1[m1 − n1,m2 − (2k − n1)]〉 = 0
which can be rewritten in the Z domain as
H0(z1, z2)H1(z−11 , z−1
2 ) + H0(−z1,−z2)H1(−z−11 ,−z−1
2 ) = 0
Using the relations between the lowpass and highpass filter, we have
H0(z1, z2) {−z1H0(−z1,−z2)} + H0(−z1,−z2) {z1H0(z1, z2)} = 0
which completes the proof. This orthonormal system is a basis for l2(Z2) if
〈hi, y〉 = 0 for i = 0, 1 =⇒ y = 0
In the time-domain, the zero inner product condition is equivalent to (assuming real filters andsignals) ∑
m1
∑m2
y[m1,m2]hi[m1,m2 − (2k − n1)] = 0
which can be expressed in the Z domain as (see the above formulation)
Y (z1, z2)H0(z−11 , z−1
2 ) + Y (−z1,−z2)H0(−z−11 ,−z−1
2 ) = 0Y (z1, z2)H1(z−1
1 , z−12 ) + Y (−z1,−z2)H1(−z−1
1 ,−z−12 ) = 0
Using the lowpass-highpass relations, we have
Y (z1, z2)H0(z−11 , z−1
2 ) + Y (−z1,−z2)H0(−z−11 ,−z−1
2 ) = 0Y (z1, z2)H0(−z1,−z2) − Y (−z1,−z2)H0(z1, z2) = 0
43
Multiplying the top equation byH0(z1, z2) and the bottom equation byH0(−z−11 ,−z−1
2 ) and addingthe resulting equations gives
Y (z1, z2)[H0(z1, z2)H0(z−1
1 , z−12 ) + H0(−z1,−z2)H0(−z−1
1 ,−z−12 )
]= 0
=⇒ Y (z1, z2)[2] = 0 =⇒ Y (z1, z2) = 0 =⇒ y[n1, n2] = 0
which completes the proof that the impulse responses with shifts on a quincunx lattice are anorthonormal basis for l2(Z2).
3.28 Linear phase diamond-shaped quincunx filters(a) Note that the origin is located at the center of the leftmost column for both h0[n1, n2] andh1[n1, n2].
H0(z1, z2) = (1 + bz−11 z−1
2 + bz−11 z2 + z−2
1 )︸ ︷︷ ︸H00(z1,z2)
+ z−11 a︸︷︷︸
H01(z1,z2)
H1(z1, z2) =
(bc
a(1 + z−4
1 ) + (b+c
a)(z−1
1 z−12 + z−1
1 z2 + z−31 z−1
2 + z−31 z2) + (z−2
1 z−22 + z2
1z22) + dz−2
1 )︸ ︷︷ ︸H10(z1,z2)
+ z−11 (c(1 + z−2
1 ) + a(z−11 z−1
2 + z1z2))︸ ︷︷ ︸H11(z1,z2)
For perfect FIR reconstruction to be possible, the determinant of the polyphase matrix has to bea monomial.
det Hp = H00(z)H11(z)−H01(z)H10(z)= (c− bc)(1 + z−4
1 ) + (−a+ ab)(z−21 z−2
2 + z−21 z2
2)+(a− ab− c+ bc)(z−3
1 z−12 + z−1
1 z−12 + z−3
1 z2 + z−11 z2)
+(2ab+ 2c− ad)z−21
For det Hp to be a monomial, the first 3 terms must be zero, while the last term cannot be zero:
c− bc = 0−a+ ab = 0
a− ab− c+ bc = 02ab+ 2c− ad �= 0
The first two constraints imply b = 1, and it is clear that a, c, d can then be anything as long asthey satisfy 2a+ 2c− ad �= 0. Hence, perfect FIR reconstruction is possible.
(b) If circular symmetry is also imposed, then b = 1 and c = a. For a = −4 and d = −28, we have
h0[n1, n2] =
⎛⎝ 0 1 0
1 −4 10 1 0
⎞⎠ ,
44
h1[n1, n2] =
⎛⎜⎜⎜⎜⎝
0 0 1 0 00 2 −4 2 01 −4 −28 −4 10 2 −4 2 00 0 1 0 0
⎞⎟⎟⎟⎟⎠ .
This set of filters yields perfect FIR reconstruction since det Hp = −128z−21 .
The filter h0[n1, n2] is a highpass filter because it takes the difference of a sample with its 4neighbors; the filter h1[n1, n2] is a lowpass filter because it resembles the shape of a sinc (lowpass)filter.
45
Chapter 4
4.1 Wavelet series expansion(a) The wavelet series expansion of a function f(t) is given by
f(t) =∑m,n
am,nψm,n(t)
where ψm,n(t) = 2−m2 ψ(2−mt− n) and am,n = 〈f(t), ψm,n(t)〉.
For the Haar expansion of f(t) = ϕ(t) (the scaling function), the series coefficients are:
am,n =∫ 1
0ψm,n(t) dt =
{2−
m2 m ≥ 1 n = 0
0 otherwise
(b)∑m
∑n
|〈ψm,n(t), f(t)〉|2 =∑m
∑n
|am,n|2 =∞∑m=1
2−m = 1
(c) All coefficients of some scale m are equal to zero if and only if 2−i = k 2m for some integerk. Thus all coefficients are equal to zero for scales m such that 2−i−m = k for some integer k. If−i−m < 0, the power 2−i−m will not be an integer. Therefore, not all of the coefficients are zerofor scales m > −i.
(d) All coefficients of some scale m are equal to zero if 1√2
= k 2m for some integer k. Since thereis no integer m for which this holds, there is no scale at which all of the coefficients are equal tozero.
4.2 Implications of the two-scale equation for an orthonormal basis(a) First note that ‖ φ(t) ‖2= 1 because {φ(t− n)} is an orthonormal set.
‖ φ(t) ‖2 = 〈φ(t), φ(t)〉= 〈
√2∑n
g0[n]φ(2t− n),√
2∑m
g0[m]φ(2t −m)〉
=∑n
∑m
g∗0 [n]g0[m] 〈√
2φ(2t− n),√
2φ(2t−m)〉︸ ︷︷ ︸δ[m−n]
=∑n
|g0[n]|2
= ‖ g0[n] ‖2
= 1
Hence ‖ g0[n] ‖= 1.
(b) Since {φ(t−n)} is an orthonormal basis for V0, {√
2φ(2t−n)} is an orthonormal basis for V−1.
√2〈φ(2t− n), φ(t)〉 = 〈
√2φ(2t− n),
√2
∞∑m=−∞
g0[m]φ(2t−m)〉
=∞∑
m=−∞g0[m] 〈
√2φ(2t− n),
√2φ(2t−m)〉︸ ︷︷ ︸
δ[m−n]
= g0[n]
46
The last equality comes from the fact that {√
2φ(2t− n)} is an orthonormal set.
4.3 More implications of the two-scale equation(a) Taking the Fourier transform of the two scale equation, we get (see equation (4.2.9))
Φ(ω) =1√2G0(ej
ω2 )Φ(
ω
2) .
Evaluating at ω = 0,
=⇒ Φ(0) = 1√2G0(1)Φ(0)
=⇒ G0(1) =∑
n g0[n] =√
2 .
(b) From equation (4.2.10),
|G(ejω)|2 + |G0(ej(ω+π))|2 = 2 .
Evaluating at ω = 0,|G(1)|2 + |G0(−1)|2 = 2 ,
implying that G0(−1) = 0. Since
G0(−1) =∑n
(−1)ng0[n] =∑n
g0[2n]−∑n
g0[2n + 1] ,
this proves that ∑n
g0[2n] =∑n
g0[2n+ 1] .
4.4 Meyer’s waveletIt is sufficient to prove that
|Ψ(2ω)|2 + |Ψ(ω)|2 = 1
for 2π3 ≤ ω ≤
4π3 . Substituting from (4.3.5) it follows that we have to prove
|Φ(ω − 2π)|2 + |Φ(ω)|2 = 1.
The proof of this last equality is already given following (4.3.3).
4.5 Meyer’s wavelet(a) The scaling function and wavelet for this choice of θ differ in the transition from 0 to 1 andvice versa. With the choice of (4.3.2) this transition is differentiable, making the scaling functionand wavelet differentiable. With the choice of θ as in this exercise, the scaling function and waveletare not differentiable in 2πk
3 for |k| = 1, 2 and |k| = 1, 2, 3, respectively. This is easily checked byconsidering the following explicit formulas for the scaling function and wavelet:
Φ(ω) =
⎧⎪⎪⎨⎪⎪⎩
0, |ω| ≥ 4π3
1, |ω| ≤ 2π3√
4π3−|ω|2π3
, 2π3 ≤ |ω| ≤
4π3
47
Ψ(ω) =
⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩
0, |ω| ≥ 8π3
0, |ω| ≤ 2π3√
8π3−|ω|4π3
, 4π3 ≤ |ω| ≤
8π3√
|ω|− 2π3
2π3
, 2π3 ≤ |ω| ≤
4π3
(b) As mentioned above, the scaling function and wavelet are only continuous in the frequencydomain. We may therefore only expect them to be of order O(1
t ) in the spatial domain. In the caseof θ(x) given by (4.3.2), we may expect an order of O( 1
t2).
(c) Using the following simple (though slow) Mathematica program, the plots of the scaling functionand wavelet are generated.
ClearAll;
theta[x_] := 0 /; (x<=0);theta[x_] := 1 /; (x >= 1);theta[x_] := x /; ((0 < x) && (x < 1));
(* The scaling function in the frequency domain *)
Phi[w_] := Sqrt[theta[2 + ((3 w) / (2 N[Pi]))]]/; (w <= 0);Phi[w_] := Sqrt[theta[2 - ((3 w) / (2 N[Pi]))]]/; (w > 0);Plot[Phi[w],{w,-10,10},AxesLabel -> {"Frequency","Magnitude Response"}];
(* The scaling function in the spatial domain *)
A = 2 Pi/3;B = 2 Pi;DL = Table[n/5,{n,-5*5,5*5}];
I1[t_] := (2/B)NIntegrate[Sqrt[((2 A-w)/A)] Cos[w t],{w,A,2 A}];I2[t_] := (2/B)NIntegrate[Cos[w t],{w,0,A}];phi[t_] := I1[t]+I2[t];PD = MapThread[List,{DL,Map[phi,DL]}];ListPlot[PD,PlotJoined->True,PlotRange ->{-0.2,1.2},AxesLabel -> {"Time","Amplitude"}];
(* The wavelet in the frequency domain *)
Psi[w_] = Exp[-I w/2] (Phi[w + 2 Pi] + Phi[w - 2 Pi]) Phi[w/2];Plot[Abs[Psi[w]],{w,-10,10},AxesLabel -> {"Frequency","Magnitude Response"}];
(* The wavelet in the spatial domain *)
I3[t_] := (2/B)NIntegrate[Cos[w/2 t] Sqrt[(4 A - w)/(2 A)],{w,2A,4A}];I4[t_] := (2/B)NIntegrate[Cos[w/2 t] Sqrt[(w - A)/A],{w,A,2A}];psi[t_] := I3[t]+I4[t];PD = MapThread[List,{DL,Map[psi,DL]}];ListPlot[PD,PlotJoined->True,PlotRange ->{-1.0,1.4}, AxesLabel -> {"Time","Amplitude"}];
48
4.6 B-splines(a) Since β(N)(t) is an even function,
β(2N+1)(t) = β(N)(t) ∗ β(N)(t) = β(N)(t) ∗ β(N)(−t) .
Thus, taking the Fourier transform yields
B(2N+1)(ω) = (B(N)(ω))2 = |B(N)(ω)|2 .
In addition, given b(2N+1)[n] = β(2N+1)(t)|t=n, sampling β(2N+1)(t) at t = kT, k ∈ Z yields
B(2N+1)(ω) =1T
∞∑k=−∞
B(2N+1)(ω + k2πT
)
=∞∑
k=−∞B(2N+1)(ω + 2πk)
=∞∑
k=−∞|B(N)(ω + 2πk)|2
where the second equality comes from letting T = 1.
(b) Since β(2N+1)(t) = β(N)(t) ∗ β(N)(t) = β(N)(t) ∗ β(N)(−t),
b(2N+1)[n] = β(2N+1)(t)|t=n= β(N)(t) ∗ β(N)(−t)|t=n
=∫ ∞
−∞β(N)(τ) ∗ β(N)(τ − t)dτ |t=n
=∫ ∞
−∞β(N)(τ) ∗ β(N)(τ − n)dτ .
(c) Using the equation in part (b), we can calculate b(2N+1)[n] for N = 1, 3. β(1)(t) and β(2)(t) areshown below.
t
t(1)β ( )
1
1−1 t
tβ ( )(2)
3/2−3/2
3/4
The expressions for β(1)(t) and β(2)(t) are
β(1)(t) ={
1− |t| , 0 ≤ |t| ≤ 10 , otherwise ,
β(2)(t) =
⎧⎨⎩
34 − t2 , 0 ≤ |t| ≤ 1
298 −
32 |t|+
12t
2 , 12 ≤ |t| ≤
32
0 , otherwise.
49
Using the expression in (b), it is straight forward to show that
b(3)[0] =23, b(3)[1] = b(3)[−1] =
16,
b(5)[0] =1120
, b(5)[1] = b(5)[−1] =1360
, b(5)[2] = b(5)[−2] =1
120.
4.7 Battle-Lemarie waveletsAccording to Section 4.3, it suffices to calculate the functions B(2)(ω), B(5)(ω) and M(ω). However,the first two of these are already explicitly given in (4.3.26) and (4.3.27). The only remaining thingto do is calculate M(ω).
From the two-scale equation
I(0)(t) = I(0)(t− 1/2) + I(0)(2t+ 1/2),
we derive thatI(0)(ω) = cos(ω/4)I(0)(ω/2).
Therefore
B(2)(ω) = e−jω/2I(2)(ω)= e−jω/2 cos3(ω/4)I(2)(ω/2)= e−jω/2 cos3(ω/4)ejω/4B(2)(ω/2)= e−jω/4 cos3(ω/4)B(2)(ω/2)= M(ω/2)B(2)(ω/2).
So we find M(ω) = e−jω/2 cos3(ω/2).As in (4.3.25) we can write
Ψ(ω) =sin3(ω/4)
(ω/4)Q(ω),
where Q is 4π-periodic. In particular, ψ(t) is piecewise quadratic over half-integer intervals.
4.8 Battle-Lemarie waveletsB2N+1(ω) can be factored as follows:
B2N+1(ω) =23
+16ejω +
16e−jω
=2 +√
36
[e−jω + (2−
√3)] [ejω + (2−
√3)]
= R(ejω) R(e−jω)
=⇒ R(ejω) =
√2 +√
36
[ejω + (2−
√3)]
Note that this factorization is not unique. R(ejω) is chosen so that Φ(ω) =G(ω)R(ejω)
is stable. In
this case
G(ω) =(
sin ω2
ω2
)2
50
=⇒ Φ(ω) =(
sin ω2
ω2
)2 1√2+
√3
6
[ejω + (2−
√3)]
From equation 4.3.17, we have G0(ejω) =√
2Φ(2ω)Φ(ω)
, so
G0(ejω) =√
24
(sin2 ω
sin2(ω2
))( ejω + (2−√
3)ej2ω + (2−
√3)
)
From equation 4.3.19, we have
Ψ(ω) = − 1√2e−
jω2 G∗
0
(ej(
ω2+π)
)Φ(ω
2
)
=−e−
jω2
[e−j(
ω2+π) + (2−
√3)]sin2(ω) sin2
(ω4
)4√
2+√
36
[e−jω + (2−
√3)] [e
jω2 + (2−
√3)] (
ω4
)2 sin2(ω2
)We can find the set {αn} as follows:
{αn} = F−1
{1
R(ejω)
}= F−1
⎧⎨⎩ 1√
2+√
36
[ejω + (2−
√3)]⎫⎬⎭
=√
6(2 −√
3)(−2 +√
3)n−1u[n − 1]
where u[n] is the unit step function. We then have
ϕ(t) =∑n∈Z
αnβ(1)(t− n) where β(1) =
{1− |t| |t| ≤ 1
0 otherwise
4.9 Implications of compact supportThe time-domain function whose discrete-time Fourier transform is D(ω) is a sampled version ofthe autocorrelation of g(t) where the samples are taken at integer times. This can be shown asfollows:
d(τ) =∫ ∞
−∞g(t)g(t − τ) dt ⇐⇒ |G(ω)|2
d[n] = d(nT ) =∫ ∞
−∞g(t)g(t − nT ) dt T=1⇐⇒
∑k∈Z|G(ω + 2πk)|2 = D(ω)
We can thus write D(ω) as
D(ω) =∑n
d[n]e−jωn =∑n
{∫ ∞
−∞g(t)g(t − n) dt
}e−jωn
Now, if g(t) has compact support, so does its autocorrelation d(τ). This implies that d(τ) will benonzero only at a finite number of integer times, which tells us that there are a finite number ofnonzero coefficients d[n] in the DTFT D(ω). We can thus conclude that D(ω) is a polynomial in
51
ejω, i.e. a trigonometric polynomial. Also, because d[n] is FIR, it does not have poles on the unitcircle, so D(ω) has a stable spectral factorization.
4.10 Orthogonality relations of Daubechies wavelets(a) It is proven in Proposition 4.4 that 〈ϕ(i)(t), ϕ(i)(t− l)〉 = δ(l) for all i. We can use this to proverelations (b) and (c) as follows:
〈ψ(i+1)(t− n), ψ(i+1)(t− n′)〉
=
⟨√
2∑k∈Z
h1(k)ϕ(i)(2t− 2n− k),√
2∑m∈Z
h1(m)ϕ(i)(2t− 2n′ −m)
⟩
= 2∑k
∑m
h1(k)h1(m)⟨ϕ(i)(2t− 2n− k), ϕ(i)(2t− 2n′ −m)
⟩=
∑k
h1(k)h1(k + 2n− 2n′) =⟨h1(k), h1(k + 2n− 2n′)
⟩= δ(n − n′)
where the scale factor is neglected in the third step. Taking the limit as i −→ ∞ gives the result:⟨ψ(t− n), ψ(t− n′)
⟩= δ(n − n′)
(b) Similarly,
〈ϕ(i+1)(t− n), ψ(i+1)(t− n′)〉
=
⟨√
2∑k∈Z
h0(k)ϕ(i)(2t− 2n− k),√
2∑m∈Z
h1(m)ϕ(i)(2t− 2n′ −m)
⟩
= 2∑k
∑m
h0(k)h1(m)⟨ϕ(i)(2t− 2n − k), ϕ(i)(2t− 2n′ −m)
⟩=
∑k
h0(k)h1(k + 2n− 2n′) = 〈h0(k), h1(k + 2n− 2n′)〉 = 0
Taking the limit as i −→∞ gives the result: 〈ϕ(t − n), ψ(t− n′)〉 = 0
4.11 Infinite products and the Haar scaling function
(a) pk =k∏i=0
abi
= a(Pk
i=0 bi) = a
�1−bk+1
1−b
�=⇒ lim
k→∞pk = a(
11−b) since |b| < 1
(b) Let Φn(ω) =n∏k=1
M0
( ω2k)
where M0(ω) =1√2G0
(ejω
)= e−
jω2 cos
(ω2
).
Substituting for M0(ω) in the Φn(ω) product yields
Φn(ω) =
[n∏k=1
e− jω
2k+1
] [n∏k=1
cos( ω
2k+1
)]
52
Using the identity cos(ω) =sin(2ω)2 sin(ω)
, the second product term can be expanded as
n∏k=1
cos( ω
2k+1
)=
[sin
(ω2
)2 sin
(ω4
)] [ sin(ω4
)2 sin
(ω8
)] [ sin(ω8
)2 sin
(ω16
)] · · ·[
sin(ω2n
)2 sin
(ω
2n+1
)]
=sin
(ω2
)2n sin
(ω
2n+1
)Taking the limit for large n yields
limn→∞
n∏k=1
e− jω
2k+1 = e−jω2 {using the result of part (a)}
limn→∞
sin(ω2
)2n sin
(ω
2n+1
) =sin
(ω2
)2n(
ω2n+1
) =sin
(ω2
)(ω2
)Finally, we have the result
Φ(ω) = limn→∞
Φn(ω) = e−jω2
sin(ω2
)(ω2
)3mm (c) From equation 4.4.15, we have Ψ(ω) = M1
(ω2
)Φ(ω
2
).
For M1(ω) =1√2G1(ω) = je−
jω2 sin
(ω2
)and Φ(ω) from (b), we have
Ψ(ω) = je−jω2
sin2(ω4
)(ω4
)4.12 Iterated filter banks(a) From the fact that |M0(ω)| ≤ 1, ω ∈ [−π, π], and the fact that M0(ω) is 2π-periodic, it followsthat |M0(ω)| ≤ 1 for all ω. Now fix some ω and define the sequence {pk} by pk = M0( ω2k ). Fromthe condition M0(0) = 1 it follows that limk→∞ pk = 1. In order to prove convergence of
∏∞k=1 pk
we may therefore assume without loss of generality that 0 < pk ≤ 1 for all k. This implies that thesequence {Pi} = {
∏ik=1 pk} is a descending sequence of positive numbers, and therefore convergent.
(b) The orthogonality condition implies
|M0(ω)|2 + |M0(ω + π)|2 = 1.
The results follow.
4.13 Maximally flat Daubechies’ filtersBy Bezout’s theorem (which can be proven using Euclid’s algorithm), we can show that P (y)is a polynomial of degree less than N . Bezout’s theorem is stated as follows (see, for example,Daubechies’ “Ten Lectures on Wavelets”):
If p1, p2 are two polynomials, of degree n1, n2, respectively, with no common zeros, thenthere exist unique polynomials q1, q2, of degree n2 − 1, n1 − 1, respectively, so that
p1(x)q1(x) + p2(x)q2(x) = 1 .
53
For our problem, since (1 − y)N and yN have no common zeros, by Bezout’s theorem, thereexist unique polynomials q1, q2, of degree ≤ N − 1, such that
(1− y)Nq1(y) + yNq2(y) = 1 .
Substituting 1− y for y in the above expression yields
(1− y)Nq2(1− y) + yNq1(1− y) = 1 .
By the uniqueness of q1 and q2, this implies that q1(y) = q2(1− y). It follows that P (y) = q1(y) isa solution for
(1− y)NP (y) + yNP (1− y) = 1 ,
and that the degree of P (y) is less than N .Now, let’s find the closed form expression for P (y). Rewrite equation (4.4.32) as
P (y) =1
(1− y)N [1− yNP (1− y)] .
The Taylor series expansion of 1(1−y)N is
1(1− y)N = 1 + Ny + N(N + 1)
y2
2!+ N(N + 1)(N + 2)
y3
3!+ . . .
=∞∑j=0
(N − 1 + j
j
)yj .
Substitute this expression into the previous equation for P (y), we have
P (y) = (1 + Ny + N(N + 1)y2
2!+ N(N + 1)(N + 2)
y3
3!+ . . .)︸ ︷︷ ︸
A
−
(yN + NyN+1 + N(N + 1)yN+2
2!+N(N + 1)(N + 2)
yN+3
3!+ · · ·) P (1− y)︸ ︷︷ ︸
B
.
Note that since we assume P (y) is a polynomial in y (in nonnegative powers only) with degreeless than N , the term B (which consists only of yi with i ≥ N) must cancel the terms in A withyi, i ≥ N . Hence,
P (y) = 1 + Ny + N(N + 1)y2
2!+ N(N + 1)(N + 2)
y3
3!+ · · ·
+N(N + 1) · · · (N +N − 2)yN−1
(N − 1)!
=N−1∑j=0
(N − 1 + j
j
)yj ,
which is equation (4.4.34) with Q(y) = 0.Note that the given expression for P (y) is the unique lowest degree solution, but there exist
many solutions of higher degrees.
54
4.14 Regularity of Daubechies filters
We represent the normalized lowpass filter M0(ω) as M0(ω) =(
1 + ejω
2
)NR(ω) and let B =
supω∈[0,2π]
|R(ω)|. The sufficient condition for the convergence of ϕ(i)(t) to a continuous function ϕ(t)
with Fourier transform Φ(ω) =∞∏k=1
M0
( ω2k)
is B < 2N−1. For the Daubechies filters in table 4.2:
N B 2N−1
2 1.73 23 3.16 44 5.92 85 11.22 166 21.49 32
All of the filters satisfy the regularity bound, so the iterated filters converge to the Fourier trans-forms of continuous functions.
For higher regularity, a sufficiency condition is that the limit is m times continuously differen-tiable if B < 2N−1−m where B is defined as in part (a). This is not satisfied for m = 1 for theN = 4 and N = 5 filters. Consider a variation of the Daubechies condition that corresponds to acascade of filters and upsampling; we define a new supremum as in the book:
Bl = supω∈[0,2π]
l−1∏k=0
|R(2kω)|
The limit function is m times continuously differentiable if
Bl < 2l(N−1)−m
For m = 1 and l = 2 this amounts to B2 < 22N−3. As an example, we consider N = 4 and N = 5,for which we obtain:
N B 22N−3
4 19.31 325 54.96 128
We can conclude from this that the limits are continuously differentiable. Other combinations areleft to the reader.
4.15 Necessity of zeros at aliasing frequenciesUsing a similar method as in the proof of Prop. 4.6, it is proved that∑
k
g0[Nk + i] =∑k
g0[Nk + j]
for all i, j = 0, · · · , N − 1. Not let w be any N th of unity, w �= 1. Then we have
G0(w) =N−1∑i=0
wi∑k
g0[Nk + i]
= (N−1∑i=0
wi)∑k
g0[Nk]
= 0.
55
Here we have used the well known equality∑N−1
i=0 wi = 0 for w �= 1.
4.16 A sufficient condition for continuitySimilarly as in the proof of Prop. 4.7, we can write
Φ(ω) =∞∏k=1
(1 + ejω/N
k+ · · ·+ ejω(N−1)/Nk
N
)K ∞∏k=1
R(ω
Nk).
In the above, the first product equals (sin(ω/2)/(ω/2))K , which leads to a decay of O(1 + |ω|)−K .Because R(0) = 1, there exists a constant C such that |R(ω)| ≤ 1 + C|ω|. So proceeding as in
the proof of Prop. 4.7, we find∏∞k=1 |R( ω
Nk )| ≤ eC
N−1 for all ω ≤ 1.For 2J−1 ≤ |ω| < 2J we find
∞∏k=1
|R(ω
Nk)| ≤ BJe
CN−1
≤ 2J(K−1−ε)eC
N−1
≤ C ′(1 + |ω|)K−1−ε,
where we have chosen ε such that B < 2K−1−ε. The required result follows.
4.17 Successive interpolation(a) Let xu[n] be the upsampled version of x[n]. The one-step output y(1)[n] isthe convolution of xu[n] with the filter impulse response g[n]:
y(1)[n] =∑
even k
xu[k]g[n − k]
where the sum is taken over even k since xu[n] is zero for odd indices. Equivalently,
y(1)[n] =∑i
xu[2i]g[n − 2i]
Since xu[2i] = x[i], we can write the problem requirement as
y(1)[2n] =∑i
x[i]g[2n − 2i] = x[n]
which implies that g[2n] = δ[n]. So g[n] must be zero for even time indices. This condition impliesthat G(z) can not have any nonzero even powers of z.
(b) If the interpolation filter g[n] meets the requirements of (a), the two-step output y(2)[n] satisfiesy(2)[2n] = y(1)[n]. Replacing n by 2n gives y(2)[4n] = y(1)[2n] = x[n]. Iterating this argumentyields the result y(i)[2in] = x[n].
(c) G(z) =12z + 1 +
12z−1 corresponds to a simple moving average where the interpolated value
is the mean of the immediate neighboring values. An example is illustrated in the following figure.
56
0 2 40
1
2
3
4
0 5 100
1
2
3
4
0 10 200
1
2
3
4
(d) We can write G(z) =(
1√2
+1√2z
)(1√2
+1√2z−1
), which is a product of Haar lowpass
filters. The iteration of G(z) converges to the product of the iterations of these filters, so the asso-ciated time function y(t) = lim y(i)(t) corresponds to a convolution of Haar lowpass functions. Wethus expect the limit function y(t) to be everywhere continuous but not everywhere differentiable.
(e) The Daubechies filter autocorrelations are given by
PN (z) = 2−2N+1(1 + z)N (1 + z−1)NR(z)
where R(z) is given in Table 4.2 for N = 2, . . . , 6. It is straightforward to numerically verify (usingMATLAB, for instance) that these functions satisfy the constraint from (a), namely they are zero-valued at even time indices (except time 0). For higher regularity, the sufficient condition given inProposition 4.7 can be used. That is, let
M(ω) =PN (ejω)PN (1)
be factored as
M(ω) =(
1 + ejω
2
)NR(ω).
Define B = supω∈[0,2π] |R(ω)|. If
B < 2N−1−�, � = 1, 2, . . . ,
then the iteration converges to ϕ(t) which is �-times continuously differentiable. With this test, wecan establish that for N = 2 the limit functions are continuous but not everywhere differentiable;for N = 3, 4 the limit functions are continuous and once differentiable; for N = 5, 6 the limitfunction is continuous and twice differentiable.
4.18 Recursive subdivision schemes(a) Using induction it is easy to show that supp(f (i)) ⊂ (−1, 1) for all i. Define
Pi(z) =∑n
p(ni)z
−n =∑n
f (i)(n
2i)z−n,
57
and
G(z) =∑k
ckz−k =
(1 + z)(1 + z−1)2
.
One now easily derives
Pi+1(z) =i∏
m=0
G(z2m).
Using the fact thati∏
m=0
(1 + z2m) =
2i+1−1∑m=1
zm,
it follows that
p(i+1)n =
12i+1
∑0≤i,j≤2i+1−1
i−j=n
1
={
2i+1−|n|2i+1 , |n| ≤ 2i+1
0, otherwise
We conclude that f (∞)(t) = max(0, 1 − |t|), the well known hat function.
(b) DefiningQ(i)(z) =
∑n
g(i)(n
2i)z−n,
we derive
Q(i+1)(z) = (i∏
m=0
G(z2m))S(z2i+1
),
where G(z) is as in (a) and where the definition of S(z) should be obvious. Taking the limits wefind that
g(∞)(t) =∑n
s[n]f (∞)(t− n),
which is nothing other than bilinear interpolation.
(c) Replacing the definition of G(z) in (a) by
G(z) =116
(−z3 + 9z + 16 + 9z−1 − z−1),
we can replay the arguments of (a) and (b). From
P (i+1)(z) = P (i)(z2)G(z),
we see that the limit function f (∞)(t) can be derived from the interpolation scheme
f (0)(n) ={
1, n = 00, otherwise
f (i)(n
2i−1) = f (i−1)(
n
2(i−1))
f (i)(2n+ 1
2i) =
916
(f (i−1)(n
2i−1) + f (i−1)(
n + 12i−1
))
− 116
(f (i−1)(n− 12i−1
) + f (i−1)(n+ 22i−1
))
58
If we start with an arbitrary sequence s[n], we can either substitute g for f in the above and replacethe definition g(0) by
g(0)(n) = s[n],
or derive a result as in (b)g(∞)(t) =
∑n
s[n]f (∞)(t− n).
4.19 Interpolation filters and functions(a) We show by induction that if g[n] is an interpolation filter, then ϕ(i)(t) has the interpolationproperty for any i. To check whether ϕ(i)(t) has the interpolation property, it is equivalent to checkthat g(i)[2in] = δ[n].
It is easier to do this in the z-transform domain. For i = 1 (the initial step of the induction),g(1)[n] = g[n], hence g(1)[2n] = δ[n]. Writing G(1)(z) in its polyphase domain, we have
G(1)(z) = 1 + zG(1)1 (z2) ,
where G(i)j (z) denotes the j-th polyphase component of G(i)(z).
Now assume that g(i)[2in] = δ[n]. Then we can write G(i)(z) in its polyphase component as
G(i)(z) = 1 +2i−1∑j=1
zjG(i)j (z2i
) .
From the iterated filter bank, we have
G(i+1)(z) = G(i)(z)G(z2i) .
Substituting in the expression for the polyphase decomposition,
G(i+1)(z) = (1 +2i−1∑j=1
zjG(i)j (z2i
)) (1 + z2iG
(1)1 (z2i+1
))
= 1 + z2iG
(1)1 (z2i+1
) +2i−1∑j=1
zjG(i)j (z2i
) +2i−1∑j=1
zj+2iG
(i)j (z2i
)G(1)1 (z2i+1
) .
We need to verify that the 0-th polyphase component of G(i+1)(z) is 1, or, equivalently, the co-efficients of zm2i+1
, m ∈ Z− {0}, are 0. Clearly, the middle two terms do not contribute to thecoefficients of zm2i+1
, m ∈ Z. The last term contributes to the coefficients of zj+2i+k2i+l2i+1for
j ∈ {1, . . . , 2i − 1}, and m, l ∈ Z. It is easy to verify that the exponent j + 2i + k2i + l2i+1 cannever be equal to multiples of 2i+1. Hence, the 0-th polyphase component is 1, and we can write
G(i+1)(z) = 1 +2i+1−1∑j=1
zjG(i+1)j (z2i+1
) .
Thus, g(i)[2in] = δ[n] and ϕ(i)(n) = δ[n] for all i. Since ϕ(i)(t) converges pointwise, taking i → ∞and sampling yields ϕ(n) = δ[n]; i.e. ϕ(t) has the interpolation property.
59
(b) Let the support of g[n] be n ∈ {0, 1, . . . , N}; that is, g[0] �= 0, g[N ] �= 0, and g[n] = 0 forn �= {0, 1, . . . , N}. Since g[n] is a lowpass filter, we cannot have N = 0 (otherwise, it becomes atrivial allpass filter), and N > 0. If g[n] is an interpolation filter, then N must be odd (becauseg[2n] = δ[n]).
The filter g[n] is orthogonal, so
〈g[n], g[n − 2k]〉 =N∑n=0
g[n]g[n − 2k] = c δ[k]
for some nonzero constant c. Since the support of g[n] is {0, 1, . . . , N} and using the fact thatg[2n] = δ[n], the inner product can be simplified to
〈g[n], g[n − 2k]〉 =N∑
n=2k
g[n]g[n − 2k]
= g[0]g[2k] +
N−1−2k2∑
n=0
g[2k + 2n+ 1]g[2n + 1] , (9)
where the second expression is simply the inner product at the odd samples of g[n] and we haveassumed, without loss of generality, that k ≥ 0. Evaluating at k = 0, and using g[0] = 1,
1 +
N−12∑
n=0
g2[2n + 1] = c .
First let’s assume that N > 1, and then we show that we must have N = 1. Evaluating (9) atk = N−1
2 yieldsg[N ]g[1] = 0 .
By assumption g[N ] �= 0, so g[1] = 0 must hold. Again, evaluating (9) at k = N−32 yields
g[N − 2]g[1] + g[N ]g[3] = 0 ,
which implies g[3] = 0. Continuing this way until k = 0, we find that all odd samples of g[n] exceptg[N ] is 0, and that
〈g[n], g[n]〉 = g2[0] + g2[N ] = c .
Since the problem specifies a lowpass filter, and for N > 1, g[n] resembles nothing like a lowpassfilter, we must have N = 1. Hence our filter is
g[n] =
⎧⎨⎩
1 , n = 0√c− 1 , n = 1
0 , otherwise.
For c = 2, we have the Haar lowpass filter (up to a scaling factor).
(c) Given that ϕ(t) has the interpolation property (i.e. ϕ(n) = δ[n]) and satisfies the two-scaleequation
ϕ(t) =∑n
cnϕ(2t− n) .
60
Then, sampling ϕ(t) at t = n, n ∈ Z yields
ϕ(n) =∑m
cmϕ(2n −m) =∑m
cmδ[2n −m] = c2n = δ[n] .
Thus, the sequence cn is an interpolation filter.
4.20 Two-scale equationWe may assume that ϕ(t) is not identically 0. We first assume that we can prove that f(t) = f(2t).Also assume that f(t) is continuous at 0. We will show that we can prove a contradiction if f(t) isnot a constant.
Assume that there exists a t1 �= 0 such that f(t1) �= f(0), and let ε = |f(t1)−f(0)|. Let δ be anypositive constant. Then for J sufficiently large |t1/2J | < δ, and |f(t1/2J )−f(0)| = |f(t1)−f(0)| =ε �< ε, contradicting the continuity assumption.
It remains to prove the two assumptions. First we show continuity at 0. Fix an ε > 0. Becauseof the decay of ϕ(t), we can find an N ∈ N such that
∑|n|>N ϕ(t − n) < 1/4ε for all t ∈ [−1, 1].
For each n ∈ {−N, · · · , N} we choose a δn such that |ϕ(n) − ϕ(n + τ)| < ε/6N for all τ < δn. Setδ = minn δn. Then we have for all τ < δ
|f(0)− f(τ)| ≤∑n
|ϕ(n) − ϕ(n+ τ)|
=∑
|n|≤N|ϕ(n)− ϕ(n + τ)|+
∑|n|>N
|ϕ(n)− ϕ(n+ τ)|
≤∑
|n|≤Nε/6N +
∑|n|>N
(|ϕ(n)| + |ϕ(n + τ)|)
< ε/2 + ε/2= ε.
This proves continuity at 0.The second assumption is taken care of as follows:
f(t) =∑n
ϕ(t− n)
=∑n
∑m
cmϕ(2t − 2n−m)
=∑k
(∑n
ck−2n)ϕ(2t− k)
=∑k
ϕ(2t− k)
= f(2t).
4.21 Two-scale equation
ϕ′(t) satisfies the two-scale equation
ϕ(t) =∑n
ϕ(2t− n) (*)
61
with∑n
c2n =∑n
c2n+1 = 1. Differentiate (*) we get
ϕ′(t) =∑n
2cnϕ′(2t− n). (**)
If we take the hat function as an example, then its derivative is a scaled version of the Haar wavelet.Pictorially this situation is presented in Figure P4.21.
������
��
-1
1-1
1
ϕ(t)
-1 1
1
-1
ϕ′(t)
(a) hat function (b) derivative of hat function
. . . . . . . . . . . . ....... . . . ............ . . . ..
.
.
.
.. . . .
ϕ′(2t − 1)
2ϕ′(2t)
ϕ′(2t + 1)
-1
2
-2
1
(c) two scale equation for derivative
Figure P4.21: Two scale equation for the derivative of the hat function.
If ϕ is Cn−1(R) and ϕ(n)(t) exists, then differentiating n-times yields
ϕ(n)(t) =∑n
2ncnφ(n)(2t− n).
4.22 Orthogonality of local cosine basesWe proceed as in Section 4.8.2. First we show that ϕj−1,k and ϕj,m are orthogonal.
〈ϕj−1,k, ϕj,m〉 =2√
Lj−1Lj
∫ +ηj
−ηj
wj−1(x− aj)wj(x+ aj)
cos[
π
Lj−1(k +
12)(x+ Lj−1)
]×
cos[π
Lj(m+
12)x]dx
= ±wj(aj − x)wj(aj + x)2√
Lj−1Lj
∫ +ηj
−ηj
62
sin[
π
Lj−1(k +
12)x]×
cos[π
Lj(m+
12)x]dx
Since the window part of this last integral is even, while the rest is odd, the above inner productis zero.
It remains to evaluate 〈ϕj,k, ϕj,m〉. As in Section 4.8.2, we split this integral in three parts:from aj − ηj to aj + ηj, from aj + ηj to aj+1 − ηj+1, and from aj+1 − ηj+1 to aj+1 + ηj+1. Weconsider the the first integral and third integral:
2Lj
∫ ηj
−ηj
w2j (aj + x) cos
[π
Lj(k +
12)x]
cos[π
Lj(m+
12)x]dx
=2Lj
∫ ηj
0(w2
j (aj + x) + w2j (aj − x)) cos
[π
Lj(k +
12)x]
cos[π
Lj(m+
12)x]dx
=2Lj
∫ ηj
0cos
[π
Lj(k +
12)x]
cos[π
Lj(m+
12)x]dx
and
2Lj
∫ ηj+1
−ηj+1
w2j (aj + Lj + x) cos
[π
Lj(k +
12)(x+ Lj)
]cos
[π
Lj(m+
12)(x+ Lj)
]dx
=2Lj
∫ 0
−ηj+1
(w2j (aj+1 + x) +w2
j (aj+1 − x))×
cos[π
Lj(k +
12)(Lj + x)
]cos
[π
Lj(m+
12)(Lj + x)
]dx
=2Lj
∫ Lj
Lj−ηj+1
cos[π
Lj(k +
12)x]
cos[π
Lj(m+
12)x]dx.
Combining this with the second integral, we find
2Lj
∫ Lj
0cos
[π
Lj(k +
12)x]
cos[π
Lj(m+
12)x]dx = δkm,
which was to be proved.
63
Chapter 5
5.1 Characterization of local regularity
(a) The continuous wavelet transform is given by CWTf (a, b) =∫ +∞
−∞ψ∗a,bf(t) dt.
For the Haar wavelet ψ(t) ={
1 −12 ≤ t ≤ 0
−1 0 ≤ t ≤ 12
the CWT of some function f(t) is
CWTf (a, b) =
⎧⎪⎪⎨⎪⎪⎩− 1√
a
∫ a2+b
0f(t) dt b < 0
− 1√a
∫ b
0f(t) dt − 1√
a
∫ a2+b
bf(t) dt b ≥ 0
For f(t) ={
0 t < 0tn t ≥ 0
the CWT is
CWTf (a, b) =
⎧⎪⎪⎨⎪⎪⎩− 1√
a
1n+ 1
(a2
+ b)n+1
b < 0
− 1√a
1n+ 1
((a2
+ b)n+1
− 2bn+1
)b ≥ 0
For b << a2 this amounts to CWTf (a, b) ≈ −
a(2n+1)
2
(n+ 1) 2n+1∼ a
(2n+1)2 .
Substituting n = 1 gives CWT ∼ a 32 .
(b) The derivation in (a) shows that CWT ∼ a 2n+12 . (This can also be derived using the reasoning
of section 5.1.2, specifically the subsection on characterization of regularity.)
5.2 Autocorrelation of the Haar wavelet
(a) The Haar wavelet is given by ψ(t) =
⎧⎪⎪⎨⎪⎪⎩
1 −12≤ t ≤ 0
−1 0 ≤ t ≤ 12
0 otherwise
Because ψ(t) is piecewise constant, its autocorrelation a(t) =∫ ∞
−∞ψ(τ)ψ(τ − t)dτ is piecewise
linear (like the convolution of two boxcars, for instance). Thus a(t) is specified completely by itsbreakpoints, which are easy to find:
a(t) =
⎧⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎩
0 t ≤ −1 no overlap between ψ(τ) and ψ(τ − t)−1
2t = −1
2half overlap
1 t = 0 full overlap
−12
t = 12 half overlap
0 t ≥ 1 no overlap
Connecting the breakpoints by straight lines gives a(t) =
⎧⎪⎪⎨⎪⎪⎩
0 |t| ≥ 1
|t| − 112≤ |t| ≤ 1
−3|t|+ 1 |t| ≤ 12
64
The graph of a(t) is:
−1.5 −1 −0.5 0 0.5 1 1.5−0.5
0
0.5
1
time
(b) The autocorrelation a(t) is everywhere continuous but not everywhere differentiable; this isevident from the above graph. We can find the decay by noting that a(t) = ψ(t) ∗ ψ(−t) and thus
A(ω) = |Ψ(ω)|2. Since Ψ(ω) has a1ω
decay, A(ω) has a1ω2
decay as ω −→∞.
5.3 Nondownsampled filter bank(a) We have G0(z) = H0(z−1) and G1(z) = H1(z−1), meaning that the synthesis filters are time-reversed versions of the analysis filters. Orthogonality yields:G0(z)G0(z−1) + G0(−z)G0(−z−1) = 2 and G1(z) = −z−2k−1 G0(−z−1).The filter bank output is related to the input by
Y (z) = [G0(z)H0(z) + G1(z)H1(z)]X(z)=
[G0(z)G0(z−1) + G1(z)G1(z−1)
]X(z)
=[G0(z)G0(z−1) +
{−z−2k−1G0(−z−1)
}{−z2k+1G0(−z)
}]X(z)
=[G0(z)G0(z−1) + G0(−z)G0(−z−1)
]X(z) = 2X(z)
In the time domain, y[n] = 2x[n].
(b) By Parseval’s Theorem,
‖x0‖2 + ‖x1‖2 =12π
∫ π
−π
∣∣X0
(ejω
)∣∣2 dω +∫ π
−π
∣∣X1
(ejω
)∣∣2 dω=
12π
∫ π
−π
∣∣X (ejω
)∣∣2 {∣∣H0
(ejω
)∣∣2 +∣∣H1
(ejω
)∣∣2} dω=
12π
∫ π
−π
∣∣X (ejω
)∣∣2 {∣∣G0
(e−jω
)∣∣2 +∣∣G1
(e−jω
)∣∣2} dωBy the relationship of G0(z) and G1(z) used in part (a), we have∣∣G1(e−jω)
∣∣2 =∣∣G0(−ejω)
∣∣2 =∣∣∣G0
(ej(ω+π)
)∣∣∣2. For real g0[n], we have G0(e−jω) = G∗0(e
jω) by
conjugate symmetry, and thus∣∣G0(e−jω)
∣∣2 =∣∣G0(ejω)
∣∣2. We can then write
‖x0‖2 + ‖x1‖2 =12π
∫ π
−π
∣∣X (ejω
)∣∣2{∣∣G0
(ejω
)∣∣2 +∣∣∣G0
(ej(ω+π)
)∣∣∣2} dωSince g0[n] and its modulated version are power-complementary (due to orthogonality),
‖x0‖2 + ‖x1‖2 = 2[
12π
∫ π
−π
∣∣X(ejω)∣∣2 dω] = 2‖x‖2
65
(c) Given H0(z) andG0(z), we want to findH1(z) andG1(z) such that y[n] = x[n]; in the Z-domain,this perfect reconstruction is given by
H0(z)G0(z) + H1(z)G1(z) = 1
H1(z)G1(z) = 1 − H0(z)G0(z)
To find H1(z) and G1(z), we spectrally factor 1−H0(z)G0(z); note that this is FIR and thus hasno poles (except at 0 or ∞). The number of degrees of freedom in the design corresponds to thenumber of different ways the zeros of 1−H0(z)G0(z) can be distributed among the two filters H1(z)and G1(z). Note that if we are restricted to real filters, any complex zeros have to be distributedin conjugate pairs. Let F (z) = 1−G0(z)H0(z) with H0(z) = G0(z−1) = 1 + 2z−1 + z−2. Then,
F (z) = − z2 − 4z − 5 − 4z−1 − z−2
= (− z2 − 3z − 1) (z−2 + z−1 + 1)
=
(z +
3 +√
52
)(z +
3−√
52
)(z−1 +
1 + j√
32
)(z−1 +
1− j√
32
)
There are(
40
)+(
41
)+(
42
)+(
43
)+(
44
)= 16 ways to spectrally factor this F (z)
into H1(z) and G1(z) if we allow complex filters. For real filters, there are only 8 factorizations.
5.4 Continuous wavelet transform(a) One easily derives that
CWTf (a, b) =1√a
(∫ b
b− 12af(t) dt−
∫ b+ 12a
bf(t) dt
).
In order to describe the CWT of f , we need to evaluate an expression of the form
∫ b
b− 12aχ
(−1
2,12
)−∫ b+ 1
2a
bχ
(−1
2,12
).
It is obvious that the above expression is piecewise linear in both a and b. In general, this piecewisefunction will be determined by its values at the six points b = ±1
2(1 + {−1, 0, 1}a). For fixed athere are four distinct cases to consider: a = 1, a < 1, 1 < a ≤ 2 and 2 < a.
For a = 1 the CWTf (a, b) is completely determined by the values at the 4 points b = ±1,±12 ,
viz. 0,±12 .
For a < 1 there are 6 values of b to consider. For b = ±12 the CWT equals ± 1
2√a
and for
b = ±12(1± a) the CWT equals zero.
For 1 < a ≤ 2 there are also 6 values to consider. For b = ±12 the CWT equals ± 1
2√a, for
b = ±12(1 + a) the CWT equals zero, and for b = ±1
2(1− a) the CWT equals ± (1−a)√a
.
For 2 < a there are again 6 values to consider. For b = ±12 the CWT equals ± 1√
a, for
b = ±12(1 + a) the CWT equals zero, and for b = ±1
2(a− 1) the CWT equals ± 1√a.
See Figure P5.4 for a graphical representation. In particular, for small values of a, the CWTof f has support in the neighborhood of ±1
2 with a peak value of ± 12√a. For very large values of
66
-1
1
1
-1
-1
1
1
-1
CWT for a=1
���
����� B
BB���
���B
BB
-1
QQ�����Q
QQQ
���
����
�
b-¿
CWT for a=1/2
-1
1
1
-1
CWT for a=2 CWT for a=4
-2 2
b-¿
b-¿b-¿
1
Figure P5.4: CWT for various values of a.
a the CWT has support from −12(1 + a) to 1
2 (1 + a) with the majority of the values being either− 1√
a(for negative b values) or 1√
a(for positive b values).
(b) The function ψ can be written as
ψ(t) = f(2t+ 1/2) − f(2t− 1/2).
Defining f±(t) by f(2t± 12) we find, using the shift and scaling properties,
CWTf±(a, b) =1√2CWTf (2a, 2b± 1).
ThereforeCWTψ(a, b) =
1√2(CWTf (2a, 2b + 1/2) − CWTf (2a, 2b − 1/2)).
In particular, for small values of a, the CWT of ψ has support around ±12 (with a negative
function value 12√a) and around 0 (with a positive function value 1√
a). For large a, the CWT
has support around −a/2 and a/2, with a negative function value − 12√a
and a peak with positive
function value 1√a
around the origin.
5.5 Tight frames for R2
Note that
N−1∑k=0
cos2(
2πkN
)=
N−1∑k=0
sin2
(2πkN
)=N
2
N−1∑k=0
sin(
2πkN
)cos
(2πkN
)= 0
67
Let x = [x1, x2] and
y = [y1, y2] =2N
n−1∑N=0
< ϕk, x > ϕk
=⇒ y1 =2N
[x1
n−1∑k=0
cos2
(2πkN
)+ x2
n−1∑k=0
sin(
2πkN
)cos
(2πkN
)]
y2 =2N
[x1
N−1∑k=0
cos(
2πkN
)sin
(2πkN
)+ x2
N−1∑k=0
sin2
(2πkN
)]
Applying the noted identities completes the proof:
y1 = x1 y2 = x2 =⇒ y = x
Comparing the reconstruction formula above (the formula for y) and equation 5.3.5 indicates thatthe redundancy factor is N/2. This is intuitively reasonable since this frame uses N vectors torepresent a 2-dimensional space; recall the example of the overcomplete basis of three vectors forR2 discussed in example 1.1 of section 1.1 and also in example 5.1, for which the redundancy factoris 3/2.
5.6 Sinc frameMore or less by definition, the space of bandlimited signal is given by
B =
{∑n∈Z
lnsinc(t− n) : l ∈ L2(Z)
}.
Moreover, for each function f ∈ B, the sequence {ln} is found as ln = f(n) and ‖f‖ = ‖l‖. Nowconsider shifted versions fi(t) = f(t+ i/N) of f , i = 0, . . ., N − 1. Then we find
f(t) = fi(t− i/N)
=∑n
fi(n) sinc((t− i/N)− n)
=∑n
f(n+ i/N) sinc((t− i/N)− n).
Thus
Nf(t) =N−1∑i=0
f(t)
=∑i
∑n
f(n+ i/N) sinc((t− i/N) − n)
=∑k∈Z
f(k/N) sinc(t− k/N).
This immediately implies that {sinc(t− k/N)} is a tight frame with redundancy N .
5.7 Properties of a full rank non-square matrix(a) Consider the function r(x) = ‖Mx‖ on the unit sphere Sn−1 ⊂ Rn. The rank of M being
68
equal to n is equivalent to M being injective. Therefore we have that Mx �= 0 on Sn−1, implying0 < r(x) for all x ∈ Sn−1. As r(x) is continuous, and as Sn−1 is compact, the function r(x) takeson a minimum A �= 0 and a maximum B �= 0 on Sn−1. Consequently for all x ∈ Rn the requiredinequalities hold:
0 < A‖x‖ ≤ ‖Mx‖ ≤ B‖x‖ <∞.
(b) The matrix MTM has size n × n and is invertible, as shown by the following sequence ofequalities.
MTMx = 0 ⇒〈MTMx, x〉 = 0 ⇒〈Mx, Mx〉 = 0 ⇒
Mx = 0 ⇒x = 0
The last equality follows from injectivity of M . Evaluating MM we find
MMx = (MTM)−1MTMx = Ix = x.
(c) If M is any other left inverse, then the difference N = M − M satisfies NM = 0 and vice versa.Such a matrix N can also be characterized by image(M) ⊂ kernel(N).
(d) Given y ∈ Rm, its projection Py (= Mz for some z) onto the range of M is characterized by
〈y − Py, Mx〉 = 0,
for all x ∈ Rn. Rewriting we find
〈y − Py, Mx〉 = 0 ∀x ⇒〈y −Mz, Mx〉 = 0 ∀x ⇒
〈MT y −MTMz, x〉 = 0 ∀x ⇒MT y −MTMz = 0 ⇒
z = (MTM)−1MT y = My ⇒Py = Mz = MMy
69
Chapter 6
6.1 Toeplitz matrix-vector productsA general Toeplitz matrix of size N × N has 1 + 2(N − 1) = 2N − 1 parameters, and a generalcirculant matrix of the same size has N parameters. Therefore the minimal extension of an N ×NToeplitz matrix to a circulant matrix needs N − 1 columns and rows.
The corresponding numbers of parameters for symmetric Toeplitz and circulant matrices are Nand �N+2
2 � respectively. So the minimal extension is by N − 2 rows and columns.For example, the matrix ⎡
⎣ a b cd a be d a
⎤⎦
is extended to ⎡⎢⎢⎢⎢⎣a b c e dd a b c ee d a b cc e d a ba c e d a
⎤⎥⎥⎥⎥⎦
and ⎡⎣ a b cb a bc b a
⎤⎦
is extended to ⎡⎢⎢⎣a b c bb a b cc b a bb c b a
⎤⎥⎥⎦ .
6.2 Block circulant matricesA matrix C with blocks Ci,j is block-circulant if and only if Ci,j = Di−j for some sequence ofmatrices {Dk}, k = 0, · · · , N − 1, where the subtraction i − j in the index of Di−j is to be takenmodulo N . Let W be a primitive N th root of unity. Then we need to show that the matrix
1N
∑i,j
W kiCi,jW−jk
is block diagonal. The proof is as follows:1N
∑i,j
W kiCi,jW−jl
=1N
∑i,j
W kiDi−jW−jl
=1N
∑i,j
W (k−l)iDi−jW(i−j)l
=1N
∑i,m
W (k−l)iDmWml
= δk,l∑m
DmWml
70
6.3 Walsh-Hadamard transformA size 2N Walsh-Hadamard transform WHT 2N is computed by evaluating a matrix product[
WHTN WHTNWHTN −WHTN
] [X
(1)N
X(2)N
].
It follows that the addition complexity γ(N) of the WHT satisfies the recursion formula γ(2N) =2γ(N) + 2N . Using the fact that γ(2) = 2, we find γ(2N ) = 2N + (N − 1)2N = N2N , and therequired N log2N complexity follows.
6.4 Complexity of MUSICAM filter bankThe number of operations per sample equals
2L
N+ 2 log2N = 2
51232
+ 2 log2 32 = 2(16 + 5) = 42.
With 44100 samples per second, this amount to 1852200 operations per second.
6.5 Iterated filters
H(i)(z) =i−1∏k=0
H(z2k
)= H(z)
i−1∏k=1
H(z2k
)= H(z)
i−1∏k=1
H((z2)2
k−1)
= H(z)i−2∏k′=0
H((z2)2
k′) {for k′ = k − 1} = H(z) H(i−1)(z2)
H(i)(z) =i−1∏k=0
H(z2k
)= H
(z2i−1
) i−2∏k=0
H(z2k
)= H
(z2i−1
)H(i−1)(z)
H(2k)(z) =2k−1∏i=0
H(z2i)
= H(z)H(z2) · · ·H(z22k−1−1
)︸ ︷︷ ︸ × H
(z22k−1
)H
(z22k−1+1
)· · ·H
(z22k−1
)︸ ︷︷ ︸
= H(2k−1)(z) × H(ξ)H(ξ2) · · ·H(ξ2
2k−1−1){for ξ = z22k−1
}
= H(2k−1)(z) H(2k−1)(ξ) = H(2k−1)(z) H(2k−1)(z22k−1)
6.6 Overlap-add/save filter banks(a) First observe that in the notation of pp. 361–363, N = 4 and M = 2. By substituting into theformulas on p. 362,
H(z) = z + 1
Hi(z) = z−1H(W i4z) for i = 0, 1, 2, 3
71
⇒
⎧⎪⎪⎨⎪⎪⎩
H0(z) = z−1(z + 1) = 1 + z−1
H1(z) = z−1(−jz + 1) = −j + z−1
H2(z) = z−1(−z + 1) = −1 + z−1
H3(z) = z−1(jz + z) = j + z−1
G(z) = 1 + z−1 + z−2 + z−3
Gi(z) = G(W i4z) for i = 0, 1, 2, 3
⇒
⎧⎪⎪⎨⎪⎪⎩
G0(z) = 1 + z−1 + z−2 + z−3
G1(z) = 1 + (−jz)−1 + (−jz)−2 + (−jz)−3 = 1 + jz−1 − z−2 − jz−3
G2(z) = 1 + (−z)−1 + (−z)−2 + (−z)−3 = 1− z−1 + z−2 − z−3
G3(z) = 1 + (jz)−1 + (jz)−2 + (jz)−3 = 1− jz−1 − z−2 + jz−3
(b) Note that L = N −M + 1 = 3. Now we simply substitute into an equation from p. 362 to get
Ci =1N
L−1∑�=0
W i�N c[�] =
14
2∑�=0
(−j)i�c[�]. (10)
Recall that the filter bank is implementing a convolution with the filter c[n].Substituting in
Y (z) = z−(M−1)C(z)X(z), (11)
givesY (z) = z−1(c[0] + c[1]z−1c[2]z−2)X(z).
Therefore, when the channel coefficients are scalars, the impulse response can at most have non-zerosamples corresponding to delays of 1, 2, and 3.
(c) The fact that filters with longer impulse responses can be implemented if the scalar Ci’s arereplaced by filters is discussed on pp. 362–363. We demonstrate this through a simple example butfirst derive a formula relating the input and output.
The output Y (z) consists of four components. For i ∈ {0, 1, 2, 3}, denote the i-th channeloutput component by Yi(z). Since Yi results from filtering by Hi, downsampling by 2, filtering byCi, upsampling by 2, and filtering by Gi,
Yi(z) = Gi(z)12[Ci(z2)Hi(z)X(z) +Ci((−z)2)Hi(−z)X(−z)
]=
12Gi(z)Ci(z2) [Hi(z)X(z) +Hi(−z)X(−z)] .
Therefore
Y (z) =
(12
3∑i=0
Ci(z2)Gi(z)Hi(z)
)X(z) +
(12
3∑i=0
Ci(z2)Gi(z)Hi(−z))X(−z). (12)
From this equation, we view the channel filters as weighting Gi(z)Hi(z) and Gi(z)Hi(−z). Bymultiplying out terms,
Gi(z)Hi(z) = G(W i4z)z
−1H(W i4z)
= W i4 + 2z−1 + 2W−i
4 z−2 + 2W−2i4 z−3 +W−3i
4 z−4
Gi(z)Hi(−z) = G(W i4z)(−z)−1H(−W i
4z)= W i
4 −W−3i4 z−4
72
for i ∈ {0, 1, 2, 3}. In general, aliasing will not be cancelled; choosing scalar channel coefficientsthrough (10) will insure that aliasing is cancelled. It is instructive to work out a simple example.If c[n] = [1 0 0], then Ci = 1
4 for all i. In this case
3∑i=0
Gi(z)Hi(z) = 8z−1, and3∑i=0
Gi(z)Hi(−z) = 0,
so the overall system is LTI with transfer function z−1, which agrees with (11).Let us now implement a filter that could not be implemented with scalar channel coefficients.
Suppose we wish to implement the simple lowpass filter c[n] = 18 [1 3 3 1]. (Recall that the actual
system will have a minimum delay of M−1 = 1.) Following the procedure on p. 363, we decomposec[n] into filters of length M = 2 and rescale, giving 1
4 [1 3] and 14 [3 1]. Taking 4-point Discrete
Fourier Transforms gives 14 [4 1 − 3j − 2 1 + 3j] and 1
4 [4 3− j 2 3 + j]. In the notation of thebook, these eight coefficients form the set {ci�}. From these we form the following filters:
C0(z) =14(4 + 4z−1)
C1(z) =14((1− 3j) + (3− j)z−1)
C2(z) =14(−2 + 2z−1)
C3(z) =14((1 + 3j) + (3 + j)z−1)
Through a tedious calculation (aided greatly by a system for symbolic mathematical calculation)we can show:
12
3∑i=0
Ci(z2)Gi(z)Hi(z) = z−1(1 + 3z−1 + 3z−2 + z−3) (13)
12
3∑i=0
Ci(z2)Gi(z)Hi(−z) = 0 (14)
Combining (12)–(14) shows that the system is LTI and implements the desired convolution witha delay of 1. A similar modulated filter bank implementation for this convolution using scalarchannel coefficients would require larger M and hence have more delay.
6.7 Three-channel filter bankWe consider the behavior for even and for odd signals separately. We start with an even signalX(z2). In the channels this gives rise to the signals {0,X(z),X(z)}, and after convolution with thechannel filters and upsampling we have
{0,X(z2)(F0(z2) + F1(z2)),X(z2)F1(z2)}.
Applying the synthesis filters and summing we find an overall outputX(z2)(F0(z2)+z−1F1(z2))z−1,showing the required LTI behavior for even signals.
Secondly we consider an odd signal zX(z2). In the channels this gives rise to the signals{X(z),X(z), 0}, and after convolution with the channel filters and upsampling we have
{X(z2)F0(z2),X(z2)(F0(z2) + F1(z2)), 0}.
73
Applying the synthesis filters and summing we find an overall output zX(z2)(F0(z2)+z−1F1(z2))z−1,showing the required LTI behavior for odd signals.
By linearity, the system is LTI for all signals, with the required impulse response.
74
Chapter 7
7.1 Distortion from uniform quantizationLet X be uniformly distributed on [a, b). Let X denote the result of quantizing X. The uniformquantizer outputs yi = a + (i + 1
2)∆ for inputs in [a + i∆, a + (i + 1)∆). The MSE distortion isgiven by
D = E(|X − X|2
)=
N−1∑i=0
∫ a+(i+1)∆
a+i∆(x− yi)2fX(x) dx
=N−1∑i=0
∫ a+(i+1)∆
a+i∆
(x−
(a+
(i+
12
)∆))2 1
b− a dx
=N−1∑i=0
1b− a
∫ 12∆
− 12∆z2 dz
= N1N∆
∆3
12=
∆2
12,
where the change of variables z = x − a − (i + 12)∆ has been used. Note that this calculation is
essentially the same as determining and variance of a uniform random variable.
7.2 Coding gain as function of number of channels(a) As suggested by the hint, consider coding a source with power spectrum given by
|X(ejω)|2 ={
4 |ω| ≤ π2
2 π2 < |ω| ≤ π
.
We assume that the subband bandwidths are equal. The subband variances can be determined byintegrating |X(ejω)|2. For the 2-channel system, the subband variances are:
σ20 = 2
12π
∫ π2
0|X(ejω)|2 dω = 2
σ21 = 2
12π
∫ π
π2
|X(ejω)|2 dω = 1
(The factors of 2 are to account for positive and negative frequencies.) Using the formula at themiddle of page 388, the coding gain is
GAIN2 =DPCM
DSBC=
(2 + 1)/2(2 · 1)1/2
=3
2√
2.
For the 3-channel system, the subband variances are σ20 = 4
3 , σ21 = 1 and σ2
2 = 23 . We find
GAIN3 =DPCM
DSBC=
(43 + 1 + 2
3)/3(43 · 1 ·
23)1/3
=32/3
2,
so the coding gain is greater for the 2-channel system.
75
(b) Now consider using a 4-channel system for the same source. The subband variances are σ20 =
σ21 = 1 and σ2
2 = σ23 = 1
2 . The coding gain is
GAIN4 =DPCM
DSBC=
(1 + 1 + 12 + 1
2)/4(1 · 1 · 1
2 ·12)1/4
=3
2√
2.
The performance is the same as the 2-channel system and better than the 3-channel system.
7.3 Coding gain in subband coding(a) The expression for the power spectrum should be
|X(ejω)|2 = 1− |ω|π
|ω| < π.
(Note the absolute value.) First consider the case N = 2. The subband variances are:
σ20 = 2
12π
∫ π2
0|X(ejω)|2 dω =
1π
∫ π2
0
(1− ω
π
)dω =
38
σ21 = 2
12π
∫ π
π2
|X(ejω)|2 dω =1π
∫ π
π2
(1− ω
π
)dω =
18
(The factors of 2 are to account for positive and negative frequencies.) Using the formula from themiddle of page 388, the coding gain is
12
(σ2
0 + σ21
)(σ2
0 · σ21
)1/2 =12
(38 + 1
8
)(
38 ·
18
)1/2 =2√3.
For the general case, calculate
σ2k = 2
12π
∫ (k+1)πN
kπN
|X(ejω)|2 dω =1π
∫ (k+1)πN
kπN
(1− ω
π
)dω =
2N − 2k − 12N2
for k = 0, 1, . . . , N − 1. The coding gain is given by
1N
∑N−1k=0 σ
2k(∏N−1
k=0 σ2k
)1/N=
1N
(2N−12N2 + 2N−3
2N2 + · · ·+ 12N2
)(
2N−12N2
2N−32N2 · · · 1
2N2
) 1N
=N
((2N − 1)(2N − 3) · · · 1)1N
.
(b) The expression for the power spectrum should be
|X(ejω)|2 = e−α|ω| |ω| < π.
(Note the absolute value.) First consider the case N = 2. The subband variances are:
σ20 = 2
12π
∫ π2
0|X(ejω)|2 dω =
1π
∫ π2
0e−αω dω =
1απ
[1− e−απ/2
]σ2
1 = 212π
∫ π
π2
|X(ejω)|2 dω =1π
∫ π
π2
e−αω dω =1απ
e−απ/2[1− e−απ/2
]
76
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21
1.2
1.4
1.6
1.8
2
2.2
2.4
2.6
2.8
3
alpha
Cod
ing
gain
(a)
(b)
Figure P7.3: Comparison of coding gains.(a) Coding gain from 7.3(a). (b) Coding gain from 7.3(b) for N = 2.
(The factors of 2 are to account for positive and negative frequencies.) The coding gain is given by
12
(σ2
0 + σ21
)(σ2
0 · σ21
)1/2 =12
(1 + e−απ/2
)1απ
(1− e−απ/2
)1απ
(1− e−απ/2
)√e−απ/2
=12
(eαπ/2 + e−απ/2
).
Notice that the coding gain is a monotonically increasing function of |α|. This coding gain iscompared to the coding gain for part (a) in Figure P7.3.
For the general N -channel case,
σ2k = 2
12π
∫ (k+1)πN
kπN
|X(ejω)|2 dω =1π
∫ (k+1)πN
kπN
e−αω dω
= − 1απ
[e−α
(k+1)πN − e−αkπ
N
]=
1απ
[1− e−απ
N
]e−
αkπN
for k = 0, 1, . . . , N − 1. Now we can calculate
1N
N−1∑k=0
σ2k =
1απ
[1− e−απ
N
] 1N
N−1∑k=0
e−αkπN =
1απ
[1− e−απ
N
] 1−(e−
απN
)N1− e−απ
N
=1απ
[1− e−απ
N
] 1− e−απ
1− e−απN
,
where we have used the formula for a finite geometric series. Also,(N−1∏k=0
σ2k
)1/N
=1απ
[1− e−απ
N
](N−1∏k=0
e−αkπN
)1/N
=1απ
[1− e−απ
N
] (ePN−1
k=0 −απN
)1/N
=1απ
[1− e−απ
N
]e−
απN
PN−1k=0 k =
1απ
[1− e−απ
N
]e−
απN
N(N−1)2
=1απ
[1− e−απ
N
]e−
α(N−1)π2 ,
77
where we have used the formula for an arithmetic series. The coding gain is therefore
1N
∑N−1k=0 σ
2k(∏N−1
k=0 σ2k
)1/N=
1− e−απ
(1− e−απN )e−
α(N−1)π2
.
7.4 Huffman and run-length coding(a) The Huffman code is constructed as follows:
code symbol prob.0 1 1
212
12
12
12
12
12
0→ 1
10 2 14
14
14
14
14
14
0→ 12
↗1
110 3 18
18
18
18
18
0→ 14
↗1
1110 4 116
116
116
116
0→ 18
↗1
11110 5 132
132
132
0→ 116
↗1
111110 6 164
164
0→ 132
↗1
1111110 7 1128
0→ 164
↗1
1111111 8 1128
↗1
The expected length of the Huffman code is:
E[length] =8∑i=1
Pr(symbol i)× (length of code i) =12764
= 1.984
The entropy is defined as H = −∑
i Pr(symbol i) log2 (Pr(symbol i)). In this case, the length ofcode i is exactly − log2 (Pr(symbol i)), so the expected length and the entropy are equal.
(b)symbol code prob.
+1 10 0.2−1 11 0.2+2 001 0.15−2 010 0.15+3 0001 0.075−3 0110 0.075+4 0111 0.05−4 00000 0.05SR 00001 0.05
The expected length is 3 bits/symbol and the entropy is 2.96 bits/symbol.
(c) Example: +1 − 1 − 3 + 2 0 0 0 + 4. The coded bit stream is 10, 11, 0110, 001, 00001, 110, 0111where the commas have been inserted to indicated the correspondence of the bits to the symbols.Note that the symbol sequence 0 0 0 is encoded as the “start of run” symbol followed by the codefor a run length of three. The bit stream can be uniquely decoded since the code is a prefix code.
78
(d) The expected length of this coding scheme is given by
E[L] =4∑
i=−4
pili + pSRLSR
where the first term is the expected length of the code for nonzero symbols and the second term isthe expected length of the code for zero runs. Note that this includes the length of the SR code.The result is the sum of the expected length from part (b) plus pSR times the expected lengthfrom part (a): E[L] = 3.0 + 0.05(1.984) = 3.1 bits/symbol. Without coding, the number of bitsper symbol required is �log2(number of symbols)� = 4 since there are 9 possible symbols. Thecompression gain is thus 4
3.1 = 1.29.
7.5 Pyramid coding(a) The expression for the power spectrum should be
|X(ejω)|2 = e−3|ω|/π |ω| < π.
(Note the absolute value.) Though not explicitly stated, we assume a cutoff frequency of π2 for the
lowpass filter. By integrating the power spectrum we get
σ2coarse = 2
12π
∫ π2
0|X(ejω)|2 dω =
1π
∫ π2
0e−3|ω|/π dω =
13
(1− e−3/2
)≈ 0.2590, and
σ2difference = 2
12π
∫ π
π2
|X(ejω)|2 dω =1π
∫ π
π2
e−3|ω|/π dω =13e−3/2
(1− e−3/2
)≈ 0.0578.
(The factors of 2 are to account for positive and negative frequencies.)
(b) We are modelling the quantization of the coarse channel as the addition of a white noise withvariance c∆2 where the noise is uncorrelated with x[n]. Assuming proper normalization of theinterpolating filter, the noise power in the prediction is c∆2. Since the noise is uncorrelated withx[n], the variance of the difference channel is
σ2difference′ = σ2
difference + c∆2,
where σ2difference is the variance calculated in part (a).
(c) The allocation of bits between the coarse channel and the difference channel was investigatedexperimentally using MATLAB. The problem statement left room for experimentation. We choseto use a first-order Gauss-Markov source with correlation factor a generated by filtering a unitvariance, white Gaussian source by the filter
√1−a2
1−az−1 . Uniform quantization was used withoutentropy coding. The quantizer design was based on a rule of thumb called the “4σ-rule”: Full-scalefor the quantizer is set at ±4σ, where σ is the standard deviation of the source. Built-in MATLABcommands were used for generating the coarse channel and the prediction. The basic code follows:
Bc = 4; % bits per coarse sampleBd = 4; % bits per difference sample
% Total bit rate is Bc/2 + Bd bits per samplet = 8192; % size of data stringa = 0.99; % correlation
% Generate source sequence
79
x(1) = randn;c = sqrt(1-a*a);for k=2:(t+1)
x(k) = c*randn + a*x(k-1);endx = x(2:(t+1));
x_coarse = decimate(x, 2);sigma_c = sqrt( mean(x_coarse .^ 2) );for i = 1:(t/2)
x_coarse_q(i) = quantize( x_coarse(i), sigma_c, Bc );endx_predict = interp(x_coarse_q, 2);x_diff = x - x_predict;sigma_d = sqrt( mean(x_diff .^ 2) );for i = 1:t
x_diff_q(i) = quantize( x_diff(i), sigma_d, Bd );endx_recon = x_predict + x_diff_q;distortion = mean( (x - x_recon) .^ 2 );
The quantization function is given by:
function y = quantize( x, sigma, bits )% QUANTIZE Quantize using the 4-sigma rule% y = quantize( x, sigma, bits )
if bits == 0y = 0;
elsedelta = 8*sigma / 2^bits;y = delta * (floor(x/delta) + 0.5);if abs(y) > 4*sigma
y = (2^(bits - 1) - 0.5)*delta*sign(y);end
end
Figure P7.5(a) shows the results for coding of three sources with a total budget of 8 bits/sample.The correlation coefficient of the source is denoted by a. The horizontal axis shows the numberof bits assigned per difference channel sample, Bd. Since there are half as many coarse channelsamples, this leaves Bc = 16 − 2Bd bits per coarse channel sample. Figures P7.5(b), P7.5(c), andP7.5(d) gives results for 6, 4, and 2 bits/sample, respectively.
A few general trends are of interest. The normalization is such that the process has unit variancefor any a. This explains the fact that the for a fixed coding rate, the curves coincide when all thebits are used for the difference channel. As a is increased, the process becomes more lowpass, somore of the energy is in the coarse channel. Since the coarse channel has half the sample rate, thiscorresponds to better performance for larger a. Since the processes we are considering are (to avarying degree) lowpass, when the total coding rate is very low, the most effective strategy is todisregard the highpass information completely. This is illustrated in Figure P7.5(d).
7.6 Zero trees(a) Consider an octave-band decomposition as in Figure 3.7(a) applied to a finite-length one-
80
0 1 2 3 4 5 6 7 810
−5
10−4
10−3
10−2
10−1
100
Bits for difference channel (Bd)
MS
E
a = 0.99
a = 0.9
a = 0.8
0 1 2 3 4 5 610
−4
10−3
10−2
10−1
100
Bits for difference channel (Bd)
MS
E
a = 0.99
a = 0.9
a = 0.8
(a) Total bit budget: 8 bits/sample (b) Total bit budget: 6 bits/sample
0 0.5 1 1.5 2 2.5 3 3.5 410
−3
10−2
10−1
100
Bits for difference channel (Bd)
MS
E
a = 0.99
a = 0.9
a = 0.8
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 210
−2
10−1
100
Bits for difference channel (Bd)
MS
E
a = 0.99
a = 0.9
a = 0.8
(c) Total bit budget: 4 bits/sample (d) Total bit budget: 2 bits/sample
Figure P7.5: Experimental results for bit allocation in pyramid coding
dimensional signal. To maintain the analogy with the EZW algorithm, we will consider the inde-pendent variable to be a spatial variable. For a four-level decomposition, in analogy to Figure 7.25,the channels of the decomposition can be ordered as in Figure P7.6(a). Define a tree structure inFigure P7.6(a) to be a set of points in successively higher frequency bands, extending to the highestfrequency band, which correspond to the same spatial region. Then we can define a zero tree to bea tree for which all the samples have the quantized value of zero. One possible zero tree is shownin Figure P7.6(b).
In contrast to the two-dimensional case, where a zero tree grows as powers of four, the treegrows as powers of two. Also, there is only a single direction of tree growth, as opposed to thethree shown in Figure 7.28. The only modification of the dominant and subordinate passes of theEZW algorithm is in the scanning. One simply scans one band (checking for possible zero trees)and then scans the next higher band.
(b) The zero tree definition in part (a) applies to an arbitrary subband decomposition. In contrastto the octave band case, the points at a fixed tree depth may not be adjacent. Figure P7.6(c) showsa possible decomposition and zero tree. The points in the HL and HH bands are at the same tree
81
HLHLLHLLLH
LLLL
Figure P7.6(a)
c cc cccc
HLHLLHLLLH
LLLL
Figure P7.6(b)
depth. Notice that the size and rate of growth of the tree are the same as in the octave-band case.
c cc cc cc
LHLLH HL HHLLLH
LLLL
Figure P7.6(c)
The zero tree concept is most powerful when zero trees are most likely to be found. Most signalsof interest have decaying spectra, ı.e. |X(ejω)| decreases with increasing |ω| for |ω| < π. For suchsignals, zero trees are most likely to be found with an octave-band decomposition. We will explainthis by comparing the decompositions in Figures P7.6(b) and P7.6(c). Assume the original signalhas length N . In the octave band case, the energy per coefficient in band H is given by
σ2H =
2π
∫ π
π/2|X(ejω)|2.
For the decomposition of Figure P7.6(c), we have:
σ2HL =
4π
∫ 3π/4
π/2|X(ejω)|2
σ2HL =
4π
∫ π
3π/4|X(ejω)|2
Because of the decay of the power spectrum, σ2HL < σ2
H < σ2HH . For a particular spatial location,
one is more likely to find a nonzero coefficient in band HL or band HH than in band H. Thereforezero trees are more likely to be found in the octave band case.
Please note that we are not claiming that an octave-band decomposition is always best. For aparticular image, a decomposition that gives less zero trees may give better rate-distortion perfor-mance.
82
(c) Using zero trees in a full tree subband decomposition gives a different scan order than thezig-zag scanning used in DCT. In DCT coding, the scan order corresponds to ascending order ofω1 +ω2, where ω1 and ω2 are the frequencies in the two spatial dimensions. In a zero tree structure(assuming Shapiro’s scan order), horizontal frequencies are scanned, followed by the vertical and45◦ diagonal directions. This is illustrated through an example.
Consider a full tree subband decomposition of depth 3 applied to a 32×32 image. The result is64 channels with four samples per channel, as shown in Figure P7.6(d). The subbands are orderedsuch that frequencies increase from left to right and from bottom to top. The circles in FigureP7.6(d) show a potential zero tree in this decomposition. The scanning order has the followingproperties:
• All spatial locations within a subband are scanned before moving to the next subband.
• Searches for a zero tree are within a spatial region.
• Scanning through the depth of a potential zero tree corresponds to frequencies with similarorientation. The frequency domain is effectively split into horizontal, vertical, and diagonalregions.
Figure P7.6(d)
Now consider a DCT decomposition of the same image. In order to have the same (2 × 2)spatial resolution as in the subband decomposition, assume that 8× 8 blocks are used. The zig-zagscanning order used with DCT coding is shown in Figure 7.15(a). This scanning has the followingproperties:
• Each spatial region is coded independently.
83
• The scanning order is non-decreasing in ω1 + ω2.
• An end-of-block (EOB) symbol is used when the remaining samples in the scan have a quan-tized value of zero, so the coded samples (approximately) form a triangle.
The relative efficacies of the two methods will depend on the power spectrum of the image.The DCT with zig-zag scan has the advantage that a single EOB symbol can be used to code allhigh frequency components, whereas at least three zero trees are needed to code all high frequencycomponents. The shortcoming of the DCT method is that power spectra do not simply decay as afunction of |ω1 +ω2|. For example, an image block with many horizontal edges will have significantenergy up to high vertical frequencies. For sake of illustration, suppose that the non-zero coefficientsform an 8×3 rectangle. Then an EOB does not occur until the 48th symbol. In the subband codingcase, many horizontal and diagonal frequency samples could be eliminated early on by zero trees.
7.7 Progressive to interlaced conversion(a) First note that for each filter in (7.4.3), the leftmost point corresponds to the origin. We willshow that the filters in (7.4.3) form a perfect reconstruction analysis pair and find the correspondingsynthesis pair through polyphase domain analysis. Taking (0, 0) and (1, 0) to be representatives ofthe cosets of the quincunx lattice, a polyphase representation of the analysis pair should take thefollowing form:
H0(z1, z2) = H00(z1, z2) + z−11 H01(z1, z2)
H1(z1, z2) = H10(z1, z2) + z−11 H11(z1, z2)
From this we can read off the polyphase components:
H00(z1, z2) = −1(1 + z−21 z−2
2 + z−21 z2
2 + z−41 )− 2(z−1
1 z−12 + z−1
1 z2 + z−31 z−1
2 + z−31 z2) + 28z−2
1
H01(z1, z2) = 4(1 + z−11 z−1
2 + z−11 z2 + z−2
1 )H10(z1, z2) = 1 + z−1
1 z−12 + z−1
1 z2 + z−21
H11(z1, z2) = −4
We can now find a synthesis pair through Gp(z1, z2)Hp(z1, z2) = I. Note that since detHp is amonomial, we get an FIR synthesis pair. (See also Problem 3.28.)
Gp(z1, z2) = [Hp(z1, z2)]−1
=1
detHp
(H11 −H01
−H10 −H00
)
=1
−128z−21
(H11 −H01
−H10 −H00
)Thus the polyphase components of the synthesis pair are described by:
128G00(z1, z2) = −z21H11(z1, z2)
= 4z21
128G01(z1, z2) = z21H10(z1, z2)
= z21 + z1z
−12 + z1z2 + 1
128G10(z1, z2) = z21H01(z1, z2)
= 4(z21 + z1z
−12 + z1z2 + 1)
128G11(z1, z2) = −z21H00(z1, z2)
= (z21 + z−2
2 + z22 + z−2
1 ) + 2(z1z−12 + z1z2 + z−1
1 z−12 + z−1
1 z2)− 28
84
UsingGi(z1, z2) = Gi0(z1, z2) + z1Gi1(z1, z2),
we find that the synthesis filters are as follows, where the origin is underlined:
g0[n1, n2] =1
128
⎛⎝ 1
1 4 1 01
⎞⎠
g1[n1, n2] =1
128
⎛⎜⎜⎜⎜⎝
12 4 2
1 4 −28 4 12 4 2
1
⎞⎟⎟⎟⎟⎠
(b) In Figure P7.7, the circles show the result of quincunx downsampling in the (time,vertical)-plane. The sequence has only even lines at even times and only odd lines at odd times and henceis an interlaced sequence. After quincunx downsampling again, we are left with the points markedwith squares. (Note that the second quincunx downsampling is with respect to the lattice formedby the circles.) The squares represent a progressive sequence with half as many frames and half asmany (vertical) lines as the original sequence.
h
h
h
h
h
h
h h h
h
h
h
h
h
h
h
h
h h
h
h h
h
h
h
h
h
h
h
h
h
h
�
�
time
vertical
Figure P7.7: Cascaded quincunx downsampling
85
7.8 FCO downsampling(a) In analogy to (3.2.46), the filter pair corresponds to an orthogonal decomposition for FCOdownsampling if
H0(z1, z2, z3)H0(z−11 , z−1
2 , z−13 ) +H0(−z1,−z2,−z3)H0(−z−1
1 ,−z−12 ,−z−1
3 ) = 2. (15)
Checking this is simply a matter of substitution:
H0(z1, z2, z3)H0(z−11 , z−1
2 , z−13 ) +H0(−z1,−z2,−z3)H0(−z−1
1 ,−z−12 ,−z−1
3 )
=1√2
(1 + z1z2z3)1√2
(1 +
1z1z2z3
)+
1√2
(1− z1z2z3)1√2
(1− 1
z1z2z3
)
=12
(2 + z1z2z3 +
1z1z2z3
)+
12
(2− z1z2z3 −
1z1z2z3
)= 2
The full justification of checking the condition (16) follows directly from part (b). We call it a“Haar” decomposition because of the similarity of the lowpass filter
h0[n1, n2, n3] =1√2
(δ[n1, n2, n3] + δ[n1 + 1, n2 + 1, n3 + 1])
to (3.1.16).
(b) Recall the two-channel analysis/synthesis system shown in Figure 3.1(a). It can be shown that
X(z) =12[G0(z)H0(z) +G1(z)H1(z)]X(z) +
12[G0(z)H0(−z) +G1(z)H1(−z)]X(−z). (16)
This is an alternative version of (3.2.14) which clearly shows that (3.2.15)-(3.2.16) are necessaryand sufficient for perfect reconstruction.
Now consider a system as in Figure 3.1(a) which processes three-dimensional signals and usesFCO downsampling and upsampling. We would like to find a relationship analogous to (16). Denotethe upsampled version of y0 by y0u. Then because of the FCO sampling,
y0u[n1, n2, n3] ={
(h0 ∗ x)[n1, n2, n3] n1 + n2 + n3 even,0 otherwise.
(17)
Taking the z-transform of (17) gives
Y0u(z1, z2, z3) =∑
n1+n2+n3 even(h0 ∗ x)[n1, n2, n3]z−n1
1 z−n22 z−n3
3
=12
∑n1,n2,n3∈Z
(1 + (−1)n1+n2+n3
)(h0 ∗ x)[n1, n2, n3]z−n1
1 z−n22 z−n3
3 (18)
=12
⎛⎝ ∑n1,n2,n3∈Z
(h0 ∗ x)[n1, n2, n3]z−n11 z−n2
2 z−n33
+∑
n1,n2,n3∈Z(h0 ∗ x)[n1, n2, n3](−z1)−n1(−z2)−n2(−z3)−n3
⎞⎠
=12
(H0(z1, z2, z3)X(z1, z2, z3) +H0(−z1,−z2,−z3)X(−z1,−z2,−z3)) ,
86
where in (18), 1 + (−1)n1+n2+n3 is introduced to cancel terms with n1 + n2 + n3 odd. Defining y1u
to be upsampled version of y1, we similarly find
Y1u(z1, z2, z3) =12
(H1(z1, z2, z3)X(z1, z2, z3) +H1(−z1,−z2,−z3)X(−z1,−z2,−z3)) .
Thus
X(z1, z2, z3) = (19)12
[G0(z1, z2, z3)H0(z1, z2, z3) +G1(z1, z2, z3)H1(z1, z2, z3)]X(z1, z2, z3) +
12[G0(z1, z2, z3)H0(−z1,−z2,−z3) +G1(z1, z2, z3)H1(−z1,−z2,−z3)]X(−z1,−z2,−z3).
Notice that (19) is the same as (16) with z replaced by z1, z2, z3 and −z replaced by −z1,−z2,−z3.This is due to the particular type of downsampling. In two dimensions, the analogous relationshipholds if quincunx downsampling is used.
7.9 Filtering of wide-sense stationary processes(a) By making repeated use of the linearity of expectation, we can make the following calculation:
KY [n,m] = cov(y[n], y[m]) = E ((y[n]− Ey[n])(y[m]− Ey[m]))= E ((y[n]−mY )(y[m]−mY ))= E (y[n]y[m])−m2
Y where mY denotes Ey[n], which is independent of n
= E
⎛⎝ ∞∑i=0
h[i]x[n − i]∞∑j=0
h[j]x[m − j]
⎞⎠ −m2
Y
=
⎛⎝ ∞∑i=0
∞∑j=0
h[i]h[j]E (x[n− i]x[m− j])
⎞⎠−m2
Y
=
⎛⎝ ∞∑i=0
∞∑j=0
h[i]h[j]E (x[n− i]x[m− j])
⎞⎠−
⎛⎝ ∞∑i=0
h[i]E (x[n− i])∞∑j=0
h[j]E (x[m− j])
⎞⎠
=∞∑i=0
∞∑j=0
h[i]h[j] (E (x[n− i]x[m− j])− E (x[n− i])E (x[m− j]))
=∞∑i=0
∞∑j=0
h[i]h[j]KX [(n − i)− (m− j)]
=∞∑i=0
∞∑j=0
h[i]h[j]KX [(n −m)− (i− j)]
Since KY [n,m] is a function of (n−m) only, {y[n]} is wide-sense stationary. Denoting n−m by kgives
KY [k] =∞∑i=0
∞∑j=0
h[i]h[j]KX [k − (i− j)].
(b)
KXY [�, n] = cov(x[�], y[n]) = E ((x[�]− Ex[�])(y[n]− Ey[n]))
87
= E (x[�]y[n]) because we have assumed a zero-mean input
= E
(x[�]
∞∑k=0
h[k]x[n − k])
=∞∑k=0
h[k]E (x[�]x[n − k])
=∞∑k=0
h[k]KX [(n− k)− �]
=∞∑k=0
h[k]KX [(n− �)− k]
Denoting n− � by m gives
KXY [m] =∞∑k=0
h[k]KX [m− k].
(c) Let
x[n] ={x[n] n ≥ 00 n < 0
,
where x[n] is a wide-sense stationary process. Let y[n] be a filtered version of x[n]. For n ≥ 0,
E(y[n]) = E
( ∞∑k=0
h[k]x[n − k])
= E
(n∑k=0
h[k]x[n − k])
because x[n] = 0 for n < 0
=n∑k=0
h[k]Ex[n − k]
=n∑k=0
h[k]mX where mX denotes Ex[n], which is independent of n
= mX
n∑k=0
h[k]. (20)
Following (7.A.7), it was shown that when the input is a (two-sided) wide-sense stationary process,the output has constant expected value equal to mX
∑∞k=0 h[k]. In contrast, (20) shows that when
the input is a one-sided wide-sense stationary process, E(y[n]) generally depends on n. If theprocessing is by an FIR filter of length L+ 1,
E(y[n]) = mX
min(n, L)∑k=0
h[k],
so E(y[n]) is constant for n ≥ L. For FIR or IIR filtering, limn→∞E(y[n]) = mY , where mY is theconstant expected value that would result from filtering the two-sided x[n].
88