Álgebra linear e aplicações - w3.impa.brw3.impa.br/~diego/teaching/2018.1/slides-dct.pdf ·...
TRANSCRIPT
Álgebra Linear e Aplicações
DISCRETE FOURIER TRANSFORM(AND THE FFT)
Motivation• Consider the propagation of sound from fixed
source to fixed destination in an environment• Sound is pressure variation through time• It is a vector space
• Each environment can be seen as an operator• Different environments, different operators
• What are the properties we can expect?
Linear, time-invariant systems• An LTI system is an operator
satisfying the two properties• Linearity
• Time invariance
• Sound can be discretized…
O : L2 ! L2
O{↵1x1 + x2} = ↵1O{x1}+O{x2}
Ta{u}(t) = u(t� a) ) O�Ta{x}
= Ta
�O{x}
Focus on the discrete, finite case• Linearity means linear operator• Time invariance could mean different things• Most powerful definition uses circular shifts• , with• T (u) = v, vi mod n = ui�1 mod n
T =
2
6664
11
. . .1
3
7775T
2
666664
u0
u1
u2...
un�1
3
777775=
2
666664
un�1
u0
u1...
un�2
3
777775
L�T (u)
�= T
�L(u)
�
L : Rn ! Rn
LT = TL
What can an LTI system do?
• Look at the effect of L on vector u
• It is completely defined by !
• What about matrix L?
vj =n�1X
i=0
ui
⇥T i(h)
⇤j =
n�1X
i=0
ui hj�i mod n
=
2
6664
h0 hn�1 · · · h1
h1 h0 · · · h2...
.... . .
...hn�1 hn�2 · · · h0
3
7775
Circulant matrix
Impulse
Circular convolution between u and h
Impulse response
v = L(u) = L
n�1X
i=0
uiei
!=
n�1X
i=0
ui L(ei) =n�1X
i=0
ui L�T i(e0)
�=
n�1X
i=0
ui Ti�L(e0)
�
h = L(e0)
L =⇥h T (h) · · · Tn�1(h)
⇤
Summary of LTI systems• Every discrete linear time-invariant system L
can be expressed as a circular convolution• The circular convolution between ,
denoted or , is
• is the impulse response of L• The corresponding matrix L is circulant• It commutes with the unit circular shift T
h = L(�) (� = e0)
vj =n�1X
i=0
ui hj�i mod n
u,h 2 Rn
u ⇤ h v 2 Rnh ⇤ u
=n�1X
k=0
uj�k mod n hk
Computational cost too high• Same as a matrix-vector product!• But matrix has only n degrees of freedom • Doesn’t make sense
• Vectors can be very large• Millions of entries• Need a faster alternative
• What if we found a better basis?
O(n2)
DiagonalizeT
• First notice that T has n distinct eigenvalues
• Find the eigenvector basis
det(T� �I) =
���������
�� 11 ��
. . .. . .1 ��
���������
= (��)n + (�1)1+n = 0
Tn = I ) (�k)n = 1
Tvk = �kvk
�k = !k, k 2 {0, 1, . . . , n� 1}, ! = e2⇡in
[Tvk]j = �k[vk]j = [vk]j�1 mod n
[vk]j = (�k)j = (!k)j
v⇤kvj =
n�1X
p=0
(!k)p(!j)p P⇤P = PP⇤ = I=n�1X
p=0
(!k�j)p =
(1�(!k�j)n
1�!k�j = 0, k 6= j
n, k = j
P =1pn
2
6664
1 1 1 · · · 11 ! !2 · · · !n�1
......
.... . .
...1 !n�1 (!2)n�1 · · · (!n�1)n�1
3
7775
Simultaneously diagonalizeL
• Eigenvectors of L and T are the same!
• Lv and v are eigenvectors of T associated to• But• So we must have • And therefore L is diagonalizable by same P!
�
dim�N(T� �I)
�= 1
Lv = �v
Tv = �vLT = TL
�) T(Lv) = LTv = �(Lv)
The Discrete Fourier Transform• Given a vector , the Discrete Fourier
Transform of v, denoted , is the vector
• The Inverse Discrete Fourier Transform, denoted , restores v
v 2 Rn
F (v)
v = P⇤v vj =1pn
n�1X
k=0
vke� 2⇡ijk
n
F�1(v)
v = Pv = PP⇤v vj =1pn
n�1X
k=0
vke2⇡ijk
n
The Convolution Theorem• If u,v are two vectors in Rn, then
• Proof
F (u ⇤ h) = F (u) � F (h)
[a � b]j = ajbj (Hadamard product, element-wise product)
F (u ⇤ h) = P⇤(u ⇤ h) = P⇤Lu = P⇤LPP⇤u = (P⇤LP)F (u)
= DF (u)
P⇤h = P⇤Le0 = P⇤LP(P⇤e0)
=⇥�0 · · · �n�1
⇤T� F (u)
= P⇤h � F (u) = F (h) � F (u)
=⇥�0 · · · �n�1
⇤T= D
⇥1 1 · · · 1
⇤T
Alternative to convolution
u,h u ⇤ hO(n2)
u � hO(n)
u, h
P⇤
FO(n2)
P
F�1O(n2)
Fast Fourier Transform (FFT)• Divide and conquer!
hj =n�1X
k=0
hk!�jkn
=
n/2�1X
k=0
h2k !�j2kn +
n/2�1X
k=0
h2k+1 !�j(2k+1)n
=m�1X
k=0
hek (!
2n)
�jk + !�jn
m�1X
k=0
hok (!
2n)
�jk
=m�1X
k=0
hek !
�jkm + !�j
n
m�1X
k=0
hok !
�jkm
= hej modm + !�j
n hoj modn
h =
2
666664
h0
h1...
hn�2
hn�1
3
777775
he =
2
666664
h0
h2...
hn�4
hn�2
3
777775
ho =
2
666664
h1
h3...
hn�3
hn�1
3
777775
Alternative to convolution
u,h u ⇤ h
u, h u � h
PP⇤
F F�1
O(n2)
O(n)
O(n log n) O(n log n)
Demo!