Introduction to Fourier Transform and Time-Frequency Analysis
Speaker: Li-Ming Chen
Advisor: Meng-Chang Chen, Yeali S. Sun
2009/10/9 Speaker: Li-Ming Chen 2
Outline
Periodic Phenomena and Fourier Series
Non-periodic Phenomena and Fourier Transform
Why Needs Time-Frequency Analysis?
Wavelet Transform and its Applications
2009/10/9 Speaker: Li-Ming Chen 3
History of Fourier Series
Fourier series Named in honor of Joseph Fourier (1768-1830) Originally used to solve “heat equation” Initially, the paper submitted in 1807 However, the theory published in 1822
A Fourier series decomposes a periodic function (or periodic signal) into a sum of simple oscillating functions, namely sines and cosines (or complex exponentials) (Wikipedia)
2009/10/9 Speaker: Li-Ming Chen 4
Example Fourier series model a general periodic phenomena by using basic building blocks;
Time (ms)50001000 2000 3000 4000
1
1
1
3
0
Am
plitu
de
-3
0sin(2*pi*t)
sin(2*pi*2*t)
sin(2*pi*5*t)
sin(2*pi*t) + sin(2*pi*2*t) + sin(2*pi*5*t)
2009/10/9 Speaker: Li-Ming Chen 5
Fourier Transform & Fourier Series We can consider:
Fourier transform as a limiting case of Fourier series in concerned with analysis of non-periodic phenomena
Applications of Fourier analysis: Physics, partial differential equations, signal processing, imaging,
acoustics, cryptography… Why so popular, so applicable? due to some properties:
Transforms are linear operators and “usually” invertible Using complex exponential (computational convenience) Convenient to compute convolution operation Has fast Fourier transform (FFT) algorithm
2009/10/9 Speaker: Li-Ming Chen 6
What is a Periodic Phenomenon? Periodic:
Some pattern that repeated and repeat regularly
Periodic
Periodic in Time
Periodic in Space
Called “Frequency”(e.g., number of repetitions of patterns in a second)
Called “Period”(e.g., Heat) The temperature
on a ring is periodic(depend on position)
※ Periodic in Time and Space!? e.g., wave motion
2009/10/9 Speaker: Li-Ming Chen 7
Using Periodic Function to Model Periodic Phenomena sin and cos are periodic fun
ctions Can configure their frequency,
amplitude, and phase
“One period, many frequencies” Let’s consider with period 1 also has perio
d 1 !! We can modify and combine th
ese building blocks to model very general periodic phenomena
)**2sin(* tkA
)*2sin( t
Time (ms)0 1
Am
plitu
de1
1
1
)**2sin( tk
2009/10/9 Speaker: Li-Ming Chen 8
Fourier Series
A periodic function f(t) can be represented as
Using 和角公式
Such that, “partial sums” of the Fourier series for ƒ
N
kkk tkAtf
1
)**2sin(*)(
N
kkkk tktkAtf
1
)sin(*)**2cos()cos(*)**2sin(*)(
these are constants
N
kkk tkbtkatf
1
)**2sin(*)**2cos(*)(
an and bn called Fourier coefficients of f.(include the info. of amplitude and phase)
2009/10/9 Speaker: Li-Ming Chen 9
Using Complex Exponential Euler’s formula
Fourier Series (General Form)
But, how to get ck ? (how to compute Fourier coefficients?)
)**2sin()**2cos(2 tkitke ikt
k
iktk ectf 2*)(
2)**2cos(
22 iktikt eetk
i
eetk
iktikt
2)**2sin(
22
Ck is also a complex number
2009/10/9 Speaker: Li-Ming Chen 10
How to Compute ck ?
Say, isolate cn and then solve cn
…(utilizing the property of complex exponential), we can get
nk
iktk
intn ectfec
!
22 *)(*
1
0
2*)( dtetfc intn
andnnn cca
)( nnn ccib
k
iktk ectf 2*)(
(Fourier series)
(Fourier coefficient)
f(t) is the function we observed[Analysis]
[Synthesis]
2009/10/9 Speaker: Li-Ming Chen 11
Outline
Periodic Phenomena and Fourier Series
Non-periodic Phenomena and Fourier Transform
Why Needs Time-Frequency Analysis?
Wavelet Transform and its Applications
2009/10/9 Speaker: Li-Ming Chen 12
Fourier Transform (FT)
Not all phenomena are periodic, and periodic phenomena will die out eventually… Let’s view non-periodic function as limiting case of
period function as period ∞
Fourier transform is invertible FT is the generalization of Fourier coefficient Inverse FT is the generalization of Fourier series
2009/10/9 Speaker: Li-Ming Chen 13
Fourier Transform (FT) (cont’d) Definition
FT:
Inverse FT:
dtetfsF ist2*)()(
dsesFtf ist2*)()(
2009/10/9 Speaker: Li-Ming Chen 14
Example (periodic function)
FT
f(t) = cos(2pi*5t) + cos(2pi*10t) + cos(2pi*20t) + cos(2pi*50t)
5 10 20 50
all the amplitudes are 1
Magnitude/amplitude of the Freq.is half of the original amplitude
Time (ms) Frequency (Hz)
2009/10/9 Speaker: Li-Ming Chen 15
Example (non-periodic function)
FT
2
*)*6cos()( tettf
Compute c3 Compute c5
Freq. (Hz)
Integrals of the function in GREEN
Integrals of the function in Red
3 5Time (s)
2009/10/9 Speaker: Li-Ming Chen 16
Outline
Periodic Phenomena and Fourier Series
Non-periodic Phenomena and Fourier Transform
Why Needs Time-Frequency Analysis?
Wavelet Transform and its Applications
2009/10/9 Speaker: Li-Ming Chen 17
Example
What if the periodic components occur at different time? Non-stationary
FT
5 10 20 50Time (ms) Frequency (Hz)
cos(2pi*5t) | cos(2pi*10t) | cos(2pi*20t) | cos(2pi*50t)
The noise is due tothe sudden change between the freq.
※ FT can still find the frequencies (4 peaks)
2009/10/9 Speaker: Li-Ming Chen 18
Why Needs Time-Frequency Analysis? Drawback of Fourier Transform
Time information is lost !! unable to tell when in time a particular event (Freq.)
took place Not a problem for signals which are stationary But when we have a signal which changes with time (non-
stationary), we need more information about the signal behavior
(Idea) can we assume that, some portion of a non-stationary signal is stationary? If so, we can do time-frequency analysis
2009/10/9 Speaker: Li-Ming Chen 19
Short Term Fourier Transform (STFT) In STFT, the signal is divided into small enough
segments, where these segments (portions) of the signal can be assumed to be stationary. Assume the signal is NOT changed for that particular
period Using window function (mask function) to cover those
periods
dtettxfXtxSTFT ift
2)()(),()}({
frequencytime
2009/10/9 Speaker: Li-Ming Chen 20
STFT
dtettxfXtxSTFT ift
2)()(),()}({
τ= t1’
τ= t2’τ= t3’
The length of this window function is pre-assigned.(the length will affect the results)
Window function could be:Rectangular function, Gaussian function, …
2009/10/9 Speaker: Li-Ming Chen 21
Example (STFT)
0~300ms: 300Hz,300~600ms: 200Hz,600~800ms: 100Hz,800~1000ms: 50Hz.
if we use FFT to analyze this signal,we might have good frequency resolution but poor time resolution.
300Hz 200Hz 100Hz 50Hz
2009/10/9 Speaker: Li-Ming Chen 22
4 Gaussian window Func. with different length
Poor freq. resolution,Good time resolution
Good freq. resolution,Poor time resolution
τf
2009/10/9 Speaker: Li-Ming Chen 23
Drawback of STFT
Heisenberg uncertainty principle ( 海森堡測不準原理 ) One can not know the exact time-frequency representati
on of a signal i.e., One can not know what spectral components exist at what i
nstances of times. What one can know are the time intervals in which certain band
of frequencies exist, which is a resolution problem.
Drawback of STFT is due to the constant length windows STFT: single-resolution for complete signal !! Need multi-resolution
2009/10/9 Speaker: Li-Ming Chen 24
Outline
Periodic Phenomena and Fourier Series
Non-periodic Phenomena and Fourier Transform
Why Needs Time-Frequency Analysis?
Wavelet Transform and its Applications
2009/10/9 Speaker: Li-Ming Chen 25
Multi-resolution Analysis (MRA) MRA is designed to give
“good time resolution and poor frequency resolution” at high frequencies
and “good frequency resolution and poor time resolution” at low frequencies.
This approach makes sense especially when the signal at hand has high frequency components for short durations and low frequency components for long durations. Common practical applications are often of this type
2009/10/9 Speaker: Li-Ming Chen 26
Continuous Wavelet Transform (CWT) 2 main differences between STFT and CWT:
1.) The Fourier transforms of the windowed signals are not taken, and therefore single peak will be seen corresponding to a sinusoid, i.e., negative frequencies are not computed.
2.) The width of the window is changed as the transform is computed for every single spectral component, which is probably the most significant characteristic of the wavelet transform.
2009/10/9 Speaker: Li-Ming Chen 27
Continuous Wavelet Transform Forward wavelet transform:
Inverse wavelet transform:
1,
t aX a b x t dt
bb
(t): mother wavelet
Scale (~ 1/freq)
Location (time, translation)
energy normalization能量守恆
,, a ba b
x t X a b t a,b(t) is dual orthogonal to (t)
output
Fourier transform X(f) or F(s), f, s: frequency (spectrum)
time-frequency analysis X(t, f), t: time, f: frequency
wavelet transform X(a, b), a: time, b: scale
2009/10/9 Speaker: Li-Ming Chen 28
Mother Wavelet, (t)
(t) is a prototype for generating the other window functions (t)(t/b) is the function with different scale derived from the moth
er wavelet b < 1, compresses the signal, (low scale detailed view) b > 1, dilates the signal, (high scale non-detailed global view)
0 10 20 300
0.5
1
0 10 20 300
0.5
1
0 10 20 300
0.5
1
0 10 20 300
0.5
1
1.5
0 10 20 300
0.2
0.4
0.6
0.8
0 10 20 300
0.2
0.4
0.6
0.8
a = 8, b=1 a = 15, b=1 a = 22, b=1
a = 8, b=0.5 a = 8, b=2 a = 8, b=3
)(1
b
at
b
能量守恆,故面積一樣
(a: 調整位置 )
(b: 調整寬度 )
2009/10/9 Speaker: Li-Ming Chen 29
Example30Hz 20Hz 10Hz 5Hz
CWT
Unlike the STFT which has a constant resolution at all times and frequencies, the WT has a good time and poor frequency resolution at high frequencies, and good frequency and poor time resolution at low frequencies
2009/10/9 Speaker: Li-Ming Chen 30
The discrete wavelet transform is very different from the continuous wavelet transform. It is simpler and more useful than the continuous one. ( 較為實用 )
x1,L[n]
x1,H[n]
g[n]
x[n]
h[n]
2
2
x[n] 的低頻成份
x[n] 的高頻成份
lowpass filter
highpass filter
down samplingxL[n]
xH[n]
1, 2Lk
x n x n k g k
1, 2Hk
x n x n k h k
down sampling
Discrete Wavelet Transform
2009/10/9 Speaker: Li-Ming Chen 31
原影像
2-D DWT
的結果
100 200 300 400 500
100
200
300
400
500
100 200 300 400 500
100
200
300
400
500
x1,L[m, n]
x1,H1[m, n]
x1,H2[m, n]
x1,H3[m, n]
Example:ImageCompression
( 低頻部分 ) (n 軸高頻 )
(m 軸高頻 )
2009/10/9 Speaker: Li-Ming Chen 3250 100 150 200 250 300 350 400 450 500
50
100
150
200
250
300
350
400
450
500
Example: Image Compression
重複對低頻部分進行 DWT
和原圖類似,但資料量僅 1/64
2009/10/9 Speaker: Li-Ming Chen 33
Example: Traffic Volume Anomaly Detection Jianning Mai, Chen-Nee Chuah Ashwin Sridharan, Tao Y
e, Hui Zang, “Is Sampled Data Sufficient for Anomaly Detection?,” IMC 2006
Discrete wavelet transform (DWT) based detection An off-line algorithm 3 steps:
Decomposition Re-synthesis Detection
2009/10/9 Speaker: Li-Ming Chen 34
(Volume Anomaly Detection)
DWT-based Detection Decomposition
Input signal X will be separate into detail information D1 or approximation information A1 (level 1)
Repeated using Ai as an input to generate Di+1 and Ai+1 at the next level
, each level j represents the strength of a particular frequency in the signal Higher value of j indicating a lower frequency
Re-synthesis Aggregate the various frequency levels into low, mid and high
bands ,
2009/10/9 Speaker: Li-Ming Chen 35
(Volume Anomaly Detection)
DWT-based Detection (cont’d) Detection
Compute local variability of the high and mid bands Compute the variance of data within a sliding window
Compute deviation score The ratio between the local variance within the window and
the global variance Windows with deviation scores higher than a threshold
are marked as volume anomalies
2009/10/9 Speaker: Li-Ming Chen 36
Reference
Wikipedia (keyword search :p) The Wavelet Tutorial (by Robi Polikar):
http://users.rowan.edu/~polikar/WAVELETS/WTtutorial.html Standford Video Course (The Fourier Transform and its Applications) (by Br
ad G. Osgood): http://academicearth.org/courses/the-fourier-transform-and-its-applicatio
ns Wavelets and Time-Frequency Analysis (by Mudasir)
http://hubpages.com/hub/wavelets1