1

Transform Techniques

Transform Techniques

Mark Stamp

2

Intro

Signal can be viewed in…o Time domain usual view, raw signalo Frequency domain transformed view

Many types of transformationso Fourier transform most well-knowno Wavelet transform some advantages

Transform Techniques

3

Intro

Fourier and wavelet transforms are reversibleo From time domain representation to

frequency domain, and vice versa Fourier transform is in terms of

functions sin(nx) and cos(nx) Wavelet can use a wide variety of

different “basis” functions

Transform Techniques

4

Fourier Series

Generally, we can write f(x) in terms of series of sin(nx) and cos(nx)o Exact, but generally need infinite

serieso Finite sum usually just an

approximation Coefficients on sin(nx) and cos(nx)

tell us “how much” of that frequencyo May not be obvious from functiono Can be very useful information

Transform Techniques

5

Fourier Series

For example, consider sawtooth function: s(x) = x / π

The graph is…

Transform Techniques

6

Sawtooth Function

1 term of Fourier series 2 terms

Transform Techniques

3 terms 4 terms

5 terms

7

Fourier Transform

A function f(x) is usually viewed in the “time domain”

Transform allows us to also view it in “frequency domain”

What does this mean?o See next slide…

Why might this be useful?o Again, reveals non-obvious structure

Transform Techniques

8

Time vs Frequency

Function f(x) written as sums of functions ansin(nx) and bncos(nx)

Coefficients (amplitudes) an and bn o Tell us “how much” of each frequency

Transform Techniques

9

Time vs Frequency

Frequency domain view gives us info about the functiono More complicated the signal, less

obvious the frequency perspective may be

Transform Techniques

Time domain Frequency domain

10

Time vs Frequency Time domain in red

o Frequency domain in blue

What does blue tell us?o Dominant low

frequencyo Some high frequencies

Note that blue tells us nothing about time…o I.e., we do not know

where frequencies occur

Transform Techniques

11

Speech Example Frequency domain info used to

extract important characteristics

Transform Techniques

Time domain signal

Sonogram

12

Fourier Transform

Many different transforms exist So, why is Fourier so popular?

o Fast, efficient algorithmso Fast Fourier Transform (FFT)

Apply transform to entire function?o May not be too informative, since we

lose track of where frequencies occuro Usually, want to understand local

behavior Transform Techniques

13

Global vs Local

Function can change a lot over time…

Global frequency info not so useful Local frequency info is much betterTransform Techniques

14

Global vs Local Use Short Time Fourier Transform

(STFT) for each windowo Note that windows can overlap

Transform Techniques

15

Window Size How big should the window be?

o Small? May not have enough freq infoo Big? May not have useful time info

Transform Techniques

about righttoo smalltoo big

16

Window Size

Looks like ideal case would be windows that match frequencyo Bigger windows for low frequency

areaso Smaller windows for high frequency

The bottom line?o Too big of window gives good

frequency resolution, but poor time resolution

o Too small of window gives good time resolution, but poor frequency resolution

Transform Techniques

17

Uncertainty Principle

Cannot have accurate frequency and time resolution simultaneouslyo Form of Heisenberg Uncertainty

Principle So, this is something we must deal

witho Since it’s the law! (of physics…)

Is there any alternative to STFT?o Yes, “multiresolution analysis”

What the … ?Transform Techniques

18

Windowing Revisited Window in STFT is really a function

o Selects f(x) within current windowo “Window function” is essentially 1

within current window, 0 outside of it For wavelets, “windows” much

fanciero Like Windows 95 vs Windows 7…o Effect is to filter based on frequencieso Can mitigate some of the problems

inherent in the uncertainty principleTransform Techniques

19

Fourier Transform In Fourier transform, frequency

resolution, but no time resolution

Transform Techniques

freq

uen

cy

time

20

Short Time Fourier Trans. In STFT, time resolution via

windowing

Transform Techniques

freq

uen

cy

time

21

Wavelet Transform Time resolution based on frequency

Transform Techniques

freq

uen

cy

time

22

Wavelet

Recall that Fourier analysis is based on sin(nx) and cos(nx) functions

Wavelet analysis based on waveletso Duh!

But, what is a wavelet? o A small wave, of course…o “Wave”, so it oscillates (integrates to

0)o “Small”, meaning acts like finite

window

Transform Techniques

23

Wavelets

Many different wavelet functions to choose fromo Select a “mother” wavelet or basiso Form translations and dilations of

basis Examples include

o Haar wavelets (piecewise constant)o Daubechies waveletso …and many others

Transform Techniques

24

Haar: Translation & Dilation

Transform Techniques

10

1

-1

10

1

-1

10

1

-1

25

Advantages of Wavelets

Wavelet basis is localo Unlike Fourier basis of sine and cosineo Local, implies better time resolution

Basis functions all mutually orthogonalo Makes computations fasto Fourier basis also orthogonal, but requires

“extreme cancellation” outside windowo In effect, “windowing” built in to wavelet

basis Wavelets faster to compute than FFT

o A recursive paradise…Transform Techniques

26

Disadvantage of Wavelets Approximation with Haar

functions…o For example, sine function is trivial in

Fourier analysis, not so easy with Haar

Transform Techniques

27

Wavelets: Bottom Line

Fourier ideal wrt frequency resolutiono But sine/cosine bad wrt time

resolution Wavelets excels at time resolution

o Since basis functions finite (compact) support, and employ translation/dilation

o In effect, filters by frequency and time Complicated mathematics

o But fairly easy to implement and useTransform Techniques

28

Discrete Transforms

In practice, apply transforms to discrete time series, a0,a1,a2,… o We assume ai = f(xi) for unknown f(x)

Discrete transforms are very fasto FFT is O(n log n) o Fast wavelet transform is O(n)

Discrete transforms based on some fancy linear algebra

Transform Techniques

29

Transform Uses

Speech processingo Construct sonogram (spectrogram)o Speech recognition

Image/video processingo Remove noise, sharpen images, etc.,

etc. Compression And many, many more…

Transform Techniques

30

What About Malware Detection?

We measure some characteristic of a exe file to obtain series a0,a1,a2,…

Compute wavelet transform and…o Filter out high frequency “noise” (i.e.,

insignificant variations)o And segment file based on where the

significant changes occur Ironically, transform used to

pinpoint significant changes wrt time

Transform Techniques

31

Malware Detection Example

1. Compute entropy measurement using ai = entropy(Bi) for i = 0,1,2,…,n o Where Bi is block of i consecutive

byteso Computed on (overlapping)

“windows”o “Window” here not same as in

transform

2. Apply discrete transform to a0,a1,…

3. Find significant changes in entropy4. Use resulting sequence for scoring

Transform Techniques

32

References

R. Polikar, The wavelet tutorial A. J. Jerri, Introduction to Wavelets G. Strang,

Wavelet transforms versus Fourier transforms, Bulletin of the American Mathematical Society, 28:288-305, 1993

Transform Techniques