time frequency analysis_journey

We value our relationshipWe value our relationships.

7 January 2016

© 2014We value our relationship7 January 2016

Chandrashekhar Padole

Title for Presentation Journey from

Vector Algebra to Wavelet Transform

(Time-Frequency Analysis)

@CopyrightPresenter: Chandra7 January 2016 2

Objective

Topics to be covered (Various Transforms):

• Vector Algebra

•Discrete Fourier Transform

•KL transform

•PCA

•Wavelet Transform

•Wigner Distribution


Applications

•Pattern Recognition Problems

• Biometrics: Face, Fingerprint, IRIS etc

• Automotive: Lane , Vehicle, Pedestrian, Sleeping Pattern, Signal

etc

• Manufacturing: object, documents etc

• GIS, Healthcare, Military and so on

•Analysis-Synthesis (Graphic Equalizer, noise removal,

image restoration etc)

•Data-Mining (content based image retrieval, audio mining

based on emotions etc)

•Transmission and Compression of data


Agenda

• Vector Algebra

•QA

•Fourier Transform

•QA

•K-L Transform

•PCA

•QA


•QA


•QA


Take-off for Vector Algebra


Preliminaries in Vector Algebra

•Vector P=3 i + 4 j

•P quantity to be analyzed or represented as P

•Projection or mapping of P onto unit vectors i and j will be

3 and 4 resp.

•Measurement of projection is obtained by using

mathematical tool or operator.

•In this case, operator is dot

product (inner product)


Contd..

• e.g. P.i=[ 3i+4j] . [ i]=[3i+4j].[1i+0j]=3

•And P.j=[ 3i+4j] . [ j]=[3i+4j].[0i+1j]=4

•Significance of i and j-

•In 2D space problem

• i is the vector which represents horizontal property

(feature/dimension/axis),

• j is for vertical property

• k- depth property in 3D


Let’s Look at some Real-life Problems

Two Cupboards


Contd..

Chemist


Contd..

Carpenter


Rewind

• P=3i + 4j

•Px = P. i= 3

•Py= P . j= 4

•Operator we used for measurement of projection is dot

product or inner product

•[ 3 4 2][2 1 2]T = 3.2 + 4.1+ 2.2=6 +4+4= 14

•It is also a correlation (zero-shift) between two vectors

• P � properties� component value/coefficient

•It’s a analysis process


Contd..

•To synthesize combine component values along with

property vectors to get original P

i.e. combine ( 3 and 4) as 3i+4j= P

•What we had till now , is 2D problem. Same terminology and

process can be extended for 3 D space problem….


3D Space Problem

• P= ai+bj+ck

• analysis : find { a,b,c}

• synthesis : combine a,b,c

•In 2D space problem – 2 analysis vectors

•In 3D space problem – 3 analysis vectors


Investigation in A-S Problem

• Can there be more than 3 analysis vectors ..?

if so , give examples..

•Can each analysis vector be multi valued ( as opposed to

double/triple valued vector in 2D /3D analysis) ?

•If both answers are positive, can vector algebra analysis

theory be extended to those problems?


Stop for Queries


Replacing VA by DFT

•Use of DFT – frequency analysis

•Why it is needed?

• to measure/analyze the variation of physical quantity

(such as pressure, intensity, voltage , current etc) with

respect to one or more independent variables

•Representation of physical quantity with respect to

independent variables is called as function or signal

•Variation need not to be periodic and mostly it is aperiodic

in real life problems

• For aperiodic DFT will analyze variations in signal


Take-off for Vector Algebra


Fourier Idea


Frequency Analysis

• Definition of DFT

• Computations steps in DFT

• Its physical significance

•Definition of DFT

for N-point DFT

∑−

=

−=

1

0

2

)()(N

n

N

nkj

enxkX

π


Computations in DFT

• Signal to be analyzed is 64 points in length

x(n) �

analog discrete

0 10 20 30 40 50 60 70-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0 10 20 30 40 50 60 70-0 .5

-0 .4

-0 .3

-0 .2

-0 .1

0

0 .1

0 .2

0 .3

0 .4

0 .5


K=0

)/2sin()/2cos(

2

NnkjNnke N

nkj

πππ

+=−

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plit

ude

n

0 10 20 30 40 50 60 700

0.5

1COSINE SEQUENCE

Am

plitu

de

n


K=1

)/2sin()/2cos(

2

NnkjNnke N

nkj

πππ

+=−

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plit

ude

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1COSINE SEQUENCE

Am

plit

ude

n


K=2

)/2sin()/2cos(

2

NnkjNnke N

nkj

πππ

+=−

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plit

ude

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1COSINE SEQUENCE

Am

plit

ude

n


K=3

)/2sin()/2cos(

2

NnkjNnke N

nkj

πππ

+=−

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plit

ude

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1COSINE SEQUENCE

Am

plit

ude

n


K=31

)/2sin()/2cos(

2

NnkjNnke N

nkj

πππ

+=−

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plit

ude

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1COSINE SEQUENCE

Am

plitu

de

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plitu

de

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1COSINE SEQUENCE

Am

plit

ude

n


DFT- Overall Physical Significance

0 1 0 2 0 3 0 4 0 5 0 6 0 7 0-0 . 5

-0 . 4

-0 . 3

-0 . 2

-0 . 1

0

0 . 1

0 . 2

0 . 3

0 . 4

0 . 5

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plitu

de

n

0 10 20 30 40 50 60 700

0.5

1

COSINE SEQUENCE

Am

plitu

de

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1

SINE SEQUENCE

Am

plitu

de

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1COSINE SEQUENCE

Am

plit

ude

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plitu

de

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1

COSINE SEQUENCE

Am

plitu

de

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plitu

de

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1COSINE SEQUENCE

Am

plit

ud

e

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1

SINE SEQUENCE

Am

plitu

de

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1COSINE SEQUENCE

Am

plit

ud

e

n

a1+jb1

a5+jb5

Dot

Product

a2+jb2

a3+jb3

a4+jb4

X(1)

X(31)

X(2)

X(3)

X(4)

∑−

=

−=

1

0

2

)()(N

n

N

nkj

enxkX

π


DFT- Phase Information

clear all;

close all;

N=64;

t=0:N-1;

f=1;

ws=sin(2*pi*f*t/N);

figure,

subplot(2,1,1)

stem(ws);

wc=cos(2*pi*f*t/N);

subplot(2,1,2)

stem(wc);

%ps=sin(2*pi*f*t/N);

%ps=sin(2*pi*f*t/N + (pi/8));


ps=sin(2*pi*f*t/N + (3*pi/8));


figure,

stem(ps);

a=sum(ps.*ws);

b=sum(ps.*wc);

ph_ang=atan(b/a);


Contd..

0 10 20 30 40 50 60 70-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1SINE SEQUENCE

Am

plit

ude

n

0 10 20 30 40 50 60 70-1

-0.5

0

0.5

1COSINE SEQUENCE

Am

plit

ude

n

Dot

Product

a=32

b= 0

ph_ang=0

a=0

b= 32

ph_ang=pi/2

a= 22.67

b= 22.67

ph_ang=pi/4

0 10 20 30 40 50 60 70-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50 60 70-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

ph=0

ph=3*pi/8

ph=pi/2

0 10 20 30 40 50 60 70-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

ph=pi/8

a= 29.56

b= 12.24

ph_ang=pi/8

0 10 20 30 40 50 60 70-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

ph=pi/4

a= 12.24

b= 29.56

ph_ang=pi/4


Relation bet VA and DFT

Vector algebra -2/3D space DFT

No of analysis vectors 2/3 N

No of elements in each

vector

2/3 N

Projection measurement

method

Inner Product Inner Product


Fourier Idea

Jean Baptiste Joseph

Fourier (1768 - 1830).

Fourier was a French

mathematician, who was

taught by Lagrange and

Laplace


Application –Example

Music Equalizer

Audio file recorded

with studio settingN-point FFT

N-point

User settingModified FFT

IFFT


Stop for Queries and Discussion


New terminology:

•quantity to be analyzed: signal /function

•Analysis vectors : basis vectors

• Thus, collection of basis vectors forms basis set

•Properties of basis set

• Completeness

• Orthogonal

• orthonormal


Completeness

• e.g. we have a function P=3i+4j+2k

• while analyzing we used only two basis vectors , i and k;

• using projection over these two vectors will not give

proper reconstruction

• But having so , we can achieve dimensionality reduction

which useful in data compression

•Dimensionality extension is possible?

XOR Problem


Orthogonal

•Information carried by one basis function should not be included in any other basis function from basis set

that’s a orthogonality

•Mathematically, if i.j=0, then I and j are orthogonal .

• In Fourier transform , and are orthogonal if

•In Fourier transform within basis function , two sub basis function ,sin and cosine, are also orthogonal and used for determining the phase of that frequency

N

nkj

e

π2

N

mkj

e

π2 nm ≠

∑∞

−∞=

=n

wmwn 0sinsin ∑∞

−∞=

=n

wnwn 0cossin


2D Fourier Transform

2D FT basis images for 8x8


Thank you

Queries?


Objective

Topics to be covered (Various Transforms):

• Vector Algebra( part I)

•Discrete Fourier Transform( part I)

•KL transform (part II)

•PCA ( part II)

•Wavelet Transform( part II)



Take-off for KL-Transform


Agenda

• Vector Algebra

•QA

•Fourier Transform

•QA

•K-L Transform

•PCA

•QA


•QA


•QA


KL Transform

• Basis vectors adopt to the data

•Powerful tool when reducing the dimensionality

•de-correlates the data => less redundancy

•Key idea: Represent the data in a more compact manner

•Used for PCA (to be discussed later)

•Thus , it provides the good and compact representation in

transformed domain

•Basis functions are derived from data itself


Contd..

Procedure to obtain KL transform basis functions

•Collect data

• subtract mean from the data (optional)

•Organize the data in matrix

• If sources are M and elements/dimensions in each

source data are N

• Then place a data in NxM matrix

•Calculate covariance of the matrix to get NxN matrix

•Calculate eigen vectors and eigen values

• N eigen vector and each vector will have N elements

• N eigen values corresponding to each eigen vector


PCA –Example


Contd..


Contd..

calculating covariance

Calculating eigen vectors and values


Contd..


Contd..

Reconstruction/restoration


Contd..

Matlab function used in PCA

• cov

• eig


Stop for Queries and Application

@CopyrightPresenter: Chandra

1st Person

7 January 2016 50

PCA- Face Recognition

1 2 7

MxNAveraging

2D-> 1D

Matrix

MNx40

2nd Person

1 2 7

40th Person

1 2 3

MNxMNCov Eig

MNxMN

MN

X

1

KL Transform

(Eigen Vectors)

Eigen Values

PCA1D-2D


WAVELET TRANSFORM


Need of WT

• Gives good resolution in time as well as frequency domain.

•It also gives locations of different frequency spectral

components during that particular instant of time.

•This is the main advantage of wavelet Transform over FT &

STFT.

•WT is used to mainly analyze non stationary signals, i.e.,

whose frequency response varies in time.


Take-off for Wavelet Transform


What is wavelet ?

• The term wavelet means a small wave , i.e. a window

function of finite length.

• It a oscillatory in which high frequency components exist

only for a short duration of time and low frequency exist

throughout the signal.

• The main feature of wavelet is that it can be of finite or

infinite duration but most of the energy of wavelet is

confined to a particular interval of time thus making it time

limited window function.


Properties of Wavelet

For any function to be a wavelet function must satisfy

following properties:

1.The function integrates to zero:

∞-∞∫Ψ(t) dt = 0.

2.It is square integrable or, equivalently has finite energy:

∞-∞∫Ψ(t)2 dt < ∞.

3.The admissibility condition:

∞C ≡ -∞∫ (Ψ(ω)2) / (ω) dω

Such that 0<C<∞.


Contd..

• Property 1 is suggestive of a function that is oscillatory or

that has a wavy appearance. Thus, in contrast to a sinusoidal

function, it is a “small wave” or a wavelet.

•Property 2 implies that most of the energy inΨ(t) is

confined to a finite duration.

•Property 3 is known as admissibility condition and is

sufficient condition that leads to the Inverse CWT but it is

not a necessary condition to obtain a mapping from the set

of CWTs back to L2R.

These properties are easily satisfied and there are an infinite

number of functions that qualify as mother wavelets.


Some Mother Wavelets

Mother wavelet: It’s a basic wavelet function at t=0 without

performing any scaling and shifting operation i.e.Ψ(t).

Morlet wavelet Haar

DB4


Definition of Wavelet Transform

∫∞

∞

−= dt

a

bt

atfbaW )(

1)(),( ψ

Where,

x(t) = Input signal which is to be

transformed

Ψ(t) = Mother wavelet

b = Time shift parameter

a = Scaling parameter

1/sqrt(a)=normalizing factor which ensures

that energy of remains constant for all values of

a & b.


MRA

Signal to be analyzed


Contd..


Chirp function


Time-Frequency Resolution


DWT Calculation

Mallat Algorithm:


Stop for Calculation of WT


2D DWT

Music Equalizer


Matlab Code

[im1,n]=imread( ‘lena.jpg‘);

im=double(im1);

wname='haar';

[A_1,H_1,V_1,D_1] = DWT2(im,wname);

[A_2,H_2,V_2,D_2] = DWT2(A_1,wname);

[A_3,H_3,V_3,D_3] = DWT2(A_2,wname);

[A_4,H_4,V_4,D_4] = DWT2(A_3,wname);


Thank you

Queries?

time frequency analysis_journey

Data & Analytics