A1 2011 1 / 1

A1 Time-Frequency Analysis

David Murray

david.murray@eng.ox.ac.ukwww.robots.ox.ac.uk/∼dwm/Courses/2TF

Hilary 2011

A1 2011 2 / 1

Course Contents

1 From Signals to Complex Fourier Series

2 From Complex Fourier Series to the Fourier Transform

3 Convolution. The impulse response and transfer functions

4 From Modulation to Sampling, Aliasing & Reconstruction

5 Power & Energy Spectra, and Correlation

6 Random Processes and Signals

A1 2011 3 / 1

Topic 5. Introduction

In Topic 1 we reviewed the notion of average signal power in aperiodic signal, and related it to An and Bn, the coefficients of aFourier series.

In this lecture we want to revisit power for the continuous timedomain, with a view to expressing it in terms of the frequencyspectrum.

First though, we should find out how to express average powerusing the complex Fourier series.

A1 2011 4 / 1

Discrete Parseval for the Complex Fourier SeriesRecall that the average power in a periodic signal with periodT = 2π/ω is

Ave power =1T

∫ +T/2

−T/2|f (t)|2 dt =

1T

∫ +T/2

−T/2f ∗(t)f (t) dt .

Now replace f (t) with its complex Fourier series

f (t) =∞∑

n=−∞Cneinωt .

It follows that

Ave power =1T

∫ +T/2

−T/2

∞∑n=−∞

Cneinωt∞∑

m=−∞C∗me−imωt dt

A1 2011 5 / 1

Discrete Parseval for the Complex Fourier Series /ctd

This can be simplified ...

Ave power =1T

∫ +T/2

−T/2

∞∑n=−∞

Cneinωt∞∑

m=−∞C∗me−imωt dt

=∞∑

n=−∞CnC∗n (because of orthogonality)

=∞∑

n=−∞|Cn|2

= |C0|2 + 2∞∑

n=1

|Cn|2 , using Cn = C∗−n.

A1 2011 6 / 1

Finite Energy signals

It is necesssary to distinguish between signals that have finiteenergy, and those that have infinite energy but finite powerFirst let us be absolutely clear that ALL signals f (t) are suchthat |f (t)|2 is a power.

An energy signal is one where the total energy is finite:

ETot =

∫ ∞−∞|f (t)|2dt 0 < ETot <∞ .

f (t) is “square integrable”.As ETot is finite, dividing by the infinite duration shows that energysignals have zero average power.Among other things, an Energy signal f (t)

always has a Fourier transform F (ω)

A1 2011 7 / 1

Finite Power signals

A power signal is one where the total energy is infinite, andinstead we consider average power

PAve = limT→∞

12T

∫ T

−T|f (t)|2dt 0 < PAve <∞ .

A Power signal f (t)may, but does not have to have a Fourier transform F (ω)

The distinction is all to do with avoiding infinities, but it results inquantities called the auto-correlation and cross-correlationhaving different dimensions.

We discuss finite energy signals first.

A1 2011 8 / 1

Parseval revisited

Let us first recall a general result from Lecture 3:

FT [f (t)g(t)] =1

2πF (ω) ∗G(ω) .

Writing out fully, this is∫ ∞−∞

f (t)g(t)e−iωt dt =1

2π

∫ ∞−∞

F (p)G(ω − p) dp .

Note that ω is not involved in the integrations above — it just afree variable on both the left and right of the above equation —and we can give it any value we wish to.

A1 2011 9 / 1

Signal energy in the continuous domain ctdChoosing ω = 0, it must be the case that∫ ∞

−∞f (t)g(t)dt =

12π

∫ ∞−∞

F (p)G(−p) dp .

Now suppose g(t) = f ∗(t). We know that∫ ∞−∞

f (t)e−iωt dt = F (ω)

⇒∫ ∞−∞

f ∗(t)e+iωt dt = F ∗(ω)

Now swap ω for −ω (not involved in integration ...)

The Fourier Transform of a complex conjugate is∫ ∞−∞

f ∗(t)e−iωt dt = F ∗(−ω)

A1 2011 10 / 1

Signal energy in the continuous domain /ctd

But in the earlier expression we had∫ ∞−∞

f (t)g(t)dt =1

2π

∫ ∞−∞

F (p)G(−p)dp

⇒∫ ∞−∞

f (t)f ∗(t)dt =1

2π

∫ ∞−∞

F (p)F ∗(p)dp

Now p is just any parameter, so it is possible to tidy theexpression by replacing it with ω.Then we arrive at the following important result

Parseval’s Theorem: The total ENERGY in a signal is

ETot =

∫ ∞−∞|f (t)|2dt =

12π

∫ ∞−∞|F (ω)|2 dω =

∫ ∞−∞|F (ω)|2 df

(NB! The df = dω/2π is a bit of frequency, and is nothing to dowith the signal being called f (t).)

A1 2011 11 / 1

The ENERGY spectral density

Reminder ...

ETot =

∫ ∞−∞|f (t)|2dt =

12π

∫ ∞−∞|F (ω)|2 dω =

∫ ∞−∞|F (ω)|2 df

So |F (ω)|2 has units of energy per Hz, and

The Energy Spectral Density of an energy signal f (t)⇔ F (ω)is

E(ω) = |F (ω)|2

If you integrate the energy spectral density over all frequency f ,you get the total energy.

This makes sense!

A1 2011 12 / 1

♣ ExampleQ: Determine the energy in the signal f (t) = u(t)e−t (i) in the time

domain, and (ii) by determining the energy spectral density andintegrating over frequency.

A: Part (i): To find the total energy in the time domain∫ ∞−∞|f (t)|2dt =

∫ ∞0

exp(−2t)dt =[

exp(−2t)−2

∣∣∣∣∞0

dt

= 0− 1−2

=12

Part (ii): In the frequency domain

F (ω) =

∫ ∞−∞

u(t)exp(−t)exp(−iωt)dt =∫ ∞

0exp(−t(1 + iω))dt

=

[−exp(−t(1 + iω))

(1 + iω)

∣∣∣∣∞0

=1

(1 + iω)

Hence the energy spectral density (ESD) is |F (ω)|2 =1

1 + ω2

A1 2011 13 / 1

♣ Example /ctd

Integration of ESD over all frequency f (not ω remember!!) givesthe total energy of∫ ∞

−∞|F (ω)|2df =

12π

∫ ∞−∞

11 + ω2 dω

Substitute tan θ = ω∫ ∞−∞|F (ω)|2df =

12π

∫ π/2

−π/2

11 + tan2 θ

sec2 θdθ

=1

2π

∫ π/2

−π/2dθ

=1

2ππ =

12

Which agrees with our earlier result.

A1 2011 14 / 1

Correlation, intro

Correlation is a tool for analysing whether processes consideredrandom a priori are in fact related.Cross-correlation Rfg indicates how similar two different signalsf (t) and g(t) are.Rfg is found by multiplying f (t) with time-shifted values g(t + τ),then summing up the products.

t

t

1 2 3 4 5 6

f(t)

g(t)

LowHigh High

Above Rfg will be low if the shift τ = 0, and high if τ = 2 or τ = 5.

A1 2011 15 / 1

Auto-Correlation

Auto-correlation Rff , describes how self-similar a signal is.Sum the products of the signal f (t) and a copy of the signal at ashifted time f (t + τ).

An auto-correlation with a high magnitude means that the valueof the signal f (t) at one instant signal has a strong bearing on thevalue at the next instant.

The cross- and auto-correlations can be derived for both finiteenergy and finite power signals, but

They have different definitions

They have different dimensions (energy and powerrespectively)

A1 2011 16 / 1

The Auto-correlation of a finite energy signal

The auto-correlation of a finite energy signal f (t)

Rff (τ) =

∫ ∞−∞

f ∗(t)f (t + τ)dt =(for real signals)

∫ ∞−∞

f (t)f (t + τ)dt

The result is an energy

then sum

f(t)

f(t )+τ

f(t)

f(t)

A1 2011 17 / 1

Properties of the Auto-correlation

1. Symmetry. The auto-correlation function is an even function

Rff (τ) = Rff (−τ) .

Proof: Substitute p = t + τ into the definition, and you will get

Rff (τ) =

∫ ∞−∞

f (t)f (t + τ)dt =∫ ∞−∞

f (p − τ)f (p)dp

=

∫ ∞−∞

f (p)f (p − τ)dp =

∫ ∞−∞

f (t)f (t − τ)dp = Rff (−τ)

2. For a non-zero signal, Rff (0) > 0.

Proof: For any non-zero signal there is at least one instant t1 forwhich f (t1)=0, and f (t1)f (t1) > 0.Hence

∫∞−∞ f (t)f (t)dt > 0.

A1 2011 18 / 1

Properties of the Auto-correlation /cdt3. The value at τ = 0 is largest: Rff (0) ≥ Rff (τ).

Proof: Consider any pair of real numbers a1 and a2. As(a1 − a2)

2 ≥ 0, we know that a21 + a2

2 ≥ a1a2 + a2a1.

Now take the pairs of numbers at random from the function f (t).Our result shows that there is no rearrangement, random or ordered,of the function values into φ(t) that would make∫

f (t)φ(t)dt >∫

f (t)2dt .

Using φ(t) = f (t + τ) is an ordered rearrangement, and so for any τ∫ ∞−∞

f (t)2dt ≥∫ ∞−∞

f (t)f (t + τ)dt

Hence Rff (0) ≥ Rff (τ).

A1 2011 19 / 1

♣ Synchronising to heartbeats in an ECG

A1 2011 20 / 1

♣ The search for Extra Terrestrial IntelligenceSETI have been looking for extra terrestrial intelligence byexamining the autocorrelation of signals from radio telescopes.

One project scans the sky around nearby (200 light years)sun-like stars chopping up the bandwidth between 1-3 GHz into2 billion channels each 1 Hz wide.

They determine the autocorrelation of each channel’s signal. Ifthe channel is noise, one would observe a very lowautocorrelation for all non-zero τ .

But if there is, say, a repeated message, one would observe aperiodic rise in the autocorrelation.

τ

increasing

A1 2011 21 / 1

The Wiener-Khinchin TheoremLet us take the Fourier transform of the cross-correlation∫

f (t)g(t + τ)dt , then switch the order of integration,

FT[∫ ∞−∞

f (t)g(t + τ)dt]

=

∫ ∞τ=−∞

∫ ∞t=−∞

f (t)g(t + τ) dt e−iωτ dτ

=

∫ ∞t=−∞

f (t)∫ ∞τ=−∞

g(t + τ) e−iωτ dτdt

But t is a constant for integration wrt τ . Substitute p = t + τ into it...

FT[∫ ∞−∞

f (t)g(t + τ)dt]

=

∫ ∞t=−∞

f (t)∫ ∞

p=−∞g(p) e−iωpe+iωt dpdt

=

∫ ∞−∞

f (t)eiωt dt∫ ∞−∞

g(p) e−iωp dp

= F ∗(ω)G(ω).

Now specialize so that g(t) = f (t) ...

A1 2011 22 / 1

The Wiener-Khinchin Theorem

We find that ...

For a finite energy signalWiener-Khinchin saysThe Fourier Transform of the Auto-Correlation isthe Energy Spectral Density

FT [Rff (τ)] = |F (ω)|2 = Eff (ω)

This statement and proof are only valid for finite energy signals

The proof rather trivializes Wiener-Khinchin. The fundamentalderivation lies in the theory of stochastic processes.

A1 2011 23 / 1

Corollary of Wiener-Khinchin

This corollary just confirms a result obtained earlier.We have just shown Rff (τ)⇔ Eff (ω)

Rff (τ) =1

2π

∫ ∞−∞Eff (ω)eiωτdω

where τ is used by convention. Now set τ = 0

The auto-correlation at τ = 0 is

Rff (0) =1

2π

∫ ∞−∞Eff (ω)dω = ETot

This is exactly as expected!

Earlier we defined the p.s.d. ETot =1

2π

∫ ∞−∞Eff (ω)dω , and we

know that for a finite energy signal Rff (0) =∫∞−∞ |f (t)|

2dt = ETot .

A1 2011 24 / 1

How is the ESD affected by passing through asystem?

If f (t) and g(t) are in the input and output of a system withtransfer function H(ω), then

G(ω) = H(ω)F (ω) .

But Eff (ω) = |F (ω)|2, and so

The ESD of the output of system H(ω) is

Egg(ω) = |H(ω)|2|F (ω)|2 = |H(ω)|2Eff (ω)

A1 2011 25 / 1

Cross-correlation

The cross-correlation describes the dependence between twodifferent signals.

Cross-correlation for finite energy signals

Rfg(τ) =

∫ ∞−∞

f (t)g(t + τ)dt

SymmetriesThe cross-correlation does not in general have a definitereflection symmetry. However, we can swap the functions,Rfg(τ) = Rgf (τ).Independent signalsIf Rfg(0) = 0, the signal f (t) and g(t) have no dependence onone another.

A1 2011 26 / 1

♣ Example[Q] Determine Rfg(τ)[A] Start by sketching g(t + τ) asfunction of t (four sketchesneeded).

1

f(t)

t

1

g(t)

t

2a 3a 4a2a

Red f (t): f = 0 then f = t/2a then f = 0Blue g(t + τ): g = 0 then g = t/a− (2 + τ/a) then g = 1 then g = 0

4a+ τ 4a+ τ 4a+ τ2a+ τ 2a+ τ 2a+ τ4a+ τ2a+ τ

t

1

0 2a

t

1

t t

0 0 02a 2a 2a

11

Left-most non-zero config valid for 0 ≤ 4a + τ ≤ a, so thatFor −4a ≤ τ ≤ −3a:

Rfg(τ) =

∫ ∞−∞

f (t)g(t + τ)dt =∫ 4a+τ

0

t2a· 1 dt =

(4a + τ)2

4a

A1 2011 27 / 1

♣ Example /ctd

4a+ τ 4a+ τ 4a+ τ2a+ τ 2a+ τ 2a+ τ4a+ τ2a+ τ

t

1

0 2a

t

1

t t

0 0 02a 2a 2a

11

For −3a ≤ τ ≤ −2a:

Rfg(τ) =

∫ ∞−∞

f (t)g(t+τ)dt =∫ 3a+τ

0

t2a·(

ta−(

2 +τ

a

))dt+

∫ 4a+τ

3a+τ

t2a·1 dt

For −2a ≤ τ ≤ −a:

Rfg(τ) =

∫ ∞−∞

f (t)g(t+τ)dt =∫ 3a+τ

2a+τ

t2a·(

ta−(

2 +τ

a

))dt+

∫ 2a

3a+τ

t2a·1 dt

For −a ≤ τ ≤ 0:

Rfg(τ) =

∫ ∞−∞

f (t)g(t + τ)dt =∫ 2a

2a+τ

t2a·(

ta−(

2 +τ

a

))dt

The integrals and finding the maximum is a DIY exercise ...

A1 2011 28 / 1

Application

Obvious that cross-correlation is useful for detecting occurencesof a “model” signal f (t) in another signal g(t).This is a 2D example where the model signal f (x , y) is the backview of a footballer, and the test signals g(x , y) are images froma match.

f (x , y) Cross-Correlation g(x , y)

A1 2011 29 / 1

W-K and Cross- Energy Spectral Density

If we looked back, we’d see that we started the finite energyderivation of Wiener-Khinchin theorem using the crosscorrelation:

FT[∫ ∞−∞

f (t)g(t + τ)dt]

= much grinding

= F ∗(ω)G(ω).

The Wiener-Khinchin Theorem shows that, for a finite energysignal,

The FT of the Cross-Correlation isthe Cross-Energy Spectral Density

FT[Rfg(τ)

]= F ∗(ω)G(ω) = Efg(ω)

A1 2011 30 / 1

Finite power signals

Let us use f (t) = sinω0t to motivate a discussion about finitepower signals.

First why is it finite power? Because one cannot evaluate∫∞−∞ | sinω0t |2dt . It is not “square-integrable”.

However, by sketching the function sinωot and using the notionof self-similarity we would hope that the auto-correlation

is positive, but decreasing, for small but increasing τ ; then negativeas the the curves are in anti-phase and dissimilar in an “organized”way, then return to being similar.

has the same period as its parent function, and large when τ = 0

Rff proportional to cos(ω0τ) would not be unreasonable?

A1 2011 31 / 1

Finite power signals /ctd

τ

increasing

A1 2011 32 / 1

Autocorrelation for Finite power signals

The autocorrelation of a finite power signal is

Rff (τ) = limT→∞

12T

∫ T

−Tf (t)f (t + τ)dt .

For this example

Rff (τ) = limT→∞

12T

∫ T

−Tsin(ω0t) sin(ω0(t + τ))dt

→1

2(T0/2)

∫ T0/2

−T0/2sin(ω0t) sin(ω0(t + τ))dt

=ω0

2π

∫ π/ω0

−π/ω0

sin(ω0t) sin(ω0(t + τ))dt

=1

2π

∫ π

−πsin(p) sin(p + ω0τ))dp

=1

2π

∫ π

−π

[sin2(p) cos(ω0τ) + sin(p) cos(p) sin(ω0τ)

]dp =

12

cos(ω0τ)

A1 2011 33 / 1

Autocorrelation for Finite power signals

For a finite energy signal, the Fourier Transform of theautocorrelation was the energy spectral density.What is the analogous result now?For our example

FT [Rff ] = FT[

12

cos(ω0τ)

]=π

2[δ(ω + ω0) + δ(ω − ω0)]

This is actually the power spectral density Sff (ω) of ourexample function sinω0t .To check, let us integrate over all frequency f :∫ ∞−∞

Sff (ω)df =

∫ ∞−∞

π

2[δ(ω + ω0) + δ(ω − ω0)] df

=

∫ ∞−∞

π

2[δ(ω + ω0) + δ(ω − ω0)]

dω2π

=14[1 + 1] =

12

A1 2011 34 / 1

Analogous definitions for finite power signals

The autocorrelation of a finite power signal is defined asRff (τ) = lim

T→∞

12T

∫ T

−Tf (t)f (t + τ)dt .

The autocorrelation and PSD are a FT PairRff (τ)⇔ Sff (ω)

We must NOT write Sff = |F (ω)|2, as F (ω) may not exist!

The average power is

PAve = Rff (0)

The PSD transfers across a system as

Sgg(ω) = |H(ω)|2Sff (ω)

A1 2011 35 / 1

Cross-correlation and power signals

Two power signals can be cross-correlated, using a similardefinition

The Cross-Correlation of a finite power signal

Rfg(τ) = limT→∞

12T

∫ T

−Tf (t)g(t + τ)dt

The crosscorrelation and Cross PSD are an FT Pair

Rfg(τ)⇔ Sfg(ω)

One very last thought. If one applies an finite power signal to asystem, it cannot be converted into a finite energy signal — orvice versa.

A1 2011 36 / 1

Summary

An energy signal f (t) has finite total energy and zero average power

ETot =

∫ ∞−∞|f (t)|2dt 0 < ETot <∞ .

An energy signal f (t)always has a Fourier transform F (ω)

always has an energy spectral density (ESD) given byEff (ω) = |F (ω)|2

always has an autocorrelation Rff (τ) =∫∞−∞ f (t)f (t + τ)dt

always has an ESD which is the FT of the autocorrelationRff (τ)⇔ Eff (ω)

always has total energy ETot = Rff (0) = 12π

∫∞−∞ Eff (ω)dω

always has an ESD which transfers as Egg(ω) = |H(ω)|2Eff (ω)

A1 2011 37 / 1

SummaryA power signal f (t) is one where the total energy is infinite, and weconsider average power

PAve = limT→∞

12T

∫ T

−T|f (t)|2dt 0 < PAve <∞ .

A power signal f (t)may have a Fourier transform F (ω), but should not rely on it.may have an power spectral density (PSD) given Sff (ω) = |F (ω)|2

always has an autocorrelationRff (τ) = limT→∞

12T

∫ T−T f (t)f (t + τ)dt

always has a PSD which is the FT of the autocorrelationRff (τ)⇔ Sff (ω)

always has integrated average power PAve = Rff (0)always has a PSD which transfers through a system asSgg(ω) = |H(ω)|2Sff (ω)