EE-M110 2006/7, EF L3&4 1/33, v2.0

Lectures 3&4: Non-Parametric System Identification: Impulse & Frequency Response

Dr Martin Brown

Room: E1k, Control Systems Centre

Email: martin.brown@manchester.ac.uk

Telephone: 0161 306 4672

http://www.eee.manchester.ac.uk/intranet/pg/coursematerial/

ControllerPlant

h(t), H(j)

Modelu(t)

y(t)

y(t)^

u(t)

EE-M110 2006/7, EF L3&4 2/33, v2.0

L3&4: ResourcesMain texts• Chapter 2, Ljung• Chapter 2, Norton• Chapters 1 & 2 in on-line notes.

Overview

In these two lectures, we’re looking at how LTI systems can be described in a non-parametric fashion

• Look at system impulse response h(t) • Look at frequency response H(j)

In both cases, we’ll investigate the representation, look at how you calculate the system response and consider how the system can be identified from data

This is sometimes known as classical system identification

We’re not doing an in-depth analysis for either case.

EE-M110 2006/7, EF L3&4 3/33, v2.0

Lecture 3: System Impulse Response

Non-parametric system impulse response

– Impulse response (t)→h(t)

– Properties of impulse response (causal, stable)

– Sifting property for signals

– System linearity & superposition property

– System response using convolution

– Examples of calculating the output using convolution

– Identifying the impulse response from data

NB, we’ll be doing the main part of the analysis for discrete time signals and LTI systems, but the work carries over to continuous time signals and systems as well.

EE-M110 2006/7, EF L3&4 4/33, v2.0

System Impulse Response

A very important way to analyse a system is to study the output signal when a unit impulse signal is used as an input

Loosely speaking, this corresponds to giving the system a kick at t=0, and then seeing what happens

This is so common, a specific notation, h(t), is used to denote the output signal, rather than the more general y(t)

This impulse response signal can be used to infer properties about the system’s structure (LHS of difference equation or unforced solution)

The system impulse response, h(t) completely characterises a linear, time invariant system

Systemh(t)

y(t)

h(t)

x(t)

(t)

EE-M110 2006/7, EF L3&4 5/33, v2.0

Properties of System Impulse Response

StableA system is stable if the impulse response is absolutely summable

CausalA system is causal if

h(t)=0 when t<0

Finite/infinite impulse responseThe system has a finite impulse response and hence no dynamics in y(t) if there exists T>0, such that:

h(t)=0 when t>T

Lineara(t) ah(t)

Time invariant(t-T) h(t-T)

t

th )(

h(t)

t0

EE-M110 2006/7, EF L3&4 6/33, v2.0

System Impulse Response Examples

Looking at system impulse responses, allows you to determine certain system properties

Causal, stable, finite impulse responsey(t) = x(t) + 0.5x(t-1) + 0.25x(t-2)

Causal, stable, infinite impulse responsey(t) = x(t) + 0.7y(t-1)

Causal, unstable, infinite impulse responsey(t) = x(t) + 1.3y(t-1)

h(t) h(t)

h(t)

t

tt

EE-M110 2006/7, EF L3&4 7/33, v2.0

Introduction to ConvolutionDefinition Convolution is an operator that takes an input

signal and returns an output signal, based on knowledge about the system’s impulse response h(t).

The basic idea behind convolution is to use the system’s response to simple input signals to calculate the response to more complex input signals

This is possible for LTI systems because they possess the superposition property

LTI system y(t) = h(t)x(t) = (t)

LTI system h(t)

y(t)x(t)

k kk txatxatxatxatx )()()()()( 332211

k kk tyatyatyatyaty )()()()()( 332211

EE-M110 2006/7, EF L3&4 8/33, v2.0

Sifting Property

Basic idea: use a (infinite) set of of DT impulses to represent any DT signal.

Consider any discrete input signal x(t). This can be written as the linear sum of a set of unit impulse signals:

Therefore, the signal can be expressed as:

In general, any discrete signal can be represented as:

k

ktkxtx )()()(

10

1)1()1()1(

00

0)0()()0(

10

1)1()1()1(

t

txtx

t

txtx

t

txtx

)1()1( tx

actual value Impulse, time shifted signal

The sifting property

)1()1()()0()1()1()2()2()( txtxtxtxtx

EE-M110 2006/7, EF L3&4 9/33, v2.0

Example: DT Signal Sifting

The discrete signal x(t)

Is decomposed into the following additive components… + x(-4)(t+4) +

x(-3)(t+3) + x(-2)(t+2) + x(-1)(t+1) + …

EE-M110 2006/7, EF L3&4 10/33, v2.0

System Linearity

Let y1(t) and y2(t) be the responses to x1(t) and x2(t), respectivelyA linear system must satisfy the two properties:

1 Additive: the response to x1(t)+x2(t) is y1(t) + y2(t)

2 Scaling: the response to ax1(t) is ay1(t) where aC

Combined: ax1(t)+bx2(t) ay1(t) + by2(t)

Examples

1) y(t) = 3x(t). Consider x(t) = ax1(t)+bx2(t),

y(t) = 3(ax1(t)+bx2(t)) = a3x1(t) + b3x2(t) = ay1(t) + by2(t). The system is linear

2) y(t) = 3x(t)+2. Consider x(t) = 2x1(t) and use the scaling property

y(t) = 3*2*x1(t)+2 = 6x1(t)+2 2y1(t). The system is not linear

3) y(t) = 3*x2(t). Consider x(t) = 2x1(t) and use the scaling property

y(t) = 3*(2x1(t))2 = 12x1(t)2 2y1(t). The system is not linear

EE-M110 2006/7, EF L3&4 11/33, v2.0

Linear Systems and SuperpositionGeneralizing the scaling/additive properties, suppose an

input signal x(t) is made of a linear sum of other (basis) signals xk(t):

then the response of a linear system is (where xk(t)yk(t)):

The basic idea is that if we understand how simple signals get affected by the system, we can work out how complex signals are processed, by expanding them as a linear sum

This is known as the superposition property which is true for linear systems in both CT & DT

Important for understanding convolution

k kk txatxatxatxatx )()()()()( 332211

k kk tyatyatyatyaty )()()()()( 332211

EE-M110 2006/7, EF L3&4 12/33, v2.0

Convolution Sum for LTI Systems

As the system is time invariant, the response to the impulse signal (t-k) is h(t-k).

Then from the superposition property of linear systems, the system’s response to a more general input signal x(t) can be derived as follows.

Input signal

System output signal is given by the convolution sum

i.e. it is the scaled sum of time shifted impulse responses.

k

ktkxtx )()()(

k

kthkxty )()()(

EE-M110 2006/7, EF L3&4 13/33, v2.0

For any LTI discrete time system, the response to an input signal x(t) is given as follows.

Using the sifting property:

h(t) is the system impulse response to (t).

Because the system is time invariant,

Using the superposition property (linear):

The system response of any LTI system can be calculated as a linear combination of impulse responses & written as

Convolution in a Nut-Shell

k

ktkxtx )()()(

k

kthkxty )()()(

)(*)()( thtxty

)()( kthkt

h(t)

y(t)

x(t)

t

t

t

EE-M110 2006/7, EF L3&4 14/33, v2.0

Interpreting the Convolution Sum

Note that convolution can be interpreted in two ways:

1. As a sum of scaled, shifted impulse response signals

2. As an algebraic formula which is a function of t

Method 1) works well when h(t) or x(t) has finite duration, but method 2) is more flexible and more widely used. See following examples

t

k

k

k

kthkxty

0

)3/1(

)()()(

h(t)x(t) y(t)

ttt

EE-M110 2006/7, EF L3&4 15/33, v2.0

Example 1: LTI ConvolutionCalculate the DT, LTI system response when:

h(t) = [0 0 1 1 1 0 0],

x(t) = [0 0 0.5 2 0 0 0]

Then:

h(t) = u(t)u(2-t)

h(t-k) = u(t-k)u(2-t+k)

and the convolution sum:

t

tk

t

k

kx

ktuktukx

kthkxty

2

)(

)2()()(

)()()(

30

32

25.2

15.2

05.0

00

)(

t

t

t

t

t

t

ty

u(t)u(2-t)

0 2 t

Very common trick!

EE-M110 2006/7, EF L3&4 16/33, v2.0

Example 2: LTI ConvolutionConsider a DT LTI system that has a

step response h(t) = u(t) to the unit impulse input signal (integrator)

What is the response when an input signal of the form

x(t) = tu(t)

where 0<<1, is applied?

)(1

1

01

100

)()()(

1

1

0

tu

t

t

ktukuty

t

t

t

k

k

k

k

h(t)

x(t)

y(t)

t

t

t

EE-M110 2006/7, EF L3&4 17/33, v2.0

Example 3: LTI ConvolutionFind the system response when:

x(t) = (1/2)tu(t+1)

h(t) = (1/3)tu(t-1)

Using DT, LTI convolution

h(t)

x(t)

y(t)

t

t

t

)(3/21)2/3(2)3/1(

03/21)2/3(2)3/1(

00

)2/3()3/1(

)1()1()2/1()3/1()3/1(

)1()3/1)(1()2/1()(

1

1

tu

t

t

ktuku

ktukuty

tt

tt

t

k

kt

k

kkt

k

ktk

EE-M110 2006/7, EF L3&4 18/33, v2.0

System Identification and PredictionNote that the system’s response to an arbitrary input signal is

completely determined by its response to the unit impulse.

Therefore, if we need to identify a particular LTI system, we can apply a unit impulse signal, (t), and measure the system’s response, h(t).

That data can then be used to predict the system’s response to any input signal

How to:

• Design impulse signal

• Collect the data (number of experiments and length)

• Make sure assumptions are satisfied (linearity)

System: h(t)y(t)x(t)

n(t)~N(0,2)

EE-M110 2006/7, EF L3&4 19/33, v2.0

How to Identify the Impulse Response

Impulse signal. Signal should be of large magnitude and small duration (unit area). How to reproduce with a physical device. NB, the system is generally continuous time

Data collection. The number of impulse tests should be large enough to reduce mean estimate, affected by measurement noise, to acceptable accuracy. The length of the signal should be large enough compared to the input signal length (FIR/IIR system). The sample period should be short enough to capture high-order dynamics (Nyquist rate)

Input amplitude. The input signal should be scaled to make the impulse response large compared to the magnitude of the noise signal. However, it should not be too large, otherwise the system may be non-linear (systems are generally only locally linear)

NB could consider step response data instead

u(t)

th(t)

t

xx

xxx xxx

xxx

xxx

xxx

xxx

xxx

xxx

()=/n0.5

ah(t)

txx

xxx xxx xx

xxxx

xxx

xxx

xxx

xxx

a<<1

EE-M110 2006/7, EF L3&4 20/33, v2.0

Lecture 4: System Frequency Response

Non-parametric system frequency response

• System eigenfunctions

• System transfer functions (stable systems)

• Sinusoidal basis functions and Fourier transforms

• Convolution in the frequency domain

• Examples of calculating the output

• Bode plots

• Identifying the frequency response curve

NB, we’ll be doing the main part of the analysis for continuous time signals and LTI systems, but the work carries over to discrete time signals and systems as well.

EE-M110 2006/7, EF L3&4 21/33, v2.0

What is a System Eigenfunction?Let’s imagine what (basis) signals k(t) have the property that:

i.e. the output signal is the same as the input signal, multiplied by the constant “gain” k (which may be complex)

For CT LTI systems, we also have that

Therefore, to make use of this theory we need:1) system identification is determined by finding {k,k}.

2) prediction, we also have to decompose x(t) in terms of k(t) by calculating the coefficients {ak}.

The {k,k} are analogous to eigenvectors/eigenvalues matrix decomposition

Systemx(t) = k(t) y(t) = kk(t)

LTISystem

x(t) = k akk(t) y(t) = k akkk(t)

EE-M110 2006/7, EF L3&4 22/33, v2.0

Sinusoidal Signals & LTI SystemsImaginary exponential (sinusoidal) signals and

stable LTI systems are very closely related because:

The signal is unchanged by any LTI system apart from a magnitude gain and a phase shift (multiply by complex number )

This means that by superposition, to calculate the system’s output, we can find the frequency content of each signal and sum the scaled responses, where k(t)=ejk0t

Sinusoidal signals are so-called eigenfunctions of LTI systems, i.e. they remain unchanged by the system apart from a gain and a phase shift

LTI systemx(t) = ejt y(t) = ()ejt

LTI systemx(t) = k akk(t) y(t) = k akkk(t)

x(t)y(t)

t

EE-M110 2006/7, EF L3&4 23/33, v2.0

Proof of Eigenfunction Property

Consider a CT LTI system with impulse response h(t) and input signal x(t)=(t) = est, for any value of sC:

Assuming that the integral on the right hand side converges to H(s) (transfer function), this becomes (for any sC):

Therefore (t)=est is an eigenfunction, with eigenvalue =H(s). When s=j, this represents frequency response data, and the system must be stable (h(t) tends to zero)

dehe

deeh

deh

dtxhty

sst

sst

ts

)(

)(

)(

)()()(

)(

stesHty )()(

dehsH s)()(

Very important for Fourier/Laplace

transforms

EE-M110 2006/7, EF L3&4 24/33, v2.0

Fourier SeriesFourier series theory says that nearly all continuous time,

periodic signal, x(t), can be represented as complex sums of harmonic (sinusoidal) basis signals

Examples (0=1, T=2)

k

kk

tjkk tkjtkaeatx ))sin()(cos()( 00

0

x(t) = sin(t)+0.2*sin(7*t)

10

1

1 /)sin()1()(k

k kkttxktk

tktxk

k

/))))12(cos(

))12(cos(()1()(5

0

0 is the fundamental frequency

EE-M110 2006/7, EF L3&4 25/33, v2.0

Fourier Transform

When x(t) is not periodic we can calculate the Fourier transform as

This is a continuous spectrum of frequenciesExample

NB the Fourier transform of periodic exists and it is a sum of delta signals, centred at the harmonic frequencies

)}({)()( 121 jXFdejXtx tj

)}({)()( txFdtetxjX tj

x(t)

tT1-T1

EE-M110 2006/7, EF L3&4 26/33, v2.0

System Prediction using Frequency Response

Taking Fourier transforms of the input and impulse response signals allows us to derive the frequency domain convolution equation

Y(j) = H(j)X(j)

This can then be inverted to obtain the time-domain system response, y(t) to a input signal x(t):

1. Calculate Fourier transform of input x(t)

2. Calculate Fourier transform of impulse response h(t)

3. Multiply together to obtain Y(j)

4. Invert the Fourier transform (generally by expressing as partial fractions)

EE-M110 2006/7, EF L3&4 27/33, v2.0

Example 1: Solving a First Order ODE

Consider the response of an LTI system with impulse response:

to the input signal:

Transforming these signals into the frequency domain:

and the frequency response:

to convert this to the time domain, express as partial fractions:

Therefore, the time domain response is:

0)()( btueth bt

0)()( atuetx at

jajX

jbjH

1)(,

1)(

))((

1)(

jajbjY

)(

1

)(

11)(

jbjaabjY ba

)()()( 1 tuetuety btatab

Try calculating using

convolution

EE-M110 2006/7, EF L3&4 28/33, v2.0

Lets design an ideal low pass filter in frequency domain:

The impulse response of this filter is the inverse Fourier transform

which is an ideal low pass filter in time domain– Non-causal, so this cannot be manufactured exactly– The time-domain oscillations may be undesirable

How to approximate the frequency selection characteristics?

Consider the 1st order LTI system impulse response:

Causal and non-oscillatory time domain response and performs a degree of low pass filtering. Higher order filters (ODEs) are usually used.

Example 2: Designing a Low Pass Filter

c

cjH

||0

||1)(

H(j)

cc

t

tdeth ctjc

c

)sin()( 2

1

jatue

Fat

1)(

h(t)

t0

)()()(1 txtyt

tya

EE-M110 2006/7, EF L3&4 29/33, v2.0

Bode PlotsA Bode Plot for a system is simply plots of log magnitude

and phase against log frequencyBoth the log magnitude and phase effects are now additiveWidely used for analysis and design of filters and

controllers

Example

Low pass, unity filter

Log mag v log freq Phase v log freq

|)(|log jH )( jH

log log

EE-M110 2006/7, EF L3&4 30/33, v2.0

Example 1: Bode Plot 1st Order SystemConsider a LTI first order system

described by:

Fourier transfer function is:

the impulse response is:

and the step response is:

Bode diagrams are shown as log/log plots on the x and y axis with =2.

0),()()(

txtydt

tdy

1

1)(

jjH

)()( /1 tueth t

)()1()(*)()( /1 tuetuthty t

|)(|log jH

)( jH

log

EE-M110 2006/7, EF L3&4 31/33, v2.0

Estimating Frequency ResponseUse sin() wave testing to estimate H(j) over the whole

range of frequencies necessary for prediction

Y(j) = H(j)X(j)

If x(t) only contains low frequencies, it is not necessary to estimate H(j) for high frequency components as the product is zero whatever the value

With a bit of work, we can show that the system response to x(t) = sin(t) where H(j)=A+jB is

y(t)=(A2+B2)0.5sin(t+tan-1(B/A))

where– gain=(A2+B2)0.5

– phase advance=tan-1(B/A)

These can be estimated from steady state response

EE-M110 2006/7, EF L3&4 32/33, v2.0

L3&4 Summary

These lectures have concentrated on “conventional” system identification methods that can be applied to LTI systems. They are non-parametric and make few assumptions about the order of the underlying ODEs/difference equations.

Impulse response• A key concept for analyzing system properties and can be

easily identified. System response is calculated using convolution

Frequency response• Derived as the Fourier transform of the impulse response

signal. Provides information about the gain and phase of the system for different input frequencies.

The introduction provided on this course has been very basic and glosses over many practical details.

EE-M110 2006/7, EF L3&4 33/33, v2.0

L3&4 LaboratoryTheory1. Using convolution, calculate the DT response for h(t)=u(t-2),

x(t)=0.5tu(t). Sketch all signals.2. Using convolution, calculate the DT response for h(t)=u(t), x(t)=2tu(-t).

Sketch all signals.3. Using Fourier transforms, calculate the CT response for h(t)=e-3(t-1)u(t-1)

and x(t)=e-(t+1)u(t+1). Sketch all signals.Matlab and SimulinkChange to your P: directory and turn your diary on!4. Use the Matlab conv() function to verify the DT convolution in

questions (1) and (2). Plot all the signals.5. Use the fourier() and ifourier() functions in the symbolic toolbox

to verify the answer to question (3).6. By evaluating the equivalent transfer functions for each of the impulse

responses in questions (1), (2) and (3), simulate the signals and systems in Simulink and verify that all the answers are the same.

7. Simulate the frequency response of a range of first and second order systems in Simulink for a range of different frequencies

EE-M110 2006/7, EF L3&4 34/33, v2.0

Appendix A: DT Eigenfunctions/values

In an analogous fashion, we can show the same for discrete-time systems, so consider a DT system with DT impulse response h(t) and input x(t).

Again, we have assumed that the (infinite) summation on the right hand side converges to H(z).

Therefore x(t)=(t)=zt is a DT eigenfunction with eigenvalue =H(z).

t

k

kt

k

kt

k

zzH

zkhz

zkh

ktxkhty

)(

)(

)(

)()()(

k

kzkhzH )()(

Very important for z-transforms

EE-M110 2006/7, EF L3&4 35/33, v2.0

Appendix B: FT of Exponential Signal Consider the (non-periodic) signal (solution to first order ODE)

Then the Fourier transform is:

0)()( atuetx at

)/(tan

222222

0

)(

0

)(

1

)(

1

)(

)(

)(

1

)(

1

)(

1

)()(

aj

tja

tjatjat

eaa

ja

aja

eja

dtedtetuejX

a = 1

If x(t) is a real signal 1) X(0) is real, 2) Im(X(-j)) = -Im(X(-j))

x(t)