1

Model Structures

2

Objective

Recognize some discrete-time model structures which are commonly used in system identification such as ARX, FIR, ARMAX, OE.

3

ARX model structure

In ARX model, the output y(k) is computed based on previous input and output values:

This equivalent to

Where e(k) is the noise at step k.Model parameters: {a1, a2, …, ana} and {b1, b2, …, bnb}.

)()()1()()1()( 11 kenbkubkubnakyakyaky nbna

)()()1()()1()(input eXogenous

1

outputs) (previous siveAutoregRes

1 kenbkubkubnakyakyaky nbna

4

Polynomial representation

Defining the Backward shift operator q−1:

Then

and similarly

)()(:

)()()()1(1

111

kuqB

kuqbqbnbkubkub nbnbnb

)1()(1 kykyq

)()(:

)()1()()1()(1

111

kyqA

kyqaqanakyakyaky nanana

5

ARX model in polynomial form• Therefore, the ARX model is written compactly:

• Which has the following symbolic representation:

• Remark: The ARX model is quite general, it can describe arbitrary linear relationships between inputs and outputs. However, the noise enters the model in a restricted way, and later we introduce models that generalize this.

)()()()()( 11 kekuqBkyqA

6

• Returning to the explicit recursive representation:

• So in fact ARX model obeys the standard model structure in linear regression!

• Regressor vector: ϕ ∈ Rna+nb , previous output and input values.• Parameter vector: θ ∈ Rna+nb , polynomial coefficients.

Linear Regression Model

)()(

)(].[

)]()1()()1([)()()1()()1()(

11

11

kek

kebbaa

nbkukunakykykenbkubkubnakyakyaky

T

Tnbna

nbna

7

• Consider now that we are given a vector of data u(k), y(k), k = 1, . . ., N, and we have to find the model parameters θ.

• For any k, we have:

• where ε(k) is now interpreted as an equation error.• Objective: minimize the mean squared error:

Identification Problem

)()()( kkky T

N

k

kN

V1

2 )(1)(

8

We can write the following set of linear equations in matrix from:

with notations Y R∈ N and Φ R∈ N×(na+nb).

YnbNuNunaNyNy

nbnaunauynaynbnaunauynay

Ny

naynay

.

)()1()()1(

)2()1()2()1()1()()1()(

)(

)2()1(

Linear System of Equations

9

• As we know, the least squares solution is given by

• Note that the form above is impractical in system identification, since the number of data points N can be very large. Better form:

YTT 1)(̂

ARX Solution

N

k

N

k

T

N

k

TN

k

TT

kykkk

kykYkk

1

1

1

11

)()()()(ˆ

)()(),()(

10

Consider we are given the following, separate, identification and validation data sets.

plot(id); and plot(val);

Remarks: Identification input: a PRBS signal, an approximation of (non-zero-mean) white noise. Validation input: a sequence of steps.

Example

11

Example, continuedIdentifying an ARX model:

model = arx(id, [na, nb, nk]);Arguments:• Identification data of type iddata format.• Array containing the orders of A and B and the minimum delay nk

between inputs and outputs (useful for systems with time delays).

)()1()()()1()(

)(

,1)(

where),()()()()(

11

1121

1

22

11

1

11

kenbnkkubnkkubnakyakyaky

qbqbbqB

qaqaqaqA

kenkkuqBkyqA

nbna

nbnb

nana

12

Example (continued): Model Validation

• Let us identify a second-order ARX model with:na = 2, nb = 1, nk = 1. • For validation: compare(model, val);

)(mean

ˆ1100fit

yyyy

13

Example (continued): Structure selection

Better idea: try many different structures and choose the best one.

Na = 1:15;Nb = 1:15;Nk = 1:5;NN = struc(Na, Nb, Nk);V = arxstruc(id, val, NN);

• struc generates all combinations of orders in Na, Nb, Nk.• arxstruc identifies for each combination an ARX model (on

the data in 1st argument), simulates it (on the data in the 2nd argument), and returns all the MSEs on the first row of V (see help arxstruc for the format of V).

14

Example (continued): Structure selection

• To choose the structure with the smallest MSE:N = selstruc(V, 0); %best fit to validation data

• For our data, N= [8, 7, 1].• Alternatively, graphical selection: N = selstruc(V, ’plot’); Then click

on bar corresponding to best (red) model and the button “Select”.

Red — Best fit minimizes the sum of the squares of the difference between the validation data output and the model output.

15

Example (continued): Validation of best ARX model

model = arx(id, N); compare(model,val);

A better fit is obtained.

16

Special case of ARX: FIR

• Setting A(q-1) = 1 (na = 0) in ARX model, we get:

• where h(j) is the impulse response of the system. • Note that here h(0), the impulse response at time 0, is

assumed 0 – i.e. system does not respond instantaneously to changes in input).

nb

j

nb

jj

kejkujh

kejkubkekuqBky

1

1

1

)()()(

)()()()()()(

17

Difference between ARX and FIR

• Since ARX includes recursive relationships between current and previous outputs, it will be sufficient to take orders na and nb (at most) equal to the order of the dynamical system.

• FIR needs a sufficiently large order nb to model the entire transient regime of the impulse response (in principle, we only recover the correct model as nb → ∞).

• ⇒ more parameters more data needed to identify them.⇒

)()()()(:FIR

)()()()()(:ARX1

11

kekuqBky

kekuqBkyqA

18

ARMAX model structure

• Number of parameters: na+nb+nc• Name: AutoRegressive, Moving Average (referring to noise model)

with eXogenous input (dependence on u)• ARX model is a special case of ARMAX model (with nc = 0, or C(q-1)=1 ).

ARMAX can model more intricate relationships between the disturbance and the inputs and outputs.

)()()()()()( 111 keqCkuqBkyqA

ncnc

nbnb

nana

qcqcqC

qbqbqB

qaqaqA

11

1

11

1

11

1

1)(

,)(

,1)(

)()1()()()1()()1()( 111 nckeckeckenbkubkubnakyakyaky ncnbna

19

Overall relationship

ARMAX ARX FIR⊃ ⊃

• ARMAX to ARX: Less freedom in modeling disturbance.

• ARX to FIR: More parameters required.

20

Example 2• Consider again the previous data on which ARX was applied

but let us now identify an ARMAX modelmARMAX = armax(id, [na, nb, nc, nk]);

• Arguments:1 Identification data.2 Array containing the orders of A, B, C and delay nk.

ncnc

nbnb

nana

qcqcqcqC

qbqbbqB

qaqaqaqA

keqCnkkuqBkyqA

22

11

1

1121

1

22

11

1

111

1)(

,)(

,1)(

where),()()()()()(

21

Example 2, continued

• Considering the system is 2nd order with na = 2, nb = 2, nc = 2, nk = 1.

• Validation: compare(val, mARMAX);

• Good results! similar to ARX model (with na=8, nb=7, nk=1), but with fewer parameters (2+2+2+1=7). Flexible noise model pays off.

22

Output error (OE) model• Other model forms are possible, e.g. Output Error, OE:

• This corresponds to simple, additive measurement noise on the output (the “output error”).

• To identify OE model we can use the command:

mOE = oe(id, [nb, nf, nk]);

)()()()()( 1

1

kekuqFqBky