![Page 1: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/1.jpg)
Trend and System Identification With Orthogonal Basis Function
OSE SEMINAR 2013
ÄMIR SHIRDEL
CENTER OF EXCELLENCE IN
OPTIMIZATION AND SYSTEMS ENGINEERING
AT ÅBO AKADEMI UNIVERSITY
ÅBO NOVEMBER 15 2013
![Page 2: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/2.jpg)
Background
– System identification is difficult when process measurements are
corrupted by structured disturbances, such as trends, outliers, level
shifts
– Standard approach: removal by data preprocessing but difficult to
separate between the effects of known system inputs and unknown
disturbances (trends, etc.)
– Orthonormal basis function models are categorized as output-error
(ballistic simulation) models
2|N
Background
Amir Shirdel: Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
2|22
![Page 3: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/3.jpg)
Present contribution
– Identification of system model parameters and disturbances
simultaneously
– Sparse optimization used in system identification problem
– Applying the method on simulated and real example
3|N
Present contribution
Amir Shirdel: Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
3|22
![Page 4: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/4.jpg)
4
4 4
Present contribution
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
4|22
Present contribution
SYSTEN
(model) U Y
0 100 200 300 400 500 600 700 800 900 1000-10
-8
-6
-4
-2
0
2
4
6
8
d
0 100 200 300 400 500 600 700 800 900 1000-25
-20
-15
-10
-5
0
5
10
15
20
0 100 200 300 400 500 600 700 800 900 1000-4
-3
-2
-1
0
1
2
3
4
![Page 5: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/5.jpg)
5
Orthogonal basis functions has some advantages:
• The corresponding approximation (representation) has simple and direct
solution.
• It corresponds to allpass filters which is robust to implement and use in
numerical computation.
• It is popular because a few parameters can describe the system.
• It is a kind of output-error model, and can be insensitive to noise.
5 5
Orthogonal basis function (Fixed-pole model)
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
5|22
Orthogonal basis function
![Page 6: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/6.jpg)
6 6 6
Orthogonal basis function (Fixed-pole model)
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
6|22
Orthogonal basis function
FIR network:
Laguerre function:
Kautz function:
![Page 7: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/7.jpg)
7
Model:
where
The parameters can be estimated using standard least squares method:
7 7
System Identification
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
7|22
System identification
2)ˆ)(( θ(k)ky T
d(k)θ(k)y(k) T
System u
d
y
T
n
T
n aaakukukuk ],...,,[,)](),...,(),([)( 2121
)()()(or)()()( kuqkukuqLku nnnn
![Page 8: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/8.jpg)
8 8 8
Problem formulation
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
8|22
Problem formulation
We consider the linear system model:
It is assumed the measured output is given by
where d(k) is a structured disturbance:
• outlier signal,
• level shifts,
• piecewise constant trends
)()()( kdkyky L
),(...)()()( 2211 kuakuakuaky nnL
![Page 9: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/9.jpg)
9 9 9
Disturbance models
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
9|22
Disturbance models
Sequence of outliers:
Level shifts:
Sequence of trends:
![Page 10: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/10.jpg)
10 10 10
Sparse representation of disturbance
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
10|22
Sparse representation of disturbance
For the structured disturbances, the vectors are sparse,
where and , depends on disturbances
Outliers:
Level shifts:
Trends:
dDi
ID 0
iD
![Page 11: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/11.jpg)
11
Identification by sparse optimization:
subject to
where
This is an intractable combinatorial optimization problem.
Instead we use and solve the convex problem
11 11 11 11
Sparse optimization approach
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
11|22
Sparse optimization approach
1
2ˆ,ˆ
ˆ))(ˆ)((min dDλkyky i
kd
2ˆ,ˆ
))(ˆ)((min k
dkyky
elements nonzero ofnumber 0
MdDi 0
relaxation1 l
![Page 12: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/12.jpg)
12
Algorithm
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
12|22
Algorithm
1. Make the basis function expansion based on given prior knowledge of system (pole and system order).
2. Solution of sparse optimization problem by iterative reweighting:
Minimize the weighted cost to give the estimates
where i = 0, 1 or 2 (user selected).
3. Calculate new weights W and go to step 2.
4. Continue until convergence.
5. Use model order reduction to get lower order model.
1
2ˆ,ˆ
ˆ))(ˆˆ)((min dDWλkdky ii
k
T
d
![Page 13: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/13.jpg)
We apply the proposed identification detrending method to the ARX model
with parameter vector
given by
u(k) and e(k) are normally distributed signals with variances 1 and 0.1, and
d(k) is unknown structured disturbance ,
For making the Kautz basis function we used N=4 as order of system and pole
is
13 13
Example1
13|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
4.07.0
![Page 14: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/14.jpg)
14
Example1
14|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0.9340ˆ 9125.0 ˆ RMSE :without
5.2937RMSE 0.7697- 1.3465 2.8084 1.5214 ˆ
0.9102RMSE 0.8412- 1.3152 2.7305 1.5428 ˆ
LS
LS
RMSEd(k)
0 100 200 300 400 500 600 700 800 900 1000 -20
-10
0
10
20
K data points
Output
Output (model)
Output (LS)
0 100 200 300 400 500 600 700 800 900 1000
-5
0
5
10
K data points
Disturbance (Identified)
Disturbance
![Page 15: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/15.jpg)
15
Example1
15|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0 1 2 3 4 50
1
2
3
4
5
6
7
8Hankel Singular Values (State Contributions)
State
Sta
te E
nerg
y
Stable modes
![Page 16: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/16.jpg)
16
Example1
16|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0.9118RMSE 0.4933- 0.6946- ˆ
0.9102RMSE 0.8412- 1.3152 2.7305 1.5428 ˆ
R
0 100 200 300 400 500 600 700 800 900 1000-20
-10
0
10
K data points
Output (model)
Output (Reduced model)
0 100 200 300 400 500 600 700 800 900 1000
-5
0
5
10
K data points
Disturbance (Identified)
Disturbance
![Page 17: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/17.jpg)
17 17
Example2: Pilot-scale distillation column data
17|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0 200 400 600 800 1000 1200 1400 1600 1800 2000 91
91.5
K data points
Top column output
0 200 400 600 800 1000 1200 1400 1600 1800 2000 0
2
K data points
Bottom column output
0 200 400 600 800 1000 1200 1400 1600 1800 2000 156
158
160
162
K data points
Reflux (L)
0 200 400 600 800 1000 1200 1400 1600 1800 2000 87
88
89
90
K data points
Reboiling flow (V)
![Page 18: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/18.jpg)
1
18 18
Example2: Identification results
18|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0 200 400 600 800 1000 1200 1400 1600 1800 2000-1
-0.5
0
0.5
1
1.5
K data points
Output (real)
Output+dd (model)
0 200 400 600 800 1000 1200 1400 1600 1800 2000-0.2
-0.1
0
0.1
0.2
0.3
K data points
Disturbance (identified)
0.0933RMSE 0.0456- 0.0353- ˆ
0.0826 RMSE 12) ...(total 0.0147- 0.0338- 0.0285- 0.0022- ˆ
0.0643RMSE 12) (total ... 0.0151- 0.0346- 0.0280- 0.0031- ˆ
RED
LS
![Page 19: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/19.jpg)
19 19
Example2: Hankel singular value for reduction
19|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0 2 4 6 8 10 120
0.1
0.2
0.3
0.4
0.5
0.6
0.7Hankel Singular Values (State Contributions)
State
Sta
te E
nerg
y
Stable modes
![Page 20: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/20.jpg)
20 20
Example2: Validation data with estimation of d(k)
20|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0.0891RMSE 0.0456- 0.0353- ˆ RED
0 200 400 600 800 1000 1200-1
0
1
Output (real)
Output(Test Model+Disturbance)
0 200 400 600 800 1000 1200
-0.2
0
0.2
Disturbance
0 200 400 600 800 1000 1200
-0.5
0
0.5
0 200 400 600 800 1000 1200-1
0
1
![Page 21: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/21.jpg)
Summary:
• Presented a method for identification of linear systems in the presence of
structured disturbances (outliers, level shifts and trends) by using sparse
optimization and orthogonal basis function as the system model
• Gives acceptable results for simulated example
• Gives acceptable results for distillation column example
• The orthogonal basis model improves the robustness and more insensitive
to noise
Future work:
• Nonlinear system identification and trends and more general disturbances
21 21
Discussion and future work
21|22
Discussion and future work
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
![Page 22: Trend and System Identification With Orthogonal Basis Functionweb.abo.fi/fak/tkf/at/ose/doc/Pres_15112013/Amir Shirdel.pdf · Shirdel : Trend and System Identification With Orthogonal](https://reader034.vdocument.in/reader034/viewer/2022050519/5fa2dc8e9cff3f4e9210772c/html5/thumbnails/22.jpg)
22 22
Thank you for your attention!
Questions?
22 22
22|22
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University