Model reduction of large-scale systems
Part I: Overview
Thanos Antoulas
Rice University and Jacobs University
email: [email protected]: www.ece.rice.edu/ aca
1st Elgersburg School on Mathematical System TheoryTU Ilmenau, 30 March - 3 April 2009
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 1 / 40
Outline
1 Introduction and model reduction problem
The big picture
Problem formulation
Projections
2 Motivating examples
3 Overview of approximation methods
SVD-based methods
Krylov methods
4 Summary – Challenges – References
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 2 / 40
Introduction and model reduction problem
Outline
1 Introduction and model reduction problem
The big picture
Problem formulation
Projections
2 Motivating examples
3 Overview of approximation methods
SVD-based methods
Krylov methods
4 Summary – Challenges – References
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 3 / 40
Introduction and model reduction problem The big picture
Recall: the big picture
@@@@@@R
���
���
@@@@@@R
� discretization
Modeling
Model reduction
���*
HHHj
Simulation
Controlreduced # of ODEs
ODEs PDEs
���
���
���
���
��
��
��
��
���
���
��
��
��
��
���
���
��
��
��
���
��
���
��
Physical System and/or Data
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 4 / 40
Introduction and model reduction problem Problem formulation
Dynamical systems
x1(·)x2(·)
...xn(·)
u1(·) −→u2(·) −→
...um(·) −→
−→ y1(·)−→ y2(·)
...−→ yp(·)
We consider (implicit) state equations
Σ : f(x(t) ,x(t), u(t)) = 0, y(t) = h(x(t), u(t))
with state (internal vartiable) x(·) of dimension n� m,p.
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 5 / 40
Introduction and model reduction problem Problem formulation
Problem statement
Given: dynamical system
Σ = (f,h) with: u(t) ∈ Rm, x(t) ∈ Rn, y(t) ∈ Rp.
Problem: Approximate Σ with:
Σ = (f, h) with : u(t) ∈ Rm, x(t) ∈ Rk , y(t) ∈ Rp, k � n :
(1) Approximation error small - global error bound
(2) Preservation of stability/passivity
(3) Procedure must be computationally efficient
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 6 / 40
Introduction and model reduction problem Projections
Approximation by projection
Unifying feature of approximation methods: projections.
Let V, W ∈ Rn×k , such that W∗V = Ik ⇒ Π = VW∗ is a projection.
Define x = W∗x ⇒ x ≈ Vx. Then
Σ :
{ddt x(t) = W∗f(Vx(t), u(t))
y(t) = h(Vx(t), u(t))
Thus Σ is ”good” approximation of Σ, if x− Πx is small.
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 7 / 40
Introduction and model reduction problem Projections
Special case: linear dynamical systems
Σ: Ex(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t)
Σ =
(E,A BC D
)Problem: Approximate Σ by projection: Π = VW∗
Σ =
(E, A BC D
)=
(W∗EV,W∗AV W∗B
CV D
), k � n
Norms:• H∞-norm:worst output error‖y(t)− y(t)‖ for ‖u(t)‖ = 1.
• H2-norm: ‖h(t)− h(t)‖
E,An
n
C
B
D
⇒ E, Ak
k
C
B
D
Σ: : Σ
= Vn
k
W∗
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 8 / 40
Motivating examples
Outline
1 Introduction and model reduction problem
The big picture
Problem formulation
Projections
2 Motivating examples
3 Overview of approximation methods
SVD-based methods
Krylov methods
4 Summary – Challenges – References
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 9 / 40
Motivating examples
Motivating Examples: Simulation/Control
1. Passive devices • VLSI circuits• Thermal issues• Power delivery networks
2. Data assimilation • North sea forecast• Air quality forecast
3. Neural models • Simulations4. CVD reactor • Bifurcations5. Mechanical systems: •Windscreen vibrations
• Buildings6. Optimal cooling • Steel profile7. MEMS: Micro Electro-
-Mechanical Systems • Elf sensor8. Nano-Electronics • Plasmonics
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 10 / 40
Motivating examples
Passive devices: VLSI circuits
1960’s: IC 1971: Intel 4004 2001: Intel Pentium IV10µ details 0.18µ details2300 components 42M components64KHz speed 2GHz speed
2km interconnect7 layers
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 11 / 40
Motivating examples
Passive devices: VLSI circuits
Typical GateDelay
0.1
1.0D
elay
(ns)
1.01.3 0.8 0.30.5Technology (μm)
0.1 0.08
Average WiringDelay
≈ 0.25 μm
Today’s Technology: 65 nm
65nm technology: gate delay < interconnect delay!
Conclusion: Simulations are required to verify that internal electromagneticfields do not significantly delay or distort circuit signals. Thereforeinterconnections must be modeled.
⇒ Electromagnetic modeling of packages and interconnects⇒ resultingmodels very complex: using PEEC methods (discretization of Maxwell’sequations): n ≈ 105 · · · 106 ⇒ SPICE: inadequate
• Source: van der Meijs (Delft)Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 12 / 40
Motivating examples
Power delivery network for VLSI chips
VDD
r r r rr r r r
r r r rr r r r�
��
��
��
���
��
��
��
��
��
��
��
��
��
��
��
��
��
��
��
AA AA AA AA
AA AA AA AA
AA AA AA AA
AA AA AA AA
n? n?r r'
&
$
%
'&
$%
?
R L
C C
#states ≈ 8 · 106
#inputs/outputs ≈ 1 · 106
1
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 13 / 40
Motivating examples
Mechanical systems: carsCar windscreen simulation subject to acceleration load.
Problem: compute noise at points away from the window.
PDE: describes deformation of a structure of a specific material; FEdiscretization: 7564 nodes (3 layers of 60 by 30 elements). Material:glass with Young modulus 7·1010 N/m2; density 2490 kg/m3; Poissonratio 0.23⇒ coefficients of FE model determined experimentally.The discretized problem has dimension: 22,692.
Notice: this problem yields 2nd order equations:
Mx(t) + Cx(t) + Kx(t) = f(t).
• Source: Meerbergen (Free Field Technologies)
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 14 / 40
Motivating examples
Mechanical Systems: Buildings, Earthquake prevention
Taipei 101: 508m Damper between 87-91 floors 730 ton damper
Building Height Control mechanism Damping frequencyDamping mass
CN Tower, Toronto 533 m Passive tuned mass damperHancock building, Boston 244 m Two passive tuned dampers 0.14Hz, 2x300tSydney tower 305 m Passive tuned pendulum 0.1,0.5z, 220tRokko Island P&G, Kobe 117 m Passive tuned pendulum 0.33-0.62Hz, 270tYokohama Landmark tower 296 m Active tuned mass dampers (2) 0.185Hz, 340tShinjuku Park Tower 296 m Active tuned mass dampers (3) 330tTYG Building, Atsugi 159 m Tuned liquid dampers (720) 0.53Hz, 18.2t
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 15 / 40
Motivating examples
Neural models: How a cell distinguishes between inputs
Image from Neuromart (Rice-Baylor archive of neural morphology)
Goal: • simulation of systems containing a few million neurons• simulations currently limited to systems with ≈10K neurons
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 16 / 40
Motivating examples
Example: neuron simulationPropagation of action potentials
The Hodgkin-Huxley model: C · dVdt = Iext − ICa − INa − IL
V is the potential across the membrane. The currents ICa , INa und IL (with given potentials ECa , ENa , EL) are:
ICa = gCa · (V − ECa), INa = gNa · (V − ENa) and IL = gL · (V − EL),
The conductivities gCa und gNa depend on time, while gL is constant. H-H postulated the existence of gate variables, whichrepresent the dynamics of the ion channels (i.e. the active channels). Experimental results led to three gate variables: n, m, h.Together with the ion conductivity g, gate variables describe the current which flows through the membrane:
ICa = gCa · n4 · (V − ECa) and INa = gNa · m3 · h · (V − ENa).
The three gate variables are described as follows:dndt = αn(V ) · (1− n)− βn(V ) · n, dm
dt = αm(V ) · (1− m)− βm(V ) · m, dhdt = αh(V ) · (1− h)− βh(V ) · h,
where αn(V ), αm(V ), αh(V ), βn(V ), βm(V ), βh(V ), are exponential functions.
Example. The model size is dictated by synapse density and total dendritic length. For instance, with approximately one synapseper micron on a cell with 5mm of dendrite we obtain 5000 compartments. Under usual conditions there are 10 equations percompartment (1 voltage and 9 gating) ⇒ 50,000 coupled equations.
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 17 / 40
Overview of approximation methods
Outline
1 Introduction and model reduction problem
The big picture
Problem formulation
Projections
2 Motivating examples
3 Overview of approximation methods
SVD-based methods
Krylov methods
4 Summary – Challenges – References
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 19 / 40
Overview of approximation methods
Large-scale systems
What is the problem with very large systems?
⇓
• Storage
• Computational speed
• Accuracy
• System theoretic properties
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 20 / 40
Overview of approximation methods
Approximation methods: OverviewPPPPPPPPq
������
Krylov
• Realization• Interpolation• Lanczos• Arnoldi
SVD
@@@R
���
Nonlinear systems Linear systems• POD methods • Balanced truncation• Empirical Gramians • Hankel approximation
@@@@R
���
Krylov/SVD Methods
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 21 / 40
Overview of approximation methods SVD-based methods
Approximation methods: a prototype problem
The Singular value decomposition: SVD
A = UΣV ∗ ∈ Rn×m
Singular values: Σ = diag (σ1, · · · , σn), σ1 ≥ · · · ≥ σn ≥ 0
⇒ σi =√λi(A∗A)
left singular vectors: U = (u1 u2 · · · un), UU∗ = Inright singular vectors: V = (v1 v2 · · · vm), VV ∗ = ImDyadic decomposition:
A = σ1u1v∗1 + σ2u2v∗2 + · · ·+ σnunv∗n
σ1: 2-induced norm of A
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 22 / 40
Overview of approximation methods SVD-based methods
Reminder: 2-norm
Vectors:
x =
x1...
xn
⇒ ‖ x ‖2 =√
x21 + · · ·+ x2
n
Matrices/Operators: induced 2-norm
A
⇒ ‖ A ‖2 = max‖x‖=1 ‖ Ax ‖2 = σ1
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 23 / 40
Overview of approximation methods SVD-based methods
Optimal approximation in the 2-norm
Given: A ∈ Rn×m
find: X ∈ Rn×m, rank X = k < rank ACriterion: norm(error) is minimized, whereerror: E = A− X , norm: 2-norm
Theorem (Schmidt-Mirsky, Eckart-Young)
minrankX≤k
‖ A− X ‖2 = σk+1(A)
Minimizer (non-unique): truncation of dyadic decomposition of A:
X# = σ1u1v∗1 + σ2u2v∗2 + · · ·σkukv∗k
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 24 / 40
Overview of approximation methods SVD-based methods
Remarks.
(a) Importance of Schmidt-Mirsky: establishes a relationship betweenthe rank k of the approximant, and the (k + 1)st singular value of A.
(b) Other minimizers:
X (η1, · · · , ηk ) :=∑k
i=1 (σi − ηi)uiv∗i ,
where 0 ≤ ηi ≤ σk+1.
(c) The problem of minimizing the 2-induced norm of A− X over allmatrices X of rank at most k , is non-convex.
(d) The SVD cannot be used to approximate unitary matrices, whereσk = 1, for all k .
(e) Problem can also be solved in the Frobenius norm.
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 25 / 40
Overview of approximation methods SVD-based methods
SVD Approximation methods: two static examples
Supernova Clown
Singular values provide trade-off between accuracy and complexity
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 26 / 40
Overview of approximation methods Krylov methods
Projectors for Krylov and rational Krylov methods
Given:
Σ =
(E,A BC D
)by projection: Π = VW∗, Π2 = Π obtain
Σ =
(E, A BC D
)=
(W∗EV,W∗AV W∗B
CV D
), where k < n.
Krylov (Lanczos, Arnoldi): let E = I and
V =[B, AB, · · · , Ak−1B
]∈ Rn×k
W∗ =
C
CA...
CAk−1
∈ Rk×n
⇒ W∗ = (W∗V)−1W∗
then the Markov parameters match:
CAiB = CAiB
Rational Krylov: let
V =[
(λ1E− A)−1B · · · (λk E− A)−1B]∈ Rn×k
W∗ =
C(λk+1E− A)−1
C(λk+2E− A)−1
.
.
.C(λ2k E− A)−1
∈ Rk×n
⇒ W∗ = (W∗V)−1W∗
then the moments of G match those of G at λi :
G(λi) = D + C(λiE−A)−1B = D + C(λiE− A)−1B = G(λi)
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 35 / 40
Overview of approximation methods Krylov methods
Properties of Krylov methods
(a) Number of operations: O(kn2) or O(k2n) vs. O(n3)⇒ efficiency
(b) Only matrix-vector multiplications are required. No matrix factorizationsand/or inversions. No need to compute transformed model and then truncate.
(c) Drawbacks
• global error bound?• Σ may not be stable.
Q: How to choose the projection points?
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 36 / 40
Summary – Challenges – References
Outline
1 Introduction and model reduction problem
The big picture
Problem formulation
Projections
2 Motivating examples
3 Overview of approximation methods
SVD-based methods
Krylov methods
4 Summary – Challenges – References
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 37 / 40
Summary – Challenges – References
Approximation methods: SummaryPPPPPPPPq
������
Krylov
• Realization• Interpolation• Lanczos• Arnoldi
SVD
@@@R
���
Nonlinear systems Linear systems• POD methods • Balanced truncation• Empirical Gramians • Hankel approximation
@@@@R
���
Krylov/SVD Methods
��
r@@R
rProperties
• numerical efficiency
• n� 103
• choice of matching moments
Properties
• Stability
• Error bound
• n ≈ 103
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 38 / 40
Summary – Challenges – References
(Some) Challenges in complexity reduction
Model reduction of uncertain systems
Model reduction of differential-algebraic (DAE) systems
Domain decomposition methods
Parallel algorithms for sparse computations in model reduction
Development/validation of control algorithms based on reducedmodels
Model reduction and data assimilation (weather prediction)
Active control of high-rise buildings
MEMS and multi-physics problems
VLSI design
Molecular Dynamics (MD) simulations
CAD tools for nanoelectronics
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 39 / 40
Summary – Challenges – References
ReferencesPassivity preserving model reduction
Antoulas SCL (2005)Sorensen SCL (2005)Ionutiu, Rommes, Antoulas IEEE TCAD (2008)
Optimal H2 model reductionGugercin, Antoulas, Beattie SIMAX (2008)
Low-rank solutions of Lyapunov equationsGugercin, Sorensen, Antoulas, Numerical Algorithms (2003)Sorensen (2006)
Model reduction from dataMayo, Antoulas LAA (2007)Lefteriu, Antoulas, Tech. Report (2008)
General reference: Antoulas, SIAM (2005)
Thanos Antoulas (Rice U. & Jacobs U.) Model reduction of large-scale systems 40 / 40