18-660: numerical methods for engineering design and ...xinli/classes/cmu_18660/lec06.pdf18-660:...
TRANSCRIPT
Slide 1
18-660: Numerical Methods for Engineering Design and Optimization
Xin Li Department of ECE
Carnegie Mellon University Pittsburgh, PA 15213
Slide 2
Overview
Linear Equation Solver LU decomposition Cholesky decomposition
Slide 3
Linear Equation Solver
Gaussian elimination solves a linear equation
Sometimes we want to repeatedly calculate the solutions for different right-hand-side vectors
=
⋅
A 1X 1B
=
⋅
A 2X 2B
Case 1 Case 2
=
⋅
A X B
Slide 4
Ordinary Differential Equation Example
Backward Euler integration for linear ordinary differential equation with constant time step ∆t
( ) ( ) ( ) ( ) 00 =⋅+⋅= xtuBtxAtx
( ) ( ) ( ) ( )[ ] ( ) 0011
1 =⋅⋅∆+⋅⋅∆−= +−
+ txtuBttxAtItx nnn
Identical @ all tn’s
Different @ all tn’s
Slide 5
LU Factorization
It would be expensive to repeatedly run Gaussian elimination for many times How can we save and re-use the intermediate results? LU factorization is to address this problem
=
⋅
A 1X 1B
=
⋅
A 2X 2B
Case 1 Case 2
Slide 6
LU Factorization
Key idea: Represent A as the product of L (lower triangular) and U
(upper triangular) via Gaussian-elimination-like steps All diagonal elements in U are set to 1 by proper scaling
⋅
=
A UL
LU factorization is unchanged as long as A is unchanged (i.e., independent of the right-hand-side vector B)
=
⋅
A X B
Slide 7
LU Factorization
⋅
=
1
11
2
112
21
2221
11
21
22221
11211
N
N
NNNNNNNN
N
N
uuu
lll
lll
aaa
aaaaaa
( )Nial ii ,,2,11 11 ==⋅
Slide 8
LU Factorization
⋅
=
1
11
2
112
21
2221
11
21
22221
11211
N
N
NNNNNNNN
N
N
uuu
lll
lll
aaa
aaaaaa
( )Niaul ii ,,3,21111 ==⋅
Slide 9
LU Factorization
⋅
=
1
11
2
112
21
2221
11
21
22221
11211
N
N
NNNNNNNN
N
N
uuu
lll
lll
aaa
aaaaaa
( )Nialul iii ,,3,21 22121 ==⋅+⋅
Continue iteration until all elements in L and U are solved
Slide 10
Memory Storage
The matrix A can be iteratively replaced by L and U No additional memory is required
NNNN
N
N
aaa
aaaaaa
21
22221
11211
−
NNNN
NN
N
lllu
lluul
21
,1
2221
11211
Slide 11
A Simple LU Example
⋅
=
−−−
−
11
1
212213112
23
1312
333231
2221
11
uuu
lllll
l
313121211111 111 alalal =⋅=⋅=⋅
−−−
−
212213112
Slide 12
A Simple LU Example
131311121211 aulaul =⋅=⋅
−−−
−
212213
21212
⋅
11
1
23
1312
333231
2221
11
uuu
lllll
l
−−−
−
212213112
Slide 13
A Simple LU Example
3232123122221221 alulalul =+⋅=+⋅
−−
−
2222213
21212
⋅
11
1
23
1312
333231
2221
11
uuu
lllll
l
−−−
−
212213
21212
Slide 14
A Simple LU Example
2323221321 aulul =⋅+⋅
−−
−
2221213
21212
⋅
11
1
23
1312
333231
2221
11
uuu
lllll
l
−−
−
2222213
21212
Slide 15
A Simple LU Example
333323321331 alulul =+⋅+⋅
−−−
−
1221213
21212
⋅
11
1
23
1312
333231
2221
11
uuu
lllll
l
−−
−
2221213
21212
Slide 16
LU Factorization
Given L and U, solve linear equation via two steps
BXA =⋅
LUA =
BXUL =⋅⋅V
VXUBVL=⋅=⋅
Slide 17
LU Factorization
Only the above two steps are repeated if the right-hand-side vector B is changed LU factorization is not repeated More efficient than Gaussian elimination
=
⋅
V BL
=
⋅
X VU
Forward substitution Backward substitution
Slide 18
⋅
=
Cholesky Factorization
If the matrix A is symmetric and positive definite, Cholesky factorization is preferred over LU factorization
Cholesky factorization is cheaper than LU Only needs to find a single triangular matrix L (instead of two
different matrices L and U)
A TLL
Slide 19
Cholesky Factorization
A must be symmetric and positive definite to make Cholesky factorization applicable
A symmetric matrix A is positive definite if
Sufficient and necessary condition for a symmetric matrix A to be positive definite: All eigenvalues of A are positive
0>⋅⋅ PAPT for any real-valued vector P ≠ 0
Slide 20
Partial Differential Equation Example
1-D rod discretized into 4 segments
T1 T2 T3 T4 T5
( )
10030
0,
51
2
2
==
=∂
∂⋅
TTx
txTκ
( ) ( )4202 11 ≤≤=−+−⋅ +− iTTT iiiκ
01002020230
43
432
32
=−+−=−+−=−+−
TTTTTTT
Slide 21
Partial Differential Equation Example
Eigenvalues of A
01002020230
43
432
32
=−+−=−+−=−+−
TTTTTTT
=
⋅
−−−
−
1000
30
21121
12
4
3
2
TTT
A
58.000.241.3
3
2
1
===
λλλ
(A is positive definite)
Slide 22
Partial Differential Equation Example
In practice, we never calculate eigenvalues to check if a matrix is positive definite or not Eigenvalue decomposition is much more expensive than
solving a linear equation
If we apply finite difference to discretize steady-state heat
equation, the resulting linear equation is positive definite
Slide 23
Partial Differential Equation Example
⋅
=
−−−
−
33
3222
312111
333231
2221
11
21121
12
llllll
lllll
l
311131211121111111 allallall =⋅=⋅=⋅
−−−
−
2101221
12
Slide 24
Partial Differential Equation Example
23322231212222222121 allllallll =⋅+⋅=⋅+⋅
−−−
−
232012321
12
⋅
33
3222
312111
333231
2221
11
llllll
lllll
l
−−−
−
2101221
12
Slide 25
Partial Differential Equation Example
33333332323131 allllll =⋅+⋅+⋅
−−−
−
3432012321
12
⋅
33
3222
312111
333231
2221
11
llllll
lllll
l
−−−
−
232012321
12
Slide 26
Summary
Linear equation solver LU decomposition Cholesky decomposition