6.2 pivoting strategies 1/17 chapter 6 direct methods for solving linear systems -- pivoting...
TRANSCRIPT
6.2 Pivoting Strategies
1/17
Chapter 6 Direct Methods for Solving Linear Systems -- Pivoting Strategies
Example: Solve the linear system
using 4-digit arithmetic with rounding.
78.46130.6291.5
17.5914.59003000.0
21
21
xx
xx
Solution: The exact solutions are .000.1 and ,00.10 21 xx
1764003000.0/291.5/ 112121 aam
10430014.59130.6 2122 ma
10440017.5978.46 212 mb
Apply Gaussian elimination:
1044001043000
17.5914.59003000.0
001.12 x 00.10003000.0
001.114.5917.591
x
Trouble maker
Small pivot element may cause trouble.
Chapter 6 Direct Methods for Solving Linear Systems -- Pivoting Strategies
Partial Pivoting (or maximal column pivoting)
-- Determine the smallest p k such that ||max|| )()( kik
nik
kpk aa
and interchange the pth and the kth rows.
78.46130.6291.5
17.5914.59003000.0
17.5914.59003000.0
78.46130.6291.5
14.5914.590
78.46130.6291.500.10;000.1 12 xx
Example: Solve the linear system
using 4-digit arithmetic with rounding.
78.46130.6291.5
59170059140000.30
21
21
xx
xx
Small relative to the entries in its row.
2/17
Chapter 6 Direct Methods for Solving Linear Systems -- Pivoting Strategies
Scaled Partial Pivoting (or scaled-column pivoting)
-- Place the element in the pivot position that is largest relative to the entries in its row.
||max1
ijnj
i as
Step 1: Define a scale factor si for each row as
Step 2: Determine the smallest p k such that i
kik
nikp
kpk
s
a
s
a ||max
|| )()(
and interchange the pth and the kth rows.
Note: Note:
The scaled factors The scaled factors ssii must be computed must be computed only onceonly once, otherwise , otherwise
this method would be too slow.this method would be too slow.
3/17
Chapter 6 Direct Methods for Solving Linear Systems -- Pivoting Strategies
Complete Pivoting (or maximal pivoting)
-- Search all the entries aij for i, j = k, …, n, to find the entry with the largest magnitude. Both row and column interchanges are performed to bring this entry to the pivot position.
Result of solving 3 by 3 linear systems with direct Gaussian elimination
Result of solving 3 by 3 linear systems with complete pivoting
4/17
Chapter 6 Direct Methods for Solving Linear Systems -- Pivoting Strategies
Amount of Computation
Partial Pivoting: Requires about O(n2) additional comparisons.
Scaled Partial Pivoting: Requires about O(n2) additional comparisons and O(n2) divisions.
Complete Pivoting: Requires about O(n3/3) additional comparisons.
Note: Note:
If the new scaled factors were determined each time a row If the new scaled factors were determined each time a row
interchange decision was to be made, then the scaled partial interchange decision was to be made, then the scaled partial
pivoting would add pivoting would add O(O(nn33/3) comparisons/3) comparisons in addition to the in addition to the O(O(nn22) )
divisions.divisions.
5/17
Chapter 6 Direct Methods for Solving Linear Systems -- Matrix Factorization
6.5 Matrix Factorization
Matrix Form of Gaussian Elimination
Step 1: )0(/ 111111 aaam ii
Let L1 =
1...
1
1
1
21
nm
m
, then ][ )1()1(1 bAL
)1(1
)1(1
)1(11 ... baa n
)2(A )2(b
Step n 1:
)(
)2(2
)1(1
)(
)2(2
)2(22
)1(1
)1(12
)1(11
121
..
.......
...
...
...
nn
nnn
n
n
nn
b
b
b
a
aa
aaa
bALLL
where
Lk =
1...
1
1
,
,1
kn
kk
m
m
6/17
Chapter 6 Direct Methods for Solving Linear Systems -- Matrix Factorization
1kL
1...
1
1
,
,1
kn
kk
m
m
1
11
21
1 ... nLLL
1
1
1
jim ,L
unitary lower-triangular matrix
Let U =
)(
)2(2
)2(22
)1(1
)1(12
)1(11
..
....
...
...
nnn
n
n
a
aa
aaa
LUA
LU factorization of A
Hey hasn’t GE given me enough headache? Why do I have to know its matri
x form??!
When you have to solve the system for
different with a fixed
A.bCould you be more specific, please?
Factorize A first, then for every you only have to
solve two simple triangular systems and
.
b
byL
yxU
7/17
Chapter 6 Direct Methods for Solving Linear Systems -- Matrix Factorization
Theorem: If Gaussian elimination can be performed on the linear system Ax = b without row interchanges, then the matrix A can be factored into the product of a lower-triangular matrix L and an upper-triangular matrix U.
If L has to be unitary, then the factorization is unique.
Proof (for uniqueness): If the factorization is NOT unique, then there exist L1, U1, L2 and U2 such that A = L1U1 = L2U2 .
121 UU 2
11
1222
11 LLUULL
Upper-triangularLower-triangular
with diagonal entries 1
I
Note: The factorization with Note: The factorization with UU being unitary being unitary is called the is called the Crout’Crout’s factorizations factorization. Crout’s factorization can be obtained by the . Crout’s factorization can be obtained by the LULU factorization of factorization of AAtt. That is, find . That is, find AAtt = = LULU, then , then AA = = UUtt L Ltt is the Cro is the Crout’s factorization of ut’s factorization of AA..
8/17
Chapter 6 Direct Methods for Solving Linear Systems -- Matrix Factorization
Doolittle Factorization – a compact form of LU factorization
Repeated computations.What a waste!
nn
n
nnnn
n
u
uu
l
l
aa
aa
..
.
..
....
...
1...
......
1
1
......
..
....
..
.... 111
1
21
1
111
),min(
1
ji
kjkki ul jia
9/17
Chapter 6 Direct Methods for Solving Linear Systems -- Matrix Factorization
),min(
1
ji
kjkkiji ula
Fix i :For j = i, i+1, …, n we have ijkj
i
kikij uula
1
1
lii = 1
kj
i
kikijij ulau
1
1 a
Interchange i and j . For j = i, i+1, …, n we have iijiki
i
kjkji ulula
1
1
ii
i
kkijkjiji uulal /)(
1
1
b
Algorithm: Doolittle FactorizationStep 1: u1j = a1j; lj1 = aj1 / u11; ( j = 1, …, n )
Step 2: compute and for i = 2, …, n1;
Step 3:
a b
1
1
n
kknnknnnn ulau
HW: p.397 #7
10/17
Chapter 6 Direct Methods for Solving Linear Systems -- Special Types of Matrices
6.6 Special Types of Matrices
Strictly Diagonally Dominant Matrix
for each i = 1, …, n.
n
ijj
ijii aa,1
||||
Theorem: A strictly diagonally dominant matrix A is nonsingular. Moreover, Gaussian elimination can be performed without row or column interchanges, and the computations will be stable with respect to the growth of roundoff errors.
Proof: A is nonsingular – proof by contradiction.
Gaussian elimination can be performed without row or column interchanges – proof by induction: each of the matrices A(2), A(3), …, A(n) generated by the Gaussian elimination is strictly diagonally dominant.
Omitted.
11/17
Chapter 6 Direct Methods for Solving Linear Systems -- Special Types of Matrices
Choleski’s Method for Positive Definite Matrix
Review: A is positive definite
Definition: A matrix A is positive definite if it is symmetric and if
x tA x > 0 for every n-dimensional vector x 0.
A1 is positive definite as well, and aii > 0.
max | aij | max | akk |; ( aij )2 < aii ajj for each i j.
Each of A’s leading principal submatrices Ak has a positive determinant.
HW: Read the proofs onp. 401-402
12/17
Chapter 6 Direct Methods for Solving Linear Systems -- Special Types of Matrices
Consider the LU factorization of a positive definite A:
U =uij
=
u11 uij / uii
11
1
u22
unn
UD~
A is symmetric tUL~ tLDLA
Let D1/2 =
11u
22u
nnu
2/1~LDL
is still a lower-triangular matrix
Why is uii > 0?Since det(Ak) > 0
tLLA~~
A is positive definiteA can be factored in the form LDLt, where L is a unitary lower-triangular matrix and D is a diagonal matrix with positive diagonal entries.
A can be factored in the form LLt, where L is lower-triangular with nonzero diagonal entries.
13/17
Chapter 6 Direct Methods for Solving Linear Systems -- Special Types of Matrices
Algorithm: Choleski’s MethodTo factor the symmetric positive definite nn matrix A into LLt, where L is lower-triangular.
Input: the dimension n; entries aij for 1 i, j n of A.
Output: the entries lij for 1 j i and 1 i n of L.
Step 1 Set ;
Step 2 For j = 2, …, n, set ;
Step 3 For i = 2, …, n1, do steps 4 and 5
Step 4 Set ;
Step 5 For j = i+1, …, n, set ;
Step 6 Set ;
Step 7 Output ( lij for j = 1, …, i and i = 1, …, n );
STOP.
1111 al
1111 / lal jj
1
1
2i
k ikiiii lal
ii
i
k ikjkjiji lllal
1
1
1
1
2n
k nknnnn lal
LDLt is faster, but must be modified to
solve Ax = b.
14/17
Chapter 6 Direct Methods for Solving Linear Systems -- Special Types of Matrices
Crout Reduction for Tridiagonal Linear System
nnnn
nnn
f
f
f
x
x
x
ba
cba
cba
cb
2
1
2
1
111
222
11
Step 1: Find the Crout factorization of A
1
1
1
1
2
1
n
nn
A
Step 2: Solve fyL ,
1
11
fy ),...,2(
)( 1 niyrf
yi
iiii
Step 3: Solve yxU )1,...,1(, 1 nixyxyx iiiinn
The process cannot continue if i = 0. Hence not all the
tridiagonal linear system can be solvedby this method.
15/17
Chapter 6 Direct Methods for Solving Linear Systems -- Special Types of Matrices
Theorem: If A is tridiagonal and diagonally dominant. Moreover, if 0,0,0||||,0|||| 11 iinn caabcb
Then A is nonsingular, and the linear system can be solved.
Note: Note:
If If AA is strictly diagonally dominant, then it is not necessary to is strictly diagonally dominant, then it is not necessary to
have all the entries have all the entries aaii, , bbii, and , and ccii being nonzero. being nonzero.
The method is stable in a sense that all the values obtained duThe method is stable in a sense that all the values obtained du
ring the process will be bounded by the values of the original entring the process will be bounded by the values of the original ent
ries.ries.
The amount of computation is O(The amount of computation is O(nn).).
HW: p.412 #17
16/17
Chapter 6 Direct Methods for Solving Linear Systems -- Special Types of Matrices
Lab 03. There is No Free Lunch
Time Limit: 1 second; Points: 4
One day, CYJJ found an interesting piece of commercial from newspaper: the Cyber-restaurant was offering a kind of "Lunch Special" which was said that one could "buy one get two for free". That is, if you buy one of the dishes on their menu, denoted by di
with price pi , you may get the two neighboring dishes di1 and di+1 for
free! If you pick up d1, then you may get d2 and the last one dn for
free, and if you choose the last one dn, you may get dn1 and d1 for
free.
However, after investigation CYJJ realized that there was no free lunch at all. The price pi of the i-th dish was actually calculated
by adding up twice the cost ci of the dish and half of the costs of the
two "free" dishes. Now given all the prices on the menu, you are asked to help CYJJ find the cost of each of the dishes.
17/17