chap. 9 gauss elimination
TRANSCRIPT
Chapter 9Chapter 9
Gauss EliminationGauss Elimination
Gauss EliminationGauss Elimination
9.1 Solving small numbers of equations 9.2 Naive Gauss Elimination 9.3 Pivoting 9.4 Tridiagonal Systems
MATLAB M-files GaussNaive, GaussPivot, Tridiag
Small MatricesSmall Matrices
For small numbers of equations, can be solved by hand
Graphical Cramer's rule Elimination
nnnn22n11n
2nn2222121
1nn1212111
bxaxaxa
bxaxaxab xaxaxa
Graphical MethodGraphical Method
12
12
21
21
x3x3x2x rearrange 3xx
3xx2
2x1 – x2 = 3x1 + x2 = 3
One solution
Graphical MethodGraphical Method
2x1 – x2 = 3
2x1 – x2 = – 1
No solution
Graphical MethodGraphical Method
6x1 – 3x2 = 92x1 – x2 = 3
Infinite many solution
Graphical MethodGraphical Method
2x1 – x2 = 3
2.1x1 – x2 = 3
Ill conditioned
Cramer’s RuleCramer’s RuleCompute the determinant D2 x 2 matrix
3 x 3 matrix 21122211
2221
1211 aaaaaaaa
D
3231
222113
3331
232112
3332
232211
333231
232221
131211
aaaa
aaaaa
aaaaa
a
aaaaaaaaa
D
Cramer’s RuleCramer’s RuleTo find xk for the following system
Replace kth column of as with bs (i.e., aik bi ) nnnn22n11n
2nn2222121
1nn1212111
bxa...xaxa
bxa...xaxabxa...xaxa
))(
ijk D(a
matrix newDx
ExampleExample3 x 3 matrix
333231
232221
131211
aaaaaaaaa
D
33231
22221
112113
3
33331
23221
131112
2
33323
23222
131211
1
baabaabaa
D1
DDx
abaabaaba
D1
DDx
aabaabaab
D1
DDx
Ill-Conditioned SystemIll-Conditioned System
What happen if the determinant D is very small or zero?
Divided by zero (linearly dependent system)Divided by a small number: Round-off errorLoss of significant digits
0AdetD
Eliminate x2
Subtract to get
2222121
1212111
bxaxabxaxa
2122221212112
1222122211122
baxaaxaabaxaaxaa
aaaababax
aaaababax
babaxaaxaa
21122211
1212112
21121122
2121221
2121221211211122
Elimination MethodElimination Method
Not very practical for large number (> 4) of equations
MATLAB’s MethodsMATLAB’s Methods
Forward slash ( / )Back-slash ( \ )Multiplication by the inverse of the
quantity under the slash
b*AinvxbAxbAx
bAx1
)(\
Gauss EliminationGauss EliminationManipulate equations to eliminate one of the unknownsDevelop algorithm to do this repeatedlyThe goal is to set up upper triangular matrix
Back substitution to find solution (root)
nn
n333
n22322
n1131211
a
aaaaaaaaa
U
Basic Gauss EliminationBasic Gauss Elimination
Direct Method (no iteration required)Forward eliminationColumn-by-column elimination of the
below-diagonal elementsReduce to upper triangular matrixBack-substitution
Naive Gauss EliminationNaive Gauss EliminationBegin with
Multiply the first equation by a21 / a11 and subtract from second equation
nnnn22n11n
2nn2222121
1nn1212111
bxa...xaxa
bxa...xaxabxa...xaxa
nnnn22n11n
111
212nn1
11
21n2212
11
2122111
11
2121
1nn212111
bxa...xaxa
baabxa
aaa...xa
aaaxa
aaa
bxa...xaxa
Gauss EliminationGauss EliminationReduce to
Repeat the forward elimination to get
nnnn22n11n
2nn2222
1nn1212111
bxa...xaxa
bxa...xa bxa...xaxa
nnnn22n
2nn2222
1nn1212111
bxa...xa
bxa...xa bxa...xaxa
Gauss EliminationGauss Elimination
First equation is pivot equationa11 is pivot elementNow multiply second equation by a'32 /a'22
and subtract from third equation
222
323nn2
22
32n3323
22
3233
2nn2323222
1nn1313212111
baabxa
aaaxa
aaa
bxaxaxabxaxaxaxa
Gauss EliminationGauss EliminationRepeat the elimination of ai2 and get
Continue and get nnnn33n
3nn3333
2nn2323222
1nn1313212111
bxaxa
bxaxabxaxaxabxaxaxaxa
)()( 1nnn
1nnn
3nn3333
2nn2323222
1nn1313212111
bxa
bxaxabxaxaxabxaxaxaxa
Now we can perform back substitution to get {x}By simple division
Substitute this into (n-1)th equation
Solve for xn-1
Repeat the process to solve for xn-2 , xn-3 , …. x2, x1
)()(,
)(,
2n1nn
2nn1n1n
2n1n1n bxaxa
Back SubstitutionBack Substitution
)(
)(
1nnn
1nn
n abx
Back substitution: starting with xn Solve for xn1 , xn2 , … , 3, 2, 1
Back SubstitutionBack Substitution
a
xabx
abx
1iii
n
1ijj
1iij
1ii
i
1nnn
1nn
n
)(
)()(
)(
)(
for i = n1, n2, …, 1
0a 1iii )( Naive Gauss EliminationNaive Gauss Elimination
Elimination of first columnElimination of first column
)()()()()()(
1f41f31f2
baaa0baaa0baaa0baaaa
41
31
21
4444342
3343332
2242322
114131211
114141
113131
112121
444434241
334333231
224232221
114131211
aafaafaaf
baaaabaaaabaaaabaaaa
///
Elimination of second columnElimination of second column
)()()()(
//
2f42f3
baa00baa00baaa0baaaa
aafaaf
baaa0baaa0baaa0baaaa
42
32
44443
33433
2242322
114131211
224242
223232
4444342
3343332
2242322
114131211
Elimination of third columnElimination of third column
Upper triangular matrix
)()(
/
3f4
ba000baa00baaa0baaaa
aaf
aaa00aaa00baaa0baaaa
43444
33433
2242322
114131211
33434344443
33433
2242322
114131211
1141431321211
2242432322
3343433
4444
a/)xaxaxab(xa/)xaxab(x
a/)xab(xa/bx
Back-SubstitutionBack-Substitution
Upper triangular matrix
0a,a,a,a 44332211
ba000baa00baaa0baaaa
444
33433
2242322
114131211
ExampleExample
41
31
21
41
31
21
f14f13f12
5141020241100042013201
6f0f
1f
14226241101322113201
)()()()()()(
Forward EliminationForward Elimination
42
32
42
32
f24f23
5141400241000042013201
1f1/2f
5141020241100042013201
)()()()(
Upper Triangular MatrixUpper Triangular Matrix
43
43
f34
3370000241000042013201
14f
5141420241100042013201
)()(
Back-SubstitutionBack-Substitution
13/70x3x21x
8/35x2x
4/352x4x
33/707033/x
431
32
43
4
33/704/358/35
13/70
x
3370000241000042013201
MATLAB Script File: MATLAB Script File: GaussNaiveGaussNaive
Print all factor and Aug(do not suppress output)
Eliminate first column
Eliminate second column
Eliminate third column Back-substitution
Aug = [A, b]
>> format short>> x = GaussNaive(A,b)m = 4n = 4Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1factor = -1Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 0Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 6Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 0 2 -10 -14 -5
factor = 0.5000Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 2 -10 -14 -5factor = 1Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 -14 -14 -5
factor = 14Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 0 -70 -33
x4
x3
x2
x1
x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x = -0.1857 0.2286 -0.1143 0.4714
Algorithm for Gauss eliminationAlgorithm for Gauss elimination1. Forward elimination for each equation j, j = 1 to n-1
for all equations k greater than j (a) multiply equation j by akj /ajj
(b) subtract the result from equation kThis leads to an upper triangular matrix2. Back-Substitution
(a) determine xn from (b) put xn into (n-1)th equation, solve for xn-1
(c) repeat from (b), moving back to n-2, n-3, etc. until all equations are solved
)1n(nn
)1n(nn a/bx
Operation CountOperation Count
Important as matrix gets large
For Gauss elimination
Elimination routine uses on the order of O(n3/3) operations
Back-substitution uses O(n2/2)
Operation CountOperation Count
))(())((,
))(())((,
))(())((,))(())((,
3121nn1n
2knkn1knknn1kk
n2n1n2nn321n1nn1nn21
flopsiontion/DivisMultiplica
flopsubtractionAddition/S
iLoop Inner
kLoop Outer
Total operation counts for elimination stage = 2n3/3 + O(n2)Total operation counts for back substitution stage = n2 + O(n)
Operation CountOperation Count Number of flops (floating-point operations) for
Naive Gauss elimination
%....%.%.
859910676106861000000106761000539866666768155010000671550100588766780510070510
nEliminatio toDue Percentage
32n
FlopsTotal
onSbustitutiBack
nElimination
868
3
Computation time increase rapidly with n Most of the effort is incurred in the elimination step Improve efficiency by reducing the elimination effort
Partial PivotingPartial Pivoting
Problems with Gauss elimination division by zero round off errors ill conditioned systems
Use “Pivoting” to avoid this Find the row with largest absolute coefficient below the
pivot element Switch rows (“partial pivoting”) complete pivoting switch columns also (rarely used)
Round-off ErrorsRound-off ErrorsA lot of chopping with more than n3/3 operations
More important - error is propagated
For large systems (more than 100 equations), round-off error can be very important (machine dependent)
Ill conditioned systems - small changes in coefficients lead to large changes in solution
Round-off errors are especially important for ill-conditioned systems
Ill-conditioned SystemIll-conditioned System
2x1 – x2 = 3
2.1x1 – x2 = 3
Consider
Since slopes are almost equal
22
21
22
212
12
11
12
112
2222121
1212111
abx
aax
abx
aax
bxaxabxaxa
22
21
12
11
aa
aa
Ill-Conditioned SystemIll-Conditioned System
0aaaa
D2221
1211 Divided by small number
)1n(nn
n333
n22322
n1131211
a
aaaaaaaaa
U
)(detdet 1nnn332211 aaaaUA
DeterminantDeterminant
Calculate determinant using Gauss elimination
Gauss Elimination with Partial PivotingGauss Elimination with Partial Pivoting
Forward elimination
for each equation j, j = 1 to n-1
first scale each equation k greater than j then pivot (switch rows) Now perform the elimination
(a) multiply equation j by akj /ajj
(b) subtract the result from equation
Partial (Row) PivotingPartial (Row) Pivoting
1x4x2x2x62x4xx1x3x2x2x1 x3x2x
4321
432
4321
431
14226241101322113201
bA
Forward EliminationForward EliminationInterchange rows 1 & 4
41
31
21
41
31
21
f14f13f12
5/67/35/31/3024110
5/67/37/37/3014226
1/6f0f
1/6f
13201241101322114226
)()()()()()(
Forward EliminationForward Elimination
No interchange required
42
32
42
32
f24f23
5/7220033/145000
5/67/37/37/3014226
1/7f3/7f
5/67/35/31/3024110
5/67/37/37/3014226
)()()()(
Back-SubstitutionBack-Substitution
Interchange rows 3 & 4
13/70/6x 2x 2x 41x8/357/3/x 7/3x 7/35/6x
4/35/2x 25/7x33/70/533/14x
2341
342
43
4
)()()(
)()(
33/704/358/35
13/70
x
0f
33/1450005/722005/67/37/37/30
14226
43
MATLAB M-File: MATLAB M-File: GaussPivotGaussPivot
Partial Pivoting (switch rows)
[big,i] = max(x)
largest element in {x}
index of the largest element
Partial Pivoting
>> format short>> x=GaussPivot0(A,b)Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1big = 6i = 4ipr = 4Aug = 6 2 2 4 1 -1 2 2 -3 -1 0 1 1 4 2 1 0 2 3 1factor = -0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 0 -0.3333 1.6667 2.3333 0.8333
Aug = [A b]
Interchange rows 1 and 4
Find the first pivot element and its index
Eliminate first columnNo need to interchange
big = 2.3333i = 1ipr = 2factor = 0.4286Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 -0.3333 1.6667 2.3333 0.8333factor = -0.1429Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 0 2.0000 2.0000 0.7143big = 2i = 2ipr = 4Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571
x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x =
-0.1857 0.2286 -0.1143 0.4714
Back substitutionSecond pivot element and index
Third pivot element and index
Eliminate second column
Eliminate third column
Interchange rows 3 and 4
No need to interchange
Save factors fij for
LU Decomposition
1
4
2 3
5
F14
F23F12
F24
F45
H1
F35
F25
V3V1
TRUSSTRUSS
W = 100 kg
Statics: Force BalanceStatics: Force Balance
0FFFF
0FFF
0FFFF
0FFF
0FFF
0FVF
0FFFFF
100FFF
0FFHF
0FVF
4535255x
35255y
4524144x
24144y
35233x
3533y
252423122x
25242y
141211x
1411y
coscos
sinsin
coscos
sinsin
cos
sin
coscos
sinsin
sin
sin
,
,
,
,
,
,
,
,
,
,
Node 1
Node 2
Node 3
Node 4
Node 5
Example: Forces in a Simple TrussExample: Forces in a Simple Truss
0000000
10000
FFFFFFFVHV
1coscos00000000sinsin0000000100cos0cos0000000sin0sin00000cos001000000sin0000010000coscos10100000sinsin00000000000cos101000000sin0001
45
35
25
24
23
14
12
3
1
1
function [A,b]=Truss(alpha,beta,gamma,delta)
A=zeros(10,10);A(1,1)=1; A(1,5)=sin(alpha);A(2,2)=1; A(2,4)=1; A(2,5)=cos(alpha);A(3,7)=sin(beta); A(3,8)=sin(gamma);A(4,4)=-1; A(4,6)=1; A(4,7)=-cos(beta); A(4,8)=cos(gamma);A(5,3)=1; A(5,9)=sin(gamma);A(6,6)=-1; A(6,9)=-cos(delta);A(7,5)=-sin(alpha); A(7,7)=-sin(beta);A(8,5)=-cos(alpha); A(8,7)=cos(beta); A(8,10)=1;A(9,8)=-sin(gamma); A(9,9)=-sin(delta);A(10,8)=-cos(gamma); A(10,9)=cos(delta); A(10,10)=-1;
b=zeros(10,1); b(3,1)=100;
Define Matrices A and b in script fileDefine Matrices A and b in script file
>> alpha=pi/6; beta=pi/3; gamma=pi/4; delta=pi/3;>> [A,b] = Truss(alpha,beta,gamma,delta)A = 1.0000 0 0 0 0.5000 0 0 0 0 0 0 1.0000 0 1.0000 0.8660 0 0 0 0 0 0 0 0 0 0 0 0.8660 0.7071 0 0 0 0 0 -1.0000 0 1.0000 -0.5000 0.7071 0 0 0 0 1.0000 0 0 0 0 0 0.7071 0 0 0 0 0 0 -1.0000 0 0 -0.5000 0 0 0 0 0 -0.5000 0 -0.8660 0 0 0 0 0 0 0 -0.8660 0 0.5000 0 0 1.0000 0 0 0 0 0 0 0 -0.7071 -0.8660 0 0 0 0 0 0 0 0 -0.7071 0.5000 -1.0000b = 0 0 100 0 0 0 0 0 0 0>> x = GaussPivot(A,b)x = 40.5827 0 48.5140 70.2914 -81.1655 34.3046 46.8609 84.0287 -68.6091 -93.7218
Gauss Elimination with Partial PivotingGauss Elimination with Partial Pivoting
Simple truss
Banded MatrixBanded Matrix
HBW: Half Band Width
•Banded Matrix
xwvut
srqpzk
nmlkji
hgfedc
ba
0000000000000
0000000000000000000000000000000000000000000
ai,j= 0if j > i + HBor j< i - HB
HB: Half BandwidthB: Bandwidth
B = 2*HB + 1
In this exampleHB = 1 & B = 3
Tridiagonal MatrixTridiagonal Matrix
Only three nonzero elements in each equation (3n instead of n2 elements)
Subdiagonal, diagonal, superdiagonal
Row Scaling (not implemented in textbook) -- scale the diagonal element to aii = 1Solve by Gauss elimination
n
1n
i
3
2
1
n
1n
i
3
2
1
nn
1n1n1n
iii
333
222
11
rr
r
rrr
xx
x
xxx
fegfe
gfe
gfegfe
gf
Tridiagonal MatrixTridiagonal Matrix
Special case of banded matrix with bandwidth = 3 Save storage, 3 n instead of n n
Tridiagonal MatrixTridiagonal Matrix Forward elimination
Back substitution
n32k r
ferr
gfeff
1k1k
kkk
1k1k
kkk
,,,
1232n1nk f
xgrx
frx
k
1kkkk
n
nn
,,,,,
Use factor = ek / fk1
to eliminate subdiagonal element
Apply the same matrix operations to right hand side
Hand Calculations: Tridiagonal MatrixHand Calculations: Tridiagonal Matrix
53253
xxxx
25150005021001520021
4
3
2
1
....
411
5053rferr
1501
50251gfeff
11112r
ferr
11112g
feff
13125r
ferr
12125g
feff
33
444
33
444
11
223
22
333
11
222
11
222
)(..
).(..
)(
)(
)(
)(
11
223f
xgrx
21
311f
xgrx
31
4501f
xgrx
414
frx
1
2111
2
3222
3
4333
4
44
))((
))((
))(.(
(a) Forward elimination (b) Back substitution
MATLAB M-file:MATLAB M-file: Tridiag Tridiag
» [e,f,g,r] = example
e = 0 -2.0000 4.0000 -0.5000 1.5000 -3.0000f = 1.0000 6.0000 9.0000 3.2500 1.7500 13.0000g = -2.0000 4.0000 -0.5000 1.5000 -3.0000 0r = -3.0000 22.0000 35.5000 -7.7500 4.0000 -33.0000
» x = Tridiag (e, f, g, r)x = 1 2 3 -1 -2 -3
Example: Tridiagonal matrixExample: Tridiagonal matrixfunction [e,f,g,r] = example
e=[ 0 -2 4 -0.5 1.5 -3];f=[ 1 6 9 3.25 1.75 13];g=[-2 4 -0.5 1.5 -3 0];r=[-3 22 35.5 -7.75 4 -33];
334
75.75.35
223
xxxxxx
133375.15.1
5.125.35.05.094
46221
6
5
4
3
2
1
Note: e(1) = 0 and g(n) = 0
CVEN 302-501CVEN 302-501Homework No. 6Homework No. 6
Chapter 8Problem 8.2 (20), 8.3(10), 8.7(15)Chapter 9Problem 9.5 (15), 9.8(20), 9.11(20).
Due Monday 10/06/08 at the beginning Due Monday 10/06/08 at the beginning of the periodof the period