se301: numerical methods topic 3: solution of systems of linear equations lectures 12-17:
DESCRIPTION
SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:. KFUPM Read Chapter 9 of the textbook. Lecture 12 Vector, Matrices, and Linear Equations. VECTORS. MATRICES. MATRICES. Determinant of a MATRICES. Adding and Multiplying Matrices. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/1.jpg)
CISE301_Topic3 1
SE301: Numerical Methods
Topic 3: Solution of Systems of Linear
Equations Lectures 12-17:
KFUPM
Read Chapter 9 of the textbook
![Page 2: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/2.jpg)
CISE301_Topic3 2
Lecture 12Vector, Matrices, and
Linear Equations
![Page 3: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/3.jpg)
CISE301_Topic3 3
VECTORSVECTORS
1
0
0
0
,
0
1
0
0
,
0
0
1
0
,
0
0
0
1
1
2tor column vec 241 vector row
:Examples
numbers ofarray ldimensiona one a:Vector
4321 eeeevectorsIdentity
![Page 4: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/4.jpg)
CISE301_Topic3 4
MATRICESMATRICES
1200
1410
0143
0021
lTridiagona,
6000
0000
0040
0001
diagonal
10
01matrix identity
0
0
00
00matrix zero
:Examples
numbers ofarray ldimensiona twoa:Matrix
![Page 5: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/5.jpg)
CISE301_Topic3 5
MATRICESMATRICES
1000
1400
0140
3121
ngular upper tria,
451
501
112
symmetric
:Examples
![Page 6: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/6.jpg)
CISE301_Topic3 6
Determinant of a Determinant of a MATRICESMATRICES
82)015(1)512(1)25(2
50
1-31-
45
1-31-
45
502
451
501
132
det
:Examples
only matrices squarefor Defined
![Page 7: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/7.jpg)
CISE301_Topic3 7
Adding and Multiplying Adding and Multiplying MatricesMatrices
ji
ji
m
k
,bacBAC *
pm ifonly defined is ABCproduct The *
q)B(p and m)(nA matrices twooftion Multiplica
,bacBAC *
size same thehave they ifonly Defined *
B andA matrices twoofaddition The
1kjikij
ijijij
![Page 8: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/8.jpg)
CISE301_Topic3 8
Systems of Linear Equations
formMatrix form Standard
7
5
3
601
315.2
342
76
535.2
3342
formsdifferent in
presented becan equationslinear of systemA
3
2
1
31
321
321
x
x
x
xx
xxx
xxx
![Page 9: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/9.jpg)
CISE301_Topic3 9
Solutions of Linear Equations
52
3
:equations following theosolution t a is 2
1
21
21
2
1
xx
xx
x
x
![Page 10: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/10.jpg)
CISE301_Topic3 10
Solutions of Linear Equations A set of equations is inconsistent if there
exists no solution to the system of equations:
ntinconsiste are equations These
542
32
21
21
xx
xx
![Page 11: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/11.jpg)
CISE301_Topic3 11
Solutions of Linear Equations Some systems of equations may have infinite
number of solutions
allfor solution ais)3(5.0
solutions ofnumber infinite have
642
32
2
1
21
21
aa
a
x
x
xx
xx
![Page 12: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/12.jpg)
CISE301_Topic3 12
Graphical Solution of Systems ofLinear Equations
52
3
21
21
xx
xx
Solution
x1=1, x2=2
![Page 13: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/13.jpg)
CISE301_Topic3 13
Cramer’s Rule is Not Practical
way efficient in computed are tsdeterminan theif used be canIt
needed. are tionsmultiplica 102.38 system, 30by 30a solve To
tions.multiplica 1)N!-1)(N(N requires system Nby N solve To
. systems largefor practicalnot is RulesCramer'
2
21
1151
31
,1
21
1125
13
system thesolve toused be can RulesCramer'
35
21
xx
![Page 14: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/14.jpg)
CISE301_Topic3 14
Naive Gaussian Elimination Examples
Lecture 13 Naive Gaussian
Elimination
![Page 15: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/15.jpg)
CISE301_Topic3 15
Naive Gaussian Elimination The method consists of two steps:
Forward Elimination: the system is reduced to upper triangular form. A sequence of elementary operations is used.
Backward Substitution: Solve the system starting from the last variable.
'
'
'00
''0
3
2
1
3
2
1
33
2322
131211
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
a
aa
aaa
b
b
b
x
x
x
aaa
aaa
aaa
![Page 16: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/16.jpg)
CISE301_Topic3 16
Elementary Row Operations
Adding a multiple of one row to another
Multiply any row by a non-zero constant
![Page 17: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/17.jpg)
CISE301_Topic3 17
ExampleForward Elimination
18
27
6
16
14320
18120
2240
4226
4 3, 2,equationsfrom Eliminate:Step1
nEliminatio Forward:1Part
34
19
26
16
18146
39133
106812
4226
4
3
2
1
1
4
3
2
1
x
x
x
x
x
x
x
x
x
![Page 18: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/18.jpg)
CISE301_Topic3 18
ExampleForward Elimination
3
9
6
16
3000
5200
2240
4226
4equation from Eliminate:Step3
21
9
6
16
13400
5200
2240
4226
4 ,3equationsfrom Eliminate:Step2
4
3
2
1
3
4
3
2
1
2
x
x
x
x
x
x
x
x
x
x
![Page 19: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/19.jpg)
CISE301_Topic3 19
ExampleForward Elimination
3
9
6
16
3000
5200
2240
4226
34
19
26
16
18146
39133
106812
4226
:nEliminatio Forward theofSummary
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x
![Page 20: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/20.jpg)
CISE301_Topic3 20
ExampleBackward Substitution
36
)1(4)2(2)1(216,1
4
)1(2)2(26
22
59,1
3
3
for solve ,...for solvethen ,for Solve
3
9
6
16
3000
5200
2240
4226
12
34
134
4
3
2
1
xx
xx
xxx
x
x
x
x
![Page 21: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/21.jpg)
CISE301_Topic3 21
Forward Elimination
ni
ba
abb
njaa
aaa
x
ni
ba
abb
njaa
aaa
x
iii
ji
ijij
iii
ji
ijij
3
)2(
eliminate To
2
)1(
eliminate To
222
2
222
2
2
111
1
111
1
1
![Page 22: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/22.jpg)
CISE301_Topic3 22
Forward Elimination
.eliminated is until Continue
1
)(
eliminate To
1
n
kkk
ikii
kjkk
ikijij
k
x
nik
ba
abb
njkaa
aaa
x
![Page 23: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/23.jpg)
CISE301_Topic3 23
Backward Substitution
ii
n
ijjjii
i
nn
nnnnnnnn
nn
nnnnn
nn
nn
a
xab
x
a
xaxabx
a
xabx
a
bx
,
1,
2,2
11,2,222
1,1
,111
,
![Page 24: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/24.jpg)
CISE301_Topic3 24
Summary of the Naive Gaussian Elimination Example Problems with Naive Gaussian Elimination
Failure due to zero pivot element Error
Pseudo-Code
Lecture 14Naive Gaussian
Elimination
![Page 25: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/25.jpg)
CISE301_Topic3 25
Naive Gaussian Elimination
o The method consists of two stepso Forward Elimination: the system is reduced to
upper triangular form. A sequence of elementary operations is used.
o Backward Substitution: Solve the system starting from the last variable. Solve for xn ,xn-1,…x1.
'
'
'00
''0
3
2
1
3
2
1
33
2322
131211
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
a
aa
aaa
b
b
b
x
x
x
aaa
aaa
aaa
![Page 26: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/26.jpg)
CISE301_Topic3 26
Example 1
1775
64
832
11
333723
11
22210232
)(1832
3 ,2equations from Eliminate:Step1___nEliminatio Forward:1Part
:nEliminatioGaussian Naive using Solve
32
32
321
321
321
321
1
xx
xx
xxx
eqeqeqxxx
eqeqeqxxx
equationpivotunchangedeqxxx
x
![Page 27: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/27.jpg)
CISE301_Topic3 27
Example 1
1313
64
832
21
5331775
)(264
1832
3equationfrom Eliminate:Step2nEliminatio Forward:1Part
3
32
321
32
32
321
2
x
xx
xxx
eqeqeqxx
equationpivotunchangedeqxx
unchangedeqxxx
x
![Page 28: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/28.jpg)
CISE301_Topic3 28
Example 1Backward Substitution
1
2
1
issolution The
1328
21
46
113
13
3
2
1
1,1
32
1,1
33,122,111
3
2,2
33,222
3,3
33
x
x
x
a
xx
a
xaxabx
x
a
xabx
a
bx
![Page 29: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/29.jpg)
CISE301_Topic3 29
Determinant
13detdet
1300
410
321
A'
213
232
321
A
:Example
tdeterminan affect thenot do operations elementary The
operations Elementary
(A')(A)
![Page 30: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/30.jpg)
CISE301_Topic3 30
How Many Solutions Does a System of Equations AX=B Have?
0 elements0 elements
B ingcorrespondB ingcorrespond
rows zerorows zero
moreor one has moreor one hasrows zero no has
matrix reducedmatrix reducedmatrix reduced
0det(A)0det(A)0det(A)
Infinitesolution NoUnique
![Page 31: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/31.jpg)
CISE301_Topic3 31
Examples
5.1
!105.0
0
#:
0
2
00
21
1
2
00
21
1
1
20
21
4
2
42
21
3
2
42
21
2
1
43
21
solutions of # infintesolution NoUnique
XimpossibleX
solutionsInfinitesolutionNosolution
XXX
XXX
![Page 32: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/32.jpg)
Pseudo-Code: Forward EliminationDo k = 1 to n-1
Do i = k+1 to nfactor = ai,k / ak,k
Do j = k+1 to nai,j = ai,j – factor * ak,j
End Dobi = bi – factor * bk
End DoEnd DoCISE301_Topic3 32
![Page 33: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/33.jpg)
Pseudo-Code: Back Substitutionxn = bn / an,n
Do i = n-1 downto 1sum = bi
Do j = i+1 to nsum = sum – ai,j * xj
End Doxi = sum / ai,i
End Do
CISE301_Topic3 33
![Page 34: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/34.jpg)
CISE301_Topic3 34
Lectures 15-16:Gaussian Elimination
with Scaled Partial Pivoting
Problems with Naive Gaussian Elimination Definitions and Initial step Forward Elimination Backward substitution Example
![Page 35: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/35.jpg)
CISE301_Topic3 35
Problems with Naive Gaussian Elimination
o The Naive Gaussian Elimination may fail for very simple cases. (The pivoting element is zero).
o Very small pivoting element may result in serious computation errors
2
1
11
10
2
1
x
x
2
1
11
110
2
110
x
x
![Page 36: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/36.jpg)
CISE301_Topic3 36
Example 2
1
1
1
1
3524
3685
4123
1211
: Pivoting PartialScaled with
on EliminatiGaussian using system following theSolve
4
3
2
1
x
x
x
x
![Page 37: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/37.jpg)
CISE301_Topic3 37
Example 2Initialization step
4321LVectorIndex
5842S vectorScale
1
1
1
1
3524
3685
4123
1211
4
3
2
1
x
x
x
xScale vector:
disregard sign
find largest in magnitude in each row
![Page 38: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/38.jpg)
CISE301_Topic3 38
Why Index Vector? Index vectors are used because it is much
easier to exchange a single index element compared to exchanging the values of a complete row.
In practical problems with very large N, exchanging the contents of rows may not be practical.
![Page 39: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/39.jpg)
CISE301_Topic3 39
Example 2Forward Elimination-- Step 1: eliminate x1
]1324[
Exchangeequation pivot first theis 4equation
toscorrespondmax 5
4,
8
5,
4
3,
2
14,3,2,1
]4321[
]5842[
1
1
1
1
3524
3685
4123
1211
equationpivot theofSelection
14
4
1,
4
3
2
1
L
landl
liS
aRatios
L
S
x
x
x
x
i
i
l
l
![Page 40: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/40.jpg)
CISE301_Topic3 40
Example 2Forward Elimination-- Step 1: eliminate x1
1
25.2
75.1
25.1
3524
75.025.05.50
75.175.25.00
25.075.05.10
1
1
1
1
3524
3685
4123
1211
B andA Update
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x
First pivot equation
![Page 41: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/41.jpg)
CISE301_Topic3 41
Example 2Forward Elimination-- Step 2: eliminate x2
]2314[2
5.1
8
5.5
4
5.04,3,2:Ratios
]1324[]5842[
1
25.2
75.1
25.1
3524
75.025.05.50
75.175.25.00
25.075.05.10
equationpivot second theofSelection
2,
4
3
2
1
LiS
a
LS
x
x
x
x
i
i
l
l
![Page 42: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/42.jpg)
CISE301_Topic3 42
Example 2Forward Elimination-- Step 3: eliminate x3
1
9
1667.2
25.1
3524
2000
8333.15.200
25.075.05.10
]3214[
1
8333.6
1667.2
25.1
3524
6667.125.000
8333.15.200
25.075.05.10
4
3
2
1
4
3
2
1
x
x
x
x
L
x
x
x
xThird pivot equation
![Page 43: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/43.jpg)
CISE301_Topic3 43
Example 2Backward Substitution
7.23334
2531
1.13335.1
75.025.025.1
2.43275.28333.11667.2
,5.429
]3214[
1
9
1667.2
25.1
3524
2000
8333.15.200
25.075.05.10
234
1,
22,433,444,441
34
2,1
33,144,112
4
3,2
44,223
4,3
34
4
3
2
1
1
xxx
a
xaxaxabx
xx
a
xaxabx
xa
xabx
a
bx
L
x
x
x
x
l
![Page 44: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/44.jpg)
CISE301_Topic3 44
Example 3
1
1
1
1
3524
3685
4123
1211
Pivoting Partial Scaledwith
nEliminatioGaussian using sytstem following theSolve
4
3
2
1
x
x
x
x
![Page 45: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/45.jpg)
CISE301_Topic3 45
Example 3Initialization step
4321LVectorIndex
5842S vectorScale
1
1
1
1
3524
3685
4123
1211
4
3
2
1
x
x
x
x
![Page 46: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/46.jpg)
CISE301_Topic3 46
Example 3Forward Elimination-- Step 1: eliminate x1
]1324[
Exchangeequation pivot first theis 4equation
toscorrespondmax 5
4,
8
5,
4
3,
2
14,3,2,1
]4321[
]5842[
1
1
1
1
3524
3685
4123
1211
equationpivot theofSelection
14
41,
4
3
2
1
L
landl
liS
aRatios
L
S
x
x
x
x
i
i
l
l
![Page 47: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/47.jpg)
CISE301_Topic3 47
Example 3Forward Elimination-- Step 1: eliminate x1
1
25.2
75.1
25.1
3524
75.025.05.100
75.175.25.00
25.075.05.10
1
1
1
1
3524
3685
4133
1211
B andA Update
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x
![Page 48: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/48.jpg)
CISE301_Topic3 48
Example 3Forward Elimination-- Step 2: eliminate x2
]1234[2
5.1
8
5.10
4
5.04,3,2:Ratios
]1324[]5842[
1
25.2
75.1
25.1
3524
75.025.05.100
75.175.25.00
25.075.05.10
equationpivot second theofSelection
2,
4
3
2
1
LiS
a
LS
x
x
x
x
i
i
l
l
![Page 49: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/49.jpg)
CISE301_Topic3 49
Example 3Forward Elimination-- Step 2: eliminate x2
1
2.25
1.8571
0.9286
3524
75.025.05.100
1.71432.7619-00
0.35710.785700
]2314[
1
25.2
75.1
25.1
3524
75.025.05.100
75.175.25.00
25.075.05.10
B andA Updating
4
3
2
1
4
3
2
1
x
x
x
x
L
x
x
x
x
![Page 50: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/50.jpg)
CISE301_Topic3 50
Example 3Forward Elimination-- Step 3: eliminate x3
]1234[2
0.7857
4
2.76194,3:Ratios
]1234[]5842[
1
2.25
1.8571
0.9286
3524
75.025.05.100
1.71432.761900
0.35710.785700
equationpivot third theofSelection
3,
4
3
2
1
LiS
a
LS
x
x
x
x
i
i
l
l
![Page 51: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/51.jpg)
CISE301_Topic3 51
Example 3Forward Elimination-- Step 3: eliminate x3
1
2.25
1.8571
1.4569
3524
75.025.05.100
1.71432.761900
0.8448000
]1234[
1
2.25
1.8571
0.9286
3524
75.025.05.100
1.71432.761900
0.35710.785700
4
3
2
1
4
3
2
1
x
x
x
x
L
x
x
x
x
![Page 52: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/52.jpg)
CISE301_Topic3 52
Example 3Backward Substitution
1.86734
2531
0.3469
0.39802.7619
1.71431.8571,1.7245
0.84481.4569
]1234[
1
2.25
1.8571
1.4569
3524
75.025.05.100
1.71432.761900
0.8448000
234
1,
22,33,44,1
2,
33,44,2
4
3,
44,3
4,4
4
3
2
1
1
1111
2
222
3
33
4
4
xxx
a
xaxaxabx
a
xaxabx
xa
xabx
a
bx
L
x
x
x
x
l
llll
l
lll
l
ll
l
l
![Page 53: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/53.jpg)
CISE301_Topic3 53
How Do We Know If a Solution is Good or Not
Given AX=BX is a solution if AX-B=0
Compute the residual vector R= AX-B
Due to rounding error, R may not be zero
ii
rmax if acceptable issolution The
![Page 54: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/54.jpg)
CISE301_Topic3 54
How Good is the Solution?
0.001
0.003
0.002
0.005
:Residues
1.7245
0.3980
0.3469
1.8673
1
1
1
1
3524
3685
4123
1211
4
3
2
1
4
3
2
1
R
x
x
x
x
solution
x
x
x
x
![Page 55: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/55.jpg)
CISE301_Topic3 55
Remarks: We use index vector to avoid the need to move
the rows which may not be practical for large problems.
If we order the equation as in the last value of the index vector, we have a triangular form.
Scale vector is formed by taking maximum in magnitude in each row.
Scale vector does not change. The original matrices A and B are used in
checking the residuals.
![Page 56: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/56.jpg)
CISE301_Topic3 56
Lecture 17 Tridiagonal & Banded
Systems and Gauss-Jordan
Method
Tridiagonal Systems Diagonal Dominance Tridiagonal Algorithm Examples Gauss-Jordan Algorithm
![Page 57: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/57.jpg)
CISE301_Topic3 57
Tridiagonal Systems: The non-zero elements are
in the main diagonal, super diagonal and subdiagonal.
aij=0 if |i-j| > 1
5
4
3
2
1
5
4
3
2
1
61000
14100
02620
00143
00015
b
b
b
b
b
x
x
x
x
x
Tridiagonal Systems
![Page 58: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/58.jpg)
CISE301_Topic3 58
Occur in many applications Needs less storage (4n-2 compared to n2 +n for the general cases)
Selection of pivoting rows is unnecessary (under some conditions)
Efficiently solved by Gaussian elimination
Tridiagonal Systems
![Page 59: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/59.jpg)
CISE301_Topic3 59
Based on Naive Gaussian elimination. As in previous Gaussian elimination algorithms
Forward elimination step Backward substitution step
Elements in the super diagonal are not affected. Elements in the main diagonal, and B need
updating
Algorithm to Solve Tridiagonal Systems
![Page 60: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/60.jpg)
CISE301_Topic3 60
Tridiagonal System
'
'3
'2
1
3
2
1
'1
'3
2'2
11
3
2
1
3
2
1
1
1
32
221
11
updatednot are elements The
elements and theupdate toneed zeros, be willelements theAll
nnn
n
nnnn
n
b
b
b
b
x
x
x
x
d
c
d
cd
cd
b
b
b
b
x
x
x
x
da
c
da
cda
cd
c
bda
![Page 61: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/61.jpg)
CISE301_Topic3 61
Diagonal Dominance
row. ingcorrespond in the elements of sum the
nlarger tha iselement diagonaleach of magnitude The
)1(aa
ifdominant diagonally is matrix A
,1ijii nifor
An
ijj
![Page 62: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/62.jpg)
CISE301_Topic3 62
Diagonal Dominance
dominant DiagonallyNot dominant Diagonally
121
232
103
521
161
103
:Examples
![Page 63: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/63.jpg)
CISE301_Topic3 63
Diagonally Dominant Tridiagonal System
A tridiagonal system is diagonally dominant if
Forward Elimination preserves diagonal dominance
)1(1 niacd iii
![Page 64: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/64.jpg)
CISE301_Topic3 64
Solving Tridiagonal System
1,...,2,1for 1
onSubstituti Backward
2
nEliminatio Forward
1
11
1
11
1
nnixcbd
x
d
bx
nibd
abb
cd
add
iiii
i
n
nn
ii
iii
ii
iii
![Page 65: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/65.jpg)
CISE301_Topic3 65
Example
1,2,3for 1
,
onSubstituti Backward
42,
nEliminatio Forward
6
8
9
12
,2
2
2
,1
1
1
,
5
5
5
5
6
8
9
12
51
251
251
25
Solve
1
11
11
1
1
4
3
2
1
ixcbd
xd
bx
ibd
abbc
d
add
BCAD
x
x
x
x
iiii
in
nn
ii
iiii
i
iii
![Page 66: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/66.jpg)
CISE301_Topic3 66
Example
5619.45652.4
5652.616,5619.4
5652.4
215
5652.66.4
6.618,5652.4
6.4
215
6.65
1219,6.4
5
215
nEliminatio Forward
6
8
9
12
,2
2
2
,1
1
1
,
5
5
5
5
33
3443
3
344
22
2332
2
233
11
1221
1
122
bd
abbc
d
add
bd
abbc
d
add
bd
abbc
d
add
BCAD
![Page 67: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/67.jpg)
CISE301_Topic3 67
ExampleBackward Substitution
After the Forward Elimination:
Backward Substitution:
25
1212
16.4
126.6
15652.4
125652.6
,15619.4
5619.4
5619.45652.66.612,5619.45652.46.45
1
2111
2
3222
3
4333
4
44
d
xcbx
d
xcbx
d
xcbx
d
bx
BD TT
![Page 68: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/68.jpg)
CISE301_Topic3 68
Gauss-Jordan Method The method reduces the general system of
equations AX=B to IX=B where I is an identity matrix.
Only Forward elimination is done and no backward substitution is needed.
It has the same problems as Naive Gaussian elimination and can be modified to do partial scaled pivoting.
It takes 50% more time than Naive Gaussian method.
![Page 69: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/69.jpg)
CISE301_Topic3 69
Gauss-Jordan MethodExample
2
7
0
200
560
111
11
233
11
422
2/11
32fromxEleminate1
2
7
0
422
124
222
3
2
1
1
3
2
1
x
x
x
eqeqeq
eqeqeq
eqeq
andequationsStep
x
x
x
![Page 70: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/70.jpg)
CISE301_Topic3 70
Gauss-Jordan MethodExample
2
1.1667
1.1667
200
0.833310
1667.001
21
033
21
111
6/22
31fromxEleminate2
2
7
0
200
560
111
3
2
1
2
3
2
1
x
x
x
eqeqeq
eqeqeq
eqeq
andequationsStep
x
x
x
![Page 71: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/71.jpg)
CISE301_Topic3 71
Gauss-Jordan MethodExample
1
2
1
100
010
001
31
0.833322
31
1667.011
2/33
21fromxEleminate3
2
1.1667
1.1667
200
0.833310
1667.001
3
2
1
3
3
2
1
x
x
x
eqeqeq
eqeqeq
eqeq
andequationsStep
x
x
x
![Page 72: SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17:](https://reader036.vdocument.in/reader036/viewer/2022062308/56812bd1550346895d9032de/html5/thumbnails/72.jpg)
CISE301_Topic3 72
Gauss-Jordan MethodExample
1
2
1
1
2
1
100
010
001
2
7
0
422
124
222
3
2
1
3
2
1
3
2
1
x
x
x
issolution
x
x
x
todtransformeis
x
x
x