ch 7. iterative techniques in matrix...
TRANSCRIPT
![Page 1: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/1.jpg)
Ch 7. Iterative Techniques in Matrix Algebra
Jeong-hwan Gwak
![Page 2: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/2.jpg)
2
Contents
7.1 Norms of Vectors and Matrices 7.1 Norms of Vectors and Matrices 7.2 7.2 EigenvaluesEigenvalues and Eigenvectorsand Eigenvectors7.3 Iterative Techniques for Solving Linear Systems7.3 Iterative Techniques for Solving Linear Systems7.4 Error Bounds and Iterative Refinement7.4 Error Bounds and Iterative Refinement7.5 The Conjugate Gradient Method7.5 The Conjugate Gradient Method
![Page 3: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/3.jpg)
3
![Page 4: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/4.jpg)
4
![Page 5: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/5.jpg)
5
![Page 6: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/6.jpg)
6
![Page 7: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/7.jpg)
7
![Page 8: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/8.jpg)
8
![Page 9: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/9.jpg)
9
![Page 10: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/10.jpg)
10
![Page 11: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/11.jpg)
11
![Page 12: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/12.jpg)
12
![Page 13: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/13.jpg)
13
![Page 14: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/14.jpg)
14
![Page 15: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/15.jpg)
15
![Page 16: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/16.jpg)
16
![Page 17: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/17.jpg)
17
![Page 18: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/18.jpg)
18
(1)(1)
(2)(2)
![Page 19: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/19.jpg)
19
![Page 20: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/20.jpg)
20
![Page 21: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/21.jpg)
21
Jacobi Method
![Page 22: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/22.jpg)
22
![Page 23: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/23.jpg)
23
![Page 24: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/24.jpg)
24
Gauss-Seidel Method
![Page 25: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/25.jpg)
25
![Page 26: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/26.jpg)
26
![Page 27: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/27.jpg)
27
![Page 28: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/28.jpg)
28
![Page 29: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/29.jpg)
29
Convergence of general iterative technique
![Page 30: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/30.jpg)
30
![Page 31: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/31.jpg)
31
SOR Method
if w=1 : Gauss-Seidel method
![Page 32: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/32.jpg)
32
![Page 33: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/33.jpg)
33
![Page 34: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/34.jpg)
34
![Page 35: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/35.jpg)
35
How the appropriate value of w is chosen?
![Page 36: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/36.jpg)
36
![Page 37: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/37.jpg)
37
![Page 38: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/38.jpg)
38
![Page 39: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/39.jpg)
39
![Page 40: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/40.jpg)
40
![Page 41: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/41.jpg)
41
![Page 42: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/42.jpg)
42
![Page 43: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/43.jpg)
43
![Page 44: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/44.jpg)
44
![Page 45: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/45.jpg)
Department of Mathematics
The conjugate gradient Method
• To solve positive definite linear systemn n×
• More computationally expensive than those in Gaussian elimination
• Very useful when employed as an iterative approximation method for solving large sparse systems with nonzero entries occurring inpredictable patterns
• When the matrix has been preconditioned to make the calculationsmore effective, good results are obtained in only about steps.n
![Page 46: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/46.jpg)
Department of Mathematics
▪ Assumption matrix is positive definite
▪ Notation (inner product)where and are vectors.
▪ Some properties ( inner product )For any vectors , , and , and any real number
A ( , , 0 )tA A unless∀ ⟨ ⟩ = > =x x x x x x 0
, t⟨ ⟩ =x y x y x y dimn −
x y z αi ) , ,⟨ ⟩ = ⟨ ⟩x y y xii ) , , ,α α α⟨ ⟩ = ⟨ ⟩ = ⟨ ⟩x y x y x yiii ) , , ,⟨ + ⟩ = ⟨ ⟩ + ⟨ ⟩x z y x y z yiv) ,⟨ ⟩ ≥x x 0v) ,⟨ ⟩ = ⇔ =x x 0 x 0vi ) , ,A A when A is positive definite⟨ ⟩ = ⟨ ⟩x y x y
![Page 47: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/47.jpg)
Department of Mathematics
The vector is a solution to the positive definite linear system
iff minimizes
Forhas minimum ,
when
(i ) *x =Ax b
( ) , 2 ,g = ⟨ ⟩ − ⟨ ⟩x x Ax x b
(ii ) ,∀ ≠x v 0( )g t+x v
,,
t ⟨ − ⟩=
⟨ ⟩v b Ax
v Av
Theorem
*x
,,
⟨ ⟩=⟨ ⟩
v rv Av
![Page 48: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/48.jpg)
Department of Mathematics
· is an initial approximation to . is an initial search direction.
· For
compute
.
and choose a new search direction .
(0)x *x(1) ( )≠v 0
1 , 2 , 3 ,k =
( ) ( 1)
( ) ( )
,,
k k
k k kt−⟨ − ⟩
=⟨ ⟩
v b Axv Av
( ) ( 1) ( )k k kkt
−= +x x v
( 1)k+v
![Page 49: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/49.jpg)
Department of Mathematics
· Choice of the search directions (method of steepest descent)
1 2( , , , )tnx x x= ⋅ ⋅ ⋅x
1 2( ) ( , , , )ng g x x x=x
, 2 ,A= ⟨ ⟩ − ⟨ ⟩x x x b
1 1 1
2n n n
ij i j i ii j i
a x x x b= = =
= −∑∑ ∑
1( ) 2 2
n
ki i kik
g a x bx =
∂⇒ = −
∂ ∑x
1 2
( ) ( ), ( ), , ( )t
n
g g ggx x x
⎛ ⎞∂ ∂ ∂⇒∇ = ⎜ ⎟∂ ∂ ∂⎝ ⎠
x x x x 2( ) 2A= − = −x b r
( 1) ( ) ( )k k kA+⇒ = = −v r b x
The direction of greatest decrease in the value of is the direction given by (i.e. in the direction of the residual )
( )g x( )g−∇ x r
![Page 50: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/50.jpg)
Department of Mathematics
Alternative approach
( ) ( 1) ( ) ( 1)
( ) ( ) ( ) ( )
, ,
, ,
k k k k
k k k k k
At
A A
− −−⇒ = =
v b x v r
v v v v
Def. A - orthogonality condition
A set of nonzero direction vectors { }(1) (2) ( ), , , nv v vi i i
that satisfy ( ) ( ), 0i jA =v v i j≠if
( ) ( 1) ( )k k kkt
−= +x x v
![Page 51: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/51.jpg)
Department of Mathematics
( ) ( 1)
( ) ( )
,
,
k k
k k k
At
A
−−=
v b x
v v
Theorem
Then, assuming exact arithmetic,
{ }(1) (2) ( ), , , nv v v
: positive definite matrix
( ) ( 1) ( )k k kkt
−= +x x v
A(0)x : arbitrary
Define
for 1 , 2 , 3 ,k =
( )n =Ax b
: A –orthogonal set of nonzero vectors
![Page 52: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/52.jpg)
Department of Mathematics
( ) ( ), 0k j =r v
Theorem
The residual vectors ( )kr
for each
1 , 2 , 3 , ,k n=
1 , 2 , 3 , ,j k=
, for a conjugate direction method, satisfy
, where
![Page 53: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/53.jpg)
Department of Mathematics
1. Initial approximation
Conjugate gradient method(0)x
First search direction (1) (0) (0)= = −v r b Ax
( 1) ( 2) ( 1)1
k k kkt
− − −−= +x x v2.
( ) ( ), 0 ,i j⟨ ⟩ =v Av ( ) ( ), 0i j⟨ ⟩ =r r
and
i j≠for
⇒ (1) ( 1), , k−⋅⋅ ⋅v vfind (1) ( 1), , k−⋅⋅ ⋅x x
3. If is the solution to ,=Ax b done .
Otherwise, ( 1) ( 1)k k− −= − ≠r b Ax 0( 1) ( ), 0k i−⟨ ⟩ =r v for 1, 2, , 1.i k= ⋅⋅⋅ −
⇒ ( ) ( 1) ( 1)1
k k kks− −−= +v r v
( 1)k−x
where
![Page 54: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/54.jpg)
Department of Mathematics
4. choose 1,ks − so that ( 1) ( ), 0k k−⟨ ⟩ =v Av
( ) ( 1) ( 1)1
k k kks− −−= +Av Ar Av
⇒ ( 1) ( ),k k−⟨ ⟩v Av ( 1) ( 1)1 ,k k
ks− −
−+ ⟨ ⟩v Ar( 1) ( 1),k k− −= ⟨ ⟩v Ar
⇒( 1) ( 1)
1 ( 1) ( 1)
,,
k k
k k ks− −
− − −
⟨ ⟩= −
⟨ ⟩v Arv Av
( 1) ( ), 0k k−⟨ ⟩ =v Av⇒
5. ( ) ( ), 0k i⟨ ⟩ =v Av for each 1, 2, , 2 .i k= ⋅⋅⋅ −
⇒ (1) ( ){ , , }k⋅ ⋅ ⋅v v is an A-orthogonal set .
![Page 55: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/55.jpg)
Department of Mathematics
6. Having chosen ( ) ,kv complete
( 1) ( 1) ( 1)( ) ( 1)1
( ) ( ) ( ) ( )
( 1) ( 1) ( 1) ( 1)
1( ) ( ) ( ) ( )
( 1) ( 1)( 1) ( 1)
( ) ( )
( ) ( 1) ( )
,,, ,
, ,, ,, ( , 0)
,
k k kk kk
k k k k k
k k k k
kk k k k
k kk k
k k
k k kk
st
s
t
− − −−−
− − − −
−
− −− −
−
⟨ + ⟩⟨ ⟩= =⟨ ⟩ ⟨ ⟩
⟨ ⟩ ⟨ ⟩= +⟨ ⟩ ⟨ ⟩
⟨ ⟩= ⟨ ⟩ =⟨ ⟩
⇒ = +
r v rv rv Av v Av
r r v rv Av v Avr r v rv Av
x x v
∵
![Page 56: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/56.jpg)
Department of Mathematics
7. Compute
8. ComputeSince ,
`
( )kr( ) ( 1) ( )k k k
kt−= +x x v
( ) ( ) ( 1) ( ) ( ) ( )
( ) ( )
, , ,
,
k k k k k kk
k kk
t
t
−⇒ ⟨ ⟩ = ⟨ ⟩ − ⟨ ⟩
= − ⟨ ⟩
r r r r Av r
r Av
( ) ( 1) ( )k k kkt
−⇒ = −r r Av
( ) ( 1) ( )k k kkt
−⇒ − = − +Ax b Ax b Av
( 1) ( )( , 0 )k k−⟨ ⟩ =r r∵
ks( 1) ( 1)
( ) ( )
,,
k k
k k kt− −⟨ ⟩
=⟨ ⟩
r rv Av
( 1) ( 1) ( ) ( ), ,k k k kkt
− −⟨ ⟩ = ⟨ ⟩r r v Av
( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
( 1) ( 1) ( 1) ( 1)
, ,, ,
(1 / ) , ,(1 / ) , ,
k k k k
k k k k k
k k k kk
k k k kk
s
tt − − − −
⟨ ⟩ ⟨ ⟩⇒ = − = −
⟨ ⟩ ⟨ ⟩
⟨ ⟩ ⟨ ⟩= =
⟨ ⟩ ⟨ ⟩
v Ar r Avv Av v Av
r r r rr r r r
![Page 57: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/57.jpg)
Department of Mathematics
Summary
for(0 ) (0 ) (1) (0 );= − =r b Ax v r 1 , 2 , 3 , ,k n=
( 1) ( 1)
( ) ( )
,,
k k
k k kt− −⟨ ⟩
=⟨ ⟩
r rv Av
( ) ( 1) ( )k k kkt
−= −x x v
( ) ( 1) ( )k k kkt
−= −r r Av
( ) ( )
( 1) ( 1)
,,
k k
k k ks − −
⟨ ⟩=
⟨ ⟩r r
r r
( 1) ( ) ( )k k kks+ = −v r v
![Page 58: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/58.jpg)
Department of Mathematics
Extend the conjugate gradient Method
• If the matrix A is ill-conditioned,the conjugate gradient method is highly susceptible to rounding errors» the exact answer shoud not be obtained in n steps
n
to include preconditioning
• If the matrix A is well-conditioned,An acceptable approximate solution is often in about steps
![Page 59: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/59.jpg)
Department of Mathematics
-1 -1( )t=A C A C
1. Choose nonsingular conditioning matrix C
2. Consider =Ax b
where t=x C x -1=b C b -1 -( ( ) )t t=C C-1 -1( )( )t t−⇒ = =Ax C AC C Ax C Ax
3. Solve =Ax b xfor-t⇒ =x C x
4. Preconditioning( ) ( )k t k=x C x( ) ( ) 1 1 ( )
1 ( ) 1 ( )
( )( )
k k t t k
k k
− − −
− −
⇒ = − = −
= − =
r b Ax C b C AC C xC b Ax C r
![Page 60: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/60.jpg)
Department of Mathematics
5. Let
6.
( ) ( ) ( ) 1 ( ),k t k k kC−= =v C v w r( ) ( ) 1 ( ) 1 ( )
( 1) ( 1) 1 ( 1) 1 ( 1)
( ) ( )
( 1) ( 1)
, ,, ,,,
k k k k
k k k k k
k k
k k
s− −
− − − − − −
− −
⟨ ⟩ ⟨ ⟩⇒ = =
⟨ ⟩ ⟨ ⟩
⟨ ⟩=
⟨ ⟩
r r C r C rr r C r C r
w ww w
( 1) ( 1) 1 ( 1) 1 ( 1)
( ) 1 ( )( ) ( )
( 1) ( 1) ( 1) ( 1)
( ) 1 ( ) ( ) ( )
, ,,,
, ,, ,
k k k k
k t k t t kk k
k k k k
t k k k k
t− − − − − −
− −
− − − −
−
⟨ ⟩ ⟨ ⟩= =
⟨ ⟩⟨ ⟩
⟨ ⟩ ⟨ ⟩= =
⟨ ⟩ ⟨ ⟩
r r C r C rC v C A C C vv A v
w w w wC v C A v v A v
![Page 61: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/61.jpg)
Department of Mathematics
7.
8.
9.
( ) ( 1) ( )k k kkt
−= +x x v( ) ( 1) ( )t k t k t k
kt−⇒ = +C x C x C v
( ) ( 1) ( )k k kkt
−⇒ = +x x v
( ) ( 1) ( )k k kkt
−= −r r Av1 ( ) 1 ( 1) 1 ( )k k t k
kt− − − − −⇒ = −C r C r C AC v
( ) ( 1) ( )k k t t kkt
− −⇒ = −r r AC C v( ) ( 1) ( )k k k
kt−⇒ = −r r Av
( 1) ( ) ( )k k kks
+ = +v r v( 1) 1 ( ) ( )t k k t k
ks+ −⇒ = +C v C r C v( 1) 1 ( ) ( )
( ) ( )
k t k kk
t k kk
s
s
+ − −
−
⇒ = +
= +
v C C r v
C w v
![Page 62: Ch 7. Iterative Techniques in Matrix Algebrakucg.korea.ac.kr/education/2005/CSE530/notes/chapter07.pdf · Department of Mathematics The conjugate gradient Method • To solve nn×](https://reader034.vdocument.in/reader034/viewer/2022050300/5f691fe9e5003279ec617edd/html5/thumbnails/62.jpg)
Department of Mathematics
Summary
( 1) ( 1)
( ) ( )
,,
k k
k k kt− −⟨ ⟩
=⟨ ⟩w wv Av
( ) ( 1) ( )k k kkt
−= +x x v( ) ( 1) ( )k k k
kt−= −r r Av
( ) ( )
( 1) ( 1)
,,
k k
k k ks − −
⟨ ⟩=
⟨ ⟩w w
w w
( 1) ( ) ( )k t k kks
+ −= +v C w v
(0 ) (0 ) (1) (0 );= − =r b Ax v r 1 , 2 , 3 , ,k n=for