Numerical Analysis

1

EE, NCKUTien-Hao Chang (Darby Chang)

In the previous slide Accelerating convergence

– linearly convergent

– Newton’s method on a root of multiplicity

– (exercise 2)

Proceed to systems of equations– linear algebra review

– pivoting strategies

2

In this slide Error estimation in system of equations

– vector/matrix norms

LU decomposition– split a matrix into the product of a lower and a upper

triangular matrices

– efficient in dealing with a lots of right-hand-side vectors

Direct factorization– as an systems of equations

– Crout decomposition

– Dollittle decomposition

3

3.3

4

Vector and matrix norms

Vector and matrix norms Pivoting strategies are designed to

reduce the impact roundoff error The size of a vector/matrix is

necessary to measure the error

5

Vector norm

6

7

The two most commonly used norms in practice

8

9

Vector norm

Equivalent One of the other uses of norms is to

establish the convergence

Two trivial questions:– converge or diverge in different norms?

– converge to different limit values in different norms?

The answer to both is no– all vector norms are equivalent

10

11

12

The Euclidean norm and the maximum norm are equivalent

Matrix norms

Similarly, there are various matrix norms, here we focus on those norms related to vector norms– natural matrix norms

13

Matrix norms

Natural matrix norms

14

15

Natural matrix norms

Computing maximum norm

16

17

18

Natural matrix norms

Computing Euclidean norm Euclidean norm, unfortunately, is not

as straightforward as computing maximum matrix norms

Requires knowledge of the eigenvalues of the matrix

19

Eigenvalue review

20

later

21

Eigenvalue review

22

23

24

In action

http://thomashawk.com/hello/209/1017/1024/Jackson%20Running.jpg

25

26

Any Questions?

27

3.3 Vector and matrix norms

3.4

28

Error estimates and condition number

Error estimation A linear system , and is an

approximate solution The error, , cannot be directly

computed ( is never known) The residue vector, , can be easily

computed

29

Any Questions?

30

Is a good estimation of ? Construct the relationship between

and From the definition

31

hint#3

hint#2

hint#1

Is a good estimation of ? Construct the relationship between

and From the definition

32

answerhint#4hint#3

hint#2

Is a good estimation of ? Construct the relationship between

and From the definition

33

answerhint#4hint#3

Is a good estimation of ? Construct the relationship between

and From the definition

34

answerhint#4

Is a good estimation of ? Construct the relationship between

and From the definition

35

answer

Is a good estimation of ? Construct the relationship between

and From the definition

36

37

38

39

40

Condition number

41

42

Perturbations (skipped)

43

.

.

.

Any Questions?

44

3.4 Error estimates and condition number

3.5

45

LU decomposition

LU decomposition

Motivation Gaussian elimination solve a linear system,

, with unknowns

– with back substitution

– the minimum number of operations

If there are a lots of right-hand-side vectors– how many operations for a new RHS?

– with Gaussian elimination, all operations are also carried out on the RHS

46

LU decomposition

Given a matrix , a lower triangular matrix and an upper triangular matrix for which are said to form an LU decomposition of

Here we replace mathematical descriptions with an example to show how Gaussian elimination is used to obtain an LU decomposition

47

48

49

50

51

Any Questions?

52

Is there any other LU decompositions in addition to using modified Gaussian elimination?– degree of freedoms (number of

unknowns)•

•

Direct factorization (3.6)– as an systems of equations

53

answer

hint

Is there any other LU decompositions in addition to using modified Gaussian elimination?– degree of freedoms (number of

unknowns)•

•

Direct factorization (3.6)– as an systems of equations

54

answer

Is there any other LU decompositions in addition to using modified Gaussian elimination?– degree of freedoms (number of

unknowns)•

•

Direct factorization (3.6)– as an systems of equations

55

Solving a linear system

When a new RHS comes

– with , actually to solve and • both steps are easy

• notice that Pb does not require real matrix-vector multiplication

56

57

Solving a linear system

In summary Anyway, the two-step algorithm (LU

decomposition) is superior to Gaussian elimination with back substitution

58

Any Questions?

59

3.5 LU decomposition

3.6Direct Factorization

60

Is there any other LU decompositions in addition to using modified Gaussian elimination?– degree of freedoms (number of

unknowns)•

•

Direct factorization (3.6)– as an systems of equations

61Recall thathttp://www.dianadepasquale.com/ThinkingMonkey.jpg

Direct factorization Just add more equations

– ex: diagonal must be

Crout decomposition– for each

Dollittle decomposition– for each

62

63

64

65

66

Any Questions?

67

3.6 Direct factorization