order and rates of convergence - boise state university · 2013-10-07 · “speed of...

Numerical AnalysisMath 465/565

Order and Rates of Convergence

1

Saturday, September 14, 13

“Speed of convergence”

2

We now have two algorithms which we can compare - bisection and the fixed-point method. Which is faster?

Hard to answer : Depends on what interval we start with, how close to a root we start with, etc. So for any two particular instances one method might converge in fewer iterations than the other.

How can we qualify more generally which method might be better in some sense?

Saturday, September 14, 13

Review of bisection

3

We showed that bisection has the property that

Or, bisection ”behaves” like the sequence

�k = 2�k

If we define the error as ek = |xk � x̄|, we can write

ek |b� a| 2�k

|xk � x̄| |b� a|2k

Saturday, September 14, 13

How fast does an algorithm converge?

4

Suppose that an algorithm produces iterates that converge as

limk!1

xk = x̄

If there exists a sequence �k that converges to zero and

a positive constant C, such that

ek = |xk � x̄| C|�k|

then xk is said to converge with rate �k.

Saturday, September 14, 13

Convergence rate - Bisection

5

rate 2�k

|xk � x̄| |b� a| 12

k

So the bisection method has a convergence rate of

12k

with |b� a| as the asymptotic convergence constant.

Saturday, September 14, 13

Approximate rate of convergence

6

Using the Matlab command polyfit, we can get a best fit line to the data

ak + b = log(ek)

log(ek) = � log(2)k + log(C)

Saturday, September 14, 13

Approximate rate of convergence

6

Using the Matlab command polyfit, we can get a best fit line to the data

ak + b = log(ek)

>> k = 1:49;

log(ek) = � log(2)k + log(C)

Saturday, September 14, 13

Approximate rate of convergence

6

Using the Matlab command polyfit, we can get a best fit line to the data

ak + b = log(ek)

>> k = 1:49; >> ab = polyfit(k,log(e),1)

log(ek) = � log(2)k + log(C)

Saturday, September 14, 13

Approximate rate of convergence

6

Using the Matlab command polyfit, we can get a best fit line to the data

ak + b = log(ek)

>> k = 1:49; >> ab = polyfit(k,log(e),1)

ab =

log(ek) = � log(2)k + log(C)

Saturday, September 14, 13

Approximate rate of convergence

6

Using the Matlab command polyfit, we can get a best fit line to the data

ak + b = log(ek)

>> k = 1:49; >> ab = polyfit(k,log(e),1)

ab =

-0.692013993991203 0.046941635188405

log(ek) = � log(2)k + log(C)

Saturday, September 14, 13

Approximate rate of convergence

6

Using the Matlab command polyfit, we can get a best fit line to the data

ak + b = log(ek)

>> k = 1:49; >> ab = polyfit(k,log(e),1)

ab =

-0.692013993991203 0.046941635188405

>> log(2)

log(ek) = � log(2)k + log(C)

Saturday, September 14, 13

Approximate rate of convergence

6

Using the Matlab command polyfit, we can get a best fit line to the data

ak + b = log(ek)

>> k = 1:49; >> ab = polyfit(k,log(e),1)

ab =

-0.692013993991203 0.046941635188405

>> log(2)

ans =

log(ek) = � log(2)k + log(C)

Saturday, September 14, 13

Approximate rate of convergence

6

Using the Matlab command polyfit, we can get a best fit line to the data

ak + b = log(ek)

>> k = 1:49; >> ab = polyfit(k,log(e),1)

ab =

-0.692013993991203 0.046941635188405

>> log(2)

ans =

0.693147180559945

log(ek) = � log(2)k + log(C)

Saturday, September 14, 13

Fixed point method

7

Suppose over the interval [x0 � ⇢, x0 + ⇢], a function

g(x) satisfies a Lipschitz condition, and furthermore,

x0 is su�ciently close so that a sequence

xk+1 = g(xk), k = 0, 1, 2, . . .

converges to a root x̄ in [x0 � ⇢, x0 + ⇢]. Then

ek = |xk � x̄| �

k⇢

where |g0(x)| � < 1.

Saturday, September 14, 13

Convergence of fixed point iteration

8

So the convergence rate is �

kwhere |g0(x)| � < 1.

Saturday, September 14, 13

Estimated rate for fixed point

9

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Estimated rate for fixed point

9

>> ab = polyfit(kvec,log(e),1)

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Estimated rate for fixed point

9

>> ab = polyfit(kvec,log(e),1)

ab =

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Estimated rate for fixed point

9

>> ab = polyfit(kvec,log(e),1)

ab =

-2.9148 0.3012

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Estimated rate for fixed point

9

>> ab = polyfit(kvec,log(e),1)

ab =

-2.9148 0.3012

>> exp(ab(1))

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Estimated rate for fixed point

9

>> ab = polyfit(kvec,log(e),1)

ab =

-2.9148 0.3012

>> exp(ab(1))

ans =

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Estimated rate for fixed point

9

>> ab = polyfit(kvec,log(e),1)

ab =

-2.9148 0.3012

>> exp(ab(1))

ans =

0.0542

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Estimated rate for fixed point

9

>> ab = polyfit(kvec,log(e),1)

ab =

-2.9148 0.3012

>> exp(ab(1))

ans =

0.0542

>> exp(ab(2))

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Estimated rate for fixed point

9

>> ab = polyfit(kvec,log(e),1)

ab =

-2.9148 0.3012

>> exp(ab(1))

ans =

0.0542

>> exp(ab(2))

ans =

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Estimated rate for fixed point

9

>> ab = polyfit(kvec,log(e),1)

ab =

-2.9148 0.3012

>> exp(ab(1))

ans =

0.0542

>> exp(ab(2))

ans =

1.3514

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k

Saturday, September 14, 13

Comparison of convergence rates

10

Bisection ⇡ 2

�k

Fixed point ⇡ �k

Which is better?

Saturday, September 14, 13

Order of convergence

11

Suppose we have that

Then the convergence of the sequence xk to x̄ is said

to be of order ↵.

limk!1

|xk+1 � x̄||xk � x̄|↵ = lim

k!1

ek+1

e

↵k

= µ

Saturday, September 14, 13

Estimating order of convergence

12

and use a best-fit-line approach to finding ↵, given a

sequence of errors ek.

Both bisection and the fixed point method are linear methods

Again, we can write

log(ek+1) = ↵ log(ek) + log(µ)

Saturday, September 14, 13

Estimating order of convergence

12

and use a best-fit-line approach to finding ↵, given a

sequence of errors ek.

>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)

Both bisection and the fixed point method are linear methods

Again, we can write

log(ek+1) = ↵ log(ek) + log(µ)

Saturday, September 14, 13

Estimating order of convergence

12

and use a best-fit-line approach to finding ↵, given a

sequence of errors ek.

>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)

ab =

Both bisection and the fixed point method are linear methods

Again, we can write

log(ek+1) = ↵ log(ek) + log(µ)

Saturday, September 14, 13

Estimating order of convergence

12

and use a best-fit-line approach to finding ↵, given a

sequence of errors ek.

>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)

ab =

0.9885 -0.8910

Both bisection and the fixed point method are linear methods

Again, we can write

log(ek+1) = ↵ log(ek) + log(µ)

Saturday, September 14, 13