Numerical AnalysisMath 465/565
Order and Rates of Convergence
1
Saturday, September 14, 13
“Speed of convergence”
2
We now have two algorithms which we can compare - bisection and the fixed-point method. Which is faster?
Hard to answer : Depends on what interval we start with, how close to a root we start with, etc. So for any two particular instances one method might converge in fewer iterations than the other.
How can we qualify more generally which method might be better in some sense?
Saturday, September 14, 13
Review of bisection
3
We showed that bisection has the property that
Or, bisection ”behaves” like the sequence
�k = 2�k
If we define the error as ek = |xk � x̄|, we can write
ek |b� a| 2�k
|xk � x̄| |b� a|2k
Saturday, September 14, 13
How fast does an algorithm converge?
4
Suppose that an algorithm produces iterates that converge as
limk!1
xk = x̄
If there exists a sequence �k that converges to zero and
a positive constant C, such that
ek = |xk � x̄| C|�k|
then xk is said to converge with rate �k.
Saturday, September 14, 13
Convergence rate - Bisection
5
rate 2�k
|xk � x̄| |b� a| 12
k
So the bisection method has a convergence rate of
12k
with |b� a| as the asymptotic convergence constant.
Saturday, September 14, 13
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49;
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203 0.046941635188405
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203 0.046941635188405
>> log(2)
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203 0.046941635188405
>> log(2)
ans =
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
Approximate rate of convergence
6
Using the Matlab command polyfit, we can get a best fit line to the data
ak + b = log(ek)
>> k = 1:49; >> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203 0.046941635188405
>> log(2)
ans =
0.693147180559945
log(ek) = � log(2)k + log(C)
Saturday, September 14, 13
Fixed point method
7
Suppose over the interval [x0 � ⇢, x0 + ⇢], a function
g(x) satisfies a Lipschitz condition, and furthermore,
x0 is su�ciently close so that a sequence
xk+1 = g(xk), k = 0, 1, 2, . . .
converges to a root x̄ in [x0 � ⇢, x0 + ⇢]. Then
ek = |xk � x̄| �
k⇢
where |g0(x)| � < 1.
Saturday, September 14, 13
Convergence of fixed point iteration
8
So the convergence rate is �
kwhere |g0(x)| � < 1.
Saturday, September 14, 13
Estimated rate for fixed point
9
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
0.0542
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
0.0542
>> exp(ab(2))
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
0.0542
>> exp(ab(2))
ans =
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Estimated rate for fixed point
9
>> ab = polyfit(kvec,log(e),1)
ab =
-2.9148 0.3012
>> exp(ab(1))
ans =
0.0542
>> exp(ab(2))
ans =
1.3514
log(ek) = log(�)k + log (⇢)
Convergence rate
⇡ 1
20k
Saturday, September 14, 13
Comparison of convergence rates
10
Bisection ⇡ 2
�k
Fixed point ⇡ �k
Which is better?
Saturday, September 14, 13
Order of convergence
11
Suppose we have that
Then the convergence of the sequence xk to x̄ is said
to be of order ↵.
limk!1
|xk+1 � x̄||xk � x̄|↵ = lim
k!1
ek+1
e
↵k
= µ
Saturday, September 14, 13
Estimating order of convergence
12
and use a best-fit-line approach to finding ↵, given a
sequence of errors ek.
Both bisection and the fixed point method are linear methods
Again, we can write
log(ek+1) = ↵ log(ek) + log(µ)
Saturday, September 14, 13
Estimating order of convergence
12
and use a best-fit-line approach to finding ↵, given a
sequence of errors ek.
>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)
Both bisection and the fixed point method are linear methods
Again, we can write
log(ek+1) = ↵ log(ek) + log(µ)
Saturday, September 14, 13
Estimating order of convergence
12
and use a best-fit-line approach to finding ↵, given a
sequence of errors ek.
>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)
ab =
Both bisection and the fixed point method are linear methods
Again, we can write
log(ek+1) = ↵ log(ek) + log(µ)
Saturday, September 14, 13
Estimating order of convergence
12
and use a best-fit-line approach to finding ↵, given a
sequence of errors ek.
>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)
ab =
0.9885 -0.8910
Both bisection and the fixed point method are linear methods
Again, we can write
log(ek+1) = ↵ log(ek) + log(µ)
Saturday, September 14, 13