fixed-point iteration douglas wilhelm harder department of electrical and computer engineering...
TRANSCRIPT
![Page 1: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/1.jpg)
Fixed-Point Iteration
Douglas Wilhelm Harder
Department of Electrical and Computer Engineering
University of Waterloo
Copyright © 2007 by Douglas Wilhelm Harder. All rights reserved.
ECE 204 Numerical Methods for Computer Engineers
![Page 2: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/2.jpg)
Fixed-Point Iteration
• In this topic, we will look at:– how fixed-point iteration can be used to solve
certain problems numerically, and– some of the various limitations of iterative
numerical methods including:• determining convergence,• slow convergence, and• non-convergence
![Page 3: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/3.jpg)
Fixed-Point Iteration
• Everyone has at some point punched in an arbitrary number on a scientific calculator and started pushing one of the buttons
• We will look at two examples: sin and cos
![Page 4: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/4.jpg)
Fixed-Point Iteration
• Starting with x = 0, if I repeatedly hit the cos button, I get the sequence:x = 0
x = 1
x = 0.540302305868140
x = 0.857553215846393
x = 0.654289790497779
x = 0.793480358742566
x = 0.701368773622757
x = 0.763959682900654
x = 0.722102425026708
x = 0.750417761763761
x = 0.731404042422510
x = 0.744237354900557
x = 0.735604740436347
x = 0.741425086610109
x = 0.737506890513243
x = 0.740147335567876
x = 0.738369204122323
x = 0.739567202212256
x = 0.738760319874211
![Page 5: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/5.jpg)
Fixed-Point Iteration
• Looking at the first three digits of the last three iterations, we note that they appear to be converging:0.738369204122323
0.739567202212256
0.738760319874211
![Page 6: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/6.jpg)
Fixed-Point Iteration
• After applying cos 100 times using Matlab, I get the sequence:x = 0.739085133215166
x = 0.739085133215157
x = 0.739085133215163
x = 0.739085133215159
x = 0.739085133215162
x = 0.739085133215160
x = 0.739085133215161
x = 0.739085133215160
x = 0.739085133215161
x = 0.739085133215160
x = 0.739085133215161
x = 0.739085133215161
x = 0.739085133215161
![Page 7: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/7.jpg)
Fixed-Point Iteration
• If, however, you set your calculator to degrees instead of radians, you would have found a different sequence:>> x = 0;
>> for i = 1:6
x = cos(x/180*pi)
end
x = 1
x = 0.999847695156391
x = 0.999847741545212
x = 0.999847741531084
x = 0.999847741531088
x = 0.999847741531088
![Page 8: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/8.jpg)
Fixed-Point Iteration
• In this case, the sequence converged significantly faster, but to what?
![Page 9: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/9.jpg)
Fixed-Point Iteration
• Given an equation of the form
x = f(x)
if |f(1)(x)| < 1 in a neighbourhood of a solution to this equation, then for any initial point x0 in this neighbourhood, the sequence xk + 1
= f(xk) will converge to the solution to this equation
![Page 10: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/10.jpg)
Fixed-Point Iteration
• The examples we have been looking at were finding points such that:
x = cos(x)
• This will differ depending on whether you are using radians or degrees:
![Page 11: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/11.jpg)
Fixed-Point Iteration
• We will now look at one of the longest known examples of fixed-point iteration
• Starting with any value and repeated apply the function
f(x) = x/2 + 1/x
• For centuries, it has been known that this converges to 2
![Page 12: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/12.jpg)
Fixed-Point Iteration
• To see why, recall that this converges to a solution of
x = x/2 + 1/x
which is also a solution ofx2 = x2/2 + 1
orx2/2 = 1
x2 = 2
![Page 13: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/13.jpg)
Newton’s Method
• We also get this sequence by applying Newton’s method which was covered in first year
• The functionf(x) = x2 – 2
has two roots, and we can find these roots using Newton’s method:
![Page 14: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/14.jpg)
Newton’s Method
• Thus, we iterate
• Looking at the right hand side, it simplifies to:
n
nn
n
nnn
x
xx
x
xxx
2
2
)(f
)f(
2
)1(1
x
x
x
xx
1
22
22
![Page 15: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/15.jpg)
Fixed-Point Iteration
• For example, consider the equation
x = 2x(x – 1)
• This has two solutions, namely
x = 0 and x = 0.5
• If we start with x0 = 0.1 and define
f(x) = 2x(x – 1), we get the sequence
![Page 16: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/16.jpg)
Fixed-Point Iteration
• Sequence is
k xk |xk – xk – 1|
0 0.1 –
1 0.18 0.08
2 0.2952 0.1152
3 0.41611392 0.1209
4 0.4859262511644672 0.06981
5 0.4996038591874287 0.01368
6 0.4999996861449132 0.0003958
7 0.4999999999998030 0.0000003139
8 0.5 0.0000000000001970
9 0.5 0
![Page 17: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/17.jpg)
Fixed-Point Iteration
• To solve any such problem, we could continue to iterate until the sequence no longer changes, however, this does not always work
• Consider the slight modification, solving
x = 2.4x(x – 1)
which has the solution x = 7/12 ≈ 0.58333⋅⋅
![Page 18: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/18.jpg)
Fixed-Point Iteration
• Starting with x0 = 0.1, we converge much more slowly, and after 32 iterations:
k xk |xk – xk – 1|
32 0.5833333333333457 0.432 10 –11
33 0.5833333333333284 0.173 10 –11
34 0.5833333333333354 0.070 10 –11
35 0.5833333333333325 0.029 10 –11
36 0.5833333333333337 0.012 10 –11
37 0.5833333333333332 0.005 10 –11
38 0.5833333333333334 0.002 10 –11
39 0.5833333333333332 0.002 10 –11
40 0.5833333333333334 0.002 10 –11
![Page 19: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/19.jpg)
Fixed-Point Iteration
• This sequence will never converge, and therefore we need a weaker halting condition
• We will continue iterating until:
|xk – xk – 1| < step
where step is a suitably chosen predetermined criteria
![Page 20: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/20.jpg)
Fixed-Point Iteration
• Unfortunately, this may also lead to erroneous results
![Page 21: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/21.jpg)
Fixed-Point Iteration
• Suppose we are trying to solve
x = sin(x)
which has only one solution x = 0
• After 1000 iterations, we still have:
x1000 = 0.04802218067056788
x1001 = 0.04800372523478609
x1002 = 0.04798529106705650
x1003 = 0.04796687812655410
![Page 22: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/22.jpg)
Fixed-Point Iteration
• In this case, the rate of convergence is very, very slow
• The following table lists the number of iterations required to satisfy the terminating condition:
![Page 23: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/23.jpg)
Fixed-Point Iteration
• The following table shows how many iterations are required
step Iterations xk
10 –4 .. 123 0.0841946299701288
10 –5 .. 1658 0.03913176037625081
10 –6 .. 8785 0.01816976184230423
10 –7 .. 41870 0.008434193162746495
10 –8 .. 195441 0.003914852569606909
10 –9.. 908257 0.001817119078995468
10 –11 4216859 0.0008434325248239295
10 –12 19574033 0.0003914867496521149
![Page 24: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/24.jpg)
Fixed-Point Iteration
• Notice that a small step size does not suggest that we are anywhere near the solution?
• After 20 million iterations:– the step size is less than 10–12, but– the value 0.0003915867·· has only three zeros
after the decimal place
![Page 25: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/25.jpg)
Fixed-Point Iteration
• Therefore, we must restrict the number of iterations required to some fixed amount, say N
• Thus, if we iterate N times and have not satisfied our halting conditions, we stop and indicate that the method failed to converge
![Page 26: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/26.jpg)
Fixed-Point Iteration
• If we stop due to iterating N times, this may mean that either:– a solution does not exist,– we chose a poor initial condition, or– we are using an inappropriate method for
finding a solution
![Page 27: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/27.jpg)
Fixed-Point Iteration
• Alternatively, to find a solution to x = sin(x),
we could define f(x) = sin(x) – x and then iterate
• Thus, we get:x0 = 0.1
x1 = – 0.0000333452420668
x2 = 0.00000000001074791837
1)cos(
)sin(31
k
kkkk x
xxxx
![Page 28: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/28.jpg)
Fixed-Point Iteration
• In this topic, we have seen a number of examples of fixed-point iteration– it may be possible to solve equations of the form x
= f(x) by repeated applying f(x)– the form of the iteration will affect the convergence– it is necessary to define a halting condition
stopping if the difference is less than step
– the sequence may not converge so we may have to halt after some fixed number of iterations
![Page 29: Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm](https://reader033.vdocument.in/reader033/viewer/2022061609/56649e3f5503460f94b2fb93/html5/thumbnails/29.jpg)
Usage Notes
• These slides are made publicly available on the web for anyone to use
• If you choose to use them, or a part thereof, for a course at another institution, I ask only three things:– that you inform me that you are using the slides,
– that you acknowledge my work, and
– that you alert me of any mistakes which I made or changes which you make, and allow me the option of incorporating such changes (with an acknowledgment) in my set of slides
Sincerely,
Douglas Wilhelm Harder, MMath