lecture 1 numerical methods: principles, algorithms and
TRANSCRIPT
![Page 1: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/1.jpg)
Lecture 1 Numerical methods: principles, algorithms andapplications: an introduction
Weinan E1,2 and Tiejun Li2
1Department of Mathematics,
Princeton University,
2School of Mathematical Sciences,
Peking University,
No.1 Science Building, 1575
![Page 2: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/2.jpg)
Outline
![Page 3: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/3.jpg)
1. Solar system (two-body problem)
Schematics of two body problem
V
Planet
0
Sun
![Page 4: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/4.jpg)
1. Solar system (two-body problem)
I Governing equations: Hamiltonian systemdx
dt= v,
d(mv)
dt= −∇V (x).
Simply take V (x) = −GmMr
. Define Hamiltonian
H =1
2mv2 + V (x)
then dHdt
= 0.
I Numerical solution of ODEs
![Page 5: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/5.jpg)
1. Solar system (two-body problem)
I Long time numerical integration (T 1)
1. Bad scheme (Forward Euler)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2. Good scheme (Symplectic scheme)
−1.5 −1 −0.5 0 0.5 1 1.5−1.5
−1
−0.5
0
0.5
1
1.5
![Page 6: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/6.jpg)
2. Transportation problem
Schematics of transportation problem
a
a
a
b
b
b
b
1
2
m
1
2
n−1
n
Cij
Destination
Origin
![Page 7: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/7.jpg)
2. Transportation problem
I Formulation:
min s =
m∑i=1
n∑j=1
cijxij
subject to the condition
m∑i=1
xij = bj , j = 1, . . . , n
n∑j=1
xij = ai, i = 1, . . . , m
xij ≥ 0, i = 1, . . . , m; j = 1, . . . , n.
where ai is the supply of the i-th origin, bj is the demand of the j-th
destinations, xij is the amount of the shipment from source i to
destination j and cij is the unit transportation cost from i to j.
I Optimization problem (Simplex method)
![Page 8: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/8.jpg)
3. Image processing
I Image restoring
infu∈L
E(u) =1
2
∫Ω
|u0 −Ru|2dx + λ
∫Ω
φ(|∇u|)dx
Here Ω is the domain, u0 is the degradated image, R is the blurring
operator which is known a priori, φ is a particularly chosen function. u will
be the recovered image.
I Nonlinear optimization problem or nonlinear PDE problem
![Page 9: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/9.jpg)
3. Image processing
I Total variation denoising
φ(|∇u|) = |∇u|
A blurred image with 80 lost packets Deblurring and error concealment by TV inpainting
Figure1: TV inpaintingfor theerrorconcealmentof ablurry image.
“objects” J ’s on all the connectedcomponents0 ’s of
0®. Thusin both the
Bayesianandvariationallanguages,thepriormodelconsistsof twoparts(see(3)):_ H _ & ¯ _ A and - 1 - & 1¥% - 1 7In theMumford-Shahmodel,theedgeregularity is specifiedby - 1 S A ,the 1-D Hausdorff measure,or as in most computationalapplications, - 1 =°|k'±² X¯ assumingthat
is Lipschitz. Thesmoothnessof the “objects” is nat-
urally characterizedby the ordinary Sobolev norm: - & 1 m (*) & & Hn M .
Therefore,in combinationwith the data model (4), the variational inpaintingmodelbasedon theMumford-Shahprior is givenbyilku³f´ Tµ ¶ - & G1 « m (*) & & n M S X¯ ·I m + ! n MO7 (6)
Figure2 shows oneapplicationof this modelfor text removal [28]. Note edgesarepreservedandsmoothregionsremainsmooth.
Numerousapplicationshave demonstratedthat, for classicalapplicationsindenoising,deblurringor segmentation,both the TV andMumford-Shahmodelsperformsufficiently well evenby thehighstandardof humanvision. But inpaint-ing doeshave its specialidentity. We have demonstratedin [15, 28] that forlarge-scaleinpaintingproblems,high orderimagemodelswhich incorporatethecurvatureinformationbecomenecessaryfor morefaithful visualeffects.
11
![Page 10: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/10.jpg)
4. Stochastic simulations
Ising model for mean field ferromagnet modeling
1 2 M Spin system
![Page 11: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/11.jpg)
4. Stochastic simulations
I Ising model in statistical physics
Define the Hamiltonian
H(σ) = −J∑
<ij>
σiσj ,
where σi = ±1, < ij > means to take sum w.r.t all neighboring spins
|i− j| = 1. The internal energy per site
UM =1
M
∑σ
H(σ)exp−βH(σ)
ZM=
1
M〈H(σ)〉 = − 1
M
∂ ln ZM
∂β,
where
ZM =∑
σ
exp−βH(σ)
is the partition function and β = (kBT )−1.
![Page 12: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/12.jpg)
4. Stochastic simulations
Finding the critical temperature Tc for 2D Ising model (Metropolis algorithm)
![Page 13: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/13.jpg)
4. Stochastic simulations
I Biological network
Suppose there are Ns species of molecules Si, i = 1, . . . , Ns, and MR
reaction channels Rj , j = 1, . . . , MR. xi is the number of molecules of
species Si. Then the state of the system is given by
x = (x1, x2, . . . , xNs).
Each reaction Rj is characterized by a rate function aj(x) and a vector νj
that describes the change of state due to reaction (after the j − th
reaction, x → x + νj). In shorthand denote
Rj = (aj , νj)
How to simulate this biological process?
![Page 14: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/14.jpg)
4. Stochastic simulations
Stochastic simulation (Kinetic Monte Carlo)
(Number of molecules vs Time)
0 20 40 60 80 1004310
4320
4330
4340
4350
4360
4370
time
Direct SSANested SSA
FIG. 2:
23
Extracted from a paper on stochastically simulating the number of one kind of molecules in a
chemical reaction.
![Page 15: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/15.jpg)
5. Signal processing
I Filtering
Given a discrete time signal ujN−1j=0 , analyze the high frequency and low
frequency part. Discrete Fourier Transform
uk =
N−1∑j=0
uje−jk 2πi
N , k = 0, 1, . . . , N − 1.
I Basic technique: FFT
I A polluted signal
0 200 400 600 800 1000 1200−1.5
−1
−0.5
0
0.5
1
1.5
![Page 16: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/16.jpg)
5. Signal processing
I High pass and low pass filter (signal and noise)
0 200 400 600 800 1000 1200−1.5
−1
−0.5
0
0.5
1
1.5
![Page 17: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/17.jpg)
Three approaches in scientific research
Schematics for the relation of theory, computation and experiment
Experiment
Theory Computation
![Page 18: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/18.jpg)
Top 10 algorithms in 20th century
— from “Computing in Science and Engineering”
1. Metropolis algorithm
2. Simplex method
3. Krylov subspace iteration methods
4. Matrix decomposition approach
5. Fortran Compiler
6. QR algorithm
7. Quicksort algorithm
8. Fast Fourier Transform (FFT)
9. Integer relation detector
10. Fast multipole method
The algorithm 1,2,4,6,8 will be taught in this course.
![Page 19: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/19.jpg)
Three different eras of computing
Numerical analysis −→ Scientific computing −→ Computational science
![Page 20: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/20.jpg)
Outline
![Page 21: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/21.jpg)
Qin Jiushao algorithm (Horner’s algorithm)
I Compute the value of polynomial
p(x) = anxn + an−1xn−1 + · · ·+ a0
naively. The computational efforts will be
(n + 1)(n + 2)
2multiplications and n additions
I Nested multiplication
p(x) = ((anx + an−1) · x + an−2) · · · )x + a0
The computational efforts will be
n multiplications and n additions
![Page 22: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/22.jpg)
Solving linear system
Solving the linear system Ax = b.
I Cramer’s rule
From Cramer’s rule, if det(A) 6= 0 it is solvable and the solution can be
explicitly represented as
xi =det(Ai)
det(A)
and Ai is the matrix with i-th column replaced by b.
I Working load for computing N -determinant with definition:
(N − 1) ·N ! multiplications and N !− 1 additions.
It is an astronomy number! When N = 20, N ! ∼ 2× 1018. If the
computer power is 10Gflops/s, we need 200 years at least, which is
impossible.
![Page 23: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/23.jpg)
Computational efficiency
Theoretically solving a problem is NOT equiv-
alent that it could be solved with computer be-
cause of the computational efficiency! In general,
an O(n4) algorithm is unacceptable!
![Page 24: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/24.jpg)
Floating point arithmetic
I Binary floating point system
F = ±0.d1d2 . . . dt × 2m ∪ 0
where d1 = 1, dj = 0 or 1(j > 1). t is called precision, L ≤ m ≤ U .
For any x ∈ R, denote fl(x) the floating point representation in computer.
We have relative error ∣∣∣fl(x)− x
x
∣∣∣ ≤ 2−t := εmach
I The floating point system has
Underflow limit UFL = 2L × 0.1
Overflow limit OFL = 2U × 0.11 · · · 1
infinity Inf and not a number Nan.
![Page 25: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/25.jpg)
Some issues for floating point arithmetic
I Cancellation
0.3256734− 0.3256721 = 0.0000013
Difference between two approximately equal reals cause loss of significants!
sin(x + ε)− sin(x) = 2 cos(x +ε
2) sin
ε
2
The right hand side is more suitable for computing than the left side hand.
I Summation
1 +
n∑i=1
1
n= 2
I The first order
1 +1
n+ · · · +
1
n= 1 in computer
I The second order
(1
n+
1
n) + · · · + (
1
n+
1
n) + 1
= (2
n+
2
n) + · · · + (
2
n+
2
n) + 1
= · · · · · · + 1 = 2
![Page 26: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/26.jpg)
Outline
![Page 27: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/27.jpg)
Course plan
I 3 hours powerpoint teaching per week;
I 2 hours (or more) computer work per week (50 points);
I Homework 10 points;
I Final exam 40 points;
I Notes to be downloaded from
http://dsec.pku.edu.cn/∼tieli
with Account and Password: lnm2005.
![Page 28: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/28.jpg)
References
1. A. Quarteroni and F. Saleri, Scientific computing with MATLAB, Berlin
and New York, Springer Verlag, 2003.
2. D. Kahaner, C. Moler and S. Nash, Numerical methods and software,
Prentice Hall, 1989.
3. E. Suli and D.F. Mayers, An introduction to numerical analysis, Cambridge
and New York, Cambridge University Press, 2003.
4. S.D. Conti and C. de Boor, Elementary numerical analysis: an algorithmic
approach, New York, McGraw-Hill, 1980.
5. G. Dahlquist and A. Bjork, Numerical methods, Prentice Hall, 1989.
6. G.H. Golub and C.F. Van Loan, Matrix computation, Johns Hopkins
University Press, 1996.
![Page 29: Lecture 1 Numerical methods: principles, algorithms and](https://reader030.vdocument.in/reader030/viewer/2022012706/61a773a049a21c67d07d4889/html5/thumbnails/29.jpg)
Homework assignment 1
Familiarize software MATLAB about the basic definition of variables, arrays,
function definition, subroutine definition, basic linear algebra operations and
graphical operations.