Chapter 1
Numerical approximation of data :
interpolation, least squares method
I. Motivation
1 Approximation of functions
Evaluation of a function
Which functions (f : R→ R) can be effectively evaluated inany point ?
Þ the power functions : f(x) = xm, m ∈ NÞ the polynomial functions :
f(x) = a0 + a1x+ a2x2 + · · ·+ amx
m
How can we evaluate other functions in a given point ?
for instance : f(x) = cos(x), f(x) = sin(x) exp(x),...
Þ approximation by a polynomial function :
H using a Taylor series about the given point,
H searching a polynomial having the same values as the
function in some close points
á Lagrange interpolation
Evaluation of a function
Which functions (f : R→ R) can be effectively evaluated inany point ?
Þ the power functions : f(x) = xm, m ∈ NÞ the polynomial functions :
f(x) = a0 + a1x+ a2x2 + · · ·+ amx
m
How can we evaluate other functions in a given point ?
for instance : f(x) = cos(x), f(x) = sin(x) exp(x),...
Þ approximation by a polynomial function :
H using a Taylor series about the given point,
H searching a polynomial having the same values as the
function in some close points
á Lagrange interpolation
Evaluation of a function
Which functions (f : R→ R) can be effectively evaluated inany point ?
Þ the power functions : f(x) = xm, m ∈ NÞ the polynomial functions :
f(x) = a0 + a1x+ a2x2 + · · ·+ amx
m
How can we evaluate other functions in a given point ?
for instance : f(x) = cos(x), f(x) = sin(x) exp(x),...
Þ approximation by a polynomial function :
H using a Taylor series about the given point,
H searching a polynomial having the same values as the
function in some close points
á Lagrange interpolation
Evaluation of a function
Which functions (f : R→ R) can be effectively evaluated inany point ?
Þ the power functions : f(x) = xm, m ∈ NÞ the polynomial functions :
f(x) = a0 + a1x+ a2x2 + · · ·+ amx
m
How can we evaluate other functions in a given point ?
for instance : f(x) = cos(x), f(x) = sin(x) exp(x),...
Þ approximation by a polynomial function :
H using a Taylor series about the given point,
H searching a polynomial having the same values as the
function in some close points
á Lagrange interpolation
Principles of Lagrange interpolation
f(x) = sin(πx
2)(x2 + 3)
Principles of Lagrange interpolation
f(x) = sin(πx
2)(x2 + 3)
4 points on the curve :
(−1,−4),(1, 4),(2, 0),
(3,−12)
Principles of Lagrange interpolation
f(x) = sin(πx
2)(x2 + 3) Lagrange interpolating polynomial
4 points on the curve : P polynomial of degree ≤ 3
satisfying
(−1,−4), P (−1) = −4(1, 4), P (1) = 4(2, 0), P (2) = 0
(3,−12) P (3) = −12
Principles of Lagrange interpolation, with 6 points
f(x) = sin(πx
2)(x2 + 3) Lagrange interpolating polynomial
6 points on the curve : P polynomial of degree ≤ 5
satisfying
(xi, yi)0≤i≤5, P (xi) = yi for 0 ≤ i ≤ 5
Remark : ouside the interval defined by the (xi), the Lagrangeinterpolating polynomial has nothing to do with f .
I. Motivation
1 Approximation of functions2 Curve approximation
Piecewise interpolation
f(x) = sin(πx
2)(x2 + 3)
piecewise affine approximation
Piecewise interpolation
f(x) = sin(πx
2)(x2 + 3) piecewise affine approximation
Piecewise interpolation
f(x) = sin(πx
2)(x2 + 3) piecewise affine approximation
Applications : Calculation of an approximate value of
the length of the curve
the area under the curve (here
∫ 5
−5f(x)dx)
Þ see chapter 2
Piecewise interpolation
f(x) = sin(πx
2)(x2 + 3) piecewise affine approximation
Applications : Calculation of an approximate value of
the length of the curve
the area under the curve (here
∫ 5
−5f(x)dx)
Þ see chapter 2
Cubic spline
f(x) = sin(πx
2)(x2 + 3) s cubic spline
Principle :
between two consecutive points, s is a cubic polynomial
s(xi) = f(xi)
s ∈ C2
+ two conditions on the boundary points
Cubic spline
f(x) = sin(πx
2)(x2 + 3) s cubic spline
Principle :
between two consecutive points, s is a cubic polynomial
s(xi) = f(xi)
s ∈ C2
+ two conditions on the boundary points
Bezier curves : principle
P0, P1, · · · , Pn, are n+ 1 given control points
The corresponding Bezier curve is defined by
M(t) =
n∑i=0
Bin(t)Pi, 0 ≤ t ≤ 1
where Bin are Bernstein polynomial defined by
Bin = CinX
i(1−X)n−i, with Cin =n!
i!(n− i)!.
Bezier curves : principle
P0, P1, · · · , Pn, are n+ 1 given control points
The corresponding Bezier curve is defined by
M(t) =
n∑i=0
Bin(t)Pi, 0 ≤ t ≤ 1
where Bin are Bernstein polynomial defined by
Bin = CinX
i(1−X)n−i, with Cin =n!
i!(n− i)!.
Bezier curves : with more points
Application of the Bezier curves
I. Motivation
1 Approximation of functions2 Curve approximation3 Fitting of statistical data
Study of statistical data
Some experimental measurements
H X : noise level in the factory (in dB),
H Y : time used to do a definite work (in minutes)
X 73 78 76 63 81 70 75 81 79 84 50 76 65 58Y 77 85 79 67 83 73 72 83 81 82 52 77 65 58
A physical measure alwayscontains some noise. Canwe find a law linking Y andX (Y = f(X)) ?
Can we predict the value ofY for X = 66dB ?
Linear regression
Principle
We are looking for a and b such that
d(a, b) =∑
(yi − (axi + b))2 is minimal.
The straight line y = ax+ b is the linear regression line.
The polynomial P = aX + b is the least squares fittingpolynomial of the cloud of points.
II. Lagrange interpolating polynomial :
theoretical study
1 Study of an example
Lagrange interpolation in 1 point
f(x) = sin(πx
2)(x2 + 3)
1 point on the curve : Search for P0, such that(x0, y0) = (1, 4) degP0 ≤ 0 and
P0(x0) = y0
P0 = 4
Lagrange interpolation in 1 point
f(x) = sin(πx
2)(x2 + 3)
1 point on the curve : Search for P0, such that(x0, y0) = (1, 4) degP0 ≤ 0 and
P0(x0) = y0
P0 = 4
Lagrange interpolation in 1 point
f(x) = sin(πx
2)(x2 + 3)
1 point on the curve : Search for P0, such that(x0, y0) = (1, 4) degP0 ≤ 0 and
P0(x0) = y0
P0 = 4
Lagrange interpolation in 2 points
f(x) = sin(πx
2)(x2 + 3)
2 points on the curve : Search for P1, such that(x0, y0) = (1, 4) degP1 ≤ 1 and
(x1, y1) = (−1,−4) P1(x0) = y0 and P1(x1) = y1
P1 = 4X
Lagrange interpolation in 2 points
f(x) = sin(πx
2)(x2 + 3)
2 points on the curve : Search for P1, such that(x0, y0) = (1, 4) degP1 ≤ 1 and
(x1, y1) = (−1,−4) P1(x0) = y0 and P1(x1) = y1
P1 = 4X
Lagrange interpolation in 2 points
f(x) = sin(πx
2)(x2 + 3)
2 points on the curve : Search for P1, such that(x0, y0) = (1, 4) degP1 ≤ 1 and
(x1, y1) = (−1,−4) P1(x0) = y0 and P1(x1) = y1
P1 = 4X
Lagrange interpolation in 3 points
f(x) = sin(πx
2)(x2 + 3)
3 points on the curve : Search for P2, such that(x0, y0) = (1, 4) degP2 ≤ 2 and
(x1, y1) = (−1,−4) P2(x0) = y0, P2(x1) = y1(x2, y2) = (3,−12) and P2(x2) = y2
Lagrange interpolation in 3 points
f(x) = sin(πx
2)(x2 + 3)
3 points on the curve : Search for P2, such that(x0, y0) = (1, 4) degP2 ≤ 2 and
(x1, y1) = (−1,−4) P2(x0) = y0, P2(x1) = y1(x2, y2) = (3,−12) and P2(x2) = y2
Lagrange interpolation in 3 points
Idea
Search for L0, such that degL0 = 2 and
L0(x1) = L0(x2) = 0 and L0(x0) = 1.
Search for L1, such that degL1 = 2 and
L1(x0) = L1(x2) = 0 and L1(x1) = 1.
Search for L2, such that degL0 = 2 and
L2(x0) = L2(x1) = 0 and L2(x2) = 1.
Prove that P2 = y0L0 + y1L1 + y2L2 is a solution of the pb.
Give the expression of P .
á P2 = −3X2 + 4X + 3
Lagrange interpolation in 3 points
Idea
Search for L0, such that degL0 = 2 and
L0(x1) = L0(x2) = 0 and L0(x0) = 1.
Search for L1, such that degL1 = 2 and
L1(x0) = L1(x2) = 0 and L1(x1) = 1.
Search for L2, such that degL0 = 2 and
L2(x0) = L2(x1) = 0 and L2(x2) = 1.
Prove that P2 = y0L0 + y1L1 + y2L2 is a solution of the pb.
Give the expression of P .
á P2 = −3X2 + 4X + 3
Lagrange interpolation in 3 points
Other idea
P1 = 4X satisfies P1(1) = 4, P1(−1) = −4 and degP1 = 1.
Q = (X + 1)(X − 1) satisfies Q(1) = Q(−1) = 0 anddegQ = 2 (in fact Q = −L2)
Search for P2 under the form
P2 = P1 + αQ = 4X + α(X + 1)(X − 1).
P2(3) = −12⇐⇒ α = −3 and
P2 = 4X − 3(X + 1)(X − 1) = −3X2 + 4X + 3
Lagrange interpolation in 3 points
Other idea
P1 = 4X satisfies P1(1) = 4, P1(−1) = −4 and degP1 = 1.
Q = (X + 1)(X − 1) satisfies Q(1) = Q(−1) = 0 anddegQ = 2 (in fact Q = −L2)
Search for P2 under the form
P2 = P1 + αQ = 4X + α(X + 1)(X − 1).
P2(3) = −12⇐⇒ α = −3 and
P2 = 4X − 3(X + 1)(X − 1) = −3X2 + 4X + 3
II. Lagrange interpolating polynomial :
theoretical study
1 Study of an example2 Existence and uniqueness of the Lagrange interpolating
polynomial
The mathematical problem
Formulation
Let (n+ 1) points be given :
(xi, yi)0≤i≤n with (xi, yi) ∈ R2 for all i,
xi 6= xj for all i 6= j.
Is it possible to find a polynomial P with real coefficients satisfying
P (xi) = yi ∀0 ≤ i ≤ n?
Degree of P ?
number of equations : n+ 1
Þ number of unknowns (coefficients (ai)) less that n+ 1
Þ degP ≤ n
The Lagrange basis
For 0 ≤ j ≤ n, let us define
Lj =
n∏i=0,i 6=j
X − xixj − xi
.
It satisfies :
Lj(xj) = 1 and Lj(xi) = 0 for all i 6= j
⇐⇒ Lj(xi) = δi,j .
anddegLj = n ∀0 ≤ j ≤ n.
Solution of the problem + uniqueness
Existence of a solution
Let P =
n∑j=0
yjLj , we have
degP ≤ n,
P (xi) =
n∑j=0
yjLj(xi) = yi for all 0 ≤ i ≤ n.
á P is a solution of the Lagrange interpolation problem.
Uniqueness
Can we find another solution to the problem, Q ? If Q exists,
degQ ≤ n and deg(P −Q) ≤ n,
P (xi)−Q(xi) = (P −Q)(xi) = 0 for 0 ≤ i ≤ n.
á P −Q = 0 and the solution is unique.
Main theorem
Theorem
Hypotheses :
Let us consider (n+ 1) points of R2 : (xi, yi)0≤i≤n,
xi 6= xj for all i 6= j
Then, there exists a unique polynomial P ∈ Rn[X] satisfying
P (xi) = yi ∀0 ≤ i ≤ n.
P is the Lagrange interpolating polynomial that passes through the(n+ 1) points (xi, yi)0≤i≤n.
In the case where yi = f(xi) for all 0 ≤ i ≤ n (with f a givenfunction), P is the Lagrange interpolating polynomial of f in thepoints (xi)0≤i≤n.
II. Lagrange interpolating polynomial :
theoretical study
1 Study of an example2 Existence and uniqueness of the Lagrange interpolating
polynomial3 Interpolation error result
Presentation of the problem
Comparison of f and P (4 points)
E(x) = f(x)− P (x)
Presentation of the problem
Comparison of f and P (4 points)
E(x) = f(x)− P (x)
with a zoom around the points
Presentation of the problem
Comparison of f and P (6 points)
E(x) = f(x)− P (x)
Presentation of the problem
Comparison of f and P (6 points)
E(x) = f(x)− P (x)
with a zoom around the points
What can be said about E(x) ? Can it be bounded ?...
Interpolation error
Theorem
Hypotheses :
f : [a, b]→ R, f ∈ Cn+1([a, b]),
(xi)0≤i≤n, n+ 1 distinct real numbers of [a, b].
Pn : Lagrange interpolating polynomial of f in the points (xi)0≤i≤n.
Then, for all x ∈ [a, b], there exists ξx ∈ [a, b] such that
f(x)− Pn(x) =1
(n+ 1)!Πn(x)f (n+1)(ξx),
with Πn =n∏i=0
(X − xi).
Consequence
As a consequence, we get :
∀x ∈ [a, b] |f(x)− Pn(x)| ≤ 1
(n+ 1)!Mn+1|Πn(x)|,
with Mn+1 = maxξ∈[a,b]
|f (n+1)(ξ)|.
It does not imply the convergence of Pn(x) towards f(x).
It is not necessary interesting to increase n.
III. Lagrange interpolating polynomial :
practical computation
1 Cost of the computation of the interpolating polynomial
With the Lagrange basis
Lj(x) =
n∏i=0,i 6=j
(x− xi)
n∏i=0,i 6=j
(xj − xi),
for j = 0 to n
Cost of the computation :
(n+ 1)×
(2(n− 1) mult. + 1 div.
)
P (x) =n∑j=0
yjLj(x) Þ final cost ≈ 2n2.
Other main drawback of the method : what happens if wefinally want to add one more point (xn+1, yn+1) ?
Þ All must be started again from zero.
With the Lagrange basis
Lj(x) =
n∏i=0,i 6=j
(x− xi)
n∏i=0,i 6=j
(xj − xi), for j = 0 to n
Cost of the computation : (n+ 1)×(
2(n− 1) mult. + 1 div.)
P (x) =n∑j=0
yjLj(x) Þ final cost ≈ 2n2.
Other main drawback of the method : what happens if wefinally want to add one more point (xn+1, yn+1) ?
Þ All must be started again from zero.
With the Lagrange basis
Lj(x) =
n∏i=0,i 6=j
(x− xi)
n∏i=0,i 6=j
(xj − xi), for j = 0 to n
Cost of the computation : (n+ 1)×(
2(n− 1) mult. + 1 div.)
P (x) =n∑j=0
yjLj(x) Þ final cost ≈ 2n2.
Other main drawback of the method : what happens if wefinally want to add one more point (xn+1, yn+1) ?
Þ All must be started again from zero.
With the Lagrange basis
Lj(x) =
n∏i=0,i 6=j
(x− xi)
n∏i=0,i 6=j
(xj − xi), for j = 0 to n
Cost of the computation : (n+ 1)×(
2(n− 1) mult. + 1 div.)
P (x) =n∑j=0
yjLj(x) Þ final cost ≈ 2n2.
Other main drawback of the method : what happens if wefinally want to add one more point (xn+1, yn+1) ?
Þ All must be started again from zero.
With an other basis
Idea
Write the polynomial in the basis1, (X − x0), (X − x0)(X − x1), · · · ,n−1∏j=0
(X − xj)
.
Þ P
n
= α0 + α1(X − x0) + · · ·+ αn
n−1∏j=0
(X − xj).
Now, if we add one point (xn+1, yn+1) , we have :
Pn+1 = Pn + αn+1
n∏j=0
(X − xj)
Þ we just need to calculate αn+1 to get Pn+1.
With an other basis
Idea
Write the polynomial in the basis1, (X − x0), (X − x0)(X − x1), · · · ,n−1∏j=0
(X − xj)
.
Þ Pn = α0 + α1(X − x0) + · · ·+ αn
n−1∏j=0
(X − xj).
Now, if we add one point (xn+1, yn+1) , we have :
Pn+1 = Pn + αn+1
n∏j=0
(X − xj)
Þ we just need to calculate αn+1 to get Pn+1.
With an other basis
Idea
Write the polynomial in the basis1, (X − x0), (X − x0)(X − x1), · · · ,n−1∏j=0
(X − xj)
.
Þ Pn = α0 + α1(X − x0) + · · ·+ αn
n−1∏j=0
(X − xj).
Now, if we add one point (xn+1, yn+1) , we have :
Pn+1 = Pn + αn+1
n∏j=0
(X − xj)
Þ we just need to calculate αn+1 to get Pn+1.
Cost of the computation of Pn(x)
Pn(x) = α0 + α1(x− x0) + · · ·+ αn
n−1∏j=0
(x− xj)
= α0 + (x− x0)(α1 + (x− x1)
(α2 +
(· · ·)))
.
Horner’s algorithm for the computation of p = Pn(x)
p← αn
for k from n− 1 to 0
p← αk + (x− xk)pend
Cost
n additions + n multiplications
Þ and the computation of the coefficients αi ?
Cost of the computation of Pn(x)
Pn(x) = α0 + α1(x− x0) + · · ·+ αn
n−1∏j=0
(x− xj)
= α0 + (x− x0)(α1 + (x− x1)
(α2 +
(· · ·)))
.
Horner’s algorithm for the computation of p = Pn(x)
p← αn
for k from n− 1 to 0
p← αk + (x− xk)pend
Cost
n additions + n multiplications
Þ and the computation of the coefficients αi ?
III. Lagrange interpolating polynomial :
practical computation
1 Cost of the computation of the interpolating polynomial
2 The divided difference method
Calculation of the first αi
n = 0, 1 first point (x0, y0), P0 = α0 :
P0(x0) = y0 =⇒ α0 = y0
n = 1, + (x1, y1), P1 = y0 + α1(X − x0) :
P1(x1) = y1 =⇒ α1 =y1 − y0x1 − x0
n = 2, + (x2, y2),
P2 = y0 +y1 − y0x1 − x0
(X − x0) + α2(X − x0)(X − x1)
P2(x2) = y2 =⇒ α2 =
y2 − y1x2 − x1
− y1 − y0x1 − x0
x2 − x0
Recurrence formula
Assume we have :
Interpolating points : (x0, y0) · · · (xn−1, yn−1),
Pn−1 = α0 +
n−1∑j=1
αj
j−1∏k=0
(X − xk), (αj)0≤j≤n−1 known
Interpolating points : (x1, y1) · · · (xn, yn),
Qn−1 = β0 +
n−1∑j=1
βj
j∏k=1
(X − xk), (βj)0≤j≤n−1 known
Then,X − x0xn − x0
Qn−1 +xn −Xxn − x0
Pn−1
= Pn
and
αn =1
xn − x0βn−1 −
1
xn − x0αn−1 =
βn−1 − αn−1xn − x0
.
Recurrence formula
Assume we have :
Interpolating points : (x0, y0) · · · (xn−1, yn−1),
Pn−1 = α0 +
n−1∑j=1
αj
j−1∏k=0
(X − xk), (αj)0≤j≤n−1 known
Interpolating points : (x1, y1) · · · (xn, yn),
Qn−1 = β0 +
n−1∑j=1
βj
j∏k=1
(X − xk), (βj)0≤j≤n−1 known
Then,X − x0xn − x0
Qn−1 +xn −Xxn − x0
Pn−1 = Pn
and
αn =1
xn − x0βn−1 −
1
xn − x0αn−1 =
βn−1 − αn−1xn − x0
.
Recurrence formula
Assume we have :
Interpolating points : (x0, y0) · · · (xn−1, yn−1),
Pn−1 = α0 +
n−1∑j=1
αj
j−1∏k=0
(X − xk), (αj)0≤j≤n−1 known
Interpolating points : (x1, y1) · · · (xn, yn),
Qn−1 = β0 +
n−1∑j=1
βj
j∏k=1
(X − xk), (βj)0≤j≤n−1 known
Then,X − x0xn − x0
Qn−1 +xn −Xxn − x0
Pn−1 = Pn
and
αn =1
xn − x0βn−1 −
1
xn − x0αn−1 =
βn−1 − αn−1xn − x0
.
The divided differences
x0 f(x0)
x1 f(x1)
x2 f(x2)
......
......
. . .
xn−1 f(xn−1)
. . .
xn f(xn)
· · · · · · f [x0, · · · , xn]
with
f [x0, · · · , xn] =f [x1, · · · , xn]− f [x0, · · · , xn−1]
xn − x0.
The divided differences
x0 f [x0]
x1 f [x1]
x2 f [x2]
......
......
. . .
xn−1 f [xn−1]
. . .
xn f [xn]
· · · · · · f [x0, · · · , xn]
with
f [x0, · · · , xn] =f [x1, · · · , xn]− f [x0, · · · , xn−1]
xn − x0.
The divided differences
x0 f [x0]
x1 f [x1]f [x1]− f [x0]
x1 − x0
x2 f [x2]f [x2]− f [x1]
x2 − x1...
......
.... . .
xn−1 f [xn−1]f [xn−1]− f [xn−2]
xn−1 − xn−2
. . .
xn f [xn]f [xn]− f [xn−1]
xn − xn−1
· · · · · · f [x0, · · · , xn]
with
f [x0, · · · , xn] =f [x1, · · · , xn]− f [x0, · · · , xn−1]
xn − x0.
The divided differences
x0 f [x0]
x1 f [x1] f [x0, x1]
x2 f [x2] f [x1, x2]
......
...
.... . .
xn−1 f [xn−1] f [xn−2, xn−1]
. . .
xn f [xn] f [xn−1, xn]
· · · · · · f [x0, · · · , xn]
with
f [x0, · · · , xn] =f [x1, · · · , xn]− f [x0, · · · , xn−1]
xn − x0.
The divided differences
x0 f [x0]
x1 f [x1] f [x0, x1]
x2 f [x2] f [x1, x2]f [x1, x2]− f [x0, x1]
x2 − x0...
......
...
. . .
xn−1 f [xn−1] f [xn−2, xn−1]f [xn−1, xn]− f [xn−2, xn−1]
xn − xn−2
. . .
xn f [xn] f [xn−1, xn]f [xn−1, xn]− f [xn−2, xn−1]
xn − xn−2
· · · · · · f [x0, · · · , xn]
with
f [x0, · · · , xn] =f [x1, · · · , xn]− f [x0, · · · , xn−1]
xn − x0.
The divided differences
x0 f [x0]
x1 f [x1] f [x0, x1]
x2 f [x2] f [x1, x2] f [x0, x1, x2]
......
......
. . .
xn−1 f [xn−1] f [xn−2, xn−1] f [xn−3, xn−2, xn−1]
. . .
xn f [xn] f [xn−1, xn] f [xn−2, xn−1, xn]
· · · · · · f [x0, · · · , xn]
with
f [x0, · · · , xn] =f [x1, · · · , xn]− f [x0, · · · , xn−1]
xn − x0.
The divided differences
x0 f [x0]
x1 f [x1] f [x0, x1]
x2 f [x2] f [x1, x2] f [x0, x1, x2]
......
......
. . .
xn−1 f [xn−1] f [xn−2, xn−1] f [xn−3, xn−2, xn−1]. . .
xn f [xn] f [xn−1, xn] f [xn−2, xn−1, xn] · · · · · · f [x0, · · · , xn]
with
f [x0, · · · , xn] =f [x1, · · · , xn]− f [x0, · · · , xn−1]
xn − x0.
The divided differences
x0 f [x0]
x1 f [x1] f [x0, x1]
x2 f [x2] f [x1, x2] f [x0, x1, x2]
......
......
. . .
xn−1 f [xn−1] f [xn−2, xn−1] f [xn−3, xn−2, xn−1]. . .
xn f [xn] f [xn−1, xn] f [xn−2, xn−1, xn] · · · · · · f [x0, · · · , xn]
Þ Pn = f [x0]+f [x0, x1](X−x0)+f [x0, x1, x2](X−x0)(X−x1)
+ · · ·+ f [x0, · · · , xn]
n−1∏j=0
(X − xj).
The divided differences
x0 f [x0]
x1 f [x1] f [x0, x1]
x2 f [x2] f [x1, x2] f [x0, x1, x2]
......
......
. . .
xn−1 f [xn−1] f [xn−2, xn−1] f [xn−3, xn−2, xn−1]. . .
xn f [xn] f [xn−1, xn] f [xn−2, xn−1, xn] · · · · · · f [x0, · · · , xn]
Cost of the computation ≈ n2
2div. and n2 sub.
Result
Theorem
The Lagrange interpolating polynomial of f in the points (xi)0≤i≤nreads
Pn = f [x0] +
n∑j=1
f [x0, · · · , xj ]j−1∏k=0
(X − xk),
where f [ ] denotes the divided difference of f defined by induction
f [xi] = f(xi) for 0 ≤ i ≤ n
f [xi, · · · , xi+k] =f [xi+1, · · · , xi+k]− f [xi, · · · , xi+k−1]
xi+k − xifor 0 ≤ i ≤ n− k, 1 ≤ k ≤ n.
IV. A few words about Hermite interpolation
1 Presentation of the problem
One example
f(x) = sin(πx
2)(x2 + 3)
Consider two points :
(x0, y0) = (−1,−4)
(x1, y1) = (3,−12)
P = −4− 2(X + 1) = −6− 2X
Q = P + (X + 1)(X − 3)(αX + β)
= P + (X + 1)(X − 3)(−1)
= −X2 − 3
One example
f(x) = sin(πx
2)(x2 + 3)
Consider two points :
(x0, y0) = (−1,−4)
(x1, y1) = (3,−12)
Search P such that
P (x0) = f(x0), P (x1) = f(x1)
P = −4− 2(X + 1) = −6− 2X
Q = P + (X + 1)(X − 3)(αX + β)
= P + (X + 1)(X − 3)(−1)
= −X2 − 3
One example
f(x) = sin(πx
2)(x2 + 3)
Consider two points :
(x0, y0) = (−1,−4)
(x1, y1) = (3,−12)
Search P such that
P (x0) = f(x0), P (x1) = f(x1)
P = −4− 2(X + 1) = −6− 2X
Q = P + (X + 1)(X − 3)(αX + β)
= P + (X + 1)(X − 3)(−1)
= −X2 − 3
One example
f(x) = sin(πx
2)(x2 + 3)
Consider two points :
(x0, y0) = (−1,−4)
(x1, y1) = (3,−12)
Search Q such that
Q(x0) = f(x0), Q(x1) = f(x1)
Q′(x0) = f ′(x0), Q′(x1) = f ′(x1)
P = −4− 2(X + 1) = −6− 2X
Q = P + (X + 1)(X − 3)(αX + β)
= P + (X + 1)(X − 3)(−1)
= −X2 − 3
One example
f(x) = sin(πx
2)(x2 + 3)
Consider two points :
(x0, y0) = (−1,−4)
(x1, y1) = (3,−12)
Search Q such that
Q(x0) = f(x0), Q(x1) = f(x1)
Q′(x0) = f ′(x0), Q′(x1) = f ′(x1)
P = −4− 2(X + 1) = −6− 2X
Q = P + (X + 1)(X − 3)(αX + β)
= P + (X + 1)(X − 3)(−1)
= −X2 − 3
One example
f(x) = sin(πx
2)(x2 + 3)
Consider two points :
(x0, y0) = (−1,−4)
(x1, y1) = (3,−12)
Search Q such that
Q(x0) = f(x0), Q(x1) = f(x1)
Q′(x0) = f ′(x0), Q′(x1) = f ′(x1)
P = −4− 2(X + 1) = −6− 2X
Q = P + (X + 1)(X − 3)(αX + β)
= P + (X + 1)(X − 3)(−1)
= −X2 − 3
With 2 other points
f(x) = sin(πx
2)(x2 + 3)
P = 0
Q = −π2X3 +
π
4X2 +
3π
2X
The mathematical problem
Generalities
The Hermite interpolation takes into account
the values of the function in some points (xi)0≤i≤k,
the values of the successive derivatives of the function untilorder αi in xi.
Formulation
f is a sufficiently smooth function defined on [a, b],
x0, . . . , xk are (k + 1) given points of [a, b],
α0, . . . , αk are (k + 1) integers.
Is it possible to find P satisfying
∀0 ≤ i ≤ k, P (j)(xi) = f (j)(xi),∀0 ≤ j ≤ αi?
IV. A few words about Hermite interpolation
1 Presentation of the problem
2 Main results
Analysis of the problem
Degree of P
Number of equations :
k∑i=0
(αi + 1) = k + 1 +
k∑i=0
αi.
Degree : n = k +
k∑i=0
αi.
Definition
P is the Hermite interpolating polynomial of f in thepoints (xi)0≤i≤k with the orders (αi)0≤i≤k
Theorem
Theorem : existence and uniqueness + interpolation error
Hypotheses :
(xi)0≤i≤k, (k + 1) points in [a, b],
(αi)0≤i≤k, (k + 1) integers,n = k +
k∑i=0
αi
f : [a, b]→ R, f ∈ Cn+1([a, b]),
Then, there exists a unique polynomial Pn ∈ Rn[X] such that
∀0 ≤ i ≤ k, P (j)n (xi) = f (j)(xi), ∀0 ≤ j ≤ αi.
Furthermore, for all x ∈ [a, b], there exists ξx ∈ [a, b] such that
f(x)− Pn(x) =1
(n+ 1)!Ωn(x)f (n+1)(ξx),
with Ωn =k∏i=0
(X − xi)αi+1.
V. Least squares method
1 The case of linear regression
Linear regression
We are looking for a0 and b0 suchthat the following distance is mini-mal :
d(a, b) =
n∑i=1
(yi − (axi + b))2.
Necessary condition
∂d
∂a(a0, b0) = 0 and
∂d
∂b(a0, b0) = 0
⇐⇒
a0
n∑i=1
x2i + b0
n∑i=1
xi =
n∑i=1
xiyi
a0
n∑i=1
xi + b0 n =
n∑i=1
yi
Existence of a unique candidate (a0, b0)
Matrix of the linear system
A =
n∑i=1
x2i
n∑i=1
xi
n∑i=1
xi n
= n
1
n
n∑i=1
x2i X
X 1
Invertibility
detA = n2(1
n
n∑i=1
x2i − X2) = n
n∑i=1
(xi − X)2 = n2V(X).
ConclusionAs soon as two xi are different, detA 6= 0 and there exists aunique (a0, b0) susceptible to be a minimum of d :
a0 =Cov(X,Y )
V(X)and b0 = Y − a0X.
(a0, b0) is a minimizer of d
After some computations, we prove that
d(a, b)− d(a0, b0) =
n∑i=1
((a0 − a)xi + b0 − b)2
It yields∀(a, b) ∈ R2 d(a, b) ≥ d(a0, b0).
Remark on the matrix A
A =
n∑i=1
x2i
n∑i=1
xi
n∑i=1
xi n
= BTB with B =
x1 1x2 1...
...xn 1
V. Least squares method
1 The case of linear regression
2 Generalization
Presentation of the problem
Given points
The cloud of points is still given by (xi)1≤i≤n and (yi)1≤i≤n.
A space of functionsFor some independent functions (ϕ1, · · · , ϕm), let us define
U = ϕ; ϕ =
m∑i=1
uiϕi
Search for a minimizer
We are looking for ϕ∗ ∈ U such that
n∑i=0
|yi − ϕ∗(xi)|2 = minϕ∈U
n∑i=0
|yi − ϕ(xi)|2.
Main result
Theorem
As soon as two xi are different, the least squares problem admits aunique solution
ϕ∗ =m∑i=1
u∗iϕi.
Futhermore, the vector u∗ = (u∗1, · · · , u∗m) is the unique solution ofthe linear system
BTBu∗ = BT y,
with
B =
ϕ1(x1) · · · ϕm(x1)...
...ϕ1(xn) · · · ϕm(xn)
RemarkIn the linear case, m = 2 and ϕ1(x) = x, ϕ2(x) = 1.