TU Munchen
1. Foundations of Numerics from Advanced Mathematics
Calculus
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 1
TU Munchen
1.1. Calculus
Functions Revisited
• notions of a function, its range, and its image
• graph of a function
• isolines and isosurfaces
• sums and products of functions
• composition of functions
• inverse of a function: when existing?
• simple properties: (strictly) monotonous
• explicit and implicit definition
• parametrized representations (curves, ...)
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 2
TU Munchen
Exercise Functions
Which of the following curves is a graph of a function f (x)?
x x x
Graphically determine the image of the function graph(s).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 3
TU Munchen
Exercise Functions – Solution
Which of the following curves is a graph of a function f (x)?
x x x
Graphically determine the image of the function graph(s).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 4
TU Munchen
Exercise Functions
Sketch the isolines of the function f : R2 → R, (x , y) 7→ x2 + y2.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 5
TU Munchen
Exercise Functions
Sketch the isolines of the functionf : R2 → R2, (x , y) 7→ x2 + y2.
x
y
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 6
TU Munchen
Exercise Functions
Assume, we have the functions f ,g : R→ R with
f (x) = sin(x), g(x) = cos(x).
(f 2 + g2)(x) =?
(f ◦ g)(x) = ?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 7
TU Munchen
Exercise Functions – Solution
Assume, ew have the functions f ,g : R→ R with
f (x) = sin(x), g(x) = cos(x).
(f 2 + g2)(x) = sin2(x) + cos2(x) = 1.
(f ◦ g)(x) = sin(cos(x)).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 8
TU Munchen
Examples for Functions
Explicit function:
f : R→ R, x 7→ (1− x)2 + ex
Implicit function:
f : R+0 → R+
0 , x 7→ y with x2 + y2 = 1
Parametrized function:f : R→ R, x 7→ gy(gx−1(x)) with g : R→ R2,g(t) =( gx(t),gy(t) ).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 9
TU Munchen
Continuity
• remember the ε and the δ!
• definition of local (“in x0”) and global continuity (“∀x”)
• what about sums, products, quotients, ... of continuous functions?
• what about compositions of continuous functions?
• what about continuity of the inverse?
• intermediate value theorem
• continuous functions on compact sets – maximum and minimum value
• uniform continuity
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 10
TU Munchen
Exercise Continuity
We have two continuous functions f : R→ R and g : R→ R. fis continuous iff . . .
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 11
TU Munchen
Exercise Continuity – Solution
We have two continuous functions f : R→ R and g : R→ R. fis continuous iff∀ε > 0, x ∈ R ∃δ > 0 : |f (x)− f (y)| < ε ∀y : |x − y | < δ.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 12
TU Munchen
Exercise Continuity
We have two continuous functions f : R→ R and g : R→ R.
Are f + g, f − g, f · g, fg , and f ◦ g continuous?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 13
TU Munchen
Exercise Continuity – Solution
We have two continuous functions f : R→ R and g : R→ R.
Are f + g, f − g, f · g, fg , and f ◦ g continuous?
The continuity of all these functions can be shown easily using
|f (x)± g(x)− (f (y)± g(y))| ≤ |f (x)− f (y)|+ |g(x)− g(y)|
|f (x) · g(x)− f (y) · g(y)| = |(f (x)− f (y))g(x) + f (y)(g(x)− g(y))| ≤|g(x)||f (x)− f (y)|+ |f (y)||f (x)− f (y)|∣∣∣ 1
g(x)− 1
g(y)
∣∣∣ =∣∣∣ g(y)−g(x)
g(x)·g(y)
∣∣∣Proof the continuity of f ◦ g on your own! It’s easy, but a few lines to write.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 14
TU Munchen
Limits
• meaning of ε→ 0 and N →∞ and x →∞
• accumulation point of a set
• limit (value) of a set
• limits from the left or from the right, respectively: f (x+), f (x−)
• limits at infinity: limx→∞ f (x)
• infinite limits: f (x)→∞
• how can discontinuities look like?
– jumps: f (x+) 6= f (x−)
– holes: f (x+) = f (x−) 6= f (x)
– second kind: f (x) = 0 in x = 0 and f (x) = sin(
1x
)elsewhere
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 15
TU Munchen
Exercise Limits
Determine the accumulation point of S ={1
n ;n ∈ N}
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 16
TU Munchen
Exercise Limits – Solution
Determine the accumulation point of{1
n ;n ∈ N}
.
The accumulation point is 0 since for all ε > 0 there is a e ∈ S with |e − 0| < ε.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 17
TU Munchen
Limits – Visualization
N →∞:
x →∞:
ε→ 0:
0
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 18
TU Munchen
Limits – Visualization
Examples for limx→∞ f (x):
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 19
TU Munchen
Limits – Visualization
Two examples for f (x)→∞:
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 20
TU Munchen
Limits – Visualization
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 21
TU Munchen
Sequences
• definition of a sequence: a function f defined on N
• if f (n) = an, write(an) or a1, a2, a3, ...
• bounded / monotonously increasing / monotonously decreasing sequences
• notion of convergence of a sequence: existence of a limit for n→∞
• Cauchy sequence
• subsequences
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 22
TU Munchen
Sequences – Visualization
∀ε > 0 ∃ N ∈ N :|aN − aM | < ε ∀ N,M > N.
This is a Cauchy sequence!
This is NOT!
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 23
TU Munchen
Series
• notion of an (infinite) series– elements of a series– partial sums of a series– convergence defined by convergence of the sequence of the partial sums– convergence and absolute convergence
• examples:– geometric series:
∑∞k=1 xk = 1
1−x
– harmonic series:∑∞
k=11k = ∞
– alternating harmonic series:∑∞
k=1(−1)k−1 1k = ln(2)
• criteria for convergence: quotient and root criterion
• power series:∑∞
k=0 ak (z − a)k
– coefficients ak and centre point a– radius of convergence R: absolute convergence for |z − a| < R– identity theorem for power series
• re-arrangement
• sums of series, nested series, products of series (Cauchy product)
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 24
TU Munchen
Series – Convergence Criteria
quotient criterion: lim supn→∞|an+1||an| = q < 1 (convergence)
(Cauchy’s) root criterion:
lim supn→∞n√|an| = C
{< 1 absolute conv.> 1 divergence= 1 (abs.) con-/divvergence
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 25
TU Munchen
Series – Identitity Theorem for Power Series
If the radii of convergence of the power series∑∞
n=0 an(z − z0)n
and∑∞
n=0 bn(z − z0)n are positive and the sums of the series
are equal in infinitely many points which have z0 as anaccumulation point, then the both series are identical, i.e.an = bn for each n = 0,1,2, . . ..
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 26
TU Munchen
Differentiation
• first step: functions f of one real variable, complex values allowed
• derivative or differential quotient of f :
– defined via limit process of difference quotients– write f ′ or f or df
dx– geometric meaning?– local and global differentiability– derivative from the left / from the right
• rules for the daily work:
– derivative of f + g, fg, and f/g?– derivative of f (g) (chain rule)?– derivative of the inverse function?
• higher derivatives f (k)(x); meaning
• notion of continuous differentiability
• smoothness of a function
• space of k -times continuously differentiable functions: Ck
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 27
TU Munchen
Visualization Differention
f(x+h)−f(x)
hf(x)−f(x−h)
h
x
h 0
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 28
TU Munchen
Exercise Differentiation Rules
(f + g)′ = ?
(f · g)′ = ?
(f ◦ g)′ = ?
(f−1)′ = ?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 29
TU Munchen
Exercise Differentiation Rules – Solution
(f + g)′ = f ′ + g′.
(f · g)′ = f ′ · g + f · g′. ?
(f ◦ g)′ = (f ′ ◦ g) · g′.
(f−1)′ = (f ′)−1.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 30
TU Munchen
Differential Calculus of one Real Variable
• notion of a global/local minimum/maximum
• local extrema and the first derivative
• mean value theorem:
∃ξ ∈ (a, b) : f ′(ξ) =f (b)− f (a)
b − a
• monotonous behaviour and the first derivative
• local extrema and the second derivative
• rule of de l’Hospital
• notions of convexity and concavity
• convexity/concavity and the second derivative
• notion of a turning point
• turning points and the second derivative
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 31
TU Munchen
Visualization Mean Value Theorem
a b
f(b)
f(a)
ξ a b
f(b)
f(a)
ξξ 12
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 32
TU Munchen
Meaning of Derivatives
first derivative second derivativeincreasing > 0 –decreasing < 0 –maximum = 0 < 0minimum = 0 > 0convex – > 0concave – < 0turning point – = 0 con ex conCAVE
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 33
TU Munchen
Function Classes (1)
• polynomials– definition, degree, sums and products, division with rest, identity theorem,
roots and their multiplicity• rational functions
– poles and their multiplicity, partial fraction decomposition• exponential function and logarithm
– characterising law of the exponential function:
exp(s + t) = exp(s) · exp(t) or y ′ = y
(functional equation of natural growth)– series expansion of the exponential function, speed of growth– natural logarithm as exp’s inverse:
y = exp(x) = ex , x = ln(y)
– functional equation: ln(xy) = ln(x) + ln(y)
– exponential function and logarithm for general basis a:
ax := ex ln a, loga(y) :=ln yln a
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 34
TU Munchen
Function Classes (2)
• hyperbolic functions
– cosh(z), sinh(z), ...
• trigonometric functions
– sin(x), cos(x): solutions of y (2) + y = 0– geometric meaning?– Euler’s formula: eix = cos(x) + i · sin(x)
– derivatives, addition theorem– periodicity– series expansion
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 35
TU Munchen
Integral Calculus of one Variable
• Riemann integral, upper and lower sums• approximation by staircase functions• properties:
– linearity– monotonicity
• mean value theorem:
ξ ∈ (a, b) :
∫ b
af (x)dx = (b − a) · f (ξ)
• main theorem of differential and integral calculus:– define F (x) :=
∫ xa f (t)dt
– then∫ b
a f (t)dt = F (b)− F (a)
• rules for everyday work:– partial integration: ∫
uv ′dx = uv −∫
vu′dx
– substitution: ∫ b
af (t(x))t ′(x)dx =
∫ t(b)
t(a)f (t)dt
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 36
TU Munchen
Visualization Riemann-Integral
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 37
TU Munchen
Local Approximation: Taylor Polynomials and Series
• local approximation of functions with polynomials• generalization of the tangent approximation used for the definition of the derivative• Taylor polynomials:
– let f be n-times differentiable in a– we look for a polynomial T with T (k) = f (k) for k = 0, 1, ..., n– obviously:
T (x) :=n∑
k=0
1k!
f (k)(a)(x − a)k
– unique, degree n, write Tnf (x ; a)
– remainder Rn+1(x) := f (x)− Tnf (x : a)
Rn+1(x) =f (n+1)(ξ)
(n + 1)!(x − a)n+1
• Taylor series:– for infinitely differentiable functions (exp, sin, cos, ...)– sum up to∞ instead of n only– examples
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 38
TU Munchen
Visualization Taylor Polynomials
T(1)
T(2)
aa
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 39
TU Munchen
Global Approximation: Uniform Convergence
• convergence of sequences of functions fn defined on D:
• pointwise: for each x ∈ D; then
f (x) := limn→∞
fn(x)
defines a function
– problems: are properties such as continuity or differentiability inherited fromthe fn to f , and how to calculate derivatives or integrals of f?
– i.e., can the order of limit processes be changed?
• therefore the notion of uniform convergence:
– definition: ‖fn − f‖D → 0 for n→∞– with that, the inheritance and change-order problems from above are solved!– criteria: Cauchy, ...
• approximation theorem of Weierstrass: each continuous function f on acompact set can be arbitrarily well approximated with some polynomial
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 40
TU Munchen
Simple Differential Equations
• notion of a differential equation
– ordinary: one variable– partial: more than one variable (several spatial dimensions or space and
time)
• examples:
– growth: y = k · y or y = k(t , y) · y– oscillation: y + y = 0 or similar
• example of an analytic solution strategy: separation of variables
y ′ = g(x) · h(y), y(x) = y0
– formal separation:dy
h(y)= g(x)dx
– integration of the left and right side– some requirements for applicability
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 41
TU Munchen
Periodic Functions
• target now: periodic functions, period typically 2π
• trigonometric polynomials
• definition:
T (x) :=n∑
k=−n
ck eikx =a0
2+
n∑k=1
(ak cos(kx) + bk sin(kx))
(coefficients ck , ak , and bk are unique)
• formula for the coefficients:
ck =1
2π
∫ 2π
0T (x)e−ikx dx
• T is real iff all ak , bk are real iff ck = c−k
• Weierstrass: 2π-periodic continuous functions can be arbitrarily wellapproximated by trigonometric polynomials
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 42
TU Munchen
Fourier Series
• consider vector space of 2π-periodic complex functions f on R
• Fourier coefficients: f (k) :=1
2π
∫ 2π
0f (x)e−ikx dx
• Fourier polynomial: Snf (x) :=n∑
k=−n
f (k)eikx
• Fourier series:∞∑−∞
f (k)eikx
• sine-cosine representation of Snf :
Snf (x) =a0
2+
n∑k=1
(ak cos(kx) + bk sin(kx))
• coefficients:ak = f (k) + f (−k) =
1π
∫ π
−πf (x) cos(kx)dx
bk = i(f (k)− f (−k)) =1π
∫ π
−πf (x) sin(kx)dx
• all ak vanish for odd f , all bk vanish for even f
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 43
TU Munchen
Functions of Several Variables
• f now defined on Rn or a subset of it
• notion of differentiability: now via existence of a linear map, the differential
• directional derivatives
• partial derivatives
• prominent differentiability criterion: existence and continuity of all partialderivatives
• the gradient of a scalar function f and its interpretation
• the Jacobian of a vector-valued function f
• mean value theorem
• higher partial derivatives, Taylor approximation, Hessian
• local minima and maxima, criteria
• saddle points
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 44
TU Munchen
Visualization Saddle Point
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 45
TU Munchen
Integration over Domains
• a huge field, from which we only mention a few results• theorem of Fubini:
– shows that, in many cases, a multi-dimensional integration domain can betackled dimension by dimension
– statement (we neglect the requirements, for which more integration theory isneeded):∫
X×Yf (x , y)d(x , y) =
∫Y
(∫X
f (x , y)dx)
dy =
∫X
(∫Y
f (x , y)dy)
dx
– related to Cavalieri’s principle– will also be of relevance for numerical quadrature
• transformation theorem:– a generalisation of integration by substitution– statement, again without requirements:∫
Uf (T (x)) ·
∥∥det T ′(x)∥∥ dx =
∫V
f (y)dy
– allows for a change of the coordinate system (polar coordinates), e.g.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 46
TU Munchen
Gauss Theorem
• we further generalise integration, now allowing for integration over hyper-surfaces(a sphere, e.g.)
• this is important for the physical modelling in many scenarios (heat flux through apot’s surface, ...)
• the famous Gauss theorem allows to combine integrals over volumes andsurfaces, which occurs in the derivation of many physical models (conservationlaws) and, hence, is of special relevance for CSE
• prerequisites:– a vector field: a vector-valued function on Rn (example: the velocity field in
fluid mechanics)– the divergence of a vector field F :
div F (x) =n∑
i=1
∂i Fi (x)
• finally the Gauss theorem:– several regularity assumptions needed∫
Gdiv Fdx =
∫∂G
F ~dS
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Calculus, October 25, 2012 47