tutorial 1 - technionwebcourse.cs.technion.ac.il/236861/winter2011-2012/ho/wcfiles/cov.pdf · guy...

236861 Numerical Geometry of Images

Tutorial 1

Calculus of variations

Guy Rosman

c©2011

Guy Rosman 236861 - Tutorial 1 - Calculus of variations

What is calculus of variations?

I Deals with functions of functions – functionals

I Defines optimal functions, optimal surfaces, optimal images,and so forth..



Calculus Calculus of variations

1. Function Functionalf : Rn 7→ R f : F 7→ R, in particular

f (u) =∫Ω F (x , u(x), u′(x), ...) dx

Example: Example:f (x , y) = x2 + y2 f (u) =

∫Ω ‖∇u(x , y)‖2 dxdy

2. Derivative Variationdf (x)

dx=

δf (u)

δu=

lim∆x→0f (x+∆x)−f (x)

∆x limε→0f (u+ε δu)−f (u)

ε

df = ∂∂ε f (x + ε ∆x)

∣∣ε=0

δf = ∂∂ε f (u + ε δu)

∣∣ε=0



0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

Figure: Variations of a function as high dimensional directions




3. Local (relative) minimumf (x∗) ≤ f (x) f (u∗) ≤ f (u)∀x : ‖x − x∗‖ ≤ α ∀u : max

x∈Ω|u(x)− u∗(x)| ≤ α

4. Necessary condition for local extremumdf (x)

dx= 0

δf (u)

δu= 0

5. Sufficient condition for local minimumd2f (x)

dx2≥ 0 More complex theory

∇2f (x) ≥ 0




3. Constrained local minimum

minx

f (x) minu(x)

∫ΩF (x , u(x), u′(x))dx

s.t. g(x) = 0 s.t.∫Ω G (x , u(x), u′(x)) = 0

4. Lagrangian

`(x) = f (x) + λg(x) `(u) =

∫Ω(F + λG ) dx

5. Method of Lagrange multipliers

d`(x∗)

dx= 0

δ`(u∗)

δu= 0

g(x) = 0∫Ω G (x , u∗(x), u∗′(x)) = 0


Examples of functionals

1. Curve length L(y) =

∫ x1

x0

√1 + y ′2dx

The length of a non-parametric curve y(x).

2. Curve length L(x , y) =

∫ 1

0

√x ′2 + y ′2dt

The length of a parametric curve (x(t), y(t)).



3. Surface area A(z) =

∫S

√1 + z2x + z2y dxdy

The area of a non-parametric surface S = (x , y , z(x , y)).



4. Total variation TV (y) =

∫ x1

x0

|y ′|dx

The “oscillation strength” of a non-parametric curve y(x).

0 2 4 6 8 10 12 14−1.5

−1

−0.5

0

0.5

1

1.5

0 2 4 6 8 10 12 14−1.5

−1

−0.5

0

0.5

1

1.5

0 2 4 6 8 10 12 14−1.5

−1

−0.5

0

0.5

1

1.5

Figure: Left to Right: original signal, noisy signal, denoised signal.


The Euler-Lagrange Equation

Given the functional

f (u) =

∫ x1

x0

F(x , u(x), u′(x)

)dx

with F ∈ C3 and all admissible u(x) having fixed boundary valuesu(x0) = u0 and u(x1) = u1.

An extremum of f (u) satisfies the differential equation

Fu −d

dxFu′ = 0

with the boundary conditions u(x0) = u0 and u(x1) = u1.This equation is known as the Euler-Lagrange equation.


The Euler-Lagrange Equation

Note: the second term of the equation does not have a parallel ina finite gradient expression from multivariate calculus – this term’sform expresses the relation of altering the value of u based on theeffect of its derivative. Later on we will see how this term stemsfrom the derivation of the E-L equation.


The E-L equation (independent on u(x))

If the integrand does not depend on u(x),

f (u) =

∫ x1

x0

F(x , u′(x)

)dx ,

the E-L equation becomes a first-order differential equation

d

dxFu′ = 0

or

Fu′ = const.


The E-L equation (independent on u′(x))

If the integrand does not depend on u′(x),

f (u) =

∫ x1

x0

F (x , u(x)) dx ,

the E-L equation becomes an algebraic equation

Fu(x , u(x)) = 0.


The E-L equation (high-order functionals)


f (u) =

∫ x1

x0

F(x , u(x), u′(x), ..., u(n)(x)

)dx

with F ∈ Cn+2 and fixed boundary values u(i)(x0) = u(i)0 and

u(i)(x1) = u(i)1 for i = 0, ..., n − 1.

The Euler-Lagrange equation (also known as the Euler-Poissonequation) is

Fu −d

dxFu′ +

d2

dx2Fu′′ + (−1)n

dn

dxnFu(n) = 0.


The E-L equation (multiple independent variables)


f (u) =

∫ΩF (x , u(x), ux1(x), ..., uxn(x)) dx

with x ∈ Rn and u|∂Ω = u∂Ω.

An extremum of f (u) satisfies the differential equation

∂F

∂u− d

dx1

∂F

∂ux1− ...− d

dxn

∂F

∂uxn= 0

with the boundary condition u|∂Ω = u∂Ω.


The E-L equation (multiple functions)


f (u) =

∫ x1

x0

F(x , u1(x), ..., un(x), u

′1(x), ..., u

′n(x)

)dx

with F ∈ C3 and all admissible u(x) having fixed boundary valuesui (x0) = u0i and ui (x1) = u1i .

An extremum of f (u) satisfies the system of differential equations

Fui −d

dxFu′i = 0

with the boundary conditions u(x0) = u0 and u(x1) = u1.


Proof of the Euler-Lagrange equation

For an optimal function u(x), we require

df

dε= 0, f (u) =

∫F (x , u, u′)dx , u(x) = u(x) + εη(x)

Developing the term for the variation we obtain:

df

dε=

∫dF

dε(x , u, u′)dx =∫

∂F

∂x

∂x

∂ε+

∂F

∂u

∂u

∂ε+

∂F

∂u′∂u′

∂εdx =∫

η∂F

∂u+ η′

∂F

∂u′dx


Using integration by parts, we obtain: And hence,∫η∂F

∂u+ η′

∂F

∂u′dx =∫

η

(∂F

∂u− d

dx

∂F

∂u′

)dx +

[η∂F

∂u′

]x1x0

Since this equation should take place for every η, and assuming theboundary term cancels out, we obtain the E-L equation:

∂F

∂u− d

dx

∂F

∂u′= 0


Problem 1: Minimum distance on a plane

Prove that the family of curves minimizing the distance

L =

∫ 1

−1

√1 + y ′2dx , y(−1) = a, y(1) = b

in the plane are straight lines.

−1.5 −1 −0.5 0 0.5 1 1.50.5

1

1.5

2

(−1,a)

(1,b)


Problem 1: Minimum distance on a plane (cont.)

SolutionThe Euler-Lagrange equation

0 =d

dxFy ′ − Fy =

d

dx

(y ′√

1 + y ′2

).

Integration w.r.t. x yields

y ′√1 + y ′2

= γ = const.


Problem 1: Minimum distance on a plane (cont.)

Substitute y ′ = tan θ:

y ′√1 + y ′2

=sin θcos θ√

1 + sin2 θcos2 θ

=sin θcos θ√

sin2 +cos2 θcos2 θ

=sin θ

cos θ

√cos2 θ

1= sin θ,

from where sin θ = γ. Substituting again yields

y ′ = tan θ =sin θ

cos θ=

± sin θ√1− sin2 θ

=±γ√1− γ2

= α = const,

from where

y = αx + β,

β = const. The solution describes a line in the plane; the exact values ofα, β are determined from the endpoint values.


Problem 2: Heat Equation

We would like to minimize:∫Ω|∇u|2 =

∫Ω(u2x + u2y )dΩ

The Euler-Lagrange equation is:

EL(u) =d

dx(Fux ) +

d

dy

(Fuy)

=d

dx(2ux) +

d

dy(2uy )

= 2div (∇u)

= 2∆u

A gradient descent of the functional can be described as

ut = EL(u) = 2∆u


Problem 3: Total Variation DenoisingWe would like to minimize:∫

Ω

√ε2 + u2x + u2ydΩ

The Euler-Lagrange equation is:

EL(u) =d

dx

2ux

2√

ε2 + u2x + u2y

+d

dy

2uy

2√

ε2 + u2x + u2y

= div

∇u√ε2 + u2x + u2y

Again, we can use the EL condition as a gradient term, to smooththe image:

ut = div

∇u√ε2 + u2x + u2y

≈ div

(∇u

|∇u|

)Guy Rosman 236861 - Tutorial 1 - Calculus of variations

Problem 3: Total Variation Denoising (cont.)

Figure: Left to Right: original signal, noisy signal, denoised signal, asimple linear diffusion result.


Problem 3: Total Variation Denoising (cont.)A similar problem is that of inpainting – here we assume a knownportion of the pixels were corrupted and apply the TV flow only tothem.

Figure: Left to Right: Inpainting using total variation: before (50% ofthe pixels removed) and after inpainting.


tutorial 1 - technionwebcourse.cs.technion.ac.il/236861/winter2011-2012/ho/wcfiles/cov.pdf · guy...

Documents