Variational Principles Mark Jackson, Jan 10

0. Preliminaries Problem 1

Find the curve of shortest length going from to .

Problem 2

Find curve such that , of length which gives the maximum area.

We will develop a systematic way of obtaining solutions to such problems by showing that any solution must satisfy a specific differential equation.

Analogous to the relationship in calculus between stationary points of , given by , and the minimum and maximum values of .

In problem 2 the area is

and the length is

is a functional, i.e. a function on a space of functions.

This class of problems is called the calculus of variations.

0.2 Direct method Problem 3

Show that there exists such that , for any .

Find such that . By the intermediate value theorem, there exists a solution .

Alternative method;

Define

Claim; such that . Given this claim, we know that

.

Proof of claim.

Step 1; . On the

other hand, let ; then

But

So if . This shows the minimum value is attained inside if anywhere.

Step 2; By Analysis I, with .

This is called the direct method for variational problems.

1. Functions on

1.1 Functions Let

Then is linear if for , . Also

where .

Definition 1.1.1. A function is differentiable at if it can be well approximated by a linear function close to in the sense that

where .

i.e. , such that

Relation with difference quotients (

,

In Definition 1.1.1, put , then identical argument to the case shows that if is

differentiable at then

exists and is finite, and is equal to . This is because if such that

when , this implies

because linear.

Proposition 1.1.2. (i) If is differentiable at , then the partial derivatives

exist and the linear map (in definition 1.1.1) is given by

(ii) If all partial derivatives exist exist and are continuous on , then is differentiable

at each and is given by

We will define

to be the set of real valued continuous functions on all of whose partial derivatives are continuous on

is the set of real valued continuous functions on all of whose partial derivatives up to order are continuous on .

The definition of differentiability can now be written as

In Analysis II, you’ll see a proof that implies that is differentiable at all

.

Lemma 1.1.3. If , then . Also, if , then . This is a 1st order necessary condition for max or min.

Remark. This is also true if it holds only for in some ball around , i.e. .

Terminology. If then is a stationary or critical point. If is a maximum or minimum then it is an extremum (or extreme value).

1.2 Second-order conditions for extreme values Definition. A symmetric matrix (positive) if for all non-

zero. Similarly, (non-negative) if for all non-zero.

Exercise. Show that if a symmetric matrix is , then all its eigenvalues are , and the same with signs.

Theorem 1.2.1. If and , then

(i) is a local minimum , and a local maximum

(ii) is a strict local minimum, and is a strict local

maximum. ‘Strict’ means that, around the point which is a minimum, the function is strictly greater than at (or smaller, if is a maximum).

Imagine which has a strict local minimum at . Then and . Assume is the only stationary point, i.e. . Is a global minimum?

Yes - by Rolle’s theorem, because otherwise, with , and then Rolle’s theorem would imply with , contradicting the assumption that there is only one stationary point.

In , , there are functions with only one stationary point which is a strict local minimum but not a global minimum.

Exercise. Find such a function for .

1.3 Convexity Definition. A set is convex if, given , , we have

.

A function is convex if

i.e. the line between two points on the curve lies above the curve.

Remark. Define the epigraph

Then convex convex.

Proof ( ). Let . Let and

be in , so and . For

Hence, as was to be proved.

Proof ( ). Exercise.

Remark. Since the definition of convexity involves only line segment joining two points, to check a function is convex you can check the functions are convex .

Proposition 1.3.1. If then the following are equivalent;

(i) is convex

(ii)

(iii)

Proof that (i) (ii). Let . Then

and

By the chain rule, , which is (ii).

Proof that (ii) (i). Let and . Then

Choosing gives the definition of convexity.

Proof that (i) (iii). We have and

. Adding, as

required.

Proof that (iii) (i). Exercise.

To visualise the second condition, work in .

is graph of

Normal vector

Tangent plane at is

To have a picture for (iii) consider ;

This says that is monotone non-decreasing. Thus

Proposition 1.3.1 (in words). is convex lies above all its tangent planes is monotone.

1.3.4 Strict convexity

Definition. is strictly convex if

whenever and .

Proposition 1.3.2. If , the following are equivalent;

(i) strictly convex

(ii) for

(iii) for

Corollary to 1.3.1. If is convex with stationary point , then is a global minimiser for , i.e. we have .

Proof. and use (ii).

Corollary to 1.3.2. If is strictly convex and is a stationary point, then it is a strict global minimiser for , i.e. .

Recall that to solve you can hope to achieve this by minimising .

Corollary to 1.3.2. If is strictly convex, then the equation has at most one solution.

Proof. Assume and , so by

1.3.2(ii).

Remark. There is a corresponding notion of concavity, with the inequalities reversed, i.e.

etc. etc. E.g. on .

Lemma 1.3.3. If then

(i)

(ii)

Exercise. Think of an example to show that the converse of (ii) is false.

Proof of (i) ( ).

Note this is using the chain rule in the form

Warning – when we say a symmetric matrix is , we mean

Example. Show that

is a concave function on the set of all with and (a

probability distribution on ).

Note that if are both probability distributions on this set of points, then so is , since

.

Calculate the matrix of second partial derivatives;

since

This is a negative definite matrix. QED.

1.4 Constraints and Lagrange multipliers Example. Maximise subject to the constraint set

Answer; , attained at .

But notice is perpendicular to (unit circle) at these points. To see why, solve the problem

by parametrising as . Then

At maximum values we must have . Apply the chain rule;

Since is the tangent vector to , this means when is normal to .

General situation; if we have a constraint set , then assume admits

local parametrisation . If we find a point

where a function has a maximum/minimum on , then

Then has an unconstrained maximum at

, so

is perpendicular to the tangent vectors

Alternatively, , i.e. such that

the Lagrange multiplier. Use function .

Example. Find the rectangle in the plane inscribed in the unit circle which has the largest possible area.

Area .

Augmented function where is the Lagrange multiplier.

Stationary points of ;

Assume neither nor are , then

i.e. stationary point is a square.

If say , we just have a line inside the circle, so and this is a minimum value of amongst inscribed rectangles.

For , recall that . We will assume , and that we can write in parametric form, so

Example.

To search for stationary points of a function constrained to , we look at

written as .

First-order necessay condition for stationary point;

where the are tangent vectors to .

If is perpendicular to all tangent vectors, it is to , i.e. such that at a stationary point.

For ,

Assume is a stationary point at which . Then

But

using the summation convention. Thus

Theorem 1.4.1. Let be functioms and . Let ; then if has maximum at , then for some , and

If has minimum, then the second inequality has a . Here we use the notion of a symmetric matrix being defined above.

Moral – do usual calculus on .

To solve problems with two or more constraints, , do the same thing with

Examples. (i) Find the probability distribution on which maximises

Constraint is , so , so

(ii) For , , find on the unit circle and on the line to minimise/maximise . satisfies

, so

Then .

Using

, we get

As always, give the constraints .

In vector form we have

Notice cannot be zero, as this would imply , contradicting constraints since and , .

From the first pair

But . Since , we get . Clearly gives the minimum value.

Example of 2nd derivative test. Let , and . As we proved, we need to look at

We want to check this is a negative definite matrix at the solution

so the matrix of 2nd partial derivatives is

Find eigenvalues of , i.e.

which satisfies the necessary 2nd order condition for being a maximum.

1.5 The Legendre transform For , given a function, we define its Legendre transform by

This is only defined for such that this is finite.

Examples. (i) Let , , then is defined for all and attained at where

At that point

Now define

If is convex, then .

(ii) , , then where this is finite. Here the domain of is because the is nowhere defined.

(iii) . Then , so the domain of is with .

(iv) . Then when defined, so the domain of is with .

Proposition 1.5.1. is convex (on its domain).

Proof.

So

So if lie in the domain of , then so does , and

Theorem 1.5.2. If is convex, then .

Proof. Prove for with . Then

is attained for all at unique

such that . (Recall the corollary to 1.3.2.)

Given , there is a unique point where the tangent line to has slope .

Consider, for given , the tangent line to

with slope . It has equation

or

Recall that a convex function lies above its tangent lines (or planes). Call . For given , the line lies below the graph of , and the supremum is attained when , and supremum is , i.e.

i.e. . QED.

Definition. An affine function is one of the form

Corollary. If convex, then it is a supremum of a family of affine functions.

Example. Consider . Then

Domain of is ; supremum is attained when

Then

(Young’s inequality). In fact we always have .

For we define the Legendre transform by

Exercise. Calculate the Legendre transform of

where is a positive symmetric matrix.

Applications

(i) Classical mechanics. The Lagrangian

Hamiltonian

To calculate; supremum attained when

Exercise. Newton equations are equivalent to

(ii) Economics. A company buys imports and produces a product which is sold with revenue . Let be the price of a unit of the th good. Then the total cost is and the profit

is . To maximise profit, they try to choose to get

(iii) Thermodynamics. An engine has internal energy where is the volume and is the entropy. In idealised ‘reversible change’, the heat flow . Also where is the temperature and the pressure. Then

(Maxwell relations).

Suppose the system is immersed in a constant temperature reservoir. In this case the system is described by a function . This is called the Helmholtz free energy, and is defined by

The supremum is attained at such that

This defines , and we can substitute in to get

In Helmholtz description

(Maxwell relation).

Remark. The entropy is determined implicitly by

This determines uniquely as long as

Constant volume heat capacity heat input needed to increase temperature at fixed volume

So is a convex function of you need heat to raise the temperature.

2. Calculus of variations

2.1 Differentiating functionals A functional is a function any space of functions or .

Examples. (i) Let and consider , the ‘Dirac functional’.

(ii) Let functions with . Then

Recall. If , then the directional derivative at in the direction of is

We will apply this to the functionals;

To define something analogous to , we use

or for complex functions we will often use

Now . In this case, we call the functional derivative. More

generally, for any functional we have

when such a function exists. Let’s try to apply this to .

In Methods we will write

Example.

because

Therefore

Example 3. Let . Then

Now . When we calculate a derivative, we consider smooth with , and look at

since vanishes at the end points of the integral; . So

using the notation e.g.

Problem. Let . Minimise

amongst all smooth -periodic functions .

Solution. Assume such a minimising function does exist. Obtain a condition which determines it. We’ll then show that there exists a function satisfying this condition, then finally we’ll show that this function minimises .

Assume is a solution, then for any , where is the set of smooth -periodic functions. In particular, if for any , then .

, i.e. .

Exercise. Show that

So if does minimise , then

Lemma 2.1.1. If (i)

for all smooth functions , with if is outside some sub-interval ;

(ii)

then in .

Proof. Assume, to the contrary, that with . Without loss of generality, take ; since is continuous, such that on .

Consider the function

Now , because .

Consider

Then

which is a contradiction.

Exercise. Find (for the previous example) such that – .

Jargon.

We define closure of set where .

We say is properly supported in or if .

Problem 1. Minimise

over set of smooth -periodic functions, and .

Solution. 1st stage (last time); assume there is such a minimising function , then for all and

.

Therefore

for all and

By general formula ( ),

Therefore

By Lemma 2.1.1, , i.e. if exists it solves this equation. This is called the

Euler-Lagrange equation.

When we get, with periodic,

Remark. To prove there is only one -periodic solution of – , imagine is

another, where – with . Subtract .

Multiply by and integrate;

by parts. Middle term is by periodicity and then we get , i.e. .

Final stage of minimisation problem; consider

Integrating the middle term by parts we get

for any , . So really minimises over smooth -periodic

functions.

Problem 2. Find the curve of shortest length, joining the two points in the plane, and .

Mathematically

We need to minimise amongst such curves.

Assume is a minimising curve, then exactly as before, at . Now

The Euler-Lagrange equation is therefore

constantm so . Fix so that and . This is our

candidate minimising curve.

Claim that is strictly convex

By lemma 1.3.2,

. Now

which if and . So if is another -curve joining to it has greater length than the straight line.

2.3 Multi-dimensional integrals Moral. Integration by parts in one dimension converts to using Green identities in more than

one dimension.

Consider

where some region in . Then

To achieve this we recall

Assume vanishes on , then

i.e.

but so we get Poisson’s equation; . This gives the minimum value of ‘energy’, . This gives gravitational potential of mass distribution , or electrostatic potential of charge.

Problem. Tourist El Cheapo wants to drive 3000 miles from New York to San Francisco in car, taking 100 days. Car uses gallons per mile, with . Fuel costs cents per gallon. How should he choose his speed to spend least money?

For we have with and .

Assume he pays for his fuel as he uses it (continuous ), and that .

Cost to go a distance at time is . So total cost is

A minimising will be a stationary point, so .

Note that the integrand so the functional derivative is

already, and then choose such that .

Notice that if is independent of then is constant, by the Euler-Lagrange equation.

Concepts in theoretical physics; an action in would be of the form

with and . Writing so that , the Euler-

Lagrange equation becomes

using the summation convention.

Derivation. Let be a smooth ‘variation’ function, with near the boundary of . Then

since near the boundary of . Alternative form;

Example. Let an action be

so that and . Then

Euler-Lagrange equation;

the wave equation.

2.4 Lagrange multipliers DIDO problem; find a curve such that the area under the curve and above the -axis is

maximised, for a fixed length. I.e. maximise

subject to

Consider ‘augmented function’ where is the Lagrange multiplier. Then

.

Substitute so that , i.e. which is a

circle. Therefore the solution is is arc of circle passing through of length .

Remark. The formulation of the problem disallows curves that go back on themselves in the -direction, because this is not a graph ! To get over this, we can instead consider

parametrised curves . Then integral functional is

even with .

Euler-Lagrange equations;

for .

Exercise. Find closed curve in the plane such that to maximise

the area inside the curve, with

Finite number of constraints with . Consider as for finite-dimensional problems.

Infinite number of constraints – need Lagrange ‘multiplier function’. E.g. in fluid mechanics. A velocity vector field on subject to an infinite number of constraints at each point; .

Minimise

subject to (incompressible). Then

Exercise. Work out .

Remark. For the notation

and

Constraint for each means there is a Lagrange multiplier function .

Green identities

by Green, assuming for large

so 2 lines above

1st term =

using the second Green identity;

Therefore

So the Euler-Lagrange equation is – . Now

Final construction; stationary for with if

Static Navier-Stokes equation, then approximate by putting all terms to zero, you get this equation where is the pressure.

3. Scientific examples Fermat’s principle – when light

reflects in a mirror, the angle of reflection is equal to the angle of incidence . This can be explained by postulating that light rays take the shortest time possible to get from A to B.

Time as path through

where speed of light.

This implies and

Typical problem; imagine the speed of light is . Then consider a light path .

Time is

This is the type of integral

we know about. A minimising path would solve

2. Mechanics A particle of mass has position , moving in a potential . The force so

This equation is the Euler-Lagrange equation for the ‘action’

where

Most ‘fundamental’ equations of classical physics come in this form, eg. Maxwell’s.

3. Geodesics A geodesic is a curve of shortest length, or more generally a stationary point for the length

.

(i) In the plane, curves from to ,

The only solution is the straight line. We can also write the curve and then

Since

by the change of variables and

Simplest to choose so that

constant, i.e. arc length.

Using these parametrisations, geodesics are curves which make stationary

For

, the Euler-Lagrange equation is (put , in the second

example.) This is exactly what you get by putting constant in .

Recall. We defined a geodesic curve to be a curve which makes stationary either the length

or the integral

The length is independent of parametrisation;

The second definition gives curves parametrised so that is constant. This is usually easier to work with.

Problem. Find geodesic curves on a cylinder

Solution 1. Use cylindrical coordinates; , , . Then

For a curve on , is fixed, and , . Therefore

A geodesic curve on is a curve which makes stationary

The Euler-Lagrange equations are

Conclusion; geodesic curves are ‘helicoidal’ curves. (See example sheet 2, question 9 – geodesics are great circles.)

Solution 2. Introduce as a constraint, and use the Lagrange multiplier function , and

The Euler-Lagrange equations are

To find and we must use constraints;

Find ; But by Euler-Lagrange,

Write , then we have and . But constant constant.

Solutions are therefore , . Again, we get helicoidal curves.

Problem. (i) Consider

(no -dependence!) and show that a curve which makes stationary satisfies

(ii) Find the curve such that a bead moving without friction on the curve takes the shortest possible time to go from to .

Solution. (i) Since makes stationary,

The last two terms come about by the chain rule, since . This is equal to

(ii) The bead moves under gravity, so the speed satisfies

Euler-Lagrange;

This is very hard to solve! Instead we use (i), to say

Choose ; then

The curve is a cycloid.

4. Conservation laws (Noether theorem)

Let be a smooth solution of

Theorem 4.1. (i) If (no dependence) then constant.

(ii) If (no dependence) then constant.

Proof. (i) constant by Euler-Lagrange equation.

(ii) See last lecture.

Special case from mechanics;

Newton’s law is Euler-Lagrange equation for

Conservation laws; (i) if does not depend on then

constant. We sometimes call a ‘cyclic coordinate’.

(ii)

since does not depend explicitly on .

5. The second variation When is a solution of the Euler-Lagrange equation

a (local) minimiser of

Possible methods; (i) use convexity. See 2.2 for example (shortest curve between two points).

(ii) look for condition analogous to the 2nd order condition in calculus.

If , then Taylor’s theorem gives

This tells us that (i)

for all sufficiently small (‘strict local minimum’).

(Positive symmetric means

(ii) a local minimum and is non-negative (i.e. ).

To extend this to a functional , let be a smooth function (or with . ( is analogous to .) Assume is smooth, then;

For all , such that then

In this case we get

Terminology.

Definition. is a weak local minimum for if for sufficiently small. Call it strict if inequality is strict for such .

Theorem 5.1. (i) If , and

for some , then is a strict weak local minimum for .

(ii) If is a weak local minimum for , then and .

Remark. A space of functions is an infinite-dimensional vector space, so we have to be careful which norm we use on a space of functions!

Example.

Euler-Lagrange equation;

There is a solution – is this a (weak) local minimum?

Exercise. The second variation is

Now

satisfies the criterion with . Therefore for , is a strict weak local minimum. Then consider

Recall . Try , then , so

since . Conclusion; is not a weak local minimum for .

Remark. In general the second variation can be written

Associated to this is the Sturm-Liouville operator

This has an infinite sequence of eigenvalues, where is an eigenfunction. So if then