some forward-backward and nonlocal diffusions of image...

Overview Introduction Why Do We Care? Past Results Nonlocal Regularizations A Forward-backward Regularization

Some Forward-Backward and Nonlocal Di↵usionsof Image Processing

Patrick Guidotti

Department of Mathematics, University of California, Irvine

Nonlocal PDEs, Variational Problems and their Applications

IPAM Feb. 27-Mar. 2 2012

Overview

Starting Point: The Perona-Malik Equation.

Why Do We Care?

A Brief History.

A Nonlocal Regularization.

Results.

A Forward-backward Regularization.

Results.

Overview

Why Do We Care?

A Brief History.

Results.

Overview

Why Do We Care?

A Brief History.

Results.

Overview

Why Do We Care?

A Brief History.

Results.

Overview

Why Do We Care?

A Brief History.

Results.

Overview

Why Do We Care?

A Brief History.

Results.

Overview

Why Do We Care?

A Brief History.

Results.

The Perona-Malik equation

The PM equation reads

= r · � 11+|ru|2ru

�in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(0, ·) = u0 in ⌦ ,

and was proposed as an image processing tool in discrete form in1990.

It has since captured many a mathematician’s attention. Itis the formal gradient flow associated to the convex-concaveenergy functional

E (u) =1

⌦log

�1 + |ru|2� dx =:

⌦'(ru) dx .

The Perona-Malik equation

The PM equation reads

= r · � 11+|ru|2ru

�in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(0, ·) = u0 in ⌦ ,

and was proposed as an image processing tool in discrete form in1990. It has since captured many a mathematician’s attention. Itis the formal gradient flow associated to the convex-concaveenergy functional

E (u) =1

⌦log

�1 + |ru|2� dx =:

⌦'(ru) dx .

The Perona-Malik Equation

The convexity properties of the functional are reflected in theforward-backward nature of the equation. In fact

1 + |ru|2⇥@⌧⌧u + (1� 2

|ru|21 + |ru|2| {z }

<0 if |ru|>1

)@⌫⌫u⇤.

The equation is therefore ill-posed as was formally shown byKichenassamy [’96]. So far it has proved impermeable tovariational methods mainly because '⇤⇤ ⌘ 0, that is, theconvexification of the energy functional vanishes identically.

The Perona-Malik Equation

The convexity properties of the functional are reflected in theforward-backward nature of the equation. In fact

1 + |ru|2⇥@⌧⌧u + (1� 2

|ru|21 + |ru|2| {z }

<0 if |ru|>1

)@⌫⌫u⇤.

The equation is therefore ill-posed as was formally shown byKichenassamy [’96]. So far it has proved impermeable tovariational methods mainly because '⇤⇤ ⌘ 0, that is, theconvexification of the energy functional vanishes identically.

Staircasing

While the equation is clearly ill-posed, its numericalimplementations are not as badly behaved as one would naivelyexpect in view of its non-trivial backward regime.

It holdsku(t, ·)k1 kuk1 , t > 0 .

The ill-posedness manifests itself through the onset of jumps in thesolution, a phenomenon called staircasing.This is simultaneously the blessing and the curse of the equation:As an image processing tool it can not only preserve edges, itsblessing, but also generate new ones, its curse.

Staircasing

While the equation is clearly ill-posed, its numericalimplementations are not as badly behaved as one would naivelyexpect in view of its non-trivial backward regime.It holds

ku(t, ·)k1 kuk1 , t > 0 .

Staircasing

ku(t, ·)k1 kuk1 , t > 0 .

The ill-posedness manifests itself through the onset of jumps in thesolution, a phenomenon called staircasing.

This is simultaneously the blessing and the curse of the equation:As an image processing tool it can not only preserve edges, itsblessing, but also generate new ones, its curse.

Staircasing

ku(t, ·)k1 kuk1 , t > 0 .

Staircasing

There does not seem to be any discernible pattern in the waystaircasing sets on.

Numerical simulations, however, strongly suggest that, over thelong run, coarsening takes place or, jumps successively disappear inan orderly fashion until the solution eventually settles down on atrivial steady-state.

Staircasing

There does not seem to be any discernible pattern in the waystaircasing sets on.Numerical simulations, however, strongly suggest that, over thelong run, coarsening takes place or, jumps successively disappear inan orderly fashion until the solution eventually settles down on atrivial steady-state.

Staircasing

In The Literature

Over the past 20 years many an attempt were undertaken to eitherreplace the equation with a better behaved one which would betractable mathematically or to develop a satisfactory theoreticalframework capable of explaining the main numerical findings.

Results about PM. Kichenassamy ’96, Kawohl-Kutev ’98,Taheri-Tang-Zhang ’05, Zhang ’06, Ghisi-Gobbino ’07-(one dimension⇤).

Regularizations and relaxations. Lions-Catte-Morel-Coll’92, Cottet-Germain ’93, Weickert ’96, Cottet-El Ayyadi ’98,Belahmidi ’03, Belahmidi-Chambolle ’05, Amann ’07.

Semidiscrete models. Esedoglu ’01,Bellettini-Novaga-Paolini ’06, B-N-P-Tornese ’08 and ’09,Bellettini-Fusco ’08, Colombo-Gobbino (one dimension).

In The Literature

Nonlocal Di↵usivity

Consider the equations

= 11+|(��)1�"

u|2 �u in ⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ ,

and (u

= r · � 11+|r1�"

�in ⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ .

where the local terms in the nonlinear di↵usivity are replaced bynonlocal ones of a slightly lower order.

= 11+|(��)1�"

u|2 �u in ⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ ,

and (u

= r · � 11+|r1�"

�in ⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ .

= 11+|(��)1�"

u|2 �u in ⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ ,

and (u

= r · � 11+|r1�"

�in ⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ .

Basic Properties

The equations can be thought of as regularizations of thePerona-Malik equation and of a once-integrated version of it(at least in one dimension).

They are now quasilinear but are not of a variational natureany longer.

They still admit natural Ljapunov functionals since

⌦|ru|2 dx = �

(�u)2

1 + |(��)1�"u|2 ,

⌦|u|2 dx = �

1 + |(r)1�"u|2 ,

respectively.

Basic Properties

⌦|ru|2 dx = �

(�u)2

1 + |(��)1�"u|2 ,

⌦|u|2 dx = �

1 + |(r)1�"u|2 ,

respectively.

Basic Properties

⌦|ru|2 dx = �

(�u)2

1 + |(��)1�"u|2 ,

⌦|u|2 dx = �

1 + |(r)1�"u|2 ,

respectively.

Local Existence

Local existence of classical and weak solutions, respectively, can beobtained for regular enough initial data

u0 2 W2,p(⌦) or u0 2 W1,p(⌦), respectively

for a p which is large enough and depends on " 2 (0, 1].

The proof relies on the quasilinear nature of the equation and isbased on maximal regularity for parabolic equations.

Local Existence

Local existence of classical and weak solutions, respectively, can beobtained for regular enough initial data

u0 2 W2,p(⌦) or u0 2 W1,p(⌦), respectively

for a p which is large enough and depends on " 2 (0, 1].

The proof relies on the quasilinear nature of the equation and isbased on maximal regularity for parabolic equations.

Maximal regularity

Consider the abstract Cauchy problem(

+ A(u)u = F (u) on E0 for t > 0 ,

u(0) = u0 .

It can be shown to possess a unique local solution if

A 2 C1��X↵,H(E1,E0)

F 2 C1�(X↵,E0) ,

where X↵o⇢ E↵ = (E0,E1)↵, ↵ 2 [0, 1), and H(E1,E0) is the space

of generators of analytic semigroups with common domain ofdefinition E1 endowed with the topology induced by L(E1,E0).

Maximal regularity

Consider the abstract Cauchy problem(

+ A(u)u = F (u) on E0 for t > 0 ,

u(0) = u0 .

It can be shown to possess a unique local solution if

A 2 C1��X↵,H(E1,E0)

F 2 C1�(X↵,E0) ,

where X↵o⇢ E↵ = (E0,E1)↵, ↵ 2 [0, 1), and H(E1,E0) is the space

of generators of analytic semigroups with common domain ofdefinition E1 endowed with the topology induced by L(E1,E0).

Stationary solutions

These regularizations are milder than other regularizations whichhave previously been considered for Perona-Malik.

They are essentially obtained by convolution with a non-smoothkernel.The upshot of this is that continuous piecewise a�ne functions (inone dimension) and characteristic functions of smooth sets (in anydimension) are honest stationary solutions of the correspondingequations.

In essence these regularizations do not take the edge o↵ of thePerona-Malik equation, while, however, delivering well-posedmodels.

These regularizations are milder than other regularizations whichhave previously been considered for Perona-Malik.They are essentially obtained by convolution with a non-smoothkernel.

The upshot of this is that continuous piecewise a�ne functions (inone dimension) and characteristic functions of smooth sets (in anydimension) are honest stationary solutions of the correspondingequations.

These regularizations are milder than other regularizations whichhave previously been considered for Perona-Malik.They are essentially obtained by convolution with a non-smoothkernel.The upshot of this is that continuous piecewise a�ne functions (inone dimension)

and characteristic functions of smooth sets (in anydimension) are honest stationary solutions of the correspondingequations.

These regularizations are milder than other regularizations whichhave previously been considered for Perona-Malik.They are essentially obtained by convolution with a non-smoothkernel.The upshot of this is that continuous piecewise a�ne functions (inone dimension) and characteristic functions of smooth sets (in anydimension)

are honest stationary solutions of the correspondingequations.

These regularizations are milder than other regularizations whichhave previously been considered for Perona-Malik.They are essentially obtained by convolution with a non-smoothkernel.The upshot of this is that continuous piecewise a�ne functions (inone dimension) and characteristic functions of smooth sets (in anydimension) are honest stationary solutions of the correspondingequations.

Behavior in 1d

How about global existence?

It is not clear whether (smooth) solutions

develop singularities in finite or infinite time or

never develop a singularity or

exist globally in some weaker form

exist for weaker initial data in some weaker form

Further regularization

We now have parabolic equations which can be regularized

11+|(��)1�"

u|2 + ��u in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ ,

and8>><

= r ·⇣⇥

11+|r1�"

u|2 + �⇤ru

⌘in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ .

11+|(��)1�"

u|2 + ��u in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ ,

and8>><

= r ·⇣⇥

11+|r1�"

u|2 + �⇤ru

⌘in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ .

11+|(��)1�"

u|2 + ��u in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ ,

and8>><

= r ·⇣⇥

11+|r1�"

u|2 + �⇤ru

⌘in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ .

A priori Estimates

The best uniform (in �) a priori estimates that can be obtained forthe two equations are

� 2 L1�[0,T ],W2,1(⌦)

�and

� 2 L1�[0,T ],W1,1(⌦)

respectively.

The estimates are optimal since stationary solutions do exist forwhich the Laplacian and the gradient, respectively, are justmeasures.

A priori Estimates

The best uniform (in �) a priori estimates that can be obtained forthe two equations are

� 2 L1�[0,T ],W2,1(⌦)

�and

� 2 L1�[0,T ],W1,1(⌦)

respectively.The estimates are optimal since stationary solutions do exist forwhich the Laplacian and the gradient, respectively, are justmeasures.

Global existence

In one space dimension equation

1 + |(�@xx

)1�"u|2@xx

can be shown to possess global smooth solutions provided ✏ > 1/2.

In all cases there is a transition from non-trivial to standarddi↵usive behavior at some critical value of ".

Global existence

In one space dimension equation

1 + |(�@xx

)1�"u|2@xx

can be shown to possess global smooth solutions provided ✏ > 1/2.

In all cases there is a transition from non-trivial to standarddi↵usive behavior at some critical value of ".

The equation

Consider the equation

= r · ⇥( 11+|ru|2 + �)ru

⇤in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ .

It is still a forward-backward di↵usion (!) and it has an energy

E�(u) =1

�log(1 + |ru|2) + �|ru|2 dx

⌦'�(ru) dx .

The equation

= r · ⇥( 11+|ru|2 + �)ru

⇤in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ .

It is still a forward-backward di↵usion (!)

and it has an energy

E�(u) =1

�log(1 + |ru|2) + �|ru|2 dx

⌦'�(ru) dx .

The equation

= r · ⇥( 11+|ru|2 + �)ru

⇤in ⌦ for t > 0 ,

@⌫u = 0 on @⌦ for t > 0 ,

u(t, ·) = u0 in ⌦ .

It is still a forward-backward di↵usion (!) and it has an energy

E�(u) =1

�log(1 + |ru|2) + �|ru|2 dx

⌦'�(ru) dx .

Basic Properties

While the new equation is still of forward-backward type as long as� < 1/8, it is now uniformly parabolic

�|⇠|2 q�(⇠) · ⇠ (1 + �)|⇠|2 , ⇠ 2 Rn ,

for q�(⇠) = r'�(⇠) =�

11+|⇠|2 + �

�⇠ , ⇠ 2 Rn.

The energy functional is thus of convex-concave-convex type andfits into a framework proposed by Kinderlehrer-Pedregal ’92 andformalized and extended in a particular instance by Demoulini ’96from seeds already present in Chipot-Kinderlehrer ’88.

Basic Properties

While the new equation is still of forward-backward type as long as� < 1/8, it is now uniformly parabolic as

�|⇠|2 q�(⇠) · ⇠ (1 + �)|⇠|2 , ⇠ 2 Rn ,

for q�(⇠) = r'�(⇠) =�

11+|⇠|2 + �

�⇠ , ⇠ 2 Rn.

Basic Properties

While the new equation is still of forward-backward type as long as� < 1/8, it is now uniformly parabolic as

�|⇠|2 q�(⇠) · ⇠ (1 + �)|⇠|2 , ⇠ 2 Rn ,

for q�(⇠) = r'�(⇠) =�

11+|⇠|2 + �

�⇠ , ⇠ 2 Rn.

General Case

Theorem

Let � 2 (0, 1/8). The equation8><

= r · q�(ru) in ⌦⇥ (0,1) =: Q1 ,

@⌫u = 0 on @⌦⇥ (0,1) ,

u(0, ·) = u0 in ⌦ ,

possesses a unique global solution u

� : R+ ! H1u0

(⌦), where

(⌦) := {u 2 H1(⌦) |Z

⌦u dx =

⌦u0 dx} ,

for any given u0 2 H1(⌦) \ L1(⌦) such that there exists a (ingeneral non unique) Young measure (⌫�

x ,t)(x ,t)2Q1 satisfying

� 2 L1(Q1) \ C↵,↵/2(Q1) , u

2 L2(Q1) ,

General Case

Theorem

= r · q�(ru) in ⌦⇥ (0,1) =: Q1 ,

@⌫u = 0 on @⌦⇥ (0,1) ,

u(0, ·) = u0 in ⌦ ,

� : R+ ! H1u0

(⌦), where

(⌦) := {u 2 H1(⌦) |Z

⌦u dx =

⌦u0 dx} ,

x ,t)(x ,t)2Q1

satisfying

� 2 L1(Q1) \ C↵,↵/2(Q1) , u

2 L2(Q1) ,

General Case

Theorem

= r · q�(ru) in ⌦⇥ (0,1) =: Q1 ,

@⌫u = 0 on @⌦⇥ (0,1) ,

u(0, ·) = u0 in ⌦ ,

� : R+ ! H1u0

(⌦), where

(⌦) := {u 2 H1(⌦) |Z

⌦u dx =

⌦u0 dx} ,

x ,t)(x ,t)2Q1 satisfying

� 2 L1(Q1) \ C↵,↵/2(Q1) , u

2 L2(Q1) ,

General Case

q�(⇠) d⌫�x ,t(⇠) ·r dx ,

2 H10(⌦) for a.a. t 2 R+ .

Such a solution is called a weak Young measure valued solution.Furthermore

supp(⌫�x ,t) ⇢ ['� = '⇤⇤� ] \ A('⇤⇤� ) ,

�(x , t) =

⇠ d⌫�x ,t(⇠) a.e. in Q1 .

Moreover

�(t, ·) *Z

⌦u0(x) dx in H1(⌦) as t !1 .

General Case

q�(⇠) d⌫�x ,t(⇠) ·r dx ,

2 H10(⌦) for a.a. t 2 R+ .

Such a solution is called a weak Young measure valued solution.

Furthermore

supp(⌫�x ,t) ⇢ ['� = '⇤⇤� ] \ A('⇤⇤� ) ,

�(x , t) =

⇠ d⌫�x ,t(⇠) a.e. in Q1 .

Moreover

�(t, ·) *Z

⌦u0(x) dx in H1(⌦) as t !1 .

General Case

q�(⇠) d⌫�x ,t(⇠) ·r dx ,

2 H10(⌦) for a.a. t 2 R+ .

supp(⌫�x ,t) ⇢ ['� = '⇤⇤� ] \ A('⇤⇤� ) ,

�(x , t) =

⇠ d⌫�x ,t(⇠) a.e. in Q1 .

Moreover

�(t, ·) *Z

⌦u0(x) dx in H1(⌦) as t !1 .

General Case

q�(⇠) d⌫�x ,t(⇠) ·r dx ,

2 H10(⌦) for a.a. t 2 R+ .

supp(⌫�x ,t) ⇢ ['� = '⇤⇤� ] \ A('⇤⇤� ) ,

�(x , t) =

⇠ d⌫�x ,t(⇠) a.e. in Q1 .

Moreover

�(t, ·) *Z

⌦u0(x) dx in H1(⌦) as t !1 .

Consequences – Young measures

Nature of the Gradients

Thanks to the results concerning the support supp ⌫x ,t of the

Young measure and to the rotational symmetry of the energyfunctional, it follows that

(⌫x ,t = �⇠(x ,t) ,

or ⌫x ,t = ��

x,t+ (1� �)�⇤�

a.e. in Q1, where ⇠(x , t) 2 ['� = '⇤⇤� ],

(��

x ,t = m�⇠0

|⇠0|⇤�

x ,t = M�⇠0

|⇠0|for some ⇠0 = ⇠0(x , t) 2 RN ,

and � = �(x , t) 2 [0, 1].

Consequences – Young measures

Nature of the Gradients

Thanks to the results concerning the support supp ⌫x ,t of the

Young measure and to the rotational symmetry of the energyfunctional, it follows that

(⌫x ,t = �⇠(x ,t) ,

or ⌫x ,t = ��

x,t+ (1� �)�⇤�

a.e. in Q1, where ⇠(x , t) 2 ['� = '⇤⇤� ],

(��

x ,t = m�⇠0

|⇠0|⇤�

x ,t = M�⇠0

|⇠0|for some ⇠0 = ⇠0(x , t) 2 RN ,

and � = �(x , t) 2 [0, 1].

Manifestation of the Micro-ramping Phenomenon

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−60

Plot of ux(2,⋅)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−5

Plot of u0 and u(2,⋅)

u(2,⋅)u0

What do the Results Mean?

Observe that solutions of PM� are continuous, while the samecannot be said for all solutions of limiting PM.

The concept of weak Young measure valued solution for PM�

is rather natural in hindsight. It is in fact able to capture theoscillatory behavior of the gradient caused by the attempt ofsolutions to flee the backward regime either developingsmaller or larger gradients in an alternating fashion.

“In the limit” these oscillations turn into jumps as thepreferred slopes tend to 0 and 1 in size.

While this is the generic behavior of global solutions of PM totranscritical initial data, interestingly, in two spacedimensions, there do exist families of global classicalsolutions with such initial data as shown in Ghisi-Gobbino[2011].

The long time behavior of solutions, that is, the eventualconvergence to a trivial steady-state for PM� as well as forPM has long been observed numerically. It is comforting tohave a theoretical confirmation that this is not an artifact dueto numerical di↵usion but rather a feature of the equation.

When the regularization parameter gets larger than 1/8, theequation loses its forward-backward and degenerate characterand possesses global classical solutions.

For transcritical initial data there are e↵ectively two timesscales of evolution. In the forward and fast regime, thesolution quickly tends to become constant, whereas, in thebackward and slow regime, oscillations appear which survivefor a long time because they cannot be readily damped byregular di↵usion.

Long Time Behavior u(t, ·)

Long Time Behavior ux

(t, ·)

What happens as � ! 0?

Colombo and Gobbino took our model and used �-convergencetools to identify a limit as � ! 0.As previously noted, this can be achieved by speeding up theevolution time.More precisely, rescaling the solutions of the regularized model