Inverse Problems:Introduction and Basic Concepts

I.LyulinaMarch 12, 2003

Inverse problems

Solution of an inverse problem involves determining unknown causes based on observation of their effectsThis is in contrast to the corresponding direct problem, whose solution involves finding effects based on a complete description of their causes

Direct problem: determining the effect y of a given cause xwhen a definite mathematical model K is posited K x = y .Two inverse problems may be posed:

Problem of causationgiven K and ydetermine x

Problem of identification given x and ydetermine K

In the direct problem existence, uniqueness and stability of solution is assumed. But in inverse problems none of these qualities can be taken for granted.

Examples of Inverse Problems

Inverse problem: find a polynomial P(x) of degree n with zeros .

.1 , ..., nx x

Direct problem: find the zeros of a given polynomial P(x).

Ex.1

Solution of inverse problem: 1( ) ( )...( )nP x C x x x x= − −

Ex.2 Lagrange interpolation problem.Inverse problem: find a polynomial P(x) of degree n that assumes given values at given points

Direct problem: find the values of given polynomial P(x) at given points .

1 , ..., ny y 1 , ..., nx x

1 , ..., nx x

Ex.3 Inverse scattering problem

D suiu

scattering field incident field( ) exp( )iu x ik xθ= scattering object

Direct problem: find the total field such thati su u u= +Direct problem: find the total field such that

2

( 1 ) / 2

0 \0

( )

N

ss N

u k u in Du o n D

u ik u O r fo r r xr

− +

∆ + == ∂

∂ − = = → ∞∂

!

Inverse problem: determine the shape D when the far field pattern is measured on the unit sphere in .N!

( )xux∞

xx

( 1) / 2( 1) / 2

exp( )( ) ( ) ( )s N

N

ik x xu x u O r xxx

− +∞−= + → ∞

uniformly in

Asymptoticbehavior:

Ex.4 Backwards heat equation

1D heat equation: 2

2

( , ) ( , )u x t u x tt x

∂ ∂=∂ ∂

BC: (0, ) ( , ) 0 0u t u t tπ= = ≥

IC: 0( ,0) ( ) 0u x u x x π= ≤ ≤

Solution of direct problem: find temperature at time t=T

2

01 0

2( , ) e s in ( ) ( ) s in ( )n tn n

n

u x t a n x a u y n y d yπ

π

∞−

=

= =∑ ∫Solution of inverse problem: determine the initial temperature

π 0 ( )u x

2

00

1

2( , ) ( , ) ( ) 0

( , ) e sin ( ) sin( )n T

n

u x T k x y u y dy x

k x y nx ny

ππ∞

−

=

= ≤ ≤

=

∫

∑

Ex.5 Sturm-Liouville eigenvalue problem

Consider a string of length L, mass density , fixedat the endpoints. Plucking the string produces tones due to vibration. Thedisplacement satisfies the wave equation:

( ) 0 0x x Lρ > ≤ ≤

( , )v x t2 2

2 2

( , ) ( , )( )

( 0 , ) ( , ) 0

v x t v x txt x

v t v L t

ρ ∂ ∂=∂ ∂

= =

A periodic displacement with frequency solves the boundary valueproblem if and only if and W satisfy the Sturm-Liouville eigenvalueproblem:

0ω>ω2( ) ( ) ( ) 0 (0) ( ) 0W x x W x W W Lω ρ′′ + = = =

Direct problem: compute the eigenfrequencies and correspondingeigenfunctions for known function .Inverse problem: determine a mass density from measured frequencies.

ωρ

ρ

( , ) ( ) [ c o s s i n ]v x t W x a t b tω ω= +Periodic solution:

Ex.6 Diffusion in inhomogeneous medium

Consider heat conduction in a material occupying a domain D whose temperature is kept at 0 at the boundary. The temperature distribution u aftersufficiently long time can be modelled by:

( ( , , ) ( )) ( , , ) , ( , , )0

div k x y z grad u f x y z x y z Du on D

− = ∈= ∂

Direct problem: solve the boundary value problem for this equation withgiven heat conductivity k(x,y,z) and internal heat sources f(x,y,z).

Inverse problem: determine heat conductivity k(x,y,z) from measurements

of the temperature u(x,y,z) and the flux on the boundary.k un

∂∂

This is a parameter identification problem. Very common in physics when the physical laws governing the process are known, but information aboutparameters is needed.

Mathematical Background for Inverse ProblemsX – vector space over the field R or C

Inner product( ).,. : ,X X× →! " ( ) ( ) ( )

( ) ( )( ) ( )( ) ( )( )

, , , , ,

, , ,

, , ,

, , 0

, 0 0

x y z x z y z x y z X

x y x y x y X

x y y x x y X

x x x x fo r a ll x

x x if x

α α α+ = + ∈

= ∈ ∈

= ∈

∈ ≥

> ≠

!

!

: X →i ! 0 0

,

x f o r a l l x

x x x X

x y x y x y X

α α α> ≠

= ∈ ∈

+ ≤ + ∈

!Norm

Vector space with norm is called normed space. An inner product space is always a normed space with .

Example : C[a,b] – space of continuous functions on [a,b]:

( )

( )

2

2

2

, ( ) ( ) , [ , ]

, ( ) [ , ]

b

La

b

La

x y x t y t d t x y C a b

x x x x t d t x C a b

= ∈

= = ∈

∫

∫

( ),x x x=

Every normed space carries a topology introduced by the norm:

Ball of radius r and center x: A subset is bounded if there exists : .

( , ) { : }K x r y X y x r= ∈ − <M X⊂ 0r > ( , )M K x r⊂

A subset M is open if for every there exists : . x M∈ 0ε > ( , )K x Mε ⊂A subset M is closed if complement is open . Or if the limit of every convergent sequence also belongs to M.

\X M

A subset M is compact if it is closed and bounded.(in finite dimensional space)

( ){ : lim }k kk

M x X x M x x→ ∞

= ∈ ∃ ⊂ =Closure of subset M:

A sequence is bounded if there exists : .( )k kx X⊂ 0c > kx c all k≤

A sequence is convergent if there exists .: 0kx X x x∈ − →A sequence is Cauchy sequence if for every there exists 0ε >

: ,m kN x x for all m kε∈ − < ≥$ $ .A normed space X is complete if every Cauchy sequence converges in X. A complete normed space is called Banach space. A complete inner product space is called Hilbert space. Example : the most familiar Hilbert spaces

2 ( , ) { : ( ) ; ( ) 0}b

a

L a b f p x f f dx p x= < ∞ >∫- the vector space of square integrable functions2 ( , )L a b

( , )pH a b Sobolev space - the vector space of measurable functions such that their generalized derivatives are square integrable .

1 ( 1) 2( , ) { [ , ] : ( ) ; , }t

p p pH a b f C a b f t ds Lα

α ψ α ψ− −= ∈ = + ∈ ∈∫ !

X ,Y – normed spaces , - linear operator:A X Y→

A is bounded if there exists c>0 such that: .Ax c x x X≤ ∀ ∈

0x

Axx≠

The following assertions are equivalent: -A is bounded-A is continuous at x=0:-A is continuous for every x.

0 0k kx Ax→ ⇒ →

X ,Y – Hilbert spaces , - linear, bounded. Then there exists one and only one adjoint operator : - linear and bounded with the property: .

:A X Y→:A Y X∗ →

( ) ( ), , ,Ax y x A y x y∗= ∀Banach-Schauder theorem: X ,Y – Banach spaces , - is linear, continuous, one-to-one mapping. Then the inverse is continuous.

:A X Y→1 :A Y X− →

The smallest of c is called the norm of A: .supA =

A is compact if for each bounded set S in X , the set A(S) has compact closure in Y.

Concept of a well-posed problemsX and Y - normed spaces, - mapping. The equation is called well-posed if the following holds:

Existence: for every there is (at least one) such that .

:K X Y→

y Y∈ x X∈

Kx y=

Kx y=

Uniqueness: for every there is at most one such that .y Y∈ x X∈ Kx y=Stability: the solution x depends continuously on y: for every sequence

( ) :k n nn nx X Kx Kx x x

→ ∞ → ∞∈ → ⇒ →

Important: to specify the full triple (X,Y,K).Existence and uniqueness depend on the algebraic nature of the spaces.Stability depends on topology: whether the inverse operator is continuous.If the computed solution does not depend continuously on the data, then ingeneral it has nothing to do with the true solution. No way to overcome thisdifficulty unless additional information about solution is available.

A lack of information cannot be remedied by any mathematical trickery!

Example of ill-posed problemConsider a Cauchy’s problem for the Laplace equation:

2 2

2 2( , ) 0 [0, )u uu x y inx y

∂ ∂∆ = + = × ∞∂ ∂

!

( , 0 ) ( ) 0( , 0 ) 1( ) s in ( )

u x f xu x g x n x x

y n

= =∂ = = ∈

∂!

2

1( , ) s in ( ) s in h ( ) , 0u x y n x n y x yn

= ∈ ≥!

with IC:

Unique solution:

The error in the data tends to zero while the error in the solution tends toinfinity for all y>0 :

2

1sup{ ( ) ( ) } 0,

1sup{ ( , ) } sinh( ) ,

x

x

f x g x nn

u x y ny nn

∈

∈

+ = → → ∞

= → ∞ → ∞

!

!

DifferentiationDirect problem: for a given continuous function find

[0,1]x C∈t

Inverse problem: for a given continuous differentiable function

compute . 1[0,1], (0) 0y C y∈ = x y′=0

( ) ( ) , [ 0 , 1 ]

( 0 ) 0

y t x s d s t

y

= ∈

=

∫

0

: [0 ,1] [0,1]

( )( ) ( ) , [0 ,1]t

K C C

K x t x s ds t

→

= ∈∫0 1

[0,1] max ( )t

C x x t∞ ≤ ≤

=The solution of K x = y is the derivative: . Consider a perturbed data:x y′=

2 2

0 0

1sin cos

10

t ty y y y

y y y yδ δ

δδ δ δ

δ∞ ∞→ →

′ ′= + = +

′ ′− → − → = ∞

% %

% %

11

0 1[0,1] max ( )

C tC x x t

≤ ≤′=

ill-posed

1 10 0

1 1C C

y y y yδ δδ δ→ →

′ ′− → − →% % well-posed

Integral operatorsMany inverse problems lead to integral equation. Integral operators are compactin many natural topologies:

2 2 2

2

0

: ( , ) ( , ), (( , ) ( , ))

( ) ( , ) ( ) , ( , ), ( , )t

K L a b L c d k L c d a b

Kx t k t s x s ds t c d x L a b

→ ∈ ×

= ∈ ∈∫Linear equations in the form K x=y with compact operators are always ill-posed.

K - compact

Theorem: X,Y – normed spaces, K – linear, compact( ) { : 0}

dim( \ ( ))N K x X Kx

X N K= ∈ =

infiniteThen there exists a sequence such that ,But does not converge. We can even choose such that

.

( )k kx X⊂ 0kKx →

( )kxkx x− →∞

null space

The worst-case error (differentiation)How large could be the error in the worst case if the error in the data is ?δ

[ ]

2

2 22

0 0

0

, [0,1] : ,

(0) (0) 0 ( ) 0

( ) ( ) ( ) ( ) 2 ( ) ( )

4 ( ) 4 ( ) 2

t t

t

y y C E y E y E

z y y z z z t

dx t x t z t z t ds z t z t dsds

E z t ds Ez t x x E if y yδ δ

∞ ∞

∞ ∞

′′ ′′∈ ∃ ≤ ≤′ ′= − = = ≥

′ ′ ′ ′′− = = = ≤

′≤ = ⇒ − ≤ − ≤

∫ ∫

∫

% %%

%

% %

Bound for the worst-case error is for an error in the data and the additional information .

2 Eδ δy E

∞′′ ≤

The worst-case error (general definition)

1X X⊂X and Y – Banach spaces, - linear, bounded operator.

subspace with a “stronger” norm .We define the worst-case error:

:K X Y→1 1

.. : c x c x x∃ ≤ ∀

11 1( , , ) sup{ , , }.:F E x x X Kx x Eδ δ⋅ = ∈ ≤ ≤

Sometimes is possible, but for compact operators this is not thecase. Lemma:

1.. ..=

X and Y – Banach spaces, - linear and compact operator.We assume is infinite-dimensional. Then

:K X Y→\ ( )X N K

0 00 0, 0 : ( , , ) (0, )E c F E cδ δ δ δ∀ > ∃ > > ⋅ ≥ ∀ ∈The worst-case error depends on the operator K and the norms in . 1, ,X Y X

DifferentiationConsider differentiation when spaces and norms are defined as follows:

2

2

2

11 1

22 2

(0,1){ (0,1) : (1) 0}

{ (0,1) : (1) 0, (0) 0}L

L

X Y LX x H x x x

X x H x x x x

= =′= ∈ = =

′ ′′= ∈ = = =Norms and are stronger than . Then for any E and

1x

2x 2L

x δ2 13 3

1 2( , , ) ( , , )F E E F E Eδ δ δ δ⋅ ≤ ⋅ ≤

2

2 2

111 1

2 2

0 0 0 0 01

0

:

( ) ( ) ( ) ( ) ( )

( ) ( )

tt t

Lt

L L

X

x x t dt x t x s ds dt x t x s ds

x t Kx t dt Kx x Eδ

=

=

′= = − + =

′ ′= − ≤ =

∫ ∫ ∫ ∫

∫

partial integration

( , )x y x y≤

Ex.4 Backwards heat equation

1D heat equation: 2

2

( , ) ( , )u x t u x tt x

∂ ∂=∂ ∂

BC: (0, ) ( , ) 0 0u t u t tπ= = ≥

IC: 0( ,0) ( ) 0u x u x x π= ≤ ≤

Solution of direct problem: find temperature at time t=T

2

01 0

2( , ) e s in ( ) ( ) s in ( )n tn n

n

u x t a n x a u y n y d yπ

π

∞−

=

= =∑ ∫Solution of inverse problem: determine the initial temperature

π 0 ( )u x

2

00

1

2( , ) ( , ) ( ) 0

( , ) e sin ( ) sin( )n T

n

u x T k x y u y dy x

k x y nx ny

ππ∞

−

=

= ≤ ≤

=

∫

∑

Backwards heat equation( , )u x τDetermine the temperature at time with additional

information . Solution of direct problem:(0, )Tτ ∈

2(.,0)L

u E≤2

01 0

2( , ) e sin ( ) ( ) s in ( ) , 0n t

n

u x t nx u y ny dy xπ

ππ

∞−

=

= ≤ ≤∑ ∫Defined spaces and norm as follows:

2

2

2

21 0

1 0

01

(., ) (0, )

2{ (0, ) : e sin( .) ( ) sin( ) }n

n

L

v u X Y L

X v L v n u y ny dy

v u

πτ

τ π

ππ

∞−

=

= = =

= ∈ =

=

∑ ∫

K is an integral operator with the kernel:2 ( )

1

2( , ) e sin( )sin( )n T

nk x y nx nyτ

π

∞− −

== ∑

Worst-case error:1

1( , , ) T TF E E

τ τ

δ δ−

⋅ ≤Conclusion: the worst-case error is smaller if is closer to T.τ

ConclusionsMany inverse problems can be formulated as operator equations of the form K x = y , where K is linear, compact operator between Hilbert spaces X and Y.Most inverse problems are ill-posed in the sense ofHadamard.Successful solution strategy requires additional a priori information about the solution.