introduction to inverse problems - mitweb.mit.edu/2.717/www/2.717-wk10-a.pdf · introduction to...

Introduction to Inverse Problems

• What is an image? Attributes and Representations • Forward vs Inverse• Optical Imaging as Inverse Problem

• Incoherent and Coherent limits• Dimensional mismatch: continuous vs discrete• Singular vs ill-posed

• Ill-posedness: a 2×2 example• Measures of ill-posedness; the Shannon metric

MIT 2.71704/11/05 – wk10-a-1

Basic premises• What you “see” or imprint on photographic film is a very narrow

interpretation of the word image• Image is a representation of a physical object having certain attributes• Examples of attributes

– Optical image: absorption, emission, scatter, color wrt light– Acoustic image: absorption, scatter wrt sound– Thermal image: temperature (black-body radiation)– Magnetic resonance image: oscillation in response to radio-

frequency EM field• Representation: a transformation upon a matrix of attribute values

– Digital image (e.g. on a computer file)– Analog image (e.g. on your retina)

MIT 2.71704/11/05 – wk10-a-2

How are images formed• Hardware

– elements that operate directly on the physical entity– e.g. lenses, gratings, prisms, etc. operate on the optical field– e.g. coils, metal shields, etc. operate on the magnetic field

• Software– algorithms that transform representations– e.g. a radio telescope measures the Fourier transform of the source

(representation #1); inverse Fourier transforming leads to a representation in the “native” object coordinates (representation #2); further processing such as iterative and nonlinear algorithms lead to a “cleaner” representation (#3).

– e.g. a stereo pair measures two aspects of a scene (representation #1); a triangulation algorithm converts that to a binocular image with depth information (representation #2).

MIT 2.71704/11/05 – wk10-a-3

Who does what• In optics,

– standard hardware elements (lenses, mirrors, prisms) perform a limited class of operations (albeit very useful ones); these operations are

• linear in field amplitude for coherent systems• linear in intensity for incoherent systems• a complicated mix for partially coherent systems

– holograms and diffractive optical elements in general perform a more general class of operations, but with the same linearity constraints as above

– nonlinear, iterative, etc. operations are best done with software components (people have used hardware for these purposes but it tends to be power inefficient, expensive, bulky, unreliable – hence these systems seldom make it to real life applications)

MIT 2.71704/11/05 – wk10-a-4

Imaging channels

HumansHumansPhysicsPhysicsHumanoidsHumanoidsAlgorithmsAlgorithms

Information generatorsInformation generators•• Wave sourcesWave sources•• Wave Wave scatterersscatterers

•• ImagingImaging•• CommunicationCommunication•• StorageStorage

UsersUsersProcessing elementsProcessing elements

GOAL:GOAL: Maximize Maximize informationinformation flowflow

MIT 2.71704/11/05 – wk10-a-5

Generalized (cognitive) representations

MIT 2.71704/11/05 – wk10-a-6

Situation ofinterest

YES/YES//NO/NO

encoded intoa scene

optical system produces a (geometricallysimilar) image

YES/YES//NO/NO

cognitiveprocessing answer

Classical “inverse problem” viewClassical “inverse problem” view--pointpoint

Situation ofinterest

YES/YES//NO/NO

encoded intoa scene

optical system produces an information-richlight intensity pattern

otherotherfunctionsfunctions

answer

““NonNon--imaging” or “generalized” sensor viewimaging” or “generalized” sensor view--pointpoint

Advantages: - optimum resource allocation- better reliability- adaptive, attentive operation

if necessary (requires resource reallocation)

e.g. is there a tanke.g. is there a tankin the scene?in the scene?

Forward problem

hardwarechannel

“physicalattributes”

(measurement)

object

fieldpropagation detection

object measurement

The Forward Problem answers the following question:• Predict the measurement given the object attributes

MIT 2.71704/11/05 – wk10-a-7

Inverse problem

hardwarechannel


(measurement)

object

fieldpropagation detection

objectrepresentation measurement

The Inverse Problem answers the following question:• Form an object representation given the measurement

MIT 2.71704/11/05 – wk10-a-8

Optical Inversionfree space(Fresnel)

propagation

free space(Fresnel)

propagation

MIT 2.71704/11/05 – wk10-a-9

free space(Fresnel)

propagation

lens array of point-wisesensors (camera)

lensamplitude object(dark “A” on bright

background)array ofintensity

measurements

amplituderepresentation

Optical Inversion: coherent

Nonlinear problemNonlinear problem

( )yxf , ( ) ( ) ( )2

coh dd,,, ∫ −′−′=′′ yxyyxxhyxfyxIobject

amplitude intensity measurement at the output plane

Note: I could make the problem linear if I could measureamplitudes directly (e.g. at radio frequencies)

MIT 2.71704/11/05 – wk10-a-10

Optical Inversion: incoherent

Linear problemLinear problem

( )yxI ,obj ( ) ( ) ( )∫ −′−′=′′ yxyyxxhyxIyxI dd,,, incohobjmeas

objectintensity intensity measurement at the output plane

MIT 2.71704/11/05 – wk10-a-11

Dimensional mismatch• The object is a “continuous” function (amplitude or intensity)

assuming quantum mechanical effects are at sub-nanometer scales, i.e.much smaller than the scales of interest (100nm or more)– i.e. the object dimension is uncountably infinite

• The measurement is “discrete,” therefore countable and finite• To be able to create a “1-1” object representation from the

measurement, I would need to create a 1-1 map from a finite set of integers to the set of real numbers. This is of course impossible– the inverse problem is inherently ill-posed

• We can resolve this difficulty by relaxing the 1-1 requirement– therefore, we declare ourselves satisfied if we sample the object

with sufficient density (Nyquist theorem)– implicitly, we have assumed that the object lives in a finite-

dimensional space, although it “looks” like a continuous function

MIT 2.71704/11/05 – wk10-a-12

Singularity and ill-posednessUnder the finite-dimensional object assumption, the linear inverse problem

is converted from an integral equation to a matrix equation

( ) ( ) ( ) yxyyxxhyxfyxg d d , ,, −′−′=′′ ∫⇔⇔ fg H=

• If the matrix H is rectangular, the problem may be overconstrained or underconstrained• If the matrix H is square and has det(H)=0, the problem is singular; it can only be solved partially by giving up on some object dimensions (i.e. leaving them indeterminate)• If the matrix H is square and det(H) is non-zero but small, the problem may be ill-posed or unstable: it is extremely sensitive to errors in the measurement f

MIT 2.71704/11/05 – wk10-a-13

Ill-posedness: a toy problemTwo point-sources

(object)Two point-detectors

(measurement)

d1

MIT 2.71704/11/05 – wk10-a-14

A~

B~A

B

Finite-NA imaging system

A, are Gaussian conjugatesA~

B, are Gaussian conjugatesB~

Classical view

x

A~ B~

d2

z1 z2

MIT 2.71704/11/05 – wk10-a-15

Object-measurement transformation

A~ B~

ss

( ) ( ) ( )( ) ( ) ( )BAB~

BAA~

ininout

ininout

IsII

sIII

αα

αα

+=

+=

11( )

talk"-cross"

NAjincjinc 22

2

22 ⎟⎠⎞

⎜⎝⎛=⎟⎟

⎠

⎞⎜⎜⎝

⎛=

λλd

zads

( )2NA~loss

energy=α

Object-measurement transformation( ) ( ) ( )( ) ( ) ( )BAB~

BAA~

ininout

ininout

IsII

sIII

αα

αα

+=

+=

( )( )

( )( )⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

⎛=⎟⎟

⎠

⎞⎜⎜⎝

⎛BA

11

B~A~

in

in

out

out

II

ss

II

α

MIT 2.71704/11/05 – wk10-a-16

( )( )

( )( )

⎟⎟⎠

⎞⎜⎜⎝

⎛=

⎟⎟⎠

⎞⎜⎜⎝

⎛=⎟⎟

⎠

⎞⎜⎜⎝

⎛==

11

and ,BA

,B~A~ where,or

in

in

out

out

ss

II

II

αH

fgHfg

Hopkins matrix

measurement object

Noiseless two-point inversion

⎟⎟⎠

⎞⎜⎜⎝

⎛=

11s

sαH

( ) ⎟⎟⎠

⎞⎜⎜⎝

⎛−

−−

=−

11

11

21

ss

sαH

( ) ( )22 1det s−=αH

( ) ⎟⎟⎠

⎞⎜⎜⎝

⎛⎟⎟⎠

⎞⎜⎜⎝

⎛−

−−

=⎟⎟⎠

⎞⎜⎜⎝

⎛⇒= −

2

12

2

11

11

11

gg

ss

sff

αgHf

MIT 2.71704/11/05 – wk10-a-17

Noisy two-point inversion: ill-posedness

matrix. noise theis where,ˆLet 2

1⎟⎟⎠

⎞⎜⎜⎝

⎛=+=

nn

nnHfg

( )

( )

( ) ⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

−+−−−

==

+=+==

−

−−

221

221

1

11

1

1 iserror thewhere

, ˆˆThen

snsn

ssnn

α

αδ

δ

nHf

ffnHfHgHf

MIT 2.71704/11/05 – wk10-a-18

.1

1 asmuch asby

amplifiedisnoise so and ,1then ,0 as Note

2 ∞→−

→→

s

sd

Noisy two-point inversion:the eigenvalue/eigendirection viewpoint

MIT 2.71704/11/05 – wk10-a-19

.10

01 ,

21212121 whereor

21212121

1001

21212121

Therefore, .)1 that so d(normalize

2121 ,

2121 rseigenvecto

,1 ,1 seigenvalue has 1

1

)2()1(

)2()1(

21

⎟⎟⎠

⎞⎜⎜⎝

⎛−

+=⎟⎟

⎠

⎞⎜⎜⎝

⎛

−=∆=

⎟⎟⎠

⎞⎜⎜⎝

⎛

−⎟⎟⎠

⎞⎜⎜⎝

⎛−

+⎟⎟⎠

⎞⎜⎜⎝

⎛ −=

==

⎟⎟⎠

⎞⎜⎜⎝

⎛−=⎟⎟

⎠

⎞⎜⎜⎝

⎛=

−=+=⎟⎟⎠

⎞⎜⎜⎝

⎛=

ss

ss

sss

s

T ∆QQQH

H

ξξ

ξξ

H µµ


so easily), verifiedbecan (as but

as problem inverse therewrite We).1set (or moment afor Ignore

IQQQfQQQgQfQHfg

=

∆=⇒∆==

=

T

TT

αα

( ) ( )QgQgQfQfQg⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

−

+=∆=⇔∆= −

110

01

11

s

s

MIT 2.71704/11/05 – wk10-a-20


( )

( )

.

12

12~~̂

t,measuremennoisy theof case In the

.

2

2~ ,

2

2~Let

21

21

21

21

21

21

⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

−−+

+

+=

⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

−

+

=≡⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

−

+

=≡

snn

snn

gg

gg

ff

ff

ff

QggQff

MIT 2.71704/11/05 – wk10-a-21


( )

( )

( )

( )

( )( )

( )( )

.12

1 if reliable is ~̂ whereas

;12

1 if reliable is ~̂ Hence,

variance.noiseenergy if reliable ist measuremen The

.

12

12

2

2

12

12~~

~̂

~̂

2

22

22

12

2

22

22

11

21

21

21

21

21

21

2

1

2

1

−>+

+>+

>

⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

−−+

+

+⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

−

+

=

⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

−−+

+

+⎟⎟⎠

⎞⎜⎜⎝

⎛=⎟

⎟

⎠

⎞

⎜⎜

⎝

⎛

sfff

sfff

snn

snn

ff

ff

snn

snn

ff

f

f

n

n

σ

σ

MIT 2.71704/11/05 – wk10-a-22


( )

( )⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

−−+

+

+=

12

12~~̂21

21

snn

snn

ffionamplificat noise strong

11~~

nsuppressio noise moderate 1

1~~

2

1

−

+⇒

sf

sf

δ

δ

In the “non-resolvable” case (s→1), the “average-like quantity”n1+n2 can still be determined with moderate accuracy.

On the other hand, the “difference quantity” n1–n2 remains “swamped”by the noise. So the noise did not completely destroy the measurement,

only one component of it. In general, we can say thatill-posedness reduces the dimensionality of the measurement

(from 2→1, gradually as s→1, in this case.)MIT 2.71704/11/05 – wk10-a-23

Generalizing: ill-posednessand the square Hopkins matrix

.~E

if ~ of featureth - theresolving"" of capable is system imaging The

.~ ,~t measuremen andobject ed transform theDefine

. ,

00

0000

where, form eddiagonaliz itsin matrix Hopkins theRewrite. is and ,1 are , ,ˆ where,ˆ

2

22

)(

)2(

)1(

2

1

j

nj

mm

t

f

j

mmm

µσ

µ

µµ

>⎥⎦⎤

⎢⎣⎡

==

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

=

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

=∆

∆=

××+=

f

QggQff

ξ

ξξ

Q

QQHHfngnHfg

M

L

MOMM

L

L

MIT 2.71704/11/05 – wk10-a-24

A Hopkins matrix example

MIT 2.71704/11/05 – wk10-a-25

Imaging system

measurement

imaginglens

opticalaxis

object

• magnification = 1• no aberrations (diffraction limited)• incoherent source (i.e. point sources radiate independently)• 501 point sources, spacing d=0.05×λ/(NA)

• i.e. much denser than the “Rayleigh resolution criterion” 0.61 ×λ/(NA)

MIT 2.71704/11/05 – wk10-a-26

MIT 2.71704/11/05 – wk10-a-27

The Airy disk function

MIT 2.71704/11/05 – wk10-a-28

Intensity from incoherent sources

( )⎟⎟⎠

⎞⎜⎜⎝

⎛ −=

λNA

jinc2 dkjjkH

MIT 2.71704/11/05 – wk10-a-29

The Hopkins matrix

MIT 2.71704/11/05 – wk10-a-30

Eigenvalues of the Hopkins matrix

MIT 2.71704/11/05 – wk10-a-31

The first few significant eigenvalues

Measures of ill-posedness

Rayleigh metric; H is well-posed if R≈1,

ill-posed if R→∞min

maxRµµ

=

Shannon metric,or Image Mutual Information (IMI);

H is well-posed if C→ ∞, ill-posed if C→0.

∑=

⎟⎟⎠

⎞⎜⎜⎝

⎛+=

n

k

k

12

2

1ln21C

σµ

MIT 2.71704/11/05 – wk10-a-32

Image Mutual Information (IMI)

hardwarechannel


(measurement)

object

fieldpropagation detection gHf

MIT 2.71704/11/05 – wk10-a-33

Assumptions: (a) f has Gaussian statistics;(b) white additive Gaussian noise (waGn)i.e. g=Hf+nwhere n is a Gaussian random vector, independent of f, with correlation matrix of the form σ2 I.

Then

quantifies information transfer between f and g.

( ) ∑=

⎟⎟⎠

⎞⎜⎜⎝

⎛+=

n

k

k

12

2

1ln21,C

σµgf H of seigenvalue :kµ

Significance of the eigenvalues

MIT 2.71704/11/05 – wk10-a-34

n

n-1

10

2nµ

21−nµ

22µ

21µ

(aka how manydimensions

the measurementis worth)

rank ofmeasurement ∑

=⎟⎟⎠

⎞⎜⎜⎝

⎛+=

n

k

k

12

2

1ln21C

σµ

...

...

largereigenvalues of H


n

n-1

10

2nµ

21−nµ

22µ

21µ




=⎟⎟⎠

⎞⎜⎜⎝

⎛+=

n

k

k

12

2

1ln21C

σµ

MIT 2.71704/11/05 – wk10-a-35

eigenvalues of H

...

larger

2σ...


n

n-1

10

2nµ

21−nµ

22µ

21µ




=⎟⎟⎠

⎞⎜⎜⎝

⎛+=

n

k

k

12

2

1ln21C

σµ

MIT 2.71704/11/05 – wk10-a-36

eigenvalues of H

...

larger

2σ

accuracyloss

...


n

n-1

10

2nµ

21−nµ

22µ

21µ




=⎟⎟⎠

⎞⎜⎜⎝

⎛+=

n

k

k

12

2

1ln21C

σµ

MIT 2.71704/11/05 – wk10-a-37

eigenvalues of H

...

larger

2σ

accuracyloss

2tµ

21−tµ ...

...

t

t-1

...


n

n-1

10

2nµ

21−nµ

22µ

21µ




=⎟⎟⎠

⎞⎜⎜⎝

⎛+=

n

k

k

12

2

1ln21C

σµ

MIT 2.71704/11/05 – wk10-a-38

eigenvalues of H larger

2σ

accuracyloss

...

...

Accuracy of the measurement

...1ln1ln1ln...

1ln21C

2

2

2

21

2

22

12

2

+⎟⎟⎠

⎞⎜⎜⎝

⎛++⎟⎟

⎠

⎞⎜⎜⎝

⎛++⎟⎟

⎠

⎞⎜⎜⎝

⎛++

=⎟⎟⎠

⎞⎜⎜⎝

⎛+=

−−

=∑

σµ

σµ

σµ

σµ

ttt

n

k

knoise floor

this term≤1

this term≈0

21

22−<< tt µσµ

≈accuracy of (t-2)th measurement

E.g. 0.5470839348

these digits worthlessif σ ≈10-5

MIT 2.71704/11/05 – wk10-a-39

Mutual information & degrees of freedom

n

n-1

10

2σ2nµ

21−nµ

22µ

21µ

rank ofmeasurement

∑=

⎟⎟⎠

⎞⎜⎜⎝

⎛+=

n

k

k

12

2

1ln21C

σµ

...

...

MIT 2.71704/11/05 – wk10-a-40

2σ

mutualinformation

As noise increases• one rank of H is lost wheneverσ2 overcomes a new eigenvalue• the remaining ranks lose precision

IMI for two-point resolution problem

11

2

1

−=+=

ss

µµ

⎟⎟⎠

⎞⎜⎜⎝

⎛=

11s

sH ( ) 21det s−=H

⎟⎟⎠

⎞⎜⎜⎝

⎛−

−−

=−

11

11

21

ss

sH

( ) ( ) ( )⎟⎟⎠

⎞⎜⎜⎝

⎛ +++⎟⎟

⎠

⎞⎜⎜⎝

⎛ −+= 2

2

2

2 11ln2111ln

21,C

σσssGF

MIT 2.71704/11/05 – wk10-a-41

MIT 2.71704/11/05 – wk10-a-42

IMI vs source separation

( ) 2

1SNRσ

=

s → 0s → 1

Concluding remarks

MIT 2.71704/11/05 – wk10-a-43

• “Exceeding the resolution limit” does not imply a catastrophic degradation in the ability of the system to image; it merely implies gradual loss of accuracy and, possibly, reduction in the number of “degrees of freedom” (i.e., the dimensionality) of the measurement.

• The losses of accuracy and degrees of freedom increase monotonically (i.e., become worse) with the level of measurement noise.

• The Shannon metric conveniently quantifies the gradual losses ofaccuracy and degrees of freedom in terms of the relative magnitude of the eigenvalues of the Hopkins matrix and the noise variance.

• Our formula for the Shannon metric is based on the assumption of both object and noise being Gaussian, incoherent, and completely uncorrelated to each other; it turns out this is a worst case assumption. Under more general conditions, the Shannon metric usually must be computed numerically (e.g., Monte-Carlo simulation.)

• The Shannon metric expresses an upper limit in the worst-case performance of the imaging system for the given Hopkins matrix, but does not specify the “inversion algorithm” that should be used to achieve this limit.

introduction to inverse problems - mitweb.mit.edu/2.717/www/2.717-wk10-a.pdf · introduction to...

Documents