research in progress presentation 2003 look closer to inverse problem qianqian fang thanks to : paul...

Research In Progress Presentation 2003Research In Progress Presentation 2003

Look Closer to Look Closer to Inverse Problem Inverse Problem

Qianqian FangQianqian Fang

Thanks toThanks to: Paul M. Meaney, Keith D. Paulsen, Dun Li, Marg: Paul M. Meaney, Keith D. Paulsen, Dun Li, Margaret Fanning, Sarah A. Pendergrassaret Fanning, Sarah A. Pendergrass

and all other friendsand all other friends

RIP 2003RIP 2003


OutlineOutlineNumerical Methods Linearization

Ax y=

What is MWhat is MAATRTRIIX?X?

Inverse problemInverse problem

SSingularingularVValuealueDDecompositionecomposition

Solving inverse problemSolving inverse problem

Improve the solutionsImprove the solutions

ConclusionsConclusions

SingularSingularMatricesMatrices

Multi-Freq Recon.

Time-Domain Recon.


Numerical Methods and Numerical Methods and linearizationlinearization

Modern Numerical TechniquesModern Numerical TechniquesModern Numerical

Techniques

RealityInfinitely Complicated, Dynamically Changing,Noisy and Interrelated

ModelDiff. Equ./Integral Equ.Linear Relation

Ax=b

Nonlinear methodsNN, GA, SA, Monte-Calo

Mathematical

Numerical

Accuracy Efficiency


What is MWhat is MAATRTRIIXX

from movie The Matrix, WarnerBros,1999,

Unfortunately no one can be told what the matrix is, you have to see it for yourself


What is MATRIXWhat is MATRIX Linear Transform

Map from one space to anotherMap from one space to another Stretch, Rotations, ProjectionsStretch, Rotations, Projections

Structural Information- on grid Simple data structure (comparing with Simple data structure (comparing with

list/tree/object etc)list/tree/object etc) But not that simple (comparing with But not that simple (comparing with

single variable)single variable)

10 20 30 40

10

20

30

40

11 1

1

n

m mn m n

a a

a a´

æ ö÷ç ÷ç ÷ç ÷ç ÷ç ÷ç ÷÷ç ÷÷ççè ø÷

K

M O M

L

nX Î RmY Î R


-4 -3 -2 -1 0 1 2 3

-3

-2

-1

0

1

2

Geometric Geometric InterpretationsInterpretations

2X2 matrix->Map 2D image to 2D 2X2 matrix->Map 2D image to 2D imageimage ˆ2 1

3 1 ˆ

i i

i i

x x

y y

æ öæ öæ ö ÷÷ çç ÷ç ÷÷ çç =÷ç ÷÷ çç ÷÷ç ÷÷- ÷ç ÷è ø çççè ø è ø

ˆ2 1

3 1.5 ˆ

i i

i i

x x

y y

æ öæ öæ ö ÷÷ çç ÷ç ÷÷ çç =÷ç ÷÷ çç ÷÷ç ÷÷÷ç ÷è ø çççè ø è ø

eig(A)={3.5,0}


Geometric Geometric Interpretations 2Interpretations 2

3D matrix3D matrix

2 1 3

1 2 2

0 1 2

ii

i i

i i

xx

y y

z z

æ öæ ö ¢æ ö ÷ç÷ç ÷ ÷ç ç÷ç ÷ ÷ç÷ çç ÷ ÷÷ç ç÷ ÷ç ¢÷ç- - = ç÷ ÷ç ÷ç ç÷ ÷ç ÷ç ÷ ÷ç÷ç ÷ç ÷ç÷ç ÷÷ ÷çç ç- ÷÷ç ¢è øç ÷÷ ç ÷è ø ç÷ è ø÷

1. Stretching

2. Rotation

2 1 3

1 2 2

1 2 2

æ ö÷ç ÷ç ÷ç ÷ç ÷- -ç ÷ç ÷÷ç ÷ç- - ÷÷ççè ø÷

3. Projection

• Diagonal Matrix• Orthogonal

Matrix• Projection

Matrix


Geometric Geometric Interpretations 3Interpretations 3

N-Dimensional matrix-> N-Dimensional matrix-> Hyper-Hyper-ellipsoidellipsoid

1 1us

2 2us

3 3us

4 4us

ˆN NusOrthogonal Basis

0iif s$ =

Singular Matrix

Ellipsoid will collapse To a “thin” hyperplane

Information along “Singular” directionWill be wiped outAfter the transform

Information losing


Inverse ProblemInverse Problem

Which is inverse? Which is forward?Which is inverse? Which is forward?

InformationInformation SensitivitySensitivity

X domain Y domainTransformation

kernel

input systemd outputW

´ W=ò 1442443

The latter discovered?

The more difficult one?

Integration operator has a smoothing nature

Forward?

Inverse?


Inversion: Information Inversion: Information PerspectivePerspective

From damaged information to get all. From damaged information to get all. From limited # of projected images to recover From limited # of projected images to recover

the full objectthe full object

Projections -> Related to Projections -> Related to singular singular matrixmatrix

-- From the website of "PHOTOGRAPHY CLUBS in Singapore"

Multi-view scheme:

?


SVD-the way to SVD-the way to degenerationdegeneration

Singular Value DecompositionSingular Value Decomposition

What this meansWhat this means Good/Bad, how good/how badGood/Bad, how good/how bad

Tm n m m m n n n

Tm n m n n n n n

Tm n m m m m m n

A U V

A U V

A U V

´ ´ ´ ´

´ ´ ´ ´

´ ´ ´ ´

= S

= S

= S

A U VT

A UVT

2 miles 4 miles

Thin SVDeconomy


One step further…One step further…

SVE- Singular Value ExpansionSVE- Singular Value Expansion

Solving Solving Ax=yAx=y

Given the knowledge of SVD and noise, we Given the knowledge of SVD and noise, we master the fate of the inverse problemmaster the fate of the inverse problem

1 21 2 1 2( , ,[ , , , ] [ , , , ], )n

i

Tm n n n

Ti i

i

A u u u v v vd

u

iag

v

s s s

s

´ =

= åLL L

Principal Planes

,

i

i Ti

i

u yx v

s= å 1x A y-=


Principal Planes of a matrixPrincipal Planes of a matrix

11 1Tu vs 22 2

Tu vs

33 3Tu vs 44 4

Tu vs

A


Singular ValuesSingular Values

--Diagonal Matrix {Diagonal Matrix {ii} } Ranking of importance, Ranking of importance, Ranking of ill-posednessRanking of ill-posedness How How linearly dependentlinearly dependent for equations for equations

[A] is an orthogonal matrix-> Hyper-sphere-> Perfectly linearly independent

[A] is an ill-posed matrix-> very thin hyper-ellipsoid-> decreasing spectrum

1

2

0

0 n

s

s

s

é ùê úê úê úê úê úê úê úê úë û

O

[A] is a singular matrix-> degenerated ellipsoid-> 0 singular value


Regularization, the saverRegularization, the saver

Eliminating the bad effect of small Eliminating the bad effect of small singular values, keep major singular values, keep major informationinformation

A filter, filter out high frequency A filter, filter out high frequency noise AND high freq. useful noise AND high freq. useful informationinformationTruncated SVD(T-SVD)Truncated SVD(T-SVD)

Truncation level

Tikhonov regularization (standard)Tikhonov regularization (standard)

1

,

M is the truncation level

Mi T

iii

u yx v

s=

= å[ ]

,

12 1

solve

( )

singular values becomes:

I

T T

A x y

A A A I A

ll

l l

+

-+ -

=

= +2

2 2i

ii

ss

s l+


LL-curve: A useful tool -curve: A useful tool

2log Ax b-

2log x

Over-smoothedsolution

“best solution”Under-smoothedsolution

: Regularization parameter increasing

† See reference [1]


Can we do better?Can we do better?

Adding more linearly independent Adding more linearly independent measurementmeasurement More antenna/more receiversMore antenna/more receivers Same antenna, but more frequency Same antenna, but more frequency

pointspoints


Multiple-Frequency Multiple-Frequency ReconstructionReconstruction

Project the object with differentWavelength microwave

Low frequency component stabilize the reconstructionHigh frequency component brings up details

1 12 22 ' 1 2 " 12 2

1 1

1 12 22 ' 1 2 " 12 2

1 1

2 22 22 ' 2 2 " 22 2

2 2

222 ' 2 2

( ) ( )( ) ( )

( ) ( )

( ) ( )( ) ( )

( ) ( )

( ) ( )( ) ( )

( ) ( )

( )( )

R R

R I

I I

R I

R R

R I

I

R

E EQ S S

k k

E EQ S S

k k

E EQ S S

k k

EQ S

k

e e

e e

e e

e

w ww m w w m w

w w

w ww m w w m w

w w

w ww m w w m w

w w

ww m w

¶ ¶¶ ¶

¶ ¶¶ ¶

¶ ¶¶ ¶

¶¶

g g g g g

g g g g g

g g g g g

g g g 222 " 2 2

2 2

2 2' 2 "2 2

2 2' 2 "2 2

( )( )

( ) ( )...... ......

( ) ( )( ) ( )

( ) ( )

( ) ( )( ) ( )

( ) ( )

I

I

R M R MM M M

R M I M

I M I MM M M

R M I M

ES

k

E EQ S S

k k

E EQ S S

k k

e

e e

e e

ww m w

w w

w ww m w w m w

w w

w ww m w w m w

w w

æççççççççççççççççççççç ¶ççç ¶çççççççç ¶ ¶ççç ¶ ¶ççç¶ ¶

è ¶ ¶

g g

g g g g g

g g g g g

1

1

2

2

( )

( )

( )1'

( )'' ...

( )

( )

R

I

R

I

R M

I M

E

E

E

Q E

E

E

w

w

we

we

w

w

ö÷÷÷÷÷÷÷÷÷ æD ö÷ ÷ç÷ ÷÷ ç ÷÷ ç ÷÷ çD ÷÷ ç ÷÷ ç ÷÷ ÷ç÷ ÷ç÷ D ÷ç÷æ ö ÷÷ ç÷ç ÷D÷ ç÷ ÷ç÷ ç÷ ÷ç÷ = D÷ ç ÷÷ç ÷ ç ÷÷ç ÷ ÷ç÷ç ÷D ÷÷ç ç÷è ø÷ ÷÷ ç ÷÷ ç ÷÷ ç ÷÷ ç÷ Dç÷ ç÷ ç÷ ç÷÷ çDç÷ è øç÷ ç÷ ç÷÷÷ç ÷ç ÷ç ÷ç ÷÷çç ø÷ç ÷÷ç ÷ç ÷ç ÷ç ÷

g

÷÷÷÷÷÷÷÷÷÷÷÷


Reconstruction results I: Reconstruction results I: SimulationsSimulations

High contrast(1:6)/Large objectHigh contrast(1:6)/Large object

100.0092.8685.7178.5771.4364.2957.1450.0042.8635.7128.5721.4314.29

7.140.00

I2.52.321432.142861.964291.785711.607141.428571.251.071430.8928570.7142860.5357140.3571430.1785710

100.0092.8685.7178.5771.4364.2957.1450.0042.8635.7128.5721.4314.29

7.140.00

I2.52.321432.142861.964291.785711.607141.428571.251.071430.8928570.7142860.5357140.3571430.1785710

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

Reconstructed Permitivity usingMulti-Frequency-Point Method

0 10 20 30 40 50 60 70 80 90 1000

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Reconstructed Permitivity usingMulti-Frequency-Point Method

True object Result from single freq. recon

Result from 3 freq. recon

Cross cut of reconstruction

Background

inclusion

Large object


Reconstruction results I: Reconstruction results I: PhantomPhantom

Saline Background/Agar Phantom with iSaline Background/Agar Phantom with inclusionnclusion

100.0092.8685.7178.5771.4364.2957.1450.0042.8635.7128.5721.4314.29

7.140.00

I2.52.321432.142861.964291.785711.607141.428571.251.071430.8928570.7142860.5357140.3571430.1785710

100.0092.8685.7178.5771.4364.2957.1450.0042.8635.7128.5721.4314.29

7.140.00

I2.52.321432.142861.964291.785711.607141.428571.251.071430.8928570.7142860.5357140.3571430.1785710

Results from Single frequencyReconstructorAt 900MHz

Results from Multi-frequencyReconstructor500/700/900MHz


Time-Domain solverTime-Domain solver

A vehicle to get full-spectrum by one-A vehicle to get full-spectrum by one-runrun

0.5 1 1.5 2 2.5 3

0.1

0.2

0.3

0.4

0.5

0.6

A pulse signal is transmittedFrom source

Interacting with inhomogeneity

A distorted pulse is received At receivers

FFT

Full Spectrum Responseretrieved


AnimationsAnimationsMicrowave scattered by Microwave scattered by

objectobject

Source: Diff Gaussian Pulse

Object


ConclusionsConclusions SVD gives us a scale to measure the SVD gives us a scale to measure the

Difficulties for solving inverse problemDifficulties for solving inverse problem SVD gives us a microscope that shows the SVD gives us a microscope that shows the

very details of how each components very details of how each components affects the inversionaffects the inversion

Incorporate noise and Incorporate noise and a prioria priori information, information, SVD provide the complete information (in SVD provide the complete information (in linear sense)linear sense)

Regularization is necessary to by Regularization is necessary to by suppressing noisesuppressing noise

Difficulties can be released by adding Difficulties can be released by adding more linearly independent measurementsmore linearly independent measurements


Key IdeasKey Ideas

Decomposing a complex problem into Decomposing a complex problem into some building blocks, they are some building blocks, they are simplesimple, , invariant to inputinvariant to input, but , but addableaddable, which can , which can create certain degree of complexity, but create certain degree of complexity, but manageable.manageable.

Find out the unchanged part from Find out the unchanged part from changing, that are the rules we are looking changing, that are the rules we are looking forfor

It is impossible to get something from It is impossible to get something from nothingnothing


ReferencesReferences

Rank-Deficient and Discrete Ill-Posed Rank-Deficient and Discrete Ill-Posed ProblemsProblems, Per Christian Hansen, SIAM 1, Per Christian Hansen, SIAM 1998998

Regularization Methods for Ill-Posed PRegularization Methods for Ill-Posed Problemsroblems, Morozov, Morozov

Matrix ComputationsMatrix Computations, G. Golub, 1989, G. Golub, 1989 Linear Algebra and it’s applicationsLinear Algebra and it’s applications, ,

G. StrangG. Strang


Questions?

,

i

i Ti

i

u y yx v

sd+

= å

A UVT


Eigen-values vs. Singular valueEigen-values vs. Singular value

-4 -3 -2 -1 0 1 2 3

-3

-2

-1

0

1

2

Eigen-vectorsDirections:

Invariant of rotations

SingularSingular-vectorsDirections:

Maximum span


Outline detailsOutline details Numerical Methods and linearizationNumerical Methods and linearization What is MWhat is MAATRTRIIX? Geometric interpretationsX? Geometric interpretations Inverse ProblemInverse Problem Singular value decomposition and Singular value decomposition and

implementations in inverse problemimplementations in inverse problem Solving inverse problemSolving inverse problem Improve the solution, can we?Improve the solution, can we? Multiple-Frequency Reconstruction & Multiple-Frequency Reconstruction &

Time-Domain ReconstructionTime-Domain Reconstruction ConclusionsConclusions


Right Singular VectorsRight Singular Vectors Eigen-modes for solutionEigen-modes for solution

Building blocks for solutions, Building blocks for solutions, if the solution is a image, vif the solution is a image, vi i are components of the imagare components of the imag

ee Less variant respect to different y=> a property of the syLess variant respect to different y=> a property of the sy

stemstem

1 2 3 4 5 6

1

0.5

0.5

1

1 1 2 3 4 5 60.25

0.250.5

0.751

1.25


Left Singular VectorsLeft Singular Vectors

A group of “basic RHS’s”-> source mA group of “basic RHS’s”-> source modeode

Arbitrary RHS Arbitrary RHS y y can be decomposed witcan be decomposed with this basish this basis


NoiseNoise

Always Noise Always Noise Small perturbation for RHSSmall perturbation for RHS Ax=yAx=y

yy==yy++yy† Modified from coca-cola’s patch

,

i

i Ti

i

u y yx v

sd+

= å

research in progress presentation 2003 look closer to inverse problem qianqian fang thanks to : paul...

Documents