SIAM Annual Meeting 2005 1

An Iterative, Projection-Based Algorithm for General Form Tikhonov Regularization

Misha Kilmer, Tufts University Per Christian Hansen, Technical University of

Denmark Malena Español, Tufts University

SIAM Annual Meeting 2005 2

Outline

Problem Background Algorithm Numerical Examples Conclusion and future work

SIAM Annual Meeting 2005 3

Discrete Ill-Posed problem

holds condition Picard Discrete

noise (white)unknown is

gap without aluessingular v Decaying

:Properties

e

matrix dconditione-ill large, a is where

,

model theand ,given , Find

nm

truetrue

true

RA

ebbAx

bAx

0 10 20 30 4010

-15

10-10

10-5

100

105

i

i|uiTbtrue|

SIAM Annual Meeting 2005 4

Need for regularization

n

ii

eui

butrue

n

i

Tiii

T

i

Ti

i

trueTix

uVUA

1

1

bygiven issolution exact The

.SVD thebe Let

0 10 20 30 4010

-15

10-10

10-5

100

105

i| uiTbtrue|

| uiTe|

SIAM Annual Meeting 2005 5

Tikhonov Regularization

. 0

min

or

min

:Problem dRegularize Tikhonov The

2

2

2

2

22

2x

bx

L

A

LxbAx

x

.0parameter tion regularizaon depends ngConditioni

SIAM Annual Meeting 2005 6

Tikhonov Method: choosing

L-curve (Lawson-Hansen)

2 log bAx

2 log Lx

101.84

101.86

101.88

101.9

101.92

101.94

102.2

102.3

102.4

102.5

SIAM Annual Meeting 2005 7

Tikhonov Regularization

employed. LSQR) (CGLS, solvers Iterative

operator. derivative discrete (scaled) aoften is L

inverse.-pseudo weighted-A theis where

, min

: If

. Tikhonov, form-standardIn

)()(2

2

22

2y

A

AA

L

yLxybyAL

IL

IL

SIAM Annual Meeting 2005 8

Bidiagonalization

'82 Saunders and Paige '96, Zha'65,Kahan -Golub

explicitlyion factorizat QR forming of need No

. of form bidiagonalupper an

and of form bidiagonallower a Obtaining

usly.simultaneo computed becan and ofization Bidiagonal

. ,

ionfactorizat QR heConsider t

L

A

LA

LTLA

TA

L

A

Q

Q

QQ

IQQQQRQ

Q

L

A

SIAM Annual Meeting 2005 9

Relating A,L

.ˆ ,

such that matrix

invertible and , bidiagonalupper an

, bidiagonallower a ,ˆmatrix

unitary ,matrix unitary exist There

. andLet :Theorem

11

ZBULUBZA

Z

nnRB

RBUpp

Umm

RL RA

np

nm

npnm

SIAM Annual Meeting 2005 10

Projected Problem

.on depend and Only

.0

min

problem projected theofsolution theis

,solution its with so )(But

.,,span with ,0

min

withproblem Tikhonov theReplace

2

11

111

1

2

kk

k

k

y

k

kkkk

kkZx

yx

ey

B

B

y

yZxbeU

zzZb

xL

A

k

SIAM Annual Meeting 2005 11

Choosing

2

2

2

2

2

211

2

2 and

.each for defined as ,for :Theorem

curve-L

kkkkkk

kk

yBLxeyBbAx

yx

101.84

101.86

101.88

101.9

101.92

101.94

102.2

102.3

102.4

102.5

SIAM Annual Meeting 2005 12

Iterative Method

e.convergencfor check 4.

norms, constraint and residual update 3.

,1 stepat computed quantities from

each for desired) if ,( compute 2.

ization,bidiagonaljoint of stepth thecompute 1.

,2 ,1For

k

xy

k

k

kk

SIAM Annual Meeting 2005 13

Regularizing algorithm

},{ of GSVD theofion approximat

an get we},{ ofion decomposit CS with the

behavior. decayingsimilar have aluessingular v the

: from properties ngconditioni inherites

. with , min

problem the toapplied LSQR

similar to worksalgorithm that theshowcan We

12

2

LA

BB

AQ

yRxbyQ

kk

A

Ay

SIAM Annual Meeting 2005 14

TGSVD of the Projected Problem

klk

kreg

l

ii

i

TiBk

l

kkB

B

k

k

kkk

k

k

y

yZxheu

y

BBHM

N

U

U

B

B

yZxe

yB

B

k

k

k

then and )()(

is problem projected theosolution t TGSVD The

}. ,{ of GSVD thebe 0

0 Let

. with ,0

min

:Problem Projected theRecall

1

11

1

2

11

SIAM Annual Meeting 2005 15

Numerical Examples

1 that so scaled ,

.or 1 dim of op. derivative where,

).01.0,0( with and rice""

)10()( blur(),

Hansen.by Toolstion Regulariza Matlab, ,deblurring Image 2D

22

11

1

9

LALA

LLIL

LIL

bNeeAxbx

OAkA

truetrue

SIAM Annual Meeting 2005 16

Original and blurred images

Original Blurred + Error

SIAM Annual Meeting 2005 17

Restoration L =derivative operator

Original Restored

SIAM Annual Meeting 2005 18

Restoration L= Laplacian

Original Restored

SIAM Annual Meeting 2005 19

Comparing with L=I and L=Laplacian

Original Blurred + Error

L=Derivative Op. L=Laplacian Op. L=Identity

SIAM Annual Meeting 2005 20

Conclusion and Future work

oningPreconditi

accuracyiteration Inner

: workFuture

priori aknown not

orecompute/st todifficult is

: whenusefulmation transfor

form-standard avoids that algorithm Iterative

AL