enee631 digital image processing (spring'04) image restoration spring ’04 instructor: min wu...

ENEE631 Digital Image Processing (Spring'04)

Image RestorationImage Restoration

Spring ’04 Instructor: Min Wu

ECE Department, Univ. of Maryland, College Park

www.ajconline.umd.edu (select ENEE631 S’04) [email protected]

Based on ENEE631 Based on ENEE631 Spring’04Spring’04Section 7Section 7

ENEE631 Digital Image Processing (Spring'04) Lec8 – Image Restoration [4]

From Matlab ImageToolbox Documentation pp12-4

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Imperfectness in Image CapturingImperfectness in Image Capturing

Blurring ~ linear spatial-invariant filter model w/ additive noise

Impulse response h(n1, n2) & H(1, 2)

– Point Spread Function (PSF) ~ positive I/O

– [No blur] h(n1, n2) = (n1, n2)

– [Linear translational motion blur] local average along motion direction

– [Uniform out-of-focus blur] local average in a circular neighborhood

– Atomspheric turbulence blur, etc.

Hu(n1, n2 ) v(n1, n2 )

N(n1, n2 )

otherwise 0

tan and

x if L

1

),;,( yx

222

Ly

Lyxh

otherwise 0

x if R

1);,(

2222

RyRyxh

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Fourier Transform of PSF for Common DistortionsFourier Transform of PSF for Common Distortions

From Bovik’s Handbook Sec.3.5 Fig.2&3 UMCP ENEE631 Slides (created by M.Wu © 2001)


Undo Linear Spatial-Invariant DistortionUndo Linear Spatial-Invariant Distortion

Assume noiseless and PSF of distortion h(n1, n2) is known

Restoration by Deconvolution / Inverse-Filtering– Often used for deblurring– Want to find g(n1, n2) satisfies h(n1, n2) g(n1, n2) = (n1, n2)

h(k1, k2) g(n1 -k1, n2-k2) = (n1, n2) for all n1, n2

– Easy to solve in spectrum domain Convolution Multiplication H(1, 2) G(1, 2) = 1 Interpretation: choose G to compensate distortions from H

),(

1),(

2121

H

G

Hu(n1, n2) v(n1, n2)

(n1, n2)=0

Gu’(n1, n2)w(n1, n2)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Problems With Inverse Filtering Under NoiseProblems With Inverse Filtering Under Noise

Zeros in H(1, 2)

– Interpretation: distortion by H removes all info. in those freq.– Inverse filter tries to “compensate” by assigning infinite gains

– Amplifies noise: W(1, 2) = H (1, 2) U (1, 2) U’ (1, 2) = (W+N) / H N/H if W=0

Solutions?

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Examples of Inverse & Pseudo-inverse FilteringExamples of Inverse & Pseudo-inverse Filtering

From Jain Fig.8.10

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Problems With Inverse Filtering Under NoiseProblems With Inverse Filtering Under Noise

Zeros in H(1, 2)

– Interpretation: distortion by H removes all info. in those freq.– Inverse filter tries to “compensate” by assigning infinite gains

– Amplifies noise: W(1, 2) = H (1, 2) U (1, 2) U’ (1, 2) = (W+N) / H N/H if W=0

Solutions ~ Pseudo-inverse Filtering

– Assign zero gain for G at spectrum nulls of H– Interpretation: not bother to make impossible compensations

|),(H| if ,0

|),(H| if ,),(

1

),(

21

212121 HG

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Handling Spectrum Nulls Via High-Freq Cut-offHandling Spectrum Nulls Via High-Freq Cut-off

Limit the restoration to lower frequency components to avoid amplifying noise at spectrum nulls

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 5)


Handling Noise in DeconvolutionHandling Noise in Deconvolution Inverse filtering is sensitive to noise

– Does not explicitly modeling and handling noise

Try to balance between deblurring vs. noise suppression– Minimize MSE between the original and restored

e = E{ [ u(n1, n2) – u’(n1, n2) ] 2 } where u’(n1, n2) is a func. of {v(m1, m2) }

– Best estimate is conditional mean E[ u(n1 , n2) | all v(m1 , m2) ] usually difficult to solve for general restoration (need conditional

probability distribution, and estimation is nonlinear in general)

Get the best linear estimate instead Wiener filtering– Consider the (desired) image and noise as random fields– Produce a linear estimate from the observed image to minimize MSE

Hu(n1, n2) v(n1, n2)

(n1, n2)

Gu’(n1, n2)w(n1, n2)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Wiener FilteringWiener Filtering Get the best linear estimate minimizing MSE

– Assume spatial-invariant filter u’(n1, n2) = g (n1, n2) v(n1, n2)

– Assume wide-sense stationarity for original signal and noise – Assume noise is zero-mean and uncorrelated with original signal

Solutions

– Bring into orthogonal condition E{ [ u(n1, n2) – u’(n1, n2) ] v(m1, m2) }=0

– Represent in correlation functions: Ruv(k,l) = g(k,l) Rvv(k,l)

– Take DFT to get representation in power spectrum density

),(

),(),(

21

2121

vv

uv

S

SG

),(),(),(

),(),(),(),( :Note

2121*

21

2121

2

2121

uuuv

uuvv

SHS

SSHS

),(),(),(

),(),(

2121

2

21

2121*

SSH

SH

uu

uu

U

MC

P E

NE

E6

31

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

1)


More on Wiener FilteringMore on Wiener Filtering

Balancing between two jobsfor deblurring noisy image– HPF filter for de-blurring

(undo H distortion)– LPF for suppressing noise

Noiseless case ~ S = 0 (inverse filter)– Wiener filter becomes pseudo-inverse filter for S 0

No-blur case ~ H = 1 (Wiener Smoothing Filter)– Zero-phase filter to attenuate noise according to SNR at each freq.

0|),(H| if ,0

0|),(H| if ,),(

1

/||),(

21

212102

*

021

HSSH

HG S

uvS

uuSH

SH

G

*

wiener21

1),(

1),(

),(

),(),(

),(),(

21

21

2121

211wiener21

SNR

SNR

uu

uuH S

S

SS

SG

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


ComparisonsComparisons

From Jain Fig.8.11

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Example: Wiener Filtering vs. Inverse FilteringExample: Wiener Filtering vs. Inverse Filtering

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)



Example (2):Example (2): Wiener Filtering Wiener Filtering vs. vs. Inverse FilteringInverse Filtering

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)



Review: Handling Noise in DeconvolutionReview: Handling Noise in Deconvolution Inverse filtering is sensitive to noise

– Does not explicitly modeling and handling noise

Try to balance between deblurring vs. noise suppression

– Consider the (desired) image and noise as random fields– Minimize MSE between the original and restored

e = E{ [ u(n1, n2) – u’(n1, n2) ] 2 } where u’(n1, n2) is a func. of {v(m1, m2) }

– Best estimate is conditional mean E[ u(n1 , n2) | all v(m1 , m2) ]

Get the best linear estimate Wiener filtering

– Produce a linear estimate from the observed image to minimize MSE

Hu(n1, n2) v(n1, n2)

(n1, n2)

Gu’(n1, n2)w(n1, n2)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Wiener Filter: From Theory to PracticeWiener Filter: From Theory to Practice

Recall: assumed p.s.d. of image & noise random fields and freq. response of distortion filter are known

Why make the assumptions?

Are these reasonable assumptions?

What do they imply in our implementation of Wiener filter?

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Wiener Filter: Issues to Be AddressedWiener Filter: Issues to Be Addressed

Wiener filter’s size– Theoretically has infinite impulse response ~ require large-size DFTs– Impose filter size constraint: find the best FIR that minimizes MSE

Need to estimate power spectrum density of orig. signal– Estimate p.s.d. of blurred image v and compensate variance due to

noise– Estimate p.s.d. from a set of representative images similar to the images

to be restored– Or use statistical model for the orig. image and estimate parameters

– Constrained least square filter ~ see Jain’s Sec.8.8 & Gonzalez Sec.5.9 Optimize smoothness in restored image (least-square of the

rough transitions) Constrain differences between blurred image and blurred

version of reconstructed image Estimate the restoration filter w/o the need of estimating p.s.d.

Unknown distortion H ~ Blind Deconvolution

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Basic Ideas of Blind DeconvolutionBasic Ideas of Blind Deconvolution

Three ways to estimate H: observation, experimentation, math. modeling

Estimate H via spectrum’s zero patterns

– Two major classes of blur (motion blur and out-of-focus)– H has nulls related to the type and the parameters of the blur

Maximum-Likelihood blur estimation

– Each set of image model and blur parameters gives a “typical” blurred output; Probability comes into picture because of the existence of noise

– Given the observation of blurred image, try to find the set of parameters that is most likely to produce that blurred output

Iteration ~ Expectation-Maximization approach (EM) Given estimated parameters, restore image via Wiener filtering Examine restored image and refine parameter estimation Get local optimums

To explore more: Bovik’s Handbook Sec.3.5 (subsection-4, pp136)

“Blind Image Deconvolution” by Kundur et al, IEEE Sig. Proc. Magazine, vol.13, 1996

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Filtering Through Transform Domain OperationFiltering Through Transform Domain Operation

E.g.1 Realize Wiener in DFT domain

E.g.2 Use zonal mask in transform domain

– Realize “ideal” LPF/BPF/HPF– Computation complexity for transform could be high for large image

From Jain Fig.7.31&7.32

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

enee631 digital image processing (spring'04) image restoration spring ’04 instructor: min wu...

Documents

humcp enee631 slides

image capturingblurring

image restorationassignment

n2 gn1

n2 hk1

w n h nh

pseudoinverse filteringassign

psf of distortion hn1