EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

1

Image Denoising: a Statistical Approach

• Linear estimation theory summary• Spatial domain denoising techniques

• Conventional Wiener filtering• Spatially adaptive Wiener filtering

• Wavelet domain denoising • Wavelet thresholding: hard vs. soft • Wavelet-domain adaptive Wiener filtering

• Experimental Results• Why transform helps?• Why spatial adaptation helps?

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

2

Denoising Problem

aYX ˆ

WXY Noisy measurements

N(0,σw2)

MMSE estimator

Wiener’s idea To simplify estimation by computing the bestestimator that is a linear scaling of Y

Difficulty: we need to know conditional pdf

]|[ˆ YXEX

N(0,σx2)

Orthogonality Principle

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

3

aYX ˆ

0})ˆ{( YXXE

A linear estimator X of a random variable X^

Minimizes E{(X-X)2} if and only if ^

Geometric Interpretation

X

Y

X-X̂

X̂

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

4

Linear MMSE Estimation

YXwx

x22

2

ˆ

),0(~ 2xNX For Gaussian signal

The optimal LMMSE estimation is given by

22

22

wx

xwMMSE

And it achieves

Note: it can be shown such linear estimator is indeedE[X|Y] for Gaussian signal

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

5

What if Signal Variance is Unknown?

222ˆ wx y

Maximum-likelihood estimation of 2x is given by

Since variance is nonnegative, we modify it

],0max[ˆ 222wx y

When multiple observations yi’s are available, we have

]1

,0max[ˆ 2

1

22w

N

iix y

N

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

6

Image Denoising

Theory of linear estimation Spatial domain denoising techniques

• Conventional Wiener filtering

• Spatially adaptive Wiener filtering Wavelet domain denoising

• Wavelet thresholding: hard vs. soft

• Wavelet-domain adaptive Wiener filtering Experimental Results

Why transform helps?Why spatial adaptation helps?

Conventional Wiener Filtering• Basic assumption: image source is modeled by a stationary

Gaussian process• Signal variance is estimated from the noisy observation data• Can be interpreted as a linear frequency weighting

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

7

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

8

Linear Frequency Weighting

),(),(

),(),(

),,(),(),(ˆ

2121

2121

212121

wwSwwS

wwSwwH

wwYwwHwwX

WX

X

22

2

,ˆwx

xaaYX

FT

Power spectrum |X|2

Image Example

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

9

Noisy, =50 (MSE=2500) denoised (MSE=1130)

Image Example (Con’d)

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

10

Noisy, =10 (MSE=100) denoised (MSE=437)

Conclusions from the Experiments

• Why did it Fail? • Nonstationary• NonGaussian• Poor modeling

• How to improve?• Achieve spatial adaptation• Use linear transform• Putting them together

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

11

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

12

Spatially Adaptive Wiener Filtering

• Basic assumption: image source is modeled by a nonstationary Gaussian process

• Signal variance is locally estimated from the windowed noisy observation data

]1

,0max[ˆ 2

1

22w

N

iix y

N

T

T

N=T2

Recall

Image Example

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

13

Noisy, =10 (MSE=100) denoised (T=3,MSE=56)

Image Example (Con’d)

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

14

Noisy, =50 (MSE=2500) denoised (MSE=354)

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

15

Image Denoising

Theory of linear estimation Spatial domain denoising techniques

• Conventional Wiener filtering

• Spatially adaptive Wiener filtering Wavelet domain denoising

• Wavelet thresholding: hard vs. soft

• Wavelet-domain adaptive Wiener filtering Experimental Results

Why transform helps? Why spatial adaptation helps?

From Scalar to Vector Case

1

022

222}||ˆ{||

N

m wm

wmXXE

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

16

1

022

2

][][ˆN

m wm

m mYnX

Suppose X is a Gaussian process whose covariance matrix is adiagonalized matrix RX=diag{ηm}(m=0,…,N-1), the linear MMSEestimator is given by

(A)

and the minimal MSE is given by

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

17

Decorrelating

XTR*

Q: What if X={X[0],…,X[N-1]} is correlated (i.e., Rx is not diagonalized)?

A: We need to transform X into a set of uncorrelated basis and then apply the above result.

The celebrated Karhunen-Loeve Transform does this job!

Diagonal matrix

XX T*'

Karhunen-Loeve Transform

Transform-Domain Denoising

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

18

ForwardTransform

InverseTransform

Denoisingoperation

e.g.,KLTDCTWT

e.g.,Linear Wiener filteringNonlinear Thresholding

Noisysignal

denoisedsignal

The performance of such transform-domain denoising is determinedby how well the assumed probability model in the transform domainmatches the true statistics of source signal (optimality can only beestablished for the Gaussian source so far).

One-Minute Tour of Wavelets

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

19

G0

G1

)(ˆ nxx(n)

H0

H1

y0(n)

y1(n)

x(n)

H0

H1

22 G0

22 G1

)(ˆ nx

s(n)

d(n)

complete expansion (with decimation)

overcomplete expansion (without decimation)

TceTce

-1

Toe Toe-1

Why Wavelet Denoising?• We need to distinguish spatially-localized events (edges) from

noise components

• More about noise components

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

20

Wavelet is such a basis because exceptional event generates identifiable exceptional coefficients due to its good localization property in both spatial and frequency domain

As long as it does not generate exceptions

Additive White Gaussian Noise is just one of them

Wavelet Thresholding

TnY

TnYTnY

TnYTnY

nX

|][|0

][][

][][

][~

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

21

otherwise

TnYifnYnX

0

|][|][][

~

DWT IWTThresholdingY X

~

Hard thresholding

Soft thresholding

Noisysignal

denoisedsignal

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

22

Choice of Threshold

NT elog2

][][

][][

~22

2

nYn

nnX

Donoho and Johnstone’1994

Gives denoising performance close to the “ideal weighting”

Reference: S. Mallat, “A Wavelet Tour of Signal Processing”, Section 10.2 (pp. 435-453)

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

23

Soft vs. Hard thresholding

|][||][~

|1][

][],[][

~22

2

nYnXn

nanaYnX

● It can be also viewed as a computationally efficient approximationof ideal weighting

|][||][~

|][,][][~

nYnXTnYTnYnX soft

ideal

● Soft-thresholding has the same upper bound as hard-thresholding asymptotically and larger error than hard-thresholding at the same threshold value, but perceptually it works better.

● Shrinking the amplitude by T guarantees with a high probability that.

|][||][~

| nXnX

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

24

Denoising Example

noisy image(σ2=100)

Wiener-filtering (ISNR=2.48dB)

Wavelet-thresholding (ISNR=2.98dB)

2

2

10||

~||

||||log10

XX

YXISNR

X: original, Y: noisy, X: denoised~

Improved SNR

What is Wrong with Wavelets?

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

25

0 1 N-1… …

x(n)

H1

T

-T

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

26

Translation Invariance (TI) Denoising

Toe Toe-1Thresholding

Tce Tce-1Thresholding

Tce Tce-1Thresholding

z

+

x(n) )(ˆ nx

x(n) )(ˆ1 nx

)(ˆ2 nx

2

)(ˆ)(ˆ)(ˆ 21 nxnxnx

Implementation based on overcomplete expansion

Implementation based on complete expansion

z-1

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

27

2D Extension

Noisy image

Tce Tce-1ThresholdingWD =

shift(mK,nK) WD shift(-mK,-nK)

shift(m1,n1) WD shift(-m1,-n1)

Avg

denoised image

(mk,nk): a pair of integers, k=1-K (K: redundancy ratio)

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

28

Example

Wavelet-thresholding (ISNR=2.98dB)

Translation-Invariant thresholding (ISNR=3.51dB)

• Challenges with wavelet thresholding• Determination of a global optimal threshold• Spatially adjusting threshold based on local statistics

• How to go beyond thresholding?• We need an accurate modeling of wavelet coefficients –

nonlinear thresholding is a computationally efficient yet suboptimal solution

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

29

Go Beyond Thresholding

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

30

Spatially Adaptive Wiener Filtering in Wavelet Domain• Wavelet high-band coefficients are modeled by a Gaussian

random variable with zero mean and spatially varying variance• Apply Wiener filtering to wavelet coefficients, i.e.,

][][

][][

~22

2

nYn

nnX

estimated in the same wayas spatial-domain (Slide 15)

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

31

Practical Implementation

YXwx

x22

2

ˆ

ˆˆ

]1

,0max[ˆ 2

1

22w

N

iix y

N

T

T

N=T2

Recall

Conceptually very similar to its counterpart in the spatial domain

In demo3.zip, you can find a C-coded example (de_noise.c)

(ML estimation of signal variance)

Example

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

32

Translation-Invariant thresholding (ISNR=3.51dB)

Spatially-adaptive wiener filtering (ISNR=4.53dB)

Further Improvements*• Gaussian scalar mixture (GSM) based denoising (Portilla et al.’

2003)• Instead of estimating the variance, it explicitly addresses the

issue of uncertainty with variance estimation• Hidden Markov Model (HMM) based denoising (Romberg et

al.’ 2001)• Build a HMM for wavelet high-band coefficients (refer to the

posted paper)

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

33

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

34

Gaussian Scalar Mixture (GSM)*

Model definition: u~N(0,1)

Noisy observation model

Gaussian pdf

scale (variance) parameter

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

35

Basic Idea*In spatially adaptive Wiener filtering, we estimate the variancefrom the data of a local window. The uncertainty with such varianceestimation is ignored. In GSM model, such uncertainty is addressedthrough the scalar z (it decides the variance of GSM). Instead of using a single z (estimated variance), we build a probabilitymodel over z, i.e., E{x|y}=Ez{E{x|y,z}}

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

36

Posterior Distribution*

where

Due to

is so-called Jeffery’s prior

Question: What is E{xc|y,z}?

Bayesian formula

(proof left as exercise)

GSM Denoising Algorithm*

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

37http://decsai.ugr.es/~javier/denoise/index.htmlMATLAB codes available at:

Image Examples

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

38

Noisy, =50 (MSE=2500) denoised (MSE=201)

Image Examples (Con’d)

EE

56

5 A

dva

nce

d I

ma

ge

P

roce

ssin

g C

op

yrig

ht

Xin

Li

20

09

-20

12

39

Noisy, =10 (MSE=100) denoised (MSE=31.7)