3.7 Adaptive filtering

Joonas VanninenAntonio Palomino Alarcos

Adaptive filtering

• Linear filtering does not take into account the local features of the image– Causes for example blurring of the edges

• An improvement: change the filter parameters according to the local statistics– Change the shape and the size of the

neighborhood– Suppress the filtering if there are features that we

want to preserve in the neighborhood

Filters

• The local LMMSE filter• The noise-updating repeated Wiener filter• The adaptive 2D LMS filter• The adaptive rectangular window LMS filter• The adaptive-neighborhood filter

The local LMMSE filter

• Local Linear Minimum Mean Squared Error filter, also known as the adaptive Wiener filter

• Assumes that the image is corrupted by additive noise• Minimizes the local mean squared error by applying a linear

operator to each pixel in the image• The local mean and variance are estimated from a rectangular

window around the pixel

LLMMSE estimation

),(),(),(

),(),(),(),(

~2

22

nmnmgnm

nmnmnmnmf g

g

gg

•g is the corrupted image•μg is the local mean•σg

2 is the local variance•ση

2 is the variance•Constant if noise is signal-independent•Varies if noise is signal-dependent•In the latter case it should be estimated locally with the knowledge of the type of the noise

MSE=1595

MSE=360

MSE=324

Interpretation of the equation

• If the area processed is uniform, the second term is small– Estimate is near the local mean of the noisy image– Noise is reduced

• If there is a edge in the processing window, the variance of the image is larger than the variance of the noise– Estimate is closer to the actual noisy value– Edge doesn’t get blurred

),(),(),(

),(),(),(),(

~2

22

nmnmgnm

nmnmnmnmf g

g

gg

A refined version

• If the local variance is high, it is assumed that there is an edge in the processing window

• The edge is assumed to be straight in a small window• The direction of the edge is calculated

– Gradient operator– 8 different directions

• The processing window is divided to two uniform sub-areas over the edge

• The statistics in the sub-area that holds processed pixel are used

• Noise is reduced near the edges without causing blurring

Sub-areas

The noise-updating repeated Wiener Filter

• An iterative application of the LLMMSE filter• The variance of noise used in the LLMMSE estimate formula

is updated in each iteration

Q

Qqqp

P

Ppg

gg

new

qnpmnm

nm

QP

nmnm

nm

QPnm

nmnm

),(),(

),(

)12)(12(

1

),(),(

),(

)12)(12(

1

),(

),(1),(

2

)0,0(),(

2

2

2

2

2

2

2

2

22

• The noise is reduced even near the edges

• To avoid blurring, a smaller window size may be choosen for each iteration

The Adaptative 2D LMS Filter

Why?

• Wiener filter has to be used with stationary and statistically independent signal and noise models

• Adaptative 2D LMS algorithm circunvent this problem– It takes into account the nonstationarity of the

given image

2D LMS Algorithm

• Same concept: fixed-window Wiener filter• New idea: filter coefficients vary depending

upon the image characteristics– Algorithm based on the steepest descent method– It tracks the variations in the local statistics

adapting to the changes in the images

1

0

1

0

),(),(),(~ P

p

P

ql qmpmgqpwnmf

• Estimate of the original pixel value f(m,n):

– Convolution

– wl(p,q) Ξ causal FIR filter

– g(m,n) Ξ noise-corrupted input image

Updating for the filter coefficients

• New coefficients determined by minimazing MSE between f(m,n) and the estimation

• Steepest descent method:

– μ controls the rate of convergence and filter stability.– Error is estimated using an approximation to the original signal d(m,n)

– d(m,n) obtained by decorrelation from the input image g(m,n)• 2D delay operator of (1,1) samples, use the previous pixel as an estimate

),(~

),( nmfnmdel

21 ),(),( lll eqpwqpw

Advantages of the Algorithm

• It does not require a priori information about:– Image– Noise statistics– Correlation properties

• It does not require averaging, differentiation and matrix operations

Image corrupted with additive Gaussian noise

50 100 150 200 250

50

100

150

200

250

Original image

50 100 150 200 250

50

100

150

200

250

Image filtered with adaptive 2D LMS filter

50 100 150 200 250

50

100

150

200

250

MSE=1590

MSE=395

The Adaptative Rectangular Window LMS Filter

ARW LMS Algorithm

• Same concept: 2D LMS filter based on standard Wiener filter

• New Idea: use of an adaptative-sized rectangular window

• Additional assumption: image processes have zero mean

Implementation

• Taking into account that now we have zero mean noise, following estimate is derived:

• σf2 is the variance of the original image (estimated)

• Idea: globally nonstationary process can be considered locally stationary and ergodic over small regions

• Target: to identify the size of a stationary square region for each pixel in the image– Sample statistics can approximate the a posteriori pareameters

needed

Updating the window size

• Large window It may include pixels form other ensembles

• Small window The statistics needed are poorly estimated

• ARW lengths (Lr and Lc) are varied using a signal activity parameter:

• A similar signal activity parameter is defined in the colum direction

Nr

Nrp

Nc

Ncqrr qnpmg

LrLcnmS 22~),(

1),(

Updating the window size (II)

TSNN

TSNN

,1

,1

• If S is large N is decremented• If S is small N is incremented

• In order to make this decision, S is compared to a threshold T as follows:

• Threshold is defined as:

• κ controls the rate at which T changes

LT

2

The adaptive-neighborhood filter

• Uses a variable-size, variable-shape neighborhood determined individually for every pixel

• Neighborhood contains only spatially connected pixels that are similar to the seed

• This way the estimated statistics are likely to be closer to the true statistics of the signal

• Adaptive neighbourhoods should not mix the pixels of an object with the pixels of the background → no blurring of the edges

Region growing• The absolute difference between each of the 8-

neighbor pixels of the region and the seed is calculated

• If the value is under a threshold T, pixel is included in the region

• The process is iterated until there are no new pixels• Additionally a background region is grown by

expanding the foreground boundary by a prespecified number of pixels

• The regions are grown for each pixel in the image

•An example of region growing

•The same neighborhood can be used for every pixel in the area with the same gray-value

Adaptive-neighborhood mean and median filters

• The mean and median values of adaptive neighborhoods can be used to filter noise

• They provide a larger population to compute local statistics than fixed 3 x 3 or 5 x 5 windows

• Edge distortion is pervented because the neighbourhood does not cross object boundaries

Adaptive-neighborhood noise subtraction (ANNS)

• Used for removing additive signal-independent noise• Estimates the noise value in the seed pixel with an adaptive

neighborhood and subtracts it to obtain an estimate of the original

),(),(),(),(

),(1),(),(

~22

2

nmnmgnmnm

nmnmnmf g

fg

•The local statistics are estimated as in the ARW-LMS –filter

Implementation

• A maximum foreground limit of Q pixels is set.• At first, the tolerance is set to the full dynamic range• An estimate of the variance of the uncorrupted

signal is calculated from the neighborhood• It is compared to the noise variance, and if σf

2 > 2ση2,

it is assumed that the neighbourhood contains a significant feature

• If so, the tolerance is modified by T = 2 σf2

Implementation (II)

• If the foreground has only one pixel, the neighborhood is enlarged to include the backround around it (3 x 3)

• Flat regions → σf2 << 2ση

2 → foreground will grow to Q

• Busy regions → σf2 >> 2ση

2 → tolerance will be reduced

Conclusion• Use of local statistics in an adaptive filter is a powerful

approach to remove noise– while retaining the edges in the images with minimal

distortion• Some of the implicit assumptions may not apply well to the

image or noise processes– It is common to try several previously established

techniques• It is quite difficult to compare the results provided by

different filters– Generally MSE or RMS error is used– In real applications, it is important to obtain an

assessment of the results by a specialist

Quote

“You can only cure retail but you can prevent wholesale”

Brock Chisholm

…any question?