computer vision group digital image filters 26 th november 2012

Computer Vision Group

Digital Image Filters26th November 2012


Outline

Definitions

Linear, shift invariant filters: convolution.

Examples of convolutional filters• Smoothing Filters (denoising)• Differentiating Filters (edge detector)

Examples of Nonlinear Filtering


Systems

A System H is considered as a black box that processes some input signal (f) and gives the output (Hf)

In our case, f is a digital image or a 1-D digital signal, but in principle could be analogic and n-D signal as well as a scalar


Linearity

A system is said to be linear i.i.f.

and this hold for all and for aribrary signals

it is the classical definition of linearity

)()()()( tHgtHftgtfH


Signal and System - Property

A system is said to be time (or shift) – invariant i.i.f.

Systems that are Linear and Translation Invariant are represented by a convolutoin.• LTI systems are characterized entirely by a single

function called the system's impulse response.• The impulse response corresponds to the filter

associated to the system.

))(()( 00 ttHfttfH


Convolution - LTI System

Let us consider a signal and a filter • Their convolution is a signal .• For continuous-domain 1D signals and filters

i.e.,

• For discrete signals and filters


Convolution LIT Systems on Images

are discrete 2D signals

h7 h8 h9

h4 h5 h6

h1 h2 h3

h9 h8 h7

h6 h5 h4

h3 h2 h1

h1 h2 h3

h4 h5 h6

h7 h8 h9

flipX

y1 y2 y3

y4 y5 y6

y7 y8 y9

=

flipY

h9 h8 h7

h6 h5 h4

h3 h2 h1

*

Point-wise product

h9y1 h8y2 h7y3

h6y4 h5y5 h4y6

h3y7 h2y8 h1y9

sum

z5 = h9y1+ h8y2 + h7y3 + h6y4 + h5y5 + h5y6 + h3y7 + h2y8 + h1y9


Linear Filters for Digital Images

Linear Filters are particular systems where the output at each pixel (sample) is given by a linear combination of neighboring pixels (samples)

10 30 10

20 11 20

11 9 1

* * *

* 5.7 *

* * *

H

Original Signal Filtered Output


Filters or Kernels

The coefficients used in the linear combination are given by the kernel

1 3 0

2 10 2

4 1 1

Image

1 0 -1

1 0.1 -1

1 0 -1

Kernel

= - 3

Filter Output

*


A well-known Test Image - Lena


Trivial example

0 0 0

0 1 0

0 0 0

*


0 0 0

1 0 0

0 0 0

*

Transalation


Linear Filtering

1 1 1

1 1 1

1 1 1

9

1* ?

-local averaging


The original Lena


Filtered Lena


Linear Filtering

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

25

1*

- local averages


What about normalization?

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

25

2*


… convolution is linear


…what about

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

25

2 *


Convolution Properties

It is a linear System; it enjoys every LTI properties

Is it Commutative ?

but it depends on the patting used. In continuous domain it holds as well as on periodic signals

It is also associative

and dissociative

)()()()( tHgtHftgtfH


2D Gaussian Filter

2

22

2 2exp

2

1,

yx

yxG

2

22

2 2

11exp

2

1,

kjki

jiH

array 1212 is , where kkjiH

Continuous Function

Discrete Function


Linear Filtering(Gaussian Filter)-weighted local averages-

*

12

34

56

7

12

34

56

7

0

0.05

0.1

0.15

0.2


Linear Filtering(Gaussian Filter)-weighted local averages-


Gaussian Vs Average

Gaussian Smoothing Support 7x7

Smoothing by Averaging On 7x7 window


31Denoising: The Issue

A Noisy Detail in Camera Raw Data.



Denoised



A Noisy Detail in Camera Raw Data.



Denoised


Observation model is

The goal is to obtain , a reliable estimate of , given and the distribution of .

For the sake of simplicity it is often assumed and independent .

The noise standard deviation is assumed as known.

Image Formation Model

Sensed image

Pixel index

Original (unknown) image

noise

July 23, 2010

35



Thus we can pursue a “regression-approach”,


July 23, 2010

36



Thus we can pursue a “regression-approach”, but on images it may not be convenient to assume a parametric expression of on


July 23, 2010

37



Thus we can pursue a “regression-approach”, but on images it may not be convenient to assume a parametric expression of on


July 23, 2010

38


Denoising Approaches

Parametric Approaches• Transform Domain Filtering, they assume the noisy-free signal is

somehow sparse in a suitable domain (e.g Fourier, DCT, Wavelet) or w.r.t. some dictionary based decomposition)

27th October 2009

39





Non Parametric Approaches• Local Smoothing / Local Approximation• Non Local Methods

27th October 2009

40






27th October 2009

41






Estimating from can be statistically treated as regression of on

27th October 2009

42


Fitting and Convolution

One can prove that the least square fit of polynomial of 0-th order (i.e constant) is given by

where

and thus

03/12/2009



Non Parametric Approaches: there are no global model for and basically • Local Methods: define, in each image pixel, the “best

neighborhood” where a simple parametric model can be enforced to perform regression

is described by a (polynomial) model

July 23, 2010

44


Ideal neighborhood – an illustrative example

Ideal in the sense that it defines the support of a pointwise Least Square Estimator of the reference point.

Typically, even in simple images, every point has its own different ideal neighborhood.

For practical reasons, the ideal neighborhood is assumed starshaped

Further details at LASIP c/o Tampere University of Technology http://www.cs.tut.fi/~lasip/

July 23, 2010

45

http://www.cs.tut.fi/~lasip/


Neighborhood discretization

A suitable discretization of this neighborhood is obtained by using a set of directional LPA kernels

where determines the orientation of the kernel support, andwhere controls by the scale of kernel support.

Ideal Neighborhood

Directional kernels

Discrete Adaptive Neighborhood

27th October 2009

46


Neighborhood discretization

A suitable discretization of this neighborhood is obtained by using a set of directional LPA kernels

where determines the orientation of the kernel support, andwhere controls by the scale of kernel support.

The initial shape optimization problem can be solved by using standard easy-to-implement varying-scale kernel techniques, such as the ICI rule.

Ideal Neighborhood

Directional kernels

Discrete Adaptive Neighborhood

27th October 2009

47


Ideal neighborhood – an illustrative example

Ideal in the sense that it defines the support of pointwise Least Square Estimator of the reference point.

July 23, 2010

48


Examples of Adaptively Selected Neighorhoods

Define, , the “ideal” neigborhood

Compute the denoised estimate at , “using”

July 23, 2010

49


Examples of adaptively selected neighorhoods

Adaptively selected neighborhoods selected using the LPA-ICI rule

July 23, 2010

50


Denoising

Additive Gaussian White Noise

After Gaussian Smoothing

After Averaging

),( N


Salt & Pepper Noise Additive Noise, 7% outliers

After Gaussian Smoothing

After Averaging

Denoising


Correlation

k l

ljkiHlkIjiHIjiI ,,,*,

A Target can be used as a filter.

Correlation between target and image assumes maximum at image pixels that match the target


Correlation for BINARY target matching

*

• Easier with binary images

• Target used as a filter

?


Correlation functionMaximum value line profile

Correlation for target matching


Examples of Non Linear Filters

• Median Filter (Weighted Median)

• Ordered Statistics based Filters

• Threshold

There are many others, such as data adaptive filtering procedures (e.g ICI)


Blockwise Median

Substititute to each pixel the median of its neighbors

),( jiN

),( jif1 3 0

2 10 2

4 1 1

2

2)1,1,4,2,10,2,0,3,1( medianm

med


Denoising with median 3x3

Salt and Pepper noise


Denoising with median 3x3

Additive Gaussian White Noise (AGWN)


What is an Edge

Lets define an edge to be a discontinuity in image intensity function.

Several Models• Step Edge• Ramp Edge• Roof Edge• Spike Edge

They can bethus detected asdiscontinuitiesof image Derivatives

03/12/2009


Differentiation and convolution

Recall

Now this is linear and shift invariant. Therefore, in discrete domain, it will be represented as a convolution

xfxf

x

f0

lim

03/12/2009


Differentiation and convolution

Recall

Now this is linear and shift invariant. Therefore, in discrete domain, it will be represented as a convolution

We could approximate this as

(which is obviously a convolution with Kernel

; it’s not a very good way to do things, as we shall see)

xfxf

x

f0

lim x

xfxf

x

f n

1

11

03/12/2009


Finite Difference in 2D

yxfyxf

x

yxf ,,lim

,0

yxfyxf

y

yxf ,,lim

,0

x

yxfyxf

x

yxf mnmn

,,, 1

x

yxfyxf

y

yxf mnmn

,,, 1

11

1

1

Discrete Approximation

Horizontal

Convolution Kernels

Vertical

03/12/2009


A 1D Example

Take a line on a grayscale image

03/12/2009


A 1D Example (II)

Filter the discrete image values, convolution against [1 -1]

03/12/2009


Differentiating Filters

)1()1()( xfxfxf x1D Discrete derivatives

1 0 -1*ff x

2D Discrete derivatives (separable)

*ff y 1 0 -1t

03/12/2009


Classical Operators : Prewitt

11

11

11

1

1

11

Smooth Differentiate

101

101

101

111

000

111

111

111

Horizontal

Vertical

03/12/2009


Classical Operators: Sobel

11

22

11

1

1

11

Smooth Differentiate

101

202

101

121

000

121

121

121

Horizontal

Vertical

03/12/2009


Another famous test image - cameraman

03/12/2009


Horizontal Derivatives using Sobel

1 * ty y h

03/12/2009


Vertical Derivatives using Sobel

2 * ty y h

03/12/2009


Gradient Magnitude and edge detectors

2 2| | * *ty y h y h

03/12/2009


A brief overview on Morphological Operators in Image Processing

Giacomo Boracchi

24/11/2010

[email protected]

home.dei.polimi.it/boracchi/teaching/IAS.htm

mailto:[email protected]


An overview on morphological operations

Erosion, Dilation

Open, Closure

We assume the image being processed is binary, as these operators are typically meant for refining “mask” images.

03/12/2009


AND operator in Binary images

03/12/2009


OR in Binary Images

03/12/2009


Erosion

General definition: Nonlinear Filtering procedure that replace to each pixel value the minimum on a given neighbor

As a consequence on binary imagesE(x)=1 iff the image in the neighbor is constantly 1

This operation reduces thus the boundaries of binary images

It can be interpreted as an AND operation of the image and the neighbor overlapped at each pixel

03/12/2009


Erosion Example

03/12/2009


Dilation

General definition: Nonlinear Filtering procedure that replace to each pixel value the maximum on a given neighbor

As a consequence on binary images

E(x)=1 iff at least a pixel in the neighbor is 1

This operation grows fat the boundaries of binary images

It can be interpreted as an OR operation of the image and the neighbor overlapped at each pixel

03/12/2009


Dilation Example

03/12/2009


In matlab

They are performed using the

bwmorph.m script passing the name of the operation as a parameter

Examples…

03/12/2009


Open and Closure

Open Erosion followed by a Dilation

Closure Dilation followed by an Erosion

03/12/2009


Open

Open Erosion followed by a Dilation• Smoothes the contours of an object• Typically eliminate thin protrusions

03/12/2009


Open

Figure to Open,

Structuring element a Disk

03/12/2009


Open

First Erode,

03/12/2009


Open

Then Dilate,

Open results, the bridge has been eliminate from the first erosion and cannot be replaced by the dilation.

Edges are smoothed

Corners are rounded

03/12/2009


Closure

Closure Dilation followed by an Erosion• Smoothes the contours of an object, typically creates• Generally fuses narrow breaks

03/12/2009


Open

Figure to Open,

Structuring element a Disk

03/12/2009


Close

First Dilate,

03/12/2009


Close

Then Erode,

Close results, the bridge has been preserved

Edges are smoothed

Corners are rounded

03/12/2009


There are several other Non Linear Filters

Ordered Statistic based• Median Filter • Weight Ordered Statistic Filter• Trimmed Mean• Hybrid Median

Thresholding

03/12/2009

computer vision group digital image filters 26 th november 2012

Documents

system h

computer vision group

computer vision group

scalar slide

lti systems

digital images linear

particular systems

invariant filters