Mestrado em Ciência de Computadores
Mestrado Integrado em Engenharia de Redes e
Sistemas Informáticos
VC 14/15 – TP7
Spatial Filters
Miguel Tavares Coimbra
VC 14/15 - TP7 - Spatial Filters
Outline
• Spatial filters
• Frequency domain filtering
• Edge detection
Acknowledgements: Most of this course is based on the excellent courses offered by Prof. Shree Nayar at
Columbia University, USA and by Prof. Srinivasa Narasimhan at CMU, USA. Please acknowledge the original
source when reusing these slides for academic purposes.
VC 14/15 - TP7 - Spatial Filters
Topic: Spatial filters
• Spatial filters
• Frequency domain filtering
• Edge detection
VC 14/15 - TP7 - Spatial Filters
Images are Discrete and Finite
yxf , yxg , yxh ,
f
MN
i
j
h
Convolution
M
m
N
n
njmihnmfjig1 1
,,,
Fourier Transform
M
m
N
n
N
nv
M
mui
enmfvuF1 1
2
,,
Inverse Fourier Transform
M
u
N
v
N
lv
M
kui
evuFMN
lkf1 1
2
,1
,
VC 14/15 - TP7 - Spatial Filters
Spatial Mask
• Simple way to
process an image.
• Mask defines the
processing function.
• Corresponds to a
multiplication in
frequency domain. Convolution – Mask
‘slides’ over the image
Mask Image
VC 14/15 - TP7 - Spatial Filters
Example
• Each mask position
has weight w.
• The result of the
operation for each
pixel is given by:
1 2 1
0 0 0
-1 -2 -1
2 2 2
4 4 4
4 5 6
Mask Image
=1*2+2*2+1*2+…
=8+0-20
=-12
a
as
b
bt
tysxftswyxg ),(),(),(
VC 14/15 - TP7 - Spatial Filters
Definitions
• Spatial filters
– Use a mask (kernel) over an image region.
– Work directly with pixels.
– As opposed to: Frequency filters.
• Advantages – Simple implementation: convolution with the
kernel function.
– Different masks offer a large variety of functionalities.
VC 14/15 - TP7 - Spatial Filters
Averaging
Let’s think
about
averaging
pixel values
For n=2, convolve pixel values with 1 2 1
2D images:
1 2 1 (a) use then
1
2
1
1 2 1 or (b) use
1
2
1
1
2
1
1
2
1
2
4
2
Which is faster? 2
1 12 nObnOa
VC 14/15 - TP7 - Spatial Filters
The convolution kernel 2n
8n
nlarge
Repeated averaging Gaussian smoothing
Averaging
VC 14/15 - TP7 - Spatial Filters
Gaussian Smoothing
2
22
2
1
22
1,
ji
ejihGaussian
kernel
N pixels
Filter size N …can be very large
(truncate, if necessary)
1 1
2
1
2,
2
1,
2
22
m n
nm
njmifejig
2D Gaussian is separable!
1 1
2
1
2
1
2,
2
1,
2
2
2
2
m n
nm
njmifeejig
Use two 1D
Gaussian
Filters!
VC 14/15 - TP7 - Spatial Filters
• A Gaussian kernel gives less weight to pixels further from the center of the window
• This kernel is an approximation of a Gaussian function:
Gaussian Smoothing
1 2 1
2 4 2
1 2 1
VC 14/15 - TP7 - Spatial Filters
2
8.2 4
original
VC 14/15 - TP7 - Spatial Filters
Mean Filtering
• We are degrading the
energy of the high spatial
frequencies of an image
(low-pass filtering).
– Makes the image
‘smoother’.
– Used in noise reduction.
• Can be implemented with
spatial masks or in the
frequency domain. 1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
VC 14/15 - TP7 - Spatial Filters Mean filter Gaussian filter
VC 14/15 - TP7 - Spatial Filters
http://www.michaelbach.de/ot/cog_blureffects/index.html
VC 14/15 - TP7 - Spatial Filters
http://www.michaelbach.de/ot/cog_blureffects/index.html
VC 14/15 - TP7 - Spatial Filters
Median Filter
• Smoothing is averaging
(a) Blurs edges
(b) Sensitive to outliers
(a)
(b)
– Sort values around the pixel
– Select middle value (median)
– Non-linear (Cannot be implemented with convolution)
• Median filtering
12 N
sort median
VC 14/15 - TP7 - Spatial Filters
3x3
5x5
7x7
Salt and pepper noise Gaussian noise
VC 14/15 - TP7 - Spatial Filters
Border Problem
What a computer sees
1 2 1
2 4 2
1 2 1
How do we apply
our mask to this
pixel?
VC 14/15 - TP7 - Spatial Filters
Border Problem
• Ignore
– Output image will be smaller than original
• Pad with constant values
– Can introduce substantial 1st order derivative values
• Pad with reflection
– Can introduce substantial 2nd order derivative values
VC 14/15 - TP7 - Spatial Filters
Topic: Frequency domain filtering
• Spatial filters
• Frequency domain filtering
• Edge detection
VC 14/15 - TP7 - Spatial Filters
Image Processing in the Fourier
Domain
Does not look anything like what we have seen
Magnitude of the FT
VC 14/15 - TP7 - Spatial Filters
Convolution in the Frequency Domain
f(x,y)
h(x,y)
g(x,y)
|F(sx,sy)|
|H(sx,sy)|
|G(sx,sy)|
VC 14/15 - TP7 - Spatial Filters
Lets the low frequencies
pass and eliminates the
high frequencies.
Generates image with overall
shading, but not much detail
Low-pass Filtering
VC 14/15 - TP7 - Spatial Filters
Lets through the high
frequencies (the detail),
but eliminates the low
frequencies (the overall
shape). It acts like an edge enhancer.
High-pass Filtering
VC 14/15 - TP7 - Spatial Filters
Boosting High Frequencies
VC 14/15 - TP7 - Spatial Filters
VC 14/15 - TP7 - Spatial Filters
VC 14/15 - TP7 - Spatial Filters
The Ringing Effect
http://homepages.inf.ed.ac.uk/rbf/HIPR2/freqfilt.htm
An ideal low-pass filter causes ‘rings’
in the spatial domain!
VC 14/15 - TP7 - Spatial Filters
Topic: Edge detection
• Spatial filters
• Frequency domain filtering
• Edge detection
VC 14/15 - TP7 - Spatial Filters
Edge Detection
• Convert a 2D image into a set of curves – Extracts
salient features of the scene
– More compact than pixels
VC 14/15 - TP7 - Spatial Filters
Origin of Edges
• Edges are caused by a variety of factors
depth discontinuity
surface color discontinuity
illumination discontinuity
surface normal discontinuity
VC 14/15 - TP7 - Spatial Filters
How can you tell that a pixel is
on an edge?
VC 14/15 - TP7 - Spatial Filters
Edge Types
Step Edges
Roof Edge Line Edges
VC 14/15 - TP7 - Spatial Filters
Real Edges
Noisy and Discrete!
We want an Edge Operator that produces:
– Edge Magnitude
– Edge Orientation
– High Detection Rate and Good Localization
VC 14/15 - TP7 - Spatial Filters
Gradient
• Gradient equation:
• Represents direction of most rapid change in intensity
• Gradient direction:
• The edge strength is given
by the gradient magnitude
VC 14/15 - TP7 - Spatial Filters
Theory of Edge Detection
1B
2B tx
yIdeal edge
0cossin, yxyxL
0,:1 yxLB
0,:2 yxLB
Unit step function:
0for 0
0for 2
1
0for 1
t
t
t
tu dsstut
Image intensity (brightness):
cossin, 121 yxuBBByxI
VC 14/15 - TP7 - Spatial Filters
• Partial derivatives (gradients):
cossincos
cossinsin
12
12
yxBBy
I
yxBBx
I
• Squared gradient:
212
22
cossin,
yxBB
y
I
x
Iyxs
Edge Magnitude:
Edge Orientation:
yxs ,
x
I
y
I/arctan
Rotationally symmetric, non-linear operator
(normal of the edge)
Theory of Edge Detection
VC 14/15 - TP7 - Spatial Filters
• Laplacian:
cossin'
122
2
2
22 yxBB
y
I
x
II
Rotationally symmetric, linear operator
I
x
x
I
2
2
x
I
x
x
zero-crossing
Theory of Edge Detection
VC 14/15 - TP7 - Spatial Filters
Discrete Edge Operators
• How can we differentiate a discrete image?
Finite difference approximations:
1, jiI 1,1 jiI
jiI , jiI ,1
jijijiji IIIIx
I,,11,1,1
2
1
jijijiji IIIIy
I,1,,11,1
2
1
1 1
1 12
1
x
I 1 1
1 12
1
y
I
Convolution masks :
VC 14/15 - TP7 - Spatial Filters
1, jiI 1,1 jiI
jiI , jiI ,1
1,1 jiI
jiI ,1
1,1 jiI 1, jiI 1,1 jiI
• Second order partial derivatives:
jijiji III
x
I,1,,122
2
21
1,,1,22
2
21
jijiji III
y
I
• Laplacian :
2
2
2
22
y
I
x
II
2
2 1
I
Convolution masks :
1 0
4 1
0
1
0 1 0
or 26
1
4 1
20 4
1
4
1 4 1
Discrete Edge Operators
(more accurate)
VC 14/15 - TP7 - Spatial Filters
The Sobel Operators
• Better approximations of the gradients exist
– The Sobel operators below are commonly used
-1 0 1
-2 0 2
-1 0 1
1 2 1
0 0 0
-1 -2 -1
VC 14/15 - TP7 - Spatial Filters
Comparing Edge Operators
-1 0 1
-2 0 2
-1 0 1
1 2 1
0 0 0
-1 -2 1
Gradient:
Roberts (2 x 2):
Sobel (3 x 3):
Sobel (5 x 5): -1 -2 0 2 1
-2 -3 0 3 2
-3 -5 0 5 3
-2 -3 0 3 2
-1 -2 0 2 1
1 2 3 2 1
2 3 5 3 2
0 0 0 0 0
-2 -3 -5 -3 -2
-1 -2 -3 -2 -1
0 1
-1 0
1 0
0 -1
Good Localization
Noise Sensitive
Poor Detection
Poor Localization
Less Noise Sensitive
Good Detection
VC 14/15 - TP7 - Spatial Filters
Effects of Noise
• Consider a single row or column of the image
– Plotting intensity as a function of position gives a signal
Where is
the edge??
VC 14/15 - TP7 - Spatial Filters Where is the edge?
Solution: Smooth First
Look for peaks in
VC 14/15 - TP7 - Spatial Filters
Derivative Theorem of Convolution
…saves us one operation.
VC 14/15 - TP7 - Spatial Filters
Laplacian of Gaussian (LoG)
Laplacian of Gaussian operator
Where is the edge? Zero-crossings of bottom graph !
fhx
fhx
2
2
2
2
Laplacian of Gaussian
VC 14/15 - TP7 - Spatial Filters
2D Gaussian Edge Operators
Laplacian of Gaussian Gaussian
Derivative of Gaussian (DoG)
Mexican Hat (Sombrero)
• is the Laplacian operator:
VC 14/15 - TP7 - Spatial Filters
Canny Edge Operator
• Smooth image I with 2D Gaussian:
• Find local edge normal directions for each pixel
• Compute edge magnitudes
• Locate edges by finding zero-crossings along the edge normal
directions (non-maximum suppression)
IG
IG
n
IG
0
2
2
n
IG
IG
VC 14/15 - TP7 - Spatial Filters
Non-maximum Suppression
• Check if pixel is local maximum along gradient direction
– requires checking interpolated pixels p and r
VC 14/15 - TP7 - Spatial Filters original image
VC 14/15 - TP7 - Spatial Filters magnitude of the gradient
VC 14/15 - TP7 - Spatial Filters
After non-maximum suppression
VC 14/15 - TP7 - Spatial Filters
Canny Edge Operator
Canny with Canny with original
• The choice of depends on desired behavior
– large detects large scale edges
– small detects fine features
VC 14/15 - TP7 - Spatial Filters
Difference of Gaussians (DoG)
• Laplacian of Gaussian can be approximated by the
difference between two different Gaussians
VC 14/15 - TP7 - Spatial Filters
DoG Edge Detection
1(a) 2(b) (b)-(a)
VC 14/15 - TP7 - Spatial Filters
Unsharp Masking
200 400 600 800
100
200
300
400
500
– =
= + a
VC 14/15 - TP7 - Spatial Filters
Resources
• Gonzalez & Woods – Chapter 3