vc 18/19 tp7 spatial filters - dccmcoimbra/lectures/vc_18... · vc 18/19 - tp7 - spatial filters...
TRANSCRIPT
Mestrado em Ciência de Computadores
Mestrado Integrado em Engenharia de Redes e
Sistemas Informáticos
VC 18/19 – TP7
Spatial Filters
Miguel Tavares Coimbra
VC 18/19 - TP7 - Spatial Filters
Outline
• Spatial filters
• Frequency domain filtering
• Edge detection
VC 18/19 - TP7 - Spatial Filters
Topic: Spatial filters
• Spatial filters
• Frequency domain filtering
• Edge detection
VC 18/19 - TP7 - Spatial Filters
Images are Discrete and Finite
yxf , yxg , yxh ,
f
MN
i
j
h
Convolution
M
m
N
n
njmihnmfjig1 1
,,,
Fourier Transform
M
m
N
n
N
nv
M
mui
enmfvuF1 1
2
,,
Inverse Fourier Transform
M
u
N
v
N
lv
M
kui
evuFMN
lkf1 1
2
,1
,
VC 18/19 - TP7 - Spatial Filters
Spatial Mask
• Simple way to
process an image.
• Mask defines the
processing function.
• Corresponds to a
multiplication in
frequency domain. Convolution – Mask
‘slides’ over the image
Mask Image
VC 18/19 - TP7 - Spatial Filters
Example
• Each mask position
has weight w.
• The result of the
operation for each
pixel is given by:
1 2 1
0 0 0
-1 -2 -1
2 2 2
4 4 4
4 5 6
Mask Image
=1*2+2*2+1*2+…
=8+0-20
=-12
a
as
b
bt
tysxftswyxg ),(),(),(
VC 18/19 - TP7 - Spatial Filters
Definitions
• Spatial filters
– Use a mask (kernel) over an image region.
– Work directly with pixels.
– As opposed to: Frequency filters.
• Advantages
– Simple implementation: convolution with the kernel function.
– Different masks offer a large variety of functionalities.
VC 18/19 - TP7 - Spatial Filters
Averaging
Let’s think
about
averaging
pixel values
For n=2, convolve pixel values with 1 2 1
2D images:
1 2 1(a) use then
1
2
1
1 2 1or (b) use
1
2
1
1
2
1
1
2
1
2
4
2
Which is faster? 2
1 12 nObnOa
VC 18/19 - TP7 - Spatial Filters
The convolution kernel2n
8n
nlarge
Repeated averaging Gaussian smoothing
Averaging
VC 18/19 - TP7 - Spatial Filters
Gaussian Smoothing
2
22
2
1
22
1,
ji
ejihGaussian
kernel
N pixels
Filter size N …can be very large
(truncate, if necessary)
1 1
2
1
2,
2
1,
2
22
m n
nm
njmifejig
2D Gaussian is separable!
1 1
2
1
2
1
2,
2
1,
2
2
2
2
m n
nm
njmifeejig
Use two 1D
Gaussian
Filters!
VC 18/19 - TP7 - Spatial Filters
• A Gaussian kernel gives less weight to pixels further from the center of the window
• This kernel is an approximation of a Gaussian function:
Gaussian Smoothing
1 2 1
2 4 2
1 2 1
VC 18/19 - TP7 - Spatial Filters
2
8.2 4
original
VC 18/19 - TP7 - Spatial Filters
Mean Filtering
• We are degrading the
energy of the high spatial
frequencies of an image
(low-pass filtering).
– Makes the image
‘smoother’.
– Used in noise reduction.
• Can be implemented with
spatial masks or in the
frequency domain. 1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
VC 18/19 - TP7 - Spatial FiltersMean filter Gaussian filter
VC 18/19 - TP7 - Spatial Filters
http://www.michaelbach.de/ot/cog_blureffects/index.html
VC 18/19 - TP7 - Spatial Filters
http://www.michaelbach.de/ot/cog_blureffects/index.html
VC 18/19 - TP7 - Spatial Filters
Median Filter
• Smoothing is averaging
(a) Blurs edges
(b) Sensitive to outliers
(a)
(b)
– Sort values around the pixel
– Select middle value (median)
– Non-linear (Cannot be implemented with convolution)
• Median filtering
12 N
sort median
VC 18/19 - TP7 - Spatial Filters
3x3
5x5
7x7
Salt and pepper noise Gaussian noise
VC 18/19 - TP7 - Spatial Filters
Border Problem
What a computer sees
1 2 1
2 4 2
1 2 1
How do we apply
our mask to this
pixel?
VC 18/19 - TP7 - Spatial Filters
Border Problem
• Ignore
– Output image will be smaller than original
• Pad with constant values
– Can introduce substantial 1st order derivative values
• Pad with reflection
– Can introduce substantial 2nd order derivative values
VC 18/19 - TP7 - Spatial Filters
Topic: Frequency domain filtering
• Spatial filters
• Frequency domain filtering
• Edge detection
VC 18/19 - TP7 - Spatial Filters
Image Processing in the Fourier
Domain
Does not look anything like what we have seen
Magnitude of the FT
VC 18/19 - TP7 - Spatial Filters
Convolution in the Frequency Domain
f(x,y)
h(x,y)
g(x,y)
|F(sx,sy)|
|H(sx,sy)|
|G(sx,sy)|
VC 18/19 - TP7 - Spatial Filters
Lets the low frequencies
pass and eliminates the
high frequencies.
Generates image with overall
shading, but not much detail
Low-pass Filtering
VC 18/19 - TP7 - Spatial Filters
Lets through the high
frequencies (the detail),
but eliminates the low
frequencies (the overall
shape). It acts like an
edge enhancer.
High-pass Filtering
VC 18/19 - TP7 - Spatial Filters
Boosting High Frequencies
VC 18/19 - TP7 - Spatial Filters
VC 18/19 - TP7 - Spatial Filters
VC 18/19 - TP7 - Spatial Filters
The Ringing Effect
http://homepages.inf.ed.ac.uk/rbf/HIPR2/freqfilt.htm
An ideal low-pass filter causes ‘rings’
in the spatial domain!
VC 18/19 - TP7 - Spatial Filters
Topic: Edge detection
• Spatial filters
• Frequency domain filtering
• Edge detection
VC 18/19 - TP7 - Spatial Filters
Edge Detection
• Convert a 2D image into a set of curves– Extracts
salient features of the scene
– More compact than pixels
VC 18/19 - TP7 - Spatial Filters
Origin of Edges
• Edges are caused by a variety of factors
depth discontinuity
surface color discontinuity
illumination discontinuity
surface normal discontinuity
VC 18/19 - TP7 - Spatial Filters
How can you tell that a pixel is
on an edge?
VC 18/19 - TP7 - Spatial Filters
Edge Types
Step Edges
Roof Edge Line Edges
VC 18/19 - TP7 - Spatial Filters
Real Edges
Noisy and Discrete!
We want an Edge Operator that produces:
– Edge Magnitude
– Edge Orientation
– High Detection Rate and Good Localization
VC 18/19 - TP7 - Spatial Filters
Gradient
• Gradient equation:
• Represents direction of most rapid change in intensity
• Gradient direction:
• The edge strength is given
by the gradient magnitude
VC 18/19 - TP7 - Spatial Filters
Theory of Edge Detection
1B
2B tx
yIdeal edge
0cossin, yxyxL
0,:1 yxLB
0,:2 yxLB
Unit step function:
0for 0
0for 2
1
0for 1
t
t
t
tu dsstut
Image intensity (brightness):
c o ss in, 121 yxuBBByxI
VC 18/19 - TP7 - Spatial Filters
• Partial derivatives (gradients):
cossincos
cossinsin
12
12
yxBBy
I
yxBBx
I
• Squared gradient:
212
22
cossin,
yxBB
y
I
x
Iyxs
Edge Magnitude:
Edge Orientation:
yxs ,
x
I
y
I/arctan
Rotationally symmetric, non-linear operator
(normal of the edge)
Theory of Edge Detection
VC 18/19 - TP7 - Spatial Filters
• Laplacian:
cossin'
122
2
2
22 yxBB
y
I
x
II
Rotationally symmetric, linear operator
I
x
x
I
2
2
x
I
x
x
zero-crossing
Theory of Edge Detection
VC 18/19 - TP7 - Spatial Filters
Discrete Edge Operators
• How can we differentiate a discrete image?
Finite difference approximations:
1, jiI 1,1 jiI
jiI , jiI ,1
jijijiji IIII
x
I,,11,1,1
2
1
jijijiji IIIIy
I,1,,11,1
2
1
1 1
1 12
1
x
I 1 1
1 12
1
y
I
Convolution masks :
VC 18/19 - TP7 - Spatial Filters
1, jiI 1,1 jiI
jiI , jiI ,1
1,1 jiI
jiI ,1
1,1 jiI 1, jiI 1,1 jiI
• Second order partial derivatives:
jijiji IIIx
I,1,,122
2
21
1,,1,22
2
21
jijiji III
y
I
• Laplacian :
2
2
2
22
y
I
x
II
2
2 1
I
Convolution masks :
1 0
4 1
0
1
0 1 0
or 26
1
4 1
20 4
1
4
1 4 1
Discrete Edge Operators
(more accurate)
VC 18/19 - TP7 - Spatial Filters
The Sobel Operators
• Better approximations of the gradients exist
– The Sobel operators below are commonly used
-1 0 1
-2 0 2
-1 0 1
1 2 1
0 0 0
-1 -2 -1
VC 18/19 - TP7 - Spatial Filters
Comparing Edge Operators
-1 0 1
-2 0 2
-1 0 1
1 2 1
0 0 0
-1 -2 1
Gradient:
Roberts (2 x 2):
Sobel (3 x 3):
Sobel (5 x 5):-1 -2 0 2 1
-2 -3 0 3 2
-3 -5 0 5 3
-2 -3 0 3 2
-1 -2 0 2 1
1 2 3 2 1
2 3 5 3 2
0 0 0 0 0
-2 -3 -5 -3 -2
-1 -2 -3 -2 -1
0 1
-1 0
1 0
0 -1
Good Localization
Noise Sensitive
Poor Detection
Poor Localization
Less Noise Sensitive
Good Detection
VC 18/19 - TP7 - Spatial Filters
Effects of Noise
• Consider a single row or column of the image
– Plotting intensity as a function of position gives a signal
Where is
the edge??
VC 18/19 - TP7 - Spatial FiltersWhere is the edge?
Solution: Smooth First
Look for peaks in
VC 18/19 - TP7 - Spatial Filters
Derivative Theorem of Convolution
…saves us one operation.
VC 18/19 - TP7 - Spatial Filters
Laplacian of Gaussian (LoG)
Laplacian of Gaussian operator
Where is the edge? Zero-crossings of bottom graph !
fhx
fhx
2
2
2
2
Laplacian of Gaussian
VC 18/19 - TP7 - Spatial Filters
2D Gaussian Edge Operators
Laplacian of GaussianGaussian
Derivative of Gaussian (DoG)
Mexican Hat (Sombrero)
• is the Laplacian operator:
VC 18/19 - TP7 - Spatial Filters
Canny Edge Operator
• Smooth image I with 2D Gaussian:
• Find local edge normal directions for each pixel
• Compute edge magnitudes
• Locate edges by finding zero-crossings along the edge normal
directions (non-maximum suppression)
IG
IG
n
IG
0
2
2
n
IG
IG
VC 18/19 - TP7 - Spatial Filters
Non-maximum Suppression
• Check if pixel is local maximum along gradient direction
– requires checking interpolated pixels p and r
VC 18/19 - TP7 - Spatial Filtersoriginal image
VC 18/19 - TP7 - Spatial Filtersmagnitude of the gradient
VC 18/19 - TP7 - Spatial Filters
After non-maximum suppression
VC 18/19 - TP7 - Spatial Filters
Canny Edge Operator
Canny with Canny with original
• The choice of depends on desired behavior
– large detects large scale edges
– small detects fine features
VC 18/19 - TP7 - Spatial Filters
Difference of Gaussians (DoG)
• Laplacian of Gaussian can be approximated by the
difference between two different Gaussians
VC 18/19 - TP7 - Spatial Filters
DoG Edge Detection
1(a) 2(b) (b)-(a)
VC 18/19 - TP7 - Spatial Filters
Unsharp Masking
200 400 600 800
100
200
300
400
500
– =
=+ a
VC 18/19 - TP7 - Spatial Filters
Resources
• Gonzalez & Woods – Chapter 3