image enhancement and filtering
TRANSCRIPT
-
8/2/2019 Image Enhancement and Filtering
1/60
EE4H, M.Sc 0407191Computer Vision
Dr. Mike Spann
[email protected]://www.eee.bham.ac.uk/spannm
mailto:[email protected]://www.eee.bham.ac.uk/spannmhttp://www.eee.bham.ac.uk/spannmmailto:[email protected] -
8/2/2019 Image Enhancement and Filtering
2/60
Introduction Images may suffer from the following degradations:
Poor contrast due to poor illumination or finitesensitivity of the imaging device
Electronic sensor noise or atmospheric disturbancesleading to broad band noise
Aliasing effects due to inadequate sampling
Finite aperture effects or motion leading to spatial
-
8/2/2019 Image Enhancement and Filtering
3/60
IntroductionWe will consider simple algorithms for image
enhancement based on lookup tables
Contrast enhancement
We will also consider simple linear filteringalgorithms
Noise removal
-
8/2/2019 Image Enhancement and Filtering
4/60
Histogram equalisation In an image of low contrast, the image has grey
levels concentrated in a narrow band
Define the grey level histogram of an image h(i)where : h(i)=number of pixels with grey level=I
For a low contrast image, the histogram will be
concentrated in a narrow band The full greylevel dynamic range is not used
-
8/2/2019 Image Enhancement and Filtering
5/60
Histogram equalisation
h i( )
i
-
8/2/2019 Image Enhancement and Filtering
6/60
Histogram equalisation Can use a sigmoid lookup to map input to output grey
levels
A sigmoid functiong(i) controls the mapping frominput to output pixel
Can easily be implemented in hardware for maximumefficiency
-
8/2/2019 Image Enhancement and Filtering
7/60
Histogram equalisationh i( )
g i( )
h i' ( )
h i h g i' ( ) ( ( ))1
g i i( ) exp 11
-
8/2/2019 Image Enhancement and Filtering
8/60
Histogram equalisation controls the position of maximum slope controls the slope
Problem - we need to determine the optimumsigmoid parameters and for each image Abetter method would be to determine the best
mapping function from the image data
-
8/2/2019 Image Enhancement and Filtering
9/60
Histogram equalisationA general histogram stretching algorithm is defined in
terms of a transormationg(i)
We require a transformation g(i) such that from anyhistogram h(i) :
constant)()(')(:
jgijjhih
-
8/2/2019 Image Enhancement and Filtering
10/60
Histogram equalisation Constraints (NxNx 8 bit image)
No crossover in grey levels after transformation
2)(' Nihi
i i g i g i1 2 1 2 ( ) ( )
-
8/2/2019 Image Enhancement and Filtering
11/60
Histogram equalisationAn adaptive histogram equalisation algorithm can be
defined in terms of the cumulative histogram H(i) :
H i( ) = number of pixels with grey levels i
i
jjhiH
0)()(
-
8/2/2019 Image Enhancement and Filtering
12/60
Histogram equalisation Since the required h(i) is flat, the required H(i) is a
ramp:
h(i) H(i)
-
8/2/2019 Image Enhancement and Filtering
13/60
Histogram equalisation Let the actual histogram and cumulative
histogram be h(i) and H(i)
Let the desired histogram and desired cumulativehistogram be h(i) and H(i)
Let the transformation beg(i)
H g i N g i' ( ( )) ( ) 2255
( ' ( ) , ' ( ) ) H N H 255 0 02
-
8/2/2019 Image Enhancement and Filtering
14/60
Histogram equalisation Sinceg(i)is an ordered transformation
i i g i g i1 2 1 2 ( ) ( )H g i H i
N g i' ( ( )) ( )
( ) 2255
g iH i
N( )
( ) 255 2
-
8/2/2019 Image Enhancement and Filtering
15/60
Histogram equalisationWorked example, 32 x 32 bit image with grey levels
quantised to 3 bits
g iH i
( )( ) 7
1024
h i h jj i g j
' ( ) ( ): ( )
-
8/2/2019 Image Enhancement and Filtering
16/60
Histogram equalisationi h i( ) H i( ) g i( )
0 197 197 1.351 -
1 256 453 3.103 1972 212 665 4.555 -
3 164 829 5.676 256
4 82 911 6.236 -
5 62 993 6.657 2126 31 1004 6.867 246
7 20 1024 7.07 113
h i' ( )
-
8/2/2019 Image Enhancement and Filtering
17/60
Histogram equalisation
0 1 2 3 4 5 6 7
0
50
100
150
200
250
300
0 1 2 3 4 5 6 7
Original histogram
Stretched histogram
-
8/2/2019 Image Enhancement and Filtering
18/60
Histogram equalisation
0.00
500.00
1000.00
1500.00
2000.00
0.00 50.00 100.00 150.00 200.00 250.00
i
h(i)
0.00
500.00
1000.00
1500.00
2000.00
0.00 50.00 100.00 150.00 200.00 250.00
i
h(i)
-
8/2/2019 Image Enhancement and Filtering
19/60
Histogram equalisation
0.00
500.00
1000.00
1500.00
2000.00
2500.00
3000.00
0.00 50.00 100.00 150.00 200.00 250.00
i
h(i)
0.00
500.00
1000.00
1500.00
2000.00
2500.00
3000.00
0.00 50.00 100.00 150.00 200.00 250.00
i
h(i)
-
8/2/2019 Image Enhancement and Filtering
20/60
Histogram equalisation ImageJ demonstration
http://rsb.info.nih.gov/ij/signed-applet
http://rsb.info.nih.gov/ij/signed-applet/http://rsb.info.nih.gov/ij/signed-applet/http://rsb.info.nih.gov/ij/signed-applet/http://rsb.info.nih.gov/ij/signed-applet/ -
8/2/2019 Image Enhancement and Filtering
21/60
Image Filtering Simple image operators can be classified as 'pointwise'
or 'neighbourhood' (filtering) operators
Histogram equalisation is a pointwise operation
More general filtering operations use neighbourhoodsof pixels
-
8/2/2019 Image Enhancement and Filtering
22/60
Image Filtering
(x,y) (x,y)
Input image Output image
(x,y) (x,y)
pointwisetransformation
neighbourhoodtransformation
Input image Output image
-
8/2/2019 Image Enhancement and Filtering
23/60
-
8/2/2019 Image Enhancement and Filtering
24/60
Image Filtering Examples of filters:
g x y h f x y h f x y
h f x y
( , ) ( , ) ( , )
..... ( , )
1 2
9
1 1 1
1 1
g x y median
f x y f x y
f x y( , )
( , ), ( , )
..... ( , )
1 1 1
1 1
-
8/2/2019 Image Enhancement and Filtering
25/60
Linear filtering and convolution Example
3x3 arithmetic mean of an input image (ignoringfloating point byte rounding)
(x,y) (x,y)
Input image f(x,y) Output image g(x,y)
(x-1,y-1)
(x+1,y+1)
-
8/2/2019 Image Enhancement and Filtering
26/60
-
8/2/2019 Image Enhancement and Filtering
27/60
Linear filtering and convolution
(x,y) (x,y)
Input image f(x,y) Output image g(x,y)
Image point
Filter mask point
-
8/2/2019 Image Enhancement and Filtering
28/60
Linear filtering and convolutionWe can define the convolution operator
mathematically
Defines a 2D convolution of an imagef(x,y) with a filterh(x,y)
g x y h x y f x x y y
f x x y y
yx
yx
( , ) ( ' , ' ) ( ' , ' )
( ' , ' )
''
''
1
1
1
1
1
1
1
11
9
-
8/2/2019 Image Enhancement and Filtering
29/60
Linear filtering and convolution Example convolution with a Gaussian filter kernel
determines the width of the filter and hence theamount of smoothing
g x yx y
g x g y
g xx
( , ) exp(( )
)
( ) ( )
( ) exp( )
2 2
2
2
2
2
2
-
8/2/2019 Image Enhancement and Filtering
30/60
Linear filtering and convolution
0.00
0.20
0.40
0.60
0.80
1.00
-6 -4 -2 0 2 4 x
g(x)
-
8/2/2019 Image Enhancement and Filtering
31/60
Linear filtering and convolution
Original Noisy
Filtered
=1.5
Filtered
=3.0
-
8/2/2019 Image Enhancement and Filtering
32/60
Linear filtering and convolution ImageJ demonstration
http://rsb.info.nih.gov/ij/signed-applet
http://rsb.info.nih.gov/ij/signed-applet/http://rsb.info.nih.gov/ij/signed-applet/http://rsb.info.nih.gov/ij/signed-applet/http://rsb.info.nih.gov/ij/signed-applet/ -
8/2/2019 Image Enhancement and Filtering
33/60
-
8/2/2019 Image Enhancement and Filtering
34/60
Linear filtering and convolution The inverse DFT is defined by
f x yN
F u v jN
ux vyy
N
x
N
( , ) ( , )exp( ( ))
1 220
1
0
1
x y N , .. 0 1
-
8/2/2019 Image Enhancement and Filtering
35/60
Linear filtering and convolutionx
y
f(x,y)
(0.0)
(N-1,N-1)
v
u(0,0)
(N-1,N-1)
F(u,v)
DFT IDFT
-
8/2/2019 Image Enhancement and Filtering
36/60
Linear filtering and convolution
log( ( , ) )1 F u v
-
8/2/2019 Image Enhancement and Filtering
37/60
Linear filtering and convolution F(u,v) is the frequency content of the image at spatial
frequency position(u,v) Smooth regions of the image contribute low frequency
components to F(u,v)Abrupt transitions in grey level (lines and edges)
contribute high frequency components toF(u,v)
-
8/2/2019 Image Enhancement and Filtering
38/60
Linear filtering and convolutionWe can compute the DFT directly using the formula
An N point DFT would require N2 floating pointmultiplications per output point
Since there are N2 output points , the computationalcomplexity of the DFT is N4
N4=4x109 for N=256
Bad news! Many hours on a workstation
-
8/2/2019 Image Enhancement and Filtering
39/60
Linear filtering and convolution The FFT algorithm was developed in the 60s for
seismic exploration
Reduced the DFT complexity to 2N2log2N
2N2log2N~106 for N=256
A few seconds on a workstation
-
8/2/2019 Image Enhancement and Filtering
40/60
Linear filtering and convolution The filtering interpretation of convolution can be
understood in terms of the convolution theorem
The convolution of an imagef(x,y) with a filter h(x,y) isdefined as:
g x y h x y f x x y y
f x y h x yy
M
x
M
( , ) ( ' , ' ) ( ' , ' )
( , )* ( , )''
0
1
0
1
-
8/2/2019 Image Enhancement and Filtering
41/60
Linear filtering and convolution
(x,y)
Input image f(x,y) Output image g(x,y)
(x,y)
Filter mask h(x,y)
-
8/2/2019 Image Enhancement and Filtering
42/60
Linear filtering and convolution Note that the filter mask is shifted and inverted prior
to the overlap multiply and add stage of theconvolution
Define the DFTs off(x,y),h(x,y), andg(x,y) asF(u,v),H(u,v) and G(u,v)
The convolution theorem states simply that :
G u v H u v F u v( , ) ( , ) ( , )
-
8/2/2019 Image Enhancement and Filtering
43/60
-
8/2/2019 Image Enhancement and Filtering
44/60
Linear filtering and convolution
DFT
IDFT
-
8/2/2019 Image Enhancement and Filtering
45/60
Linear filtering and convolution Frequency domain implementation of
convolution Imagef(x,y)NxNpixels
Filter h(x,y) MxM filter mask points UsuallyM
-
8/2/2019 Image Enhancement and Filtering
46/60
-
8/2/2019 Image Enhancement and Filtering
47/60
Linear filtering and convolutionInput image f(x,y) Output image g(x,y)
(x,y)
Filter mask h(x,y)
(x',y')
x' = x modulo N
y' = y modulo N
-
8/2/2019 Image Enhancement and Filtering
48/60
Linear filtering and convolutionWe can evaluate the computational complexity of
implementing convolution in the spatial andspatial frequency domains
Nx Nimage is to be convolved with anMx Mfilter Spatial domain convolution requiresM2 floating point
multiplications per output point orN2 M2 in total
Frequency domain implementation requires 3x(2N2
log 2N) +N2 floating point multiplications ( 2 DFTs +
1 IDFT +N2 multiplications of the DFTs)
-
8/2/2019 Image Enhancement and Filtering
49/60
Linear filtering and convolution Example 1,N=512,M=7
Spatial domain implementation requires 1.3 x 107floating point multiplications
Frequency domain implementation requires 1.4 x 107floating point multiplications
Example 2,N=512,M=32 Spatial domain implementation requires 2.7 x 108
floating point multiplications Frequency domain implementation requires1.4 x 107
floating point multiplications
-
8/2/2019 Image Enhancement and Filtering
50/60
Linear filtering and convolution For smaller mask sizes, spatial and frequency
domain implementations have about the samecomputational complexity
However, we can speed up frequency domaininterpretations by tessellating the image into sub-blocks and filtering these independently
Not quite that simple we need to overlap the filteredsub-blocks to remove blocking artefacts
Overlap and add algorithm
-
8/2/2019 Image Enhancement and Filtering
51/60
Linear filtering and convolutionWe can look at some examples of linear filters
commonly used in image processing and theirfrequency responses
In particular we will look at a smoothing filter and afilter to perform edge detection
-
8/2/2019 Image Enhancement and Filtering
52/60
Linear filtering and convolution Smoothing (low pass) filter
Simple arithmetic averaging
Useful for smoothing images corrupted by additivebroad band noise
H H3 5
1
9
1 1 1
1 1 11 1 1
1
25
1 1 1 1 1
1 1 1 1 1
1 1 1 1 11 1 1 1 1
1 1 1 1 1
etc
-
8/2/2019 Image Enhancement and Filtering
53/60
Linear filtering and convolutionh x( ) H u( )
Spatial domain Spatial frequency domain
ux
-
8/2/2019 Image Enhancement and Filtering
54/60
Linear filtering and convolution Edge detection filter
Simple differencing filter used for enhancing edged
Has a bandpass frequency response
H
1 0 1
1 0 1
1 0 1
-
8/2/2019 Image Enhancement and Filtering
55/60
Linear filtering and convolution ImageJ demonstration
http://rsb.info.nih.gov/ij/signed-applet
http://rsb.info.nih.gov/ij/signed-applethttp://rsb.info.nih.gov/ij/signed-applethttp://rsb.info.nih.gov/ij/signed-applethttp://rsb.info.nih.gov/ij/signed-applet -
8/2/2019 Image Enhancement and Filtering
56/60
Linear filtering and convolutionf x( )
p
x
f x( )* ( )1 0 1
p
x
-
8/2/2019 Image Enhancement and Filtering
57/60
Linear filtering and convolutionWe can evaluate the (1D) frequency response of
the filter h(x)={1,0,-1 } from the DFT definition
H u h xjux
N
ju
Nju
N
ju
N
ju
N
jju
N
u
N
x
N
( ) ( )exp( )
exp( )
exp( ) exp( ) exp( )
exp( ) sin( )
2
14
2 2 2
22 2
0
1
-
8/2/2019 Image Enhancement and Filtering
58/60
Linear filtering and convolution The magnitude of the response is therefore:
This has a bandpass characteristic
H u
u
N( ) sin( )
2
-
8/2/2019 Image Enhancement and Filtering
59/60
Linear filtering and convolution
H u( )
u
-
8/2/2019 Image Enhancement and Filtering
60/60