chapter 5 neighborhood processing
DESCRIPTION
Chapter 5 Neighborhood Processing. Introduction Filters (LPF, HPF, Gaussian Filter, …) Region of interest processing. Basic Image Processing Operations. Neighborhood processing process the pixel with its neighbors Point operations process according to the pixel’s value alone. - PowerPoint PPT PresentationTRANSCRIPT
SCCS 476 1
Chapter 5 Neighborhood Processing
• Introduction• Filters (LPF, HPF, Gaussian Filter, …)• Region of interest processing
SCCS 476 2
Basic Image Processing Operations
• Neighborhood processing– process the pixel with its neighbors
• Point operations– process according to the pixel’s value alone.
SCCS 476 3
Neighborhood Processing
3x3 Mask
Output derives from multiplying all elements in the mask by corresponding elements in the neighborhood and adding together all these products.
SCCS 476 4
Filter
• A rule or procedure for processing an image
• Combination of mask and function
• Goal: separating/attenuating a desired component of an observed image
• Type: – Linear (function), Nonlinear (function)– Low-pass filter (LPF), High-pass filter (HPF),
Band-pass filter (BPF)
SCCS 476 5
Steps of Linear Spatial Filtering
• Position the mask over the current pixel.
• Form all products of filter elements with the corresponding elements of the neighborhood.
• Add all products.
• Other names for mask:
filter, mask, filter mask, kernel, template or window, etc.
SCCS 476 6
Linear Spatial Filter: Example (1)
2 15 10 20
0 0
5 20
10 15
10 10
10
10
15
1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
1/9
Average Filter10
Find the output intensity of the blue pixel.
SCCS 476 7
Linear Spatial Filter: Example (2)
15 10 20
0
20
15
10 10
1/9 1/9
1/9 1/9
1/9 1/9 1/9
1/9
101/9
Multiply the number in the filter’s element with the corresponding pixel’s intensity.
SCCS 476 8
20/9
0/9
10/9
Linear Spatial Filter: Example (3)
15/9 20/9
10/9 15/9
10/9 10/9
Add all products for output.
15/9 + 10/9 + 20/9 + 0/9 + 10/9 + 15/9 + 20/9 + 10/9 + 10/9 = 12.22
Output intensity of blue pixel = 12.22
Spatial filtering is spatial convolution.
SCCS 476 9
Correlation
• Sum of the product of mask and intensity on each point.
][][][ nkhkfng functionk
inputoutput
SCCS 476 10
Example: Correlation (1)
10 20 10 15 5
5 10 15 10 0
20 10 20 15 5
10 5 0 0 10
10 20 15 0 10
1 3
2 3 4
3 4 5
2
Kernel20
SCCS 476 11
Example: Correlation (2)
10 15 10
10
5
15
0 0
1 3
2 4
3 4 5
2
20 3
Multiply and sum all products.
101 +152 +103 +102 +203 +154 +53 +04 +05= 225
SCCS 476 12
Exercise: Correlation
0 0 0 0 0
0 0 0 0 0
0 0 1 0 0
0 0 0 0 0
0 0 0 0 10
1 3
2 3 4
3 4 5
2
Kernel
Find the correlation of the green kernel to the above image.
SCCS 476 13
Convolution
• Sum of the response on each point
k
functioninputoutput knhkfng ][][][
SCCS 476 14
Example: Convolution (1)
10 20 10 15 5
5 10 15 10 0
20 10 20 15 5
10 5 0 0 10
10 20 15 0 10
5 3
4 3 2
3 2 1
4
Kernel
20
1 3
2 3 4
3 4 5
2
Reverse Kernel!!
SCCS 476 15
Example: Convolution (2)
10 15 10
10
5
15
0 0
5 3
4 2
3 2 1
4
20 3
Multiply and sum all products.
105 +154 +103 +104 +203 +152 +53 +02 +01= 285
SCCS 476 16
Exercise: Convolution
0 0 0 0 0
0 0 0 0 0
0 0 1 0 0
0 0 0 0 0
0 0 0 0 10
1 3
2 3 4
3 4 5
2
Kernel
Find the convolution of the green kernel to the above image.
SCCS 476 17
Notation
• A linear filter is represented as a matrix, e.g., the 3 x 3 averaging filter.
1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
1/9
SCCS 476 18
Notation (2)• Edges of the Image
– There are a number of different approaches to dealing with this problem.
• Ignore the edges: The mask is applied to all pixels except the edges and results in an output image that is smaller than the original. If the mask is very large, a significant amount of information may be lost.
• Pad with zeros: We assume that all necessary values outside the image are zero. It will return an output image of the same size as the original, but may have the effect of introducing unwanted artifacts, e.g., edges, around the image.
SCCS 476 19
Filtering in MATLAB
• Command: filter2• Syntax: filter2(filter, image, shape);
filter2(filter, image);– shape:
• ‘same’: pad edge with zeros. Size unchanged. (default)
• ‘valid’: apply mask only to inside pixel. Size smaller.
• ‘full’: pad edge with zeros and applying the filter at all places on and around the image where the mask intersects the image matrix. Size larger.
SCCS 476 20
Filter Construction in MATLAB
• Command: fspecial• Syntax: fspecial(type, parameter);
fspecial(type);– type: type of the filter
• ‘average’ : average filter
• ‘gaussian’ : Gaussian filter
• ‘laplacian’ : Laplacian filter …
– parameter: parameter of the filter (size, sigma, …). Default varies among filter. Try!!!
SCCS 476 21
Separable Filters
• Some filters can be implemented by the successive application of two simple filters.
n
mmn
n
mm a
a
aa
aa
.......
...
...
...
1
11
21
1211
SCCS 476 22
Separable Filters (2)
• Speed up the processing by filtering in one axis at a time.– working with 2D n x n matrix requires n2, O(n2),
multiplications, and n2 – 1, O(n2), addition for each pixel.
– working with 1D n x 1 matrix twice requires 2n, O(n), multiplications, and 2n – 2, O(n), additions for each pixel.
n
mmn
n
mm a
a
aa
aa
.......
...
...
...
1
11
21
1211
SCCS 476 23
Examples of Separable Filter
• Average filter
• Laplacian filter
1113
1
1
1
1
3
1
111
111
111
9
1
121
1
2
1
121
242
121
SCCS 476 24
Filter on Frequency Domain• Low-pass filter (LPF): filter that allows only
the low-frequency components and reduces or eliminates the high-frequency components.– E.g. Gaussian, average
• High-pass filter (HPF): filter that allows only the high-frequency components and reduces or eliminates the low-frequency components.– E.g. Laplacian, Prewitt, Sobel
SCCS 476 25
Frequency Components
• Spatial data (intensity) transformed by Fourier transform.
• Simplified version:– high-frequency indicates the abrupt changes in
intensity edges.– low-frequency indicates the intensity smoothness uniform region.
SCCS 476 26
Example: HPF
10 20 10 15 5
5 10 15 10 0
20 10 20 15 5
10 5 0 0 10
10 20 15 0 10
1 1
-2 4 -2
1 -2 1
-2
Kernel
Input image
SCCS 476 27
Example: HPF (2)
Filtered Image
10 20 10 15 5
5 0 20 -10 0
20 -40 25 15 5
10 5 -35 0 10
10 20 15 0 10
SCCS 476 28
Computing Consideration
• Filter may lead to the value outside [0,255]– Solution 1: Make negative values positive (use absolute
value) good when there are few negative values and the negative values are close to zero
– Solution 2: Clip values. Values larger than 255 become 255 and values less than 0 become 0. not good if there are many values outside the range.
– Solution 3: Scaling transformation. Rescale the range to [0,255] by pixel transform. Suppose gL and gH are the lowest and the highest values.
SCCS 476 29
Rescaling Intensity
gH
Filtered value (x)gL
0
255
Rescaled value (y)
1. Map gL to 0.2. Map gH to 255.
3. Interpolate for the remaining intensity. LH
L
gg
gxy
255
SCCS 476 30
Rescaling: MATLAB
• Manual:
>> gH = max(filtered_image(:));
>> gL = min(filtered_image(:));
>> scaled = (image – gmin)/(gmax – gmin);
Note: No need to rescale to 255 because intensity range for double image is [0,1].
SCCS 476 31
Rescaling: MATLAB (2)
• Command: mat2gray• Syntax: mat2gray(double_image);
What this command do?– scale the value in double_image to displayable
value. Output is double type.– minimum value is mapped to 0.– maximum value is mapped to 1.
SCCS 476 32
Gaussian Filter
• 1D Gaussian filter:
• 2D Gaussian filter:
2
2
2)( x
exf
2
22
2),( yx
eyxf
http://www-mmdb.iai.uni-bonn.de/lehre/BIT/ss03_DSP_Vorlesung/matlab_demos/
http://upload.wikimedia.org/wikipedia/su/thumb/3/38/Gaussian-pdf.png/300px-Gaussian-pdf.png
SCCS 476 33
Benefits of Gaussian Filter
• They are mathematically very well behaved. The Fourier transform of a Gaussian filter is another Gaussian.
• There are rotationally symmetric, so are very good starting points for some edge-detection algorithms.
• They are separable in x and y axes. This can lead to very fast implementations.
• The convolution of two Gaussians is another Gaussian.
SCCS 476 34
Gaussian Filter (2)
• Effect of Gaussian filter = blurring– larger leads to more blur.
average filter
• Construction of Gaussian filter:– command: fspecial(‘gaussian’, size, gamma);
size : size of the filter [row column], default [3 3]
gamma : , default 0.5.
SCCS 476 35
Gaussian Filter: MATLAB
>>gaussian1 = fspecial(‘gaussian’, [5 5], 5);Create the 55 Gaussian filter with the
value of 5.
>>gaussian2 = fspecial(‘gaussian’, 3, 0.75);
Create the 33 Gaussian filter with the value of 0.75.
SCCS 476 36
Effect of Gaussian Filter
= 0.5 = 2 = 5
Filter size = 5 5
SCCS 476 37
Effect of Gaussian Filter (2)
= 5 Filter size = 5 5Filter size = 3 3
SCCS 476 38
Edge Sharpening
• Also known as edge enhancement, edge crispening, unsharp masking.
• Process to make the edge slightly sharper and crisper.
• E.g. linear edge sharpening, unsharp masking, high boost filtering
SCCS 476 39
Linear Edge Sharpening
• High pass filter (HPF)
• Increase the edge power
• Example:
121
252
121
,
111
191
111
,
010
151
010
SCCS 476 40
Unsharp Masking
Original Image
Blurred with LPF
Scale with a < 1
Scale forDisplay
SCCS 476 41
Unsharp Masking: MALAB
>> f = fspecial(‘average’); Create LPF (E.g. average filter)
>> xf = filter2(f,x); Filter input image x by average filter.
>> xu = double(x) – xf/1.5; Substract the low passed components.
>> imshow (xu/70) Scale intensity for better viewing.
SCCS 476 42
Unsharp Filter
[unsharp_image] = [input] – (a [filter] [input])
= (I* – a [filter]) [input]
= [unsharp_filter] [input]
I* : matrix whose center member is 1 and the others are zero.
E.g. for 3 3 matrix I* =
000
010
000
SCCS 476 43
Unsharp Filter (2)
• For 33 matrix, after some rearranging term:
[unsharp_filter] =
MATLAB use this format for the unsharp filter with the default of 0.2.
1
151
1
1
1
SCCS 476 44
Unsharp Filter: MATLAB
• Command: fspecial• Syntax: fspecial(‘unsharp’, alpha);
– alpha : alpha value for unsharp filter– Size of the filter is fixed to 3 3
SCCS 476 45
Effect of Unsharp Masking
BEFORE AFTERhttp://ise.hansung.ac.kr/jun/DI/Chapter-5.htm
SCCS 476 46
High-Boost Filtering
Original Image
Blurred with LPF
Scale with A>1
Scale forDisplay
[high_boost] = A [input] – [low_passed]
Amplification factor. A=1, high boost = HPF
SCCS 476 47
High Boost Filter (2)
high_boost = A input – low_pass
= A input – (input - high_pass)
= (A – 1) input + high_pass
Rescale high_boost by w.
w high_boost = w (A – 1) input + w high_pass
Best result when w (A – 1) = 1.
w = 1/(A – 1)
SCCS 476 48
High Boost Filter (3)• General high boost filter:
Another version:
Best result when 3/5 A 5/6 • Used in dark image.• Boost the intensity of the original image.
)low_pass(1
1)original(
1
AA
A
)low_pass(12
1)original(
12
A
A
A
A
SCCS 476 49
Effect of High Boost Filter
BEFORE AFTERA = 5/6
SCCS 476 50
Non-linear Filter• A non-linear filter is obtained by a nonlinear function
of the grayscale values in the mask.• E.g. rank-order filters (minimum filter, maximum
filter, median filter); the elements under the mask are ordered.
• MATLAB command: nlfilter • A faster alternative: colfilt (rearranges the image into
column)• Good for removing impulse-like noise• Better tradeoff between smoothing and retention of
fine image details.
SCCS 476 51
MATLAB Command: nlfilter
• Syntax: nlfilter(Image,[m,n],func);– Image : input image
– [m n] : size of the neighborhood
– func : function to be applied to the neighborhood
• E.g. >> cmed = nlfilter(image, [3,3], ‘median(x(:))’);
Median function33 sliding box
SCCS 476 52
MATLAB Command: colfilt
• Columnwise neighborhood operation
• Syntax: colfilt(im, [m n], block_type, func)– im : input image – [m n] : size that image matrix to be arranged into
– block_type : how to rearrange im matrix.
– func : function to be applied to the neighborhood
• E.g. >> cmed = colfilt(image, [3,3], ‘sliding’, @median);
SCCS 476 53
Block_type in colfilt
• ‘distinct’: “Rearranges each distinct m-by-n block in the image A
into a column of B. im2col pads A with 0's, if necessary, so its size is an integer multiple of m-by-n. If A = [A11 A12; A21 A22], where each Aij is m-by-n, then B = [A11(:) A12(:) A21(:) A22(:)].”
• ‘sliding’“Converts each sliding m-by-n block of A into a column
of B, with no zero padding. B has m*n rows and contains as many columns as there are m-by-n neighborhoods of A.
SCCS 476 54
Rearrangement by ‘distinct’
• B = im2col(A,[m n],’distinct’);
Input Image
a1 a2 a3
a4 a5 a6
a7 a8 a9
A BmnBlock
a1 a2 a3
a4 a5 a6
a7 a8 a9
Pad this area with 0
Rearrange to column vector
SCCS 476 55
Input Image
Rearrangement by ‘sliding’
• B = im2col(A,[m n],’sliding’);
A BmnBlock
a0a1a2 anan+1
amn
SCCS 476 56
10 13 5
11 20 15
8 0 0
Rank-Order Filter
• Sort the intensities within the mask.
• Choose the intensity at ith position as output.
10 20 10 15 5
5 10 13 5 0
20 11 20 15 5
10 8 0 0 10
10 20 15 0 10
20
Sort intensityMedian filter
Min. filter
Max. filter
SCCS 476 57
Maximum Filter
• Output pixel is the maximum intensity of the pixels within the mask. (find brightest point)
BEFORE AFTERImage corrupted by pepper noise
http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/min_max.html
SCCS 476 58
Minimum Filter
• Output pixel is the minimum intensity of the pixels within the mask. (find darkest point)
BEFORE AFTERImage corrupted by salt noise
http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/max_min.html
SCCS 476 59
Median Filter
• Output pixel is the mid-intensity of the pixels within the mask (the median intensity).
www.ee.siue.edu/~sumbaug/ CVIPbook_PPLec/Chapter%209b.ppt
SCCS 476 60
Effect of Median Filter
BEFORE AFTER3x3 Kernel
http://www.ee.siue.edu/~cvip/CVIPtools_demos/
SCCS 476 61
Rank-Order Filtering: MATLAB
• Command: ordfilt2• Syntax: ordfilt2(image, order, domain);
– image : input image– order : which order of the sorted intensity
(minimum to maximum value) taken as output– domain : matrix indicating the neighborhood.
1 : pixels in the neighbor. 0 : pixels not in the neighbor
E.g. cmin = ordfilt2(image, 1, ones(3,3));
SCCS 476 62
Geometric Mean Filter
M
Mji
jixyxf
1
),(
),(),(ˆ
x(i, j): pixel’s intensityM : filter mask|M| : size of the mask
SCCS 476 63
Effect of Geometric Mean Filter
BEFORE AFTERImage corrupted by Gaussian noise with variance = 300, mean = 0
http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/geometric.html
SCCS 476 64
Bad Effect of Geometric Mean Filter
BEFORE AFTERImage corrupted by pepper noise with probability = 0.4
http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/geometric.html
SCCS 476 65
Alpha-Trimmed Mean Filter
• Output is the mean of the data after removing the first d/2 and the last d/2 ordered data.
10 13 5
11 20 15
8 0 0
10 20 10 15 5
5
10
13
5 0
20
11
20
15
5
10
8
0
0
10
10 20 15 0 10
20
Sort intensity
d =2
Trim the data by 2.(1 from the top.1 from the bottom.)
Output = average intensity of the remaining data. = 9.5
SCCS 476 66
Effect of Alpha-Trimmed Mean Filter
BEFORE AFTERImage corrupted by salt-and-pepper noise with variance = 200, mean = 0
http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/alpha.html
Trim size = 2, mask size =1
SCCS 476 67
Region of Interest Processing
• Process image only in the predefined area.
• The predefined area is called “Region Of Interest” (ROI).
• Function is the same as before but applied to only ROI.
SCCS 476 68
ROI Selection: MATLAB
• Command: roipoly• Syntax: roipoly(im);
roipoly(im, [x0 x1 …xm], [y0 y1 …ym];
– im : input image
– (x0, y0), (x1, y1), …, (xm, ym) : coordinate of the polygon covering region of interest.
– E.g. roi = roipoly(im, [60 100 100 60], [40 40 80 80]);
SCCS 476 69
Mask for ROI
• Binary image with the white (1) indicating ROI.
Mask coordinate: (222, 21) (272, 21) (300, 75) (270, 121) (221, 121) (191, 75)
http://www.mathworks.com/access/helpdesk/help/toolbox/images/roipoly.html
SCCS 476 70
ROI Filtering: MATLAB
• Command: roifilt2• Syntax: roifilt2(filter, image, roi);
– filter : 2D linear filter
– image : input image
– roi : region of interest (1: ROI, 0: not ROI)
SCCS 476 71
Effect of ROI Filtering
• Unsharp masking in ROI.
http://www.mathworks.com/access/helpdesk/help/toolbox/images/roifilt2.html
ROI at this coin