computer vision - umartalhaumartalha.weebly.com/.../8382263/02_lecture_cv_nust_pnec.pdf ·...
TRANSCRIPT
Computer Vision
EE837
NUST-PNEC Computer Vision EE 837 Fall 2014
Image processing basics
Lecture 02 Computer Vision EE837
NUST-PNEC Computer Vision EE 837 Fall 2014
• Chapter 2
David A. Forsyth and Jean Ponce, “Computer Vision A Modern Approach”, 2nd edition, Prentice Hall, Inc., 2003.
• For image processing details
Rafael C. Gonzalez, Richard E. Woods, “Digital Image Processing”, 3rd edition, Prentice Hall, 2007
Suggested readings
NUST-PNEC Computer Vision EE 837 Fall 2014
• Binary
• Gray Scale
• Color
Images
NUST-PNEC Computer Vision EE 837 Fall 2014
Image as a Function
• We can think of an image as a function, f,
• f: R2 R
– f (x, y) gives the intensity at position (x, y)
• A color image is just three functions pasted together. We can write this as a “vector-valued” function:
r(x, y)
f (x, y) = g(x, y)
b (x, y)
NUST-PNEC Computer Vision EE 837 Fall 2014
Binary Images
• Images with only two values (0 or 1)
• Simple to process and analyze
• Very useful for industrial applications
• Obtained from gray-level (or color) image f(x, y) by thresholding
• Characteristic Function
b(x, y) = 1 if f(x, y) < T
0 if f(x, y) >= T
NUST-PNEC Computer Vision EE 837 Fall 2014
0: Black1: White
p
q
X
Y
Row 1
Row q
1 1 1
0 0 0 00
1
Binary Images
NUST-PNEC Computer Vision EE 837 Fall 2014
10 5 9
100
Gray scale image
NUST-PNEC Computer Vision EE 837 Fall 2014
Gray scale image
NUST-PNEC Computer Vision EE 837 Fall 2014
Color Image
Red, Green, Blue channels
NUST-PNEC Computer Vision EE 837 Fall 2014
Image thresholding
NUST-PNEC Computer Vision EE 837 Fall 2014
Region has brighter or darker color, etc.
If pixel > threshold
then pixel = 1
else pixel = 0
Image thresholding
NUST-PNEC Computer Vision EE 837 Fall 2014
clc; clear all; close allsign = imread('StopSign.jpg','jpg'); red = (sign(:,:,1)>120) & (sign(:,:,2)<100) & (sign(:,:,3)<80); out = red*200; imwrite(out, 'SegStopSign.jpg', 'jpg');subplot(1,2,1); imshow('StopSign.jpg'); title('Stop Sign - Original');subplot(1,2,2); imshow('SegStopSign.jpg'); title('Stop Sign - Segmented');
Image thresholding
NUST-PNEC Computer Vision EE 837 Fall 2014
clc; clear all; close allsign = imread('Coins.jpg','jpg'); red = (sign(:,:,1)>80) & (sign(:,:,2)<240) & (sign(:,:,3)<140); out = red*200; imwrite(out, 'SegCoins.jpg', 'jpg');subplot(1,2,1); imshow('Coins.jpg'); title('Coins - Original');subplot(1,2,2); imshow('SegCoins.jpg'); title('Coins - Segmented');
Image histogram
NUST-PNEC Computer Vision EE 837 Fall 2014
• Light Variations
• Camera Electronics
• Surface Reflectance
• Lens
Image noise
NUST-PNEC Computer Vision EE 837 Fall 2014
• I(x,y) : the true pixel values
• n(x,y) : the noise at pixel (x,y)
ˆ , , , I x y I x y n x y
Image noise
NUST-PNEC Computer Vision EE 837 Fall 2014
2
22,
n
n x y e
Gaussian noise
NUST-PNEC Computer Vision EE 837 Fall 2014
• p is uniformly distributed random variable
• l is threshold
• smin and smax are constant
,ˆ ,
min max min
I x y p lI x y
s r s s p l
Salt and Pepper noise
NUST-PNEC Computer Vision EE 837 Fall 2014
• Derivative: Rate of change
– Speed is a rate of change of a distance
– Acceleration is a rate of change of speed
• Average (Mean)
– Dividing the sum of N values by N
Image derivatives
NUST-PNEC Computer Vision EE 837 Fall 2014
( ) ( )lim ( )0
df f x f x xf x fx x
dx x
ds
vdt
dva
dt
Derivative
Speed
Acceleration
NUST-PNEC Computer Vision EE 837 Fall 2014
2 4
32 4
y x x
dyx x
dx
sin
cos ( 1)
xy x e
dy xx edx
Examples
NUST-PNEC Computer Vision EE 837 Fall 2014
)()()(
lim 0 xfx
xxfxf
dx
dfx
)(1
)1()(xf
xfxf
dx
df
)()1()( xfxfxfdx
df
Discrete derivative
NUST-PNEC Computer Vision EE 837 Fall 2014
Take value of f(x) at present instant and subtract from the previous
)()1()1( xfxfxfdx
df
)()1()( xfxfxfdx
df
)()1()( xfxfxfdx
df Backward difference
Forward difference
Central difference
Discrete derivative – Finite difference
NUST-PNEC Computer Vision EE 837 Fall 2014
05201551050)(
005150550)(
2020202510101510)(
xf
xf
xf
Derivative Masks
Backward difference
Forward difference
Central difference
[-1 1]
[1 -1]
[-1 0 1]
Example
NUST-PNEC Computer Vision EE 837 Fall 2014
( , )f x y
y
x
f
f
y
yxfx
yxf
yxf),(
),(
),(
22),( yx ffyxf
y
x
f
f1tan
Given function
Gradient vector
Gradient magnitude
Gradient direction
Derivatives in 2-D
NUST-PNEC Computer Vision EE 837 Fall 2014
Image is 2D so we will have 2 partial derivatives
00000
0010100
0010100
0010100
00000
xI
101
101
101
3
1xfDerivative masks
101
101
101
3
1yf
2020201010
2020201010
2020201010
2020201010
2020201010
I
Derivatives of images
NUST-PNEC Computer Vision EE 837 Fall 2014
Averaging is one way to get rid of noise in any pixel
(-10+0+20) + (-10+0+20) + (-10+0+20) = 30
00000
00000
00000
00000
00000
yI
2020201010
2020201010
2020201010
2020201010
2020201010
I
Derivatives of images
NUST-PNEC Computer Vision EE 837 Fall 2014
2
2
2)(
x
exg
( ) .011 .13 .6 1 .6 .13 .011g x
2 2
2
( )
2( , )
x y
g x y e
Gaussian Filter
NUST-PNEC Computer Vision EE 837 Fall 2014
1
x=-3 x=-2 x=-1 x=0 x=1 x=2 x=3
• Most common natural model
• Smooth function, it has infinite number of derivatives
• Fourier Transform of Gaussian is Gaussian
• Convolution of a Gaussian with itself is a Gaussian
• There are cells in eye that perform Gaussian filtering
Properties of Gaussian
NUST-PNEC Computer Vision EE 837 Fall 2014
• Mean
n
I
n
IIII
n
i
i
n
121
n
Iw
n
IwIwIwI
n
i
ii
nn
12211
Averages
• Weighted mean
NUST-PNEC Computer Vision EE 837 Fall 2014
Image filtering
NUST-PNEC Computer Vision EE 837 Fall 2014
• Modify pixels in an image based on some function of a local neighborhood ofthe pixels
10 5 3
4 5 1
1 1 7
Some function
Local image data Modified image data
7
pf
• Replace each pixel by a linear combination of its neighbors
• We don’t want to only do this at a single pixel, of course, but want instead to “run the kernel over the whole image”.
1 1 2 2 3 3
4 4 5 5 6 6
7 7 8 8 9 9
f h f h f h f h
f h f h f h
f h f h f h
( , ) ( , )k l
f h f k l h i k j l
Correlation
NUST-PNEC Computer Vision EE 837 Fall 2014
f1 f2 f3
f4 f5 f6
f7 f8 f9
f = imageh = kernel/filter
h1 h2 h3
h4 h5 h6
h7 h8 h9
1 9 2 8 3 7
4 6 5 5 6 4
7 3 8 2 9 1
*f h f h f h f h
f h f h f h
f h f h f h
* ( , ) ( , )k l
f h f k l h i k j l
Convolution
NUST-PNEC Computer Vision EE 837 Fall 2014
f1 f2 f3
f4 f5 f6
f7 f8 f9
f = imageh = kernel/filter
h9 h8 h7
h6 h5 h4
h3 h2 h1
h1 h2 h3
h4 h5 h6
h7 h8 h9
h7 h8 h9
h4 h5 h6
h1 h2 h3
X-flip
Y-flip
)1,1()1,1()1,0()1,()1,1()1,1(
)0,1(),1()0,0(),()0,1(),1(
)1,1()1,1()1,0()1,()1,1()1,1(
hyxfhyxfhyxf
hyxfhyxfhyxf
hyxfhyxfhyxfhf
x
f
x
h
1
1
1
1
),(),(i j
jihiyixfhf
Convolution
NUST-PNEC Computer Vision EE 837 Fall 2014
-1,0 0,1 1,1
-1,0 0,0 1,0
-1,-1 0,-1 1,-1
Coordinates
*
2 2 2 3
2 1 3 3
2 2 1 2
1 3 2 2
Convolution example
NUST-PNEC Computer Vision EE 837 Fall 2014
1 -1 -1
1 2 -1
1 1 1
Apply
h? ? ? ?
? ? ? ?
? ? ? ?
? ? ? ?
1 1 1
-1 2 1
-1 -1 1
Rotate
f f * h
2 2 2 3
2 1 3 3
2 2 1 2
1 3 2 2
1 1 1
-1 2 1
-1 -1 1
Convolution example
NUST-PNEC Computer Vision EE 837 Fall 2014
1 -1 -1
1 2 -1
1 1 1
Apply
h5 ? ? ?
? ? ? ?
? ? ? ?
? ? ? ?
1 1 1
-1 2 1
-1 -1 1
Rotate
f
f * h
2*2 + 1*2 + (-1)*2 + 1*1 = 5
2 2 2 3
2 1 3 3
2 2 1 2
1 3 2 2
1 1 1
-1 2 1
-1 -1 1
Convolution example
NUST-PNEC Computer Vision EE 837 Fall 2014
1 -1 -1
1 2 -1
1 1 1
Apply
h5 4 ? ?
? ? ? ?
? ? ? ?
? ? ? ?
1 1 1
-1 2 1
-1 -1 1
Rotate
f
f * h
2*2+1*2+(-1)*2+1*1 = 5-1*2+2*2+1*2-1*2-1*1+1*3= 4
2 2 2 3
2 1 3 3
2 2 1 2
1 3 2 2
1 1 1
-1 2 1
-1 -1 1
Convolution example
NUST-PNEC Computer Vision EE 837 Fall 2014
1 -1 -1
1 2 -1
1 1 1
Apply
h5 4 4 ?
? ? ? ?
? ? ? ?
? ? ? ?
1 1 1
-1 2 1
-1 -1 1
Rotate
f
f * h
2*2+1*2+(-1)*2+1*1 = 5-1*2+2*2+1*2-1*2-1*1+1*3= 4-1*2+2*2+1*3-1*1-1*3+1*3= 4
2 2 2 3
2 1 3 3
2 2 1 2
1 3 2 2
1 1 1
-1 2 1
-1 -1 1
Convolution example
NUST-PNEC Computer Vision EE 837 Fall 2014
1 -1 -1
1 2 -1
1 1 1
Apply
h5 4 4 -2
? ? ? ?
? ? ? ?
? ? ? ?
1 1 1
-1 2 1
-1 -1 1
Rotate
f
f * h
2*2+1*2+(-1)*2+1*1 = 5-1*2+2*2+1*2-1*2-1*1+1*3= 4-1*2+2*2+1*3-1*1-1*3+1*3= 4-1*2+2*3-1*3-1*3 = -2
2 2 2 3
2 1 3 3
2 2 1 2
1 3 2 2
1 1 1
-1 2 1
-1 -1 1
Convolution example
NUST-PNEC Computer Vision EE 837 Fall 2014
1 -1 -1
1 2 -1
1 1 1
Apply
h5 4 4 -2
9 ? ? ?
? ? ? ?
? ? ? ?
1 1 1
-1 2 1
-1 -1 1
Rotate
f
f * h
2*2+1*2+(-1)*2+1*1 = 5-1*2+2*2+1*2-1*2-1*1+1*3= 4-1*2+2*2+1*3-1*1-1*3+1*3= 4-1*2+2*3-1*3-1*3 = -21*2+1*2+2*2+1*1-1*2+1*2= 9
Convolution example
NUST-PNEC Computer Vision EE 837 Fall 2014
And so on ……..
Boarder handling
NUST-PNEC Computer Vision EE 837 Fall 2014
for interior pixels wherethere is full overlap, we know what to do.
but what values do weuse for pixels that are “off the image” ?
Boarder handling
NUST-PNEC Computer Vision EE 837 Fall 2014
• One of the simplest methods is zero-padding
Boarder handling
NUST-PNEC Computer Vision EE 837 Fall 2014
• Replication – replace each off-image pixel with the value from the nearest pixel that is in the image.
Boarder handling
NUST-PNEC Computer Vision EE 837 Fall 2014
• Reflection – reflect pixel values at the border (as if there was a little mirror there)
Boarder handling
NUST-PNEC Computer Vision EE 837 Fall 2014
• Wraparound – when going off the right border of the image, you wrap around to the left border. Similarly, when leaving the bottom of the image you reenter at the top. Basically, the image is a big donut
0 0 0
0 1 0
0 0 0
*
NUST-PNEC Computer Vision EE 837 Fall 2014
Filtering Examples
No effect
0 0 0
0 0 1
0 0 0
*
NUST-PNEC Computer Vision EE 837 Fall 2014
Filtering Examples
1 1 1
1 1 1
1 1 19
1*
NUST-PNEC Computer Vision EE 837 Fall 2014
Filtering Examples
Averaging
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
25
1*
NUST-PNEC Computer Vision EE 837 Fall 2014
Filtering Examples
Original
Original
Pixel offset
Pixel offset
Filtered
Filtered
0
0
8
8
4
0.3
0.3
2.4
8
4
6
4.8
NUST-PNEC Computer Vision EE 837 Fall 2014
Blurring Examples
*
NUST-PNEC Computer Vision EE 837 Fall 2014
Filtering Gaussian
Gaussian Smoothing Smoothing by Averaging
NUST-PNEC Computer Vision EE 837 Fall 2014
Gaussian vs Basic Averaging
Weighted averaging Constant weight
Gaussian Noise After Gaussian SmoothingAfter Averaging
NUST-PNEC Computer Vision EE 837 Fall 2014
Noise Filtering
Salt & Pepper Noise After Gaussian SmoothingAfter Averaging
NUST-PNEC Computer Vision EE 837 Fall 2014
Noise Filtering
Does not look anything like what we have seen
Magnitude of the FT
Images in Fourier domain
NUST-PNEC Computer Vision EE 837 Fall 2014
Images in Fourier domain
Does not look anything like what we have seen
Magnitude of the FT
NUST-PNEC Computer Vision EE 837 Fall 2014
Conv. is Mult. in Fourier Domain
*
f(x,y)
h(x,y)
g(x,y)
|F(sx,sy)|
|H(sx,sy)|
|G(sx,sy)|
NUST-PNEC Computer Vision EE 837 Fall 2014
Original image FFT of original image Low-pass filter
Low-pass image FFT of low-pass image
Low-pass Filtering
Let the low frequencies pass and eliminating the high frequencies.
Generates image with overallshading, but not much detail
NUST-PNEC Computer Vision EE 837 Fall 2014
Original image FFT of original image High-pass filter
High-pass image FFT of high-pass image
High-pass Filtering
Lets through the high frequencies (the detail), but eliminates the low frequencies (the overall shape). It acts like an edge enhancer.
NUST-PNEC Computer Vision EE 837 Fall 2014
Boosting High Frequencies
Original image FFT of original image High-boost filter
High boosted image FFT of high-boosted image
NUST-PNEC Computer Vision EE 837 Fall 2014
NUST-PNEC Computer Vision EE 837 Fall 2014
Most information at low frequencies!
Fun with Fourier Spectra
NUST-PNEC Computer Vision EE 837 Fall 2014
• conv: 1-D Convolution
C = conv(A, B) convolves vectors A and B
• conv2: Two dimensional convolution
C = conv2(A, B) performs the 2-D convolution
of matrices A and B.
Some basic MATLAB functions
NUST-PNEC Computer Vision EE 837 Fall 2014
• filter2: Two-dimensional digital filter
Y = filter2(B,X) filters the data in X with the 2-D
FIR filter in the matrix B
The result, Y, is computed using 2-D correlation and is
the same size as X
filter2 uses CONV2 to do most of the work. 2-D correlation
is related to 2-D convolution by a 180 degree rotation of
the filter matrix
Some basic MATLAB functions
NUST-PNEC Computer Vision EE 837 Fall 2014
• gradient: Approximate gradient
[FX,FY] = gradient(F) returns the numerical gradient of the matrix F. FX corresponds to dF/dx, FY corresponds to dF/dy
• mean: Average or mean value
For vectors, mean(X) is the mean value (average) of the elements in X
Some basic MATLAB functions
NUST-PNEC Computer Vision EE 837 Fall 2014
• special: Create predefined 2-D filters
H = fspecial(TYPE) creates a two-dimensional filter H of the specified type. Possible values for TYPE are:
'average' averaging filter;
'gaussian' Gaussian low-pass filter
'laplacian’ filter approximating the 2-D Laplacian operator
'log' Laplacian of Gaussian filter
'prewitt' Prewitt horizontal edge-emphasizing filter
'sobel' Sobel horizontal edge-emphasizing filter
• Example: H=fspecial('gaussian',7,1)
Creates a 7x7 Gaussian filter with variance 1
Some basic MATLAB functions
NUST-PNEC Computer Vision EE 837 Fall 2014
???
Final project ideas!!!
NUST-PNEC Computer Vision EE 837 Fall 2014
• I will upload Matlab tutorial on Yahoo group
• Please take a look if you are unfamiliar with Matlab or the image toolbox
• I will upload Homework 1. Due date will be mentioned on it
Yahoo Group update and homework
NUST-PNEC Computer Vision EE 837 Fall 2014