2.3 Feature extraction

Many computer vision systems rely on first the

detection of some features in the images.

How many peppers in this image?

Detect boundaries of peppers

How many red peppers, and how

many green peppers in this

image?

Detect colors of peppers

2.3.1 Edge detection

Edges are pixels at or around which the image

values undergo a sharp variation.

However, noise can also cause intensity variations.

A good edge detector should suppress image noise,

enhance and locate true edge pixels.

Types of edge

step edge ramp edge

line edge roof edge

Real edge may have multiple types superimposed with

noise, e.g. ramp + line + noise.

Edge descriptor – result of edge detection

edge normaledge direction

edge position - coordinates

edge direction - vector

edge normal - vector

edge strength – numeric value

Edge detection methods usually rely on calculations

of the first or second derivative along the intensity

profile.

I

I’

I”

local peak

zero crossing

edge pixel

First derivative operator

∂

∂∂

∂

=

=

y

Ix

I

G

G))y,x(I(G

y

xCalculate image gradient

2

y

2

x GG))y,x(I(G +=magnitude

yx GG))y,x(I(G += )G,Gmax())y,x(I(G yx=

approximation

or

=∠

−

x

y1

G

Gtan))y,x(I(G

direction (with respect to x axis)

Threshold the gradient magnitudes to locate edges.

]j,1i[I]j,i[IG

]j,i[I]1j,i[IG

y

x

+−≅

−+≅

Discrete approximation

-1

-1

1 1Gx = Gy =

-1 1 -1

1

convolution masks

++

2

1j,

2

1i

The gradient is measured

at coordinates

To avoid having the gradient calculated about an

interpolated point between pixels, you can use a 3 x 3

neighborhood, e.g. Sobel edge detector.

2

y

2

x SS))y,x(I(S +=magnitude

-1

0

0 1Sx = Sy =

-2 0 0

2

-1 0

1

2

1 -1 -2

0

1

-1

convolution masksThe gradient is

measured at coordinates [i, j]

function sobeled(t)

% to perform Sobel edge detection

% threshold is t

[f_name, f_path] = uigetfile('*.bmp', 'Select an input image');

in_name = [f_path, f_name];

image = imread(in_name);

[height, width] = size(image);

imshow(image)

h = fspecial('sobel');

sx = imfilter(image, h);

v = h';

sy = imfilter(image, v);

for i = 1:height

for j = 1:width

mag(i,j)=sqrt(double(sx(i,j))^2+double(sy(i,j))^2);

if mag(i,j) > t

sedimage(i,j) = 255;

else

sedimage(i,j) = 0;

end

end

end

figure

imshow(sedimage)

% save edge detection result

out_name = [f_path, 'sed_', f_name];

imwrite(sedimage, out_name, 'BMP');

end

calculate Sx

calculate Sy

calculate magnitude

thresholding

Original Threshold = 100

Do you see any defects in the Sobel edge

detection result?

The errors in edge detection are false edges (false

positive FP) and missing edges (false negative FN).

Can you design other Sobel convolution masks for

measuring gradient in other directions?

Second derivative operator

The zero crossings can be

located by the Laplacian

operator 2

2

2

22

y

I

x

I)y,x(I

∂

∂+

∂

∂=∇

]j,1i[I]j,i[I2]j,1i[Iy

I

]1j,i[I]j,i[I2]1j,i[Ix

I

2

2

2

2

−+−+=∂

∂

−+−+=∂

∂

Discrete approximation0 1∇2 ≈

1 -4

0 1

0

1

0

convolution mask

The actual edge location must be determined by

interpolation.

8888822222

8888822222

8888822222

8888822222

8888822222

8888822222

Image with a

vertical step

edge

000-66000

000-66000

000-66000

000-66000

Result of the

Laplacian

operation

Edge operator involving two derivatives is affected by

noise more than an operator involving a single

derivative.

A better approach is to combine Gaussian filtering with

the second derivative – Laplacian of Gaussian (LoG).

• filter out the image noise using Gaussian filter

• enhance the edge pixels using 2D Laplacian

operator

• edge is detected when there is a zero crossing in

the second derivative with a corresponding large

peak in the first derivative

• estimate the edge location with sub-pixel

resolution using linear interpolation

Original 5 x 5 LoG mask, σ = 0.5

LoG edge detector can be implemented using the

MATLAB function edge.

An edge detector can reduce noise by smoothing the

image, but this will result in spreading of edges and

add uncertainty to the location of edge.

I

I’

I

”

An edge detector can have greater sensitivity to the

presence of edges, but this will also increase the

sensitivity of the detector to noise.

original image

Gaussian smoothed image

edge detection resultthreshold

thick edge

The best compromise between noise immunity and

edge localization is the first derivative of a Gaussian –

e.g. Canny edge detector.

• edge enhancement

• non-maximum suppression

• hysteresis thresholding

edge enhancement:

Apply Gaussian smoothing to the image.

]j,i[I];j,i[G]j,i[S ∗σ=

G is a Gaussian with zero mean and standard deviation σ

Compute the gradient of the smoothed image and

estimate the magnitude and orientation of the gradient.

2

])1j,1i[S]1j,i[S]j,1i[S]j,i[S(]j,i[Q

2

])j,1i[S]1j,1i[S]j,i[S]1j,i[S(]j,i[P

++−+++−=

+−+++−+=

22 ]j,i[Q]j,i[P]j,i[M +=

=θ

−

]j,i[P

]j,i[Qtan]j,i[ 1

non-maximum suppression:

M[i, j] may contain wide ridges around the local

maximum. This step is to thin such ridges to produce 1-

pixel wide edges. Values of M[i, j] along the edge

normal that are not peak will be suppressed.

Quantize edge normal orientations into 4, e.g. 0°, 45°,

90°, and 135° with respect to the horizontal axis. For

each pixel (i, j), find the orientation which best

approximates θ[i, j].

If M[i, j] is smaller than at least one of its two neighbors

along the quantized orientation, set M’[i, j] to zero.

Otherwise M’[i, j] = M[i, j].

hysteresis thresholding:

M’[i, j], after the non-maximum suppression step, may

still contain local maxima created by noise. To get rid of

false edges by thresholding, some false edges may still

remain if the threshold value is set too low, or some true

edges may be deleted if the threshold value is set too

high.

An effective scheme is to use 2 thresholds τl and τh,

e.g. τh = 2τl.

Scan the non-zero points of M’[i, j] in a fixed order.

If M’[i, j] is larger than τh, locate it as an edge pixel.

Else if any 8-neighbors of (i, j) have gradient

magnitude larger than τl, locate them as edge pixels.

Continue until another edge pixel is located by τh.

Original Thresholds [0.06 0.15], σ = 0.5

Canny edge detector can be implemented using the

MATLAB function edge.

2.3.2 Corner detection

Corners are quite stable across sequences of images.

They are interesting features that can be employed for

tracking objects across sequences of images.

Consider an image point p, a neighborhood Q of p, and

a matrix C defined as

=∑∑

∑∑

Q

2

y

Q

yx

Q

yx

Q

2

x

GGG

GGG

C

We can think of C as a diagonal matrix

λ

λ=

2

1

0

0C

The two eigenvalues λ1 and λ2 are non-negative. A

corner is located in Q where λ1 ≥ λ2 > 0 and λ2 is

large enough.

• compute the image gradient

• for each image point p, form matrix C over a

neighborhood Q of p, compute the smaller

eigenvalue λ2 of C, if λ2 > threshold τ save the

coordinates of p into a list L

• sort L in decreasing order of λ2

• scan L from top to bottom, for each corner point p,

delete neighboring corner points in L

Summary

♦ feature extraction – edge detection, corner

detection

♦ first derivative edge detector

♦ second derivative edge detector

♦ Canny edge detector