CS440 / ECE 448 – Fall 2006Lecture #2

CS 440 / ECE 448Introduction to Artificial Intelligence

Fall 2006

Instructor: Eyal Amir

TAs: Deepak Ramachandran (head TA), Jaesik Choi

CS440 / ECE 448 – Fall 2006Lecture #2

Today: Computer Vision

• Vision tasks– Given an image or a set of images formed by

a camera or another sensor (e.g., laser)– Identify objects– Identify scene– Identify actions– Localize the above

• Vision seems deceptively easy

CS440 / ECE 448 – Fall 2006Lecture #2

Face

Camel

BugHuman/Felix

Joe

BarbaraSteele

Problem:

Recognizing instances

Recognizing categories

CS440 / ECE 448 – Fall 2006Lecture #2

CS440 / ECE 448 – Fall 2006Lecture #2

Variability: Camera positionIlluminationInternal parameters

Within-class variations

CS440 / ECE 448 – Fall 2006Lecture #2

Object Recognition

• How to detect the class of object and its identity mechanically?– Canonical, invariant representation? Eliminate

most of the parameters that govern appearance?

– Work with the raw pixels?

• How do we use prior knowledge about the domain?– Scale, location, possible objects, texture

CS440 / ECE 448 – Fall 2006Lecture #2

Processing an Image

Image:array

of pixels

Features:lines, shapes,invariants, colors, positions,histograms

Transformationto real-world, 3Dcoordinates

Segmentation,aggregation

matching againstprototype/class

Localizing

CS440 / ECE 448 – Fall 2006Lecture #2

Today: Detecting Features

Image:array

of pixels

Features:lines, shapes,invariants, colors, positions,histograms

Segmentation,aggregation

matching againstprototype/class

CS440 / ECE 448 – Fall 2006Lecture #2

Detecting Local Features

• Slide a window over the image

• Detect– Line– Edge / Edgel– Corners– Direction of the above

CS440 / ECE 448 – Fall 2006Lecture #2

Edge detection

• Convert a 2D image into a set of curves– Extracts salient features of the scene– More compact than pixels

CS440 / ECE 448 – Fall 2006Lecture #2

Edge is Where Change Occurs

• Change is measured by derivative in 1D

• Biggest change, derivative has maximum magnitude

• Or 2nd derivative is zero.

CS440 / ECE 448 – Fall 2006Lecture #2

Image gradient• The gradient of an image: • The gradient points in the direction of most rapid change

in intensity

• The gradient direction is given by:

– how does this relate to the direction of the edge?

• The edge strength is given by the gradient magnitude

CS440 / ECE 448 – Fall 2006Lecture #2

The discrete gradient• How can we differentiate a digital image

f[x,y]?– take discrete derivative (finite difference)

• How would you implement this as a cross-correlation?

CS440 / ECE 448 – Fall 2006Lecture #2

The Sobel operator• Better approximations of the derivatives

– The Sobel operators below are very commonly used

-1 0 1

-2 0 2

-1 0 1

1 2 1

0 0 0

-1 -2 -1

– The standard defn. of the Sobel operator omits the 1/8 term• doesn’t make a difference for edge detection• the 1/8 term is needed to get the right gradient value, however

CS440 / ECE 448 – Fall 2006Lecture #2

Gradient operators

(a): Roberts’ cross operator (b): 3x3 Prewitt operator(c): Sobel operator (d) 4x4 Prewitt operator

CS440 / ECE 448 – Fall 2006Lecture #2

Effects of noise• Consider a single row or column of the image

– Plotting intensity as a function of position gives a signal

• Where is the edge?

CS440 / ECE 448 – Fall 2006Lecture #2

Where is the edge?

Solution: smooth first

Peaks in

CS440 / ECE 448 – Fall 2006Lecture #2

Derivative theorem of convolution

• This saves us one operation:

CS440 / ECE 448 – Fall 2006Lecture #2

Laplacian of Gaussian

Laplacian of Gaussianoperator

• Where is the edge? • Zero-crossings of bottom graph

CS440 / ECE 448 – Fall 2006Lecture #2

2D edge detection filters

• is the Laplacian operator:

Laplacian of Gaussian

Gaussian derivative of Gaussian

CS440 / ECE 448 – Fall 2006Lecture #2

Exercise

• Write an algorithm for detecting edges

• Congratulations! You now know how to detect edges in computer images!

• … what now?

• A little more edge detection

• Detecting lines and other shapes from edges

CS440 / ECE 448 – Fall 2006Lecture #2

Optimal Edge Detection: Canny

• Assume: – Linear filtering– Additive iid Gaussian noise

• Edge detector should have:– Good Detection. Filter responds to edge,

not noise.– Good Localization: detected edge near true

edge.– Single Response: one per edge.

CS440 / ECE 448 – Fall 2006Lecture #2

Optimal Edge Detection: Canny (continued)

• Optimal Detector is approximately Derivative of Gaussian.

• Detection/Localization trade-off– More smoothing improves detection– And hurts localization.

• This is what you might guess from (detect change) + (remove noise)

CS440 / ECE 448 – Fall 2006Lecture #2

The Canny edge detector

• original image

CS440 / ECE 448 – Fall 2006Lecture #2

The Canny edge detector

• norm of the gradient

CS440 / ECE 448 – Fall 2006Lecture #2

The Canny edge detector

• thresholding

CS440 / ECE 448 – Fall 2006Lecture #2

The Canny edge detector

• Thinning (non-maximum suppression)

CS440 / ECE 448 – Fall 2006Lecture #2

Non-maximum suppression

• Check if pixel is local maximum along gradient direction– requires checking interpolated pixels p and r

CS440 / ECE 448 – Fall 2006Lecture #2

Predictingthe nextedge point

Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s).

(Forsyth & Ponce)

CS440 / ECE 448 – Fall 2006Lecture #2

Hysteresis

• Check that maximum value of gradient value is sufficiently large– drop-outs? use hysteresis

• use a high threshold to start edge curves and a low threshold to continue them.

CS440 / ECE 448 – Fall 2006Lecture #2

Effect of (Gaussian kernel size)

Canny with Canny with original

• The choice of depends on desired behavior– large detects large scale edges– small detects fine features