Edge Detection

1

2

Roadmap

• Introduction to image analysis (computer vision) – Its connection with psychology and neuroscience

– Why is image analysis difficult?

• Theory of edge detection – Gradient operator

– Advanced operators

• Applications – Road/sign detection in intelligent driving systems

– Pupil detection in iris recognition systems

3

Computer Vision: the Grand Challenge

• Teach a computer to see is nontrivial at all

• Unlike binary images, grayscale/color images acquired by the sensor are often easy to understand by human being but difficult for a machine or a robot

• There are lots of interesting problems in the field of computer vision (image analysis)

– Image segmentation, image understanding, face detection/recognition, object tracking …

4

How does Human Vision System work?

Top-down school Bottom-up school

pixels

objects

components (such as

edges, lines etc.)

I see a human body

I expect to see a human face

I expect to see two eyes and a nose

Two hypothesis and nobody knows the answer yet!

5

An Amazing Image Example

Person A:

I see an old man with a fancy

earring and a strange hand

Person B:

I see two people on the street

and a dog lying beside

If you try really hard, you will be able

to locate at least eight different faces

from this image

6

Gestalt Theory (the Berlin School)

Emergence: the dog is perceived as a whole, all at once

7

Reification

8

Multistability (or Multistable Perception)

9

Invariance

10

Application: Face Detection

http://vasc.ri.cmu.edu/demos/faceindex/

You are strongly encouraged to try the interactive demo out yourself

11

Edge Detection

• Why detect edge?

Edges characterize object boundaries and are

useful features for segmentation, registration

and object identification in scenes.

• What is edge (to human vision system)?

Intuitively, edge corresponds to singularities in the image

(i.e. where pixel value experiences abrupt change)

No rigorous definition exists

Gradient operator

Advanced operators

Edge detection • Idea (continuous-space): Detect local gradient

• Digital image:

use finite differences

instead

12

Practical edge detectors • Edges can have any orientation

• Typical edge detection scheme uses K=2 edge templates

• Some use K>2

13

14

Gradient Operators

• Motivation: detect changes

change in the pixel value large gradient

Gradient

operator image

Thresholding

g(m,n)

edge

map f(m,n) I(m,n)

otherwise

thnmgnmI

0

|),(|1),(

MATLAB function: > help edge

𝑔 𝑚, 𝑛 = 𝑓𝑥2 + 𝑓𝑦

2

𝜃𝑔 = tan−1

𝑓𝑦𝑓𝑥

15

Common Operators

Examples: 1. Roberts operator

01

10

g1 g2

10

01

),(),(),( 2

2

2

1 nmgnmgnmg

• Gradient operator

16

Common Operators (cont’d)

2. Prewitt operator 3. Sobel operator

101

101

101

111

000

111

101

202

101

121

000

121

vertical

horizontal

17

Edge detection filters

18

Examples horizontal edge vertical edge

Prewitt operator (th=48)

original image

Sobel operator (th=48)

19

Sobel detector

20

Prewitt detector

21

Magnitude of Sobel at threshold = {450, 400, 300, 200, 100, 50}, |g|=[0,870]

22

Effect of Thresholding Parameters

threshold small large

23

Compass Operators

101

101

101

111

000

111

111

000

111

101

101

101

110

101

011

110

101

011

011

101

110

011

101

110

|}),({|max),( nmgnmg kk

24

Examples

Compass operator (th=48)

(Laplacian operator) Zero-crossings mark edge location

25

Laplacian Operators

• Gradient operator: first-order derivative

sensitive to abrupt change, but not slow change

second-order derivative: 2

2

2

22

y

f

x

ff

02

2

x

flocal extreme in f ’

010

141

010

111

181

111

• Discrete Laplacian operator =>Isotropic (rotationally invariant) operator

a=0 a=0.5

aaa

aa

aaa

a1

141

1

1

1

26

1-d illustration of 2nd derivative edge detector

f f ’ f’’

zero crossing

Laplacian

operator image zero-crossing

edge

map x(m,n) g(m,n) I(m,n)

27

Examples

zero-crossings original image

Question: why is it so sensitive to noise (many false alarms)?

Answer: a sign flip from 0.01 to -0.01 is treated the same as from 100 to -100

Sensitive to very fine detail and noise => blur image first

Responds equally to strong and weak edges

=> suppress zero-crossings with low gradient magnitude

28

Ideas to Improve Robustness

• Linear filtering – Use a Gaussian filter to smooth out noise

component Laplacian of Gaussian

• Spatially-adaptive (Nonlinear) processing – Apply different detection strategies to smooth

areas (low-variance) and non-smooth areas (high-variance) Robust Laplacian edge detector

• Return single response to edges (not multiple edge pixels) – Hysteresis thresholding Canny’s edge detector

29

Laplacian of Gaussian Blurring of image with Gaussian and Laplacian operator

can be combined into convolution with Laplacian of

Gaussian (LoG) operator

)2

exp(])(

1[),(2

22

2

22

nmnmcnmh

Laplacian

operator image edge

map x(m,n) g(m,n) I(m,n)

Gaussian

LPF ()

Pre-filtering: attenuate the noise sensitivity of the Laplacian

30

Laplacian of Gaussian

Continuous function and discrete approximation

𝜎 = 1

0 0 1 2 1 0 00 2 5 5 5 2 01 5 0 − 12 0 5 1

2 5 − 12 − 40 − 12 5 21 5 0 − 12 0 5 10 2 5 5 5 2 00 0 1 2 1 0 0

31

Zero crossings of LoG at difference scale

𝜎 = 0.5

𝜎 = 3 𝜎 = 6

32

Examples

Better than Laplacian alone but still sensitive due to zero crossing

33

Robust Laplacian-based Edge Detector

Laplacian

operator image zero

crossing?

estimate

local variance

2>th

2

not an

edge point

No

yes

No

not an

edge point

edge

point

34

Examples

More robust but return multiple edge pixels (poor localization)

35

Canny Edge Detector

• Low error rate of detection

– Well match human perception results

• Good localization of edges

– The distance between actual edges in an image and the edges found by a computational algorithm should be minimized

• Single response

– The algorithm should not return multiple edges pixels when only a single one exists

1. Smooth image with a Gaussian filter

2. Compute gradient magnitude and angle (Sobel, Prewitt . . .)

𝑔 𝑥, 𝑦 =𝜕𝑓

𝜕𝑥

2

+𝜕𝑓

𝜕𝑦

2

, 𝜃 𝑥, 𝑦 = 𝑡𝑎𝑛−1𝜕𝑓

𝜕𝑦

𝜕𝑓

𝜕𝑥

3. Apply nonmaxima suppression to gradient magnitude image

4. Double thresholding to detect strong and weak edge pixels

5. Reject weak edge pixels not connected with strong edge pixels

36

Canny Edge Detector

37

Flow-chart of Canny Edge Detector (J. Canny’1986)

Original image

Smoothing by Gaussian convolution

Differential operators along x and y axis

Non-maximum suppression

finds peaks in the image gradient

Hysteresis thresholding locates edge strings

Edge map

38

Canny nonmaxima suppression

• Quantize edge normal to one of four directions:

• If g(x,y) is smaller than either of its neighbors in edge normal direction suppress; else keep.

• Double-thresholding of gradient magnitude.

• Typical setting: 𝜃ℎ𝑖𝑔ℎ 𝜃𝑙𝑜𝑤 = 2,3,4, …

• Region labeling of edge pixels • Reject regions without strong edge pixels

39

Canny Edge Detector Example

original image vertical edges horizontal edges

norm of the gradient after thresholding after thinning

40

Marr and Hildreth’s Method*

Edge is scale-dependent

A different edge map can be generated at different scale

• Scale space representation

);,()0;,();,( syxgyxfsyxf

fine-scale

image

coarse-scale

image

Gaussian kernel

with width of s

)2

exp(2

1);,(

22

s

yx

ssyxg

41

Importance of Scale

42

Scale-Space Edge Detection Examples

fine coarse

43

Marr/Hildreth and Canny Edge Detection

44

Marr/Hildreth and Canny Edge Detection