edge detection · image segmentation: edge detection 10 a continuous approach for edge detection:...

Image Segmentation: Edge Detection 1

EDGE DETECTION

1. Edge model

2. Classical approaches

3. Analytical approaches

4. Snakes

5. Edge representation


des formes

contours

zones

textures

mesuresreconnaissance

analyse de scènes

segmentation


Détection de contours

Fermeture de contours

Détection de zones homogènes

Obtentation de contours

Formes bi-dimensionnelles

Paramétrisation

Reconnaissance des formes Classification Analyse de scène


What is an edge ?


An image as a surface


What is an edge ?

rampemarche d’escalier toit


A continuous approach for edge detection: the gradient

The image is supposed to be a continuous surface: i(x, y)

The gradient of i(x, y):−→G =

−→∇i =

∂i∂x

∂i∂y

G = |

−→∇i| =

[(∂i∂x

)2+

(∂i∂y

)2]1/2

and −→g =−→∇i

|−→∇i|

egde = maxima of the gradient in the direction of the gradient

∂G∂g = 0 and ∂2G

∂g2 < 0 with ∂∂g = −→g .

−→∇

∂i∂x .

∂∂x

√(∂i∂x

)2+

(∂i∂y

)2

+ ∂i∂y .

∂∂y

√(∂i∂x

)2+

(∂i∂y

)2

= 0


A continuous approach for edge detection: the gradient

Derivative approaches: discrete approximation

1 1

-1

-1

-1

-1

1

2

1

-1

-2

-1

gradient Roberts Prewitt Sobel

1

1

1-1

Norm approximation: G = | ∂i∂x | + | ∂i

∂y |.

Pre-processing: filtering (median), ...

Post-processing: thresholding, local maxima of the gradient, hysteresis, following, closing


A continuous approach for edge detection: the laplaciany

g

t

x

Φ

Edge: ∂2i∂g2 = 0 ⇔ ∂2i

∂x2 cos2 Φ + 2. ∂2i∂x∂y cos Φ sinΦ + ∂2i

∂y2 sin2 Φ = 0

Laplacian: ∆i = ∂2i∂g2 + ∂2i

∂t2

if ∂2i∂t2 ≈ 0 then ∂2i

∂g2 = 0 ⇔ ∆i = 0

∂2i∂t2 ≈ 0 ⇒ low curvature

Isotropy: ∆i = ∂2i∂x2 + ∂2i

∂y2


A continuous approach for edge detection: the laplacian

Interest: closed edges (separation positive / negative values)

Very sensitive to noise: need for a strong filtering

edge =(∂2

∂x2 +∂2

∂y2

)[low pass filter ∗ image ]

edge =[ (

∂2

∂x2+

∂2

∂y2

)low pass filter

]∗ image

g(x, y) =1√2πσ

exp(− (x2 + y2)2σ2

)

∆g =1√

2πσ3((x2 + y2)

2σ2− 1) exp(− (x2 + y2)

2σ2)


gaussian laplacian of gaussian


A continuous approach for edge detection: the laplacian

• laplacian of gaussian (LOG) : human visual system (Marr 80)

• difference of gaussians (DOG) σi

σe=1.6

• DOG approximated with DOB (difference of boxes)

filtre Log

filtre Dog


Ameliorations

• DOG: separable filter

DOG(x, y) = DOG(x)DOG(y)

complexity: N2 ⇒ 2N

• Huertas and Medioni 86

DOG(x, y) = H1(x)H2(y) +H1(y)H2(x)

with H1(x) = (1− x2

σ2 )exp(− x2

2σ2 ) and H2(x) = exp(− x2

2σ2 )

RIF :

H1 = [−1− 6− 17− 17 18 46 18− 17− 17− 6− 1]

H2 = [0 1 5 17 36 46 36 17 5 1 0]


Analytical approach: “optimal” edge

Canny 83: 1D step edges, linear filtering, gaussian white noise

I(x) = AU(x) + n(x)

O(xo) =∫ +∞−∞ I(x)f(xo − x)dx

f(x) ? (with f(x) = −f(−x) -derivative operator-)

• a good detection: Σ =∫∞0 f(x)dx√∫∞−∞ f2(x)dx

• a good localisation: Λ = |f ′(0)|√∫∞−∞ f ′2(x)dx

• a unique response:|f ′(0)|√∫∞

−∞ f ′′2(x)dx= k

∫ 0−∞ f(x)dx√∫∞−∞ f2(x)dx


Resolution

Maximise ΣΛ

• Euler-Lagrage equation:∫ b

aψ(x, f, f ′, f ′′)dx =

∫ b

af2 + λ1f

′2 + λ2f′′2 + λ3fdx

⇒ 2f(x)− 2λ1f′′(x) + 2λ2f

′′′′(x) + λ3 = 0

• general solution:

f(x) = a1ex/σ sinωx+ a2e

x/σ cosωx+ a3e−x/σ sinωx+ a4e

−x/σ cosωx+ .

λ2 − λ214 > 0; 1

σ2 − ω2 = λ12λ2

; 4ω2

σ2 = λ21−4λ2

4λ22

• FIR filter on [-M,M]: f(0) = 0 f(M) = 0 f ′(0) = S f ′(M) = 0

• performances: ΣΛ = 1.12


Analytical approaches around Canny

• Approximation: derivative of a gaussian

f(x) ≈ −x. exp(− x2

2σ2

)performance: ΣΛ = 0.92 (IIR)

max response to the filter = zeros of the second derivative (Marr)

• Shen et Castan (86)

maximisation of Σ

f(x) = csign(x) exp(−α|x|)

discontinuity at zero

better localisation / more sensitive to noire


Analytical approaches around Canny

Deriche 1987

• IIR with conditions at the limits

f(0) = 0; f(+∞) = 0; f ′(0) = S; f ′(+∞) = 0;

• optimal solution

f(x) = −cx exp(−α|x|)cos(ωx)

• performance: Λ =√

2α ; Σ =√

2αα2+ω2

• optimal parameters:

ω → 0 ⇒ ΣΛ = 2f(x) = −cx exp(−α|x|)

c = − [1−exp(−α)]2

exp(−α)

• link with the gaussian: α =√

πσ


Deriche operator: implementation

• Z transform

h(x) = cx exp(−α|x|) H(z) =∑+∞

n=∞ h[n]z−n

h[n] = h−[n] + h+[n]

H−(z) = ce−αz−1

1−2e−αz−1+e−2αz−2 and H+(z) = −ce−αz1−2e−αz+e−2αz2

second order recursive filters (stable)

• condition at the origine: B[0] = 1 ⇒ c = − (1−e−α)2

e−α

• finite difference equations

B1[n] = ce−αA[n− 1] + 2e−αB1[n− 1]− e−2αB1[n− 2]

B2[n] = −ce−αA[n+ 1] + 2e−αB2[n+ 1]− e−2αB2[n+ 2]

B[n] = B1[n] +B2[n]

NB : independant of α


−10 −8 −6 −4 −2 0 2 4 6 8 10−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

Gaussian / Deriche

−10 −8 −6 −4 −2 0 2 4 6 8 10−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

Deriche / Shen & Castan


2D implementation

calcul of the gradient⇒ appplication of a derivative operator

• f(x) : “optimal” derivative operator f

• g(x) : integral of f (smoothing)

g(x) = b(α|x|+ 1)e−alpha|x|

• 2D expression

∂i

∂x→ [f(x)g(y)] ∗ i(x, y) = f(x) ∗ g(y) ∗ i(x, y)

∂i

∂y→ [f(y)g(x)] ∗ i(x, y) = f(y) ∗ g(x) ∗ i(x, y)


−10 −8 −6 −4 −2 0 2 4 6 8 10−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

f(x)

−10 −8 −6 −4 −2 0 2 4 6 8 100

0.05

0.1

0.15

0.2

0.25

g(x)


2D Deriche’s filter


Snakes or active contours

Kass, Witkins and Terzopoulos 1988

Evolution of a curve under internal and external constraints: v(s) = [x(s) , y(s)]t

where s is the curviligne abscisse: s ∈ [0, 1]

The global energy takes the form:

Etotal =∫ 1

0[Einternal(v(s)) + Eimage(v(s)) + Eext(v(s))] ds

with

Einterne = α(s)(

dvds

)2+ β(s)

(d2vds2

)2

where α(s) is the tension and β(s) represents the curvature or the elasticity of the curve.


Resolution

• Euler-Lagrange Equation

−(αv′)′(s) + (βv′′)′′(s) +∇P (v) = 0

P (v) = Eimage(v) + Eext(v)

F (v) = −∇P (v)

( + conditions at the limits)

• Discretisation with the finite differences

V t = [vt0, v

t1, v

t2, ......, v

tn−1]

t

(step h, α, β cst)βh2 vi+2 − (α

h + 4 βh2 )vi+1 + ( 2α

h + 4 6βh2 )vi − (α

h + 4 βh2 )vi−1 + β

h2 vi−2 = F (vi)


Resolution

Evolution of the curve with the time

γ∂v

∂t− αv′′ + βv′′′′ = F (v)

v(t+ 1) = (τA+ I)−1(τF (v(t)) + v(t))

• initialisation

• choice of τ

• matrix inversion (circular, Toplitz or not)

• discretisation step constant

• stop criterium


Resolution

2α+6β β 0−α−4β

−α−4β 2α+6β β−α−4β

0 β −α−4β 2α+6β

0 β −α−4β 2α+6β

0

0

β

−α−4β

... ... ...

... ...

...

...

...

0

β −α−4β

0 β −α−4β

2α+6β −α−4β

2α+6β

−α−4β β 0

β −α−4β

closed snake: NxN


Resolution

-2

-2

β −α−4β 2α+6β

0

β

−α−4β

0

... ...

... ...

...

...

0

...

β −α−4β

0 β

...

2α+6β −α−4β

2α+6β

0

0 0

0 0

β β 0β

0

β

−α−4β

β β

−α−4β β2α+5β−α−2β

−α−2β2α+5β−α−4ββ

snake with free extremities: NxN, v′′(0) = v′′′(0) = 0


Resolution

-2

2α+6β β−α−4β

0 β −α−4β 2α+6β

−α−4β

0

0

β

−α−4β

... ... ...

...

...

...

...

0

β −α−4β

0 β −α−4β

2α+6β −α−4β

2α+6β

0

0 0

0 0

β 0

β...

β β

2α+6β −α−4β

snake with fixed extremities: (N-2)x(N-2)


Choice of the constraintes

• linked to the features we are looking at: F (v) = ∇P (v)

– intensity: P (v) = I(v)

– edge: P (v) = −|∇I(v)|2 (or Canny-Deriche)

– edge: P (v) = (∆(I ∗Gσ))2 (0 of the laplacian)

• minimisation with dynamique programming (Amini)

positions à l’instant t snake à t

snake à t+1positions possibles à t+1

M

N

P

1

2

3

4

1

2

3

4

1

2

3

4

1

3

4

M N P Q

0

1

2

3

4Q


Ameliorations

Balloon strength, Cohen 1991

Problems with classical snakes:

• bad initialisation: no attraction

• no constraints: retraction of the curve

F (v(s)) = k1−→n (s)− k ∇P

|∇P |−→n (s) normal vector at s

(k > k1 edge > pression strength, τk = pixel size)

initialisation : interior ou exterior of the object

(no need to be close to the object)


Ameliorations Gradient Vector Flow, Xu and Prince 1997

• Goals:

– independant to the initialisation

– convergence to concave regions

• Diffusion of the gradient all over the image

minimisation of:

E =∫ ∫

µ(u2x + u2

y + v2x + v2

y) + |F (x, y)|2|−→v − F (x, y|2dxdy

• possibility to initialise through the object


Classical snakes

Final result, iter = 500

0 10 20 30 40 50 60 70

10

20

30

40

50

60


Classical snakes


Gradient Vector Flow


Edge representation

• list of the cartesian coordinates: (xi, yi), i = 1...n

• Freeman’s code

– cartesian coordinates of the first point

– list of the displacements

04

2 13

5 6 7

A B

6 6 7 0 1 7 6 0 1 0 2 4 3 1


• Relative Freeman’s code

– first point + first displacement

– list of the direction changes

54 6 7

0 1 2 3 0 1 1 1 6 7 2 1 7 2 2 7 6


Freeman’s code

0

1

2

3

0

123

4

5 6 7

3

91011

12

13

6 75

14

4

12

80

15

4 directions : 2 bits 8 directions : 3 bits 16 directions : 4 bits

1 1 1 0 1 0 0 0 0 0 0 3 0 3 0 3 0 3 3 2 3 2 3 0 3 0 0 0 0 0 1 0 1 0

2 2 1 1 0 0 0 0 7 7 7 7 6 5 4 6 7 0 0 0 0 1 0 1

10 9 8 8 15 15 6 5 4 6 7 8 8 1 0 1 16 x 4 = 64 bits

24 x 3 = 72 bits

34 x 2 = 68 bits


Edge representation

Signature

• centre of gravity:origin of polar coordinates

• list of the distances to the origin according to the angle: ρ(θ)

• invariant to translations

• detection of small defects

• rotation = translation along θ


Signature - examples

R

edge detection · image segmentation: edge detection 10 a continuous approach for edge detection:...

Documents