Snakes - Active Contour

By: Saar Arbel

Lecturer: Hagit Hel-Or

What is our objective?

To find a contour that best approximates the perimeter of an object

Its just like segmentation and many other methods...

Subdividing or partitioning an image into its constituent regions or objects

But Wait...

Problem with other methods Not effective in presence of noise and sampling artifacts (e.g. medical images).

This is done with segmentation on a noisy image

This is done with snakes on a noisy image

What is a snake?A framework for drawing an object outline from a possibly noisy 2D image.

An energy-minimizing curve guided by external constraint forces and influenced by image forces that pull it towards features (lines, edges).

Represents an object boundary or some other salient image feature as a parametric curve

Snakes are autonomous and self-adapting in their search for a minimal energy state

So... Why snakes?

They can be easily manipulated using external image forces

They can be used to track dynamic objects in temporal as well as the spatial dimensions

ApplicationsObject tracking

Shape recognition

Segmentation

Edge detection

Segmentation example

Aspects we need to considerRepresentation of the contours

Defining the energy functions- Internal- External

Minimizing the energy function

How is a snake represented?A Snake is represented as a set of points (x,y) that defines a function V(s).

Every snake include:External Energy Function

Internal Energy Function

A set of k points (in the discreet world)or a continuous function that will represent the points

The Energy functionsThe Snake's energy functions measure the appropriateness of the contour

Good solutions correspond to the minimum of the energy functions

Our goal is to minimize this function with respect to the contour parameters

Getting the snake closer to the target

In charge of the snake curvature and smoothness

A brief example:

The Energy functions

Our goal is to find a function V that will minimize E and give us a perfect match

You can think about the energy function like a heuristic function, they are similar because they both lead to a solution in a way that lets the programmer decide what is important and what’s not.

Intuitive tip

Forces the contour to be continuous

Decide if the snake wants to shrink/expand

Forces the contour to be smooth

Internal Energy

Econt Ecurv

Econt

Minimizing the first derivative has two effects:

The snake wants to shrink

Forces the contour to be continuous

The squaring of the differences causes the differences to be as similar as possible (the points are at equal distances from each other)

On the continuous world:

On the discrete world:The contour is approximated by N points P1, P2, . .., Pn and the first derivative is approximated by a finite difference:

Example

The points are equi-distances

The points arenot equi-distances

Ecurv

Forces the contour to be smooth.By taking the second derivative which represent the difference of the differences between 2 consecutive differences.This 2 consecutive differences represent the curvature and we want to penalize if the curvature is too high.

On the continuous world:

On the discrete world:The curvature can be approximated by the following finite difference:

Example: compare curvature

That’s better!

α,β meaning

We can play with α,β and to control the relation between the first derivative and the second derivative and get different results

Reminder:

Increase the internal energy of the snake as it stretches more and more

Small alpha make the energy function insensitive to the amount of stretch

Small α Big α

When beta is large then high price for curvature so snake prefers to be smooth and not curving

Small β

Small beta causes snake to allow large curvature so that snake will curve into bends in the contour

Big β

Internal energy - Summary

On the continuous world:

On the discrete world:

(X0 , Y0)

(X3 , Y3)

(X14 , Y14)

10),( niyx iii

Discrete representationRepresent the snake with a set of n points

Open and close snake

closed snake

n 0

open snake

0

1n

Dealing with open snake is easier and our algorithm will run faster (soon…)

External Energy – The Energy map

The energy map’s task is to emphasize the areas of the contour and to weaken the other areas of the image by creating a new image. By doing this we can attract our snake near the “emphasized” pixels which represent the contour of the image.

An energy map is a function f (x, y) that we extract from the image – I(x, y)

How do we build an Energy map?

By given an image – I(x, y) , we can build an energy map – f(x, y), that will attract our snake to edges on our image.For each pixel we will do:

By doing this we can easily locate the edges on the images because the gradient there is big.

(color image)

(black-white image)

ImageEnergy map

External Energy

Attracts the contour towards the closest image edge with dependence on the energy map. determines whether the snake feels attracted to object boundaries

Initial state Final state

Intuitive example

Eexternal

In order to make snakes useful for early vision we need energy functions that attract them to salient features in images.

I will present you 2 different energy functions which attract a snake to lines & edges.

Eline

Depending on the sign of Wline the snake will be attracted either to bright lines or dark lines.

Eedge

Reminder: Gradient is a generalization of the usual concept of derivative to the functions of several variables

Clarification

Note: As long as the points on the snake are closer to the edge (where the gradient is bigger) and then the Eedge is smaller, so that’s a good thing because that's how the external energy works! this gets the snake closer and closer to the target contour.

Examle

Image

External forceZoomed in

External energy - Summary

On the continuous world:

On the discrete world:

Example – External energy

How to find the optimal function V?

Several algorithms have been proposed to fit deformable contours.

We’ll look at two: -  Greedy search -  Dynamic programming (for 2d snakes)

Greedy algorithmA greedy algorithm makes locally optimal choices, hoping that the final solutionwill be globally optimal.

For each point of the snake, we search a window around it (window size: m) and move to where the energy function is minimal.

We will stop after enough points have not changed in the last iteration

Energy function for each point in the local neighborhood is calculated

The point is moved to the next point with lowest energy function

This process is repeated for every point

Iteration is done until termination condition met

Defined number of iterations

Points are calculates by an iterative process:

)( nmOComplexity:

But...Convergence not guaranteed!!Need decent initialization

Works very well as far as the initial snake is not too far from the desired solution

Viterbi algorithmDynamic programming algorithmWe need to make the snake discrete in order to use the algorithm

The discretization of the external energy depends on the particular energy term but the internal energy is always the same and can be discretized as follows:

Our problem is to finding the set of vertex positions V that minimizes:

We will now make the assumptionthat the snake is open and that β=0

In most cases, snake energy can be rewritten as a sum of pair-wise interaction potential:

(The internal energy is a local property of neighboring vertices)

),(...),(),(),...,,( 1132221121 nnnn vvEvvEvvEvvvE

1iE iEThis means that each vertex position Vi influences the total energy only through the terms and

Each vertex has m possible moves

We now define n variables (i=2,3,…,n)iS

Each contains the lowest total energy for the first k-1 vertices of the snake for a givenvalue of . Thus the minimum energy E of the whole snake is equal to

)( kk vS

kv)(min )( nnn vsv

The globally best position for the snake is therefore computed by first computing all the , so we determine at each node the optimal position of its predecessor for each possible location of the node.

kS

When we have computed we can find the optimal position for by minimizing the expression Once we know the position of the last node we look up the optimal position for the second last node and soon until we have determined the optimal position for all the nodes.

nSnu )( nn us

),( 44 nvvE),( 433 vvE

)3(3E

)4(3E )4(4E

)3(4E

)2(4E

)1(4E

)4(nE

)3(nE

)2(nE

)1(nE

)2(3E

)1(3E

)4(2E

)3(2E

),( 322 vvE

)1(2E

)2(2E

),( 211 vvE

)( 2nmOComplexity:

0)1(1 E

0)2(1 E

0)3(1 E

0)4(1 E

states

1

2

…

m

sites

1v 2v 3v 4v nv

Now, β≠0If β≠0 then the decomposition of the energy function into local terms becomes:

1iE,iE

This means that each vertex position Vi influences the total energy only through the terms and 1iE

And now we will define again n variables (i=2,3,…,n)iS

Note that we have to compute a table of values at each node instead of only m values per node for the simpler snakes with β=0.

2m

)( 3nmOComplexity:

Now, the snake is closed

Solution:1. Can use Viterbi to optimize snake energy in case v1 = c is fixed. (in this case the energy above effectively has no loop) 2. Use Viterbi to optimize snake for all possible values of c and choose the best of the obtained m solutions.

)( 3nmOComplexity:

1n

2

1n

3 4

Snakes - Pros Vs. ConsPros:

Snakes optimize globally.

Snakes are 1-dimensional which means that we can reduce 2-dimensional optimizationproblems to 1-dimensional optimization problems.

Active contour models provide a unified solution to several image processing problems such as the detection of light and dark lines and edges.

Useful to track and fit non-rigid shapes.Flexibility in how energy function is defined.

Cons:

Must have decent initialization near true boundary, may get stuck in local minimum.

Parameters of energy function must be set well based on prior information.

ExtensionsWe will review two main applications of snakes:-  Snake splitting -  Snake expending

Snake SplittingProblem:Snakes including multiple objects can not always extract each of them individually (when initialize the contour wrong)

Solution:Using the split-and-merge contour model based on detecting self- and mutual-crossings of the contour.

The method:There is a single contour which surrounding all objects and then iteratively splits into multiple contours by cutting at self-crossing points for extracting multiple objects individually

Snake ExpandingThere is some cases when our snakes would need to expand in order to get close to the contour.

Problems:If there is no image force (F=0), the curve shrinks on itself and vanishes to a point.Often, due to noise, some isolated points are gradient maxima and can stop the curve when it passes by. We can get stuck inside a shape (if our image force=0) So we need the define new force for this cases:

Solution:

we add another force which makes the contour have a more dynamic behavior.You can think about our curve as a “Balloon” Force which inflate our snake

The new Force

Cool demonstrations

SummarySnakes include: - Two energy functions (Internal and External) - N points who represents a function V

Main problem:Finding an optimal function V that minimize the Energy function

Solved by:- Greedy algorithm- Viterbi algorithm

ReferencesM. Kass, A. Witkin, and D. Terzopoulos, "Snakes:Active contour models.“, International Journal ofComputer Vision. v. 1, n. 4, pp. 321-331, 1987.

1.

http://www.computing.edu.au/~jim/thesis/ivins02.pdf3.

Wikipedia – Active contour models (Snakes)2.

http://www.cse.unr.edu/~bebis/CS791E/Notes/DeformableContours.pdf

4.

http://www.cs.utexas.edu/~grauman/courses/378/handouts/snakes.pdf

5.

http://cs.gmu.edu/~kosecka/cs682/lect-snakes.pdf6.

Thank you