jetstream: probabilistic contour extraction with particles patrick perez, andrew blake, and michel...
Post on 14-Jan-2016
224 Views
Preview:
TRANSCRIPT
JetStream: Probabilistic Contour Extraction with Particles
Patrick Perez, Andrew Blake, and Michel Gangnet, Microsoft Research, St George House, 1 Guildhall Street, Cambridge, CB2
3NH, UK
http://research.microsoft.com/vision
Presented by:
Vladan Radosavljevic
OutlineIntroduction
Image GradientRelated Work
Probabilistic contour trackingTracking framework
DynamicsMeasurement
Iterative computation of posterior - particlesModel Ingredients
Likelihood ratioDynamicsProposal Sampling Function
Experimental ResultsConclusion
Motivation• Contour extraction
– from segmenting images with closed contours
– to the extraction of linear structures of particular interest such as roads
Introduction - image gradient• The gradient of an image:
The gradient points in the direction of most rapid change in intensity
• The gradient direction is given by:
• how does this relate to the direction of the edge?
• The edge strength is given by the gradient magnitude
Introduction• Most approaches to contour extraction rely on some
minimal cost principle
• k is the curvature, s is the arc-length• y(r(s)) scalar or vector derived at location r(s) from
the raw image I• Often y(r(s)) is the gradient norm
• This function captures some kind of regularity on candidate curves, for example - rewarding, by a lower cost, the presence of large gradients along the contour
Introduction – related work• First approach - dynamic programming
• optimal curve in the form of chain pixels• unless optimality is abandoned, and huge storage resources
are available, there are tight restrictions on the form of cost function
• Second approach - growing a contour from the seed point according to cost function• given the current contour, a new segment is appended to it
according both to a shape prior (mainly smoothness) and to the evidence provided by the data at the location under concern
• deterministic: complete discontinuous curves provided by edge detectors
Introduction – related work• A probabilistic point of view:
• the contours are seen as the paths of a stochastic process• tracking problem
• We are interesting in the method that is able:• to avoid spurious distracting contours• to track the multiple off-springs starting at branching
contours• to interpolate over transient evidence “gaps”
• JetStream – a method with particle filtering
Probabilistic contour tracking• Tracking contours in still images is an unconventional
tracking problem because of the absence of a real notion of time
• The “time” is only associated with the growing of the contour in the image plane – consider a spatial chain as a temporal chain
• Contrary to standard tracking problems where data arrive one bit after another as time passes by, the whole set of data y is standing there at once
• There is no straightforward way of tuning the “speed”, or equivalently the length of successive moves
Tracking framework - dynamics• xi – points in the plane R2
• (x0, x1,...,xn) - curve in some standard way, e.g., the xi’s are the vertices of a polyline, starting from x0 then moving along the contour to xn
• dynamics:
• assuming second order dynamics with some kernel q:
• then a priori density is:
Tracking framework - measurement
• measurement y (observed image) conditioned on x0:n is considered as independent spatial process which is not true in reality but it is a reasonable approximation:
where Ώ is a discrete set of measurements locations in the image plane
• the conditional distributions depend only on whether or not point u is contained in contour x0:n
• in particular, for any two points u along the contour the corresponding conditional distributions are identical, similarly for any two points in the background
Tracking framework - measurement
• Therefore, each is either pon if u belongs to the contour x0:n, or poff if not:
• Finally:
where
Iterative computation of posterior• Function can be
considered as a cost function• Expressed as the minimization of this function, the
contour extraction problem then amounts to seeking the maximum a posteriori (MAP) estimate
• Posterior densities can be computed recursively:
but there is no closed form of the pi
• pi can be approximated by a finite set of M sample paths (the ‘particles’)
• Best path at step i can be which is a Monte Carlo approximation of posterior expectation
Iterative computation of posterior• Prediction: each path is grown one step
by sampling from the proposal density function f
• If the paths are samples from pi then the extended paths are samples from f*pi
• Since we want samples from distribution pi+1, extended paths are weighted according to ratio
• The resulting weighted path set now provides an approximation of the target distribution pi+1
• M paths are drawn with replacement from the weighted set
• The weights are:
Likelihood ratio l• Measurement: the norm of the luminance (or color)
gradient
poff pon
• poff – exponential distribution
• pon - complex mixture, better keep as less informative as possible
Likelihood ratio l
• The direction of the gradient also retains precious information that a data model based only on gradient norm neglects: the distribution of Ψ is symmetric, and it becomes tighter as the norm of the gradient increases
Likelihood ratio l• However, at corners, the norm of the gradient is
usually large but its direction cannot be accurately measured
• Using a standard corner detector, each pixel u is associated with a label c(u) = 1 if a corner is detected, and 0 otherwise
• Where a corner has been detected assumption is that distribution is uniform
Dynamics q• Because of the absence of natural time, it is better to
consider a dynamics with fixed step length d. The definition of second order dynamics then amounts to specifying an a priori probability distribution on direction change Θi
• Finally:
• To allow for abrupt direction changes at the locations where corners have been detected, the normal distribution is mixed with a small proportion of uniform distribution
Proposal sampling function f• With choice f = q, corners will be mostly ignored since
the expected number of particles undertaking drastic direction changes is vM, where typically ν = 0.01 and M = 100
• At locations where no corners are detected, the proposal density is the normal component of the dynamics.
• If location lies on a detected corner, the next proposed location is obtained by turning of an angle picked uniformly
• Therefore:
Proposal sampling function f
JetStream - iteration
Experimental Results – Interactive cut-out
• The extraction of a region of interest from one image
• In practice, JetStream is run for a fixed number n of steps (100 in our experiments) from initial conditions x0:1 chosen by the user.
• If the result is satisfactory, n more steps are undertaken.
• If not, a restart region within the particle flow, and an associated restart direction, can be chosen by the user
Experimental Results – Interactive cut-out
• More sophisticated user interaction• Provide the user with the facility to place one or more
dams, defined as regions Rk where:
Experimental Results – Interactive cut-out
Experimental Results – Road extraction• In the specific context of road extraction, one is in
fact interested in recovering “ribbons”• Using JetStream as defined previously results in
paths jumping from one side of the ribbon to the other
Experimental Results – Road extraction• The state space is extended to include a width
variable mi, which indicates the distance at step i between the two sides of the ribbon
• It expresses that xi+ and xi
- defined as
are on a contour while xi is not:
Experimental Results – Road extraction
• A simple first order dynamics is chosen for m:
Experimental Results – Road extraction
Conclusion• JetStream is applicable if there are not sharp corners
in the images • If sharp corners exist or the image’s intensity
distribution is complex JetStream doesn’t provide good results
References[1] M. Pérez, A. Blake, and M. Gangnet.
JetStream : Probabilistic contour extraction with particles. In Int. Conf. on Computer Vision, ICCV’ 2001, Vancouver, Canada, July 2001.
top related