Scene-Consistent Detection ofScene-Consistent Detection ofFeature Points in Video SequencesFeature Points in Video Sequences

Ariel Tankus & Yehezkel Yeshurun

CVPR - Dec. 2001

Tel-AvivTel-Aviv UniversityUniversity

Outline:Outline:

Relating convexity-based detection of feature points to scene geometry.

Feature points tracking algorithm.Comparison with two other methods.Measures for evaluation of tracking

algorithms w.r.t 3D scene-consistency.

Task Definition:Task Definition:

Robust detection of scene-consistent features in video sequences.

Goals:

Object recognition. Correspondence points for recovering 3D

characteristics of the scene.

Intrinsic Property:

Convexity.

Operator for Feature DetectionOperator for Feature Detection

Detect local “circles” where the gradient of the intensity function points outward along the whole circle.

The gradient points in all orientations along the “circles”.

Detect convex or concave image domains.

Equivalently:

((motivation)motivation)

Operator for Extracting Certain Operator for Extracting Certain Gradient OrientationsGradient Orientations

)),( ),,( arctan(),(arg yxIyxIy

yxy

Yxy

derive

Gradient Argument Yarg

At the discontinuity ray of the arctan: Yarg.

Darg - An isotropic variant of Yarg.

Response of YResponse of Yargarg to the to the

Intensity SurfaceIntensity Surface

Examine Yarg in well behaving image domains.

Intensity is twice continuously differentiable.

If at , then has a jump discontinuity there.

),(θ 00 yx),( 00 yxThe basic observation:

We examine all possible intensity configurations. Four of them lead to infinite Yarg response.

The cases include: Some configurations where

is a local extrema of , and

some configurations where one side of

is flat, but the other is convex or concave.

Response of YResponse of Yargarg to the to the

Intensity Surface Intensity Surface (cont.)(cont.)

),( yxf 0),( 00 yx),( 00 yx

Only specific differential geometry structures of the intensity function causes Yarg.

Response to Local 3D Scene StructureResponse to Local 3D Scene Structure

Yarg for certain elliptic, hyperbolic or parabolic points on a Lambertian 3D surface illuminated by a point light source at infinity.

For certain intensity function configurations, if has a jump discontinuity, then z(x,y) is: elliptic, hyperbolic or parabolic there.

),(θ yx

Yarg responds to certain geometric features of the 3D scene object.

Stable points: points where .These points are the only input to the

point tracker.

Tracking AlgorithmTracking Algorithm

2argD

#8

#16

#24

#28

#36

#44

toys

#50

#100

#150

#200

#225

#250

par

kin

g

#55

#65

#70

#5

#10

#15

traf

fic

Evaluating the Performance Evaluating the Performance of the Algorithmof the Algorithm

Two measures for evaluating performance of scene-consistent point tracking algorithms.

Each measure aimed at a different task:–Maximal tracking time.– Correspondence of points in successive

frames.Their common goal: to quantify the

consistency of tracks with 3D scene.

Measures for Evaluation of Measures for Evaluation of Scene-ConsistencyScene-Consistency

Completeness:– A track is complete if the same 3D

scene point is being tracked, up to a certain level of noise, in every frame where it appears.

Completeness of track T = Time(correct 3D point is tracked)

Time(correct 3D point appears in video)

Correct 3D point of track T = 3D point tracked for the longest time under track T

Stability:

Measures for Evaluation of Measures for Evaluation of Scene-Consistency Scene-Consistency (cont.)(cont.)

Stability of tracking in frames fi, fi+1 =

#tracks following the correct 3D point in both fi, fi+1

#tracks containing points in both fi, fi+1

Tracking ComparisonTracking Comparison

We compare the Darg-based algorithm with two other algorithms:– Junction detection (Lindeberg).

with automatic scale selection.Tracking by Kalman filter.

– KLT (Kanade-Lucas-Tomasi). Tracker based on affine image change model. Features maximize tracking quality.

3I curves level of curvature

Completeness:Completeness:

Stability:Stability:

Experimental ResultsExperimental Results

Darg is more stable the Junction detection, and sometimes more than KLT. Sometimes Darg equates with KLT.

Darg completeness is at least comparable to that of Junction detection or KLT, and sometimes even better.

Darg has significantly lower no-tracking time (Darg: 4, KLT: 81, J.D.: 121 frames).

SummarySummary

Convexity-based method for scene-consistent feature points detection in video sequences.

Detection relates to specific features of the intensity surface.

These intensity features relate to geometric features of the 3D object.

Summary Summary (cont.)(cont.)

A stable point tracking algorithm is described (2D Kalman filter).

Two measures serve in a comparison with two other tracking methods.

Completeness: Maximizes tracking time of a 3D scene point.

Stability: Consistent tracking of 3D points between successive frames.