feature detection: corners and lines. edges vs. corners edges = maxima in intensity gradientedges =...

Feature Detection:Feature Detection:Corners and LinesCorners and Lines

Edges vs. CornersEdges vs. Corners

• Edges = maxima in intensity gradientEdges = maxima in intensity gradient

Edges vs. CornersEdges vs. Corners

• Corners = lots of variation in direction Corners = lots of variation in direction of gradient in a small neighborhoodof gradient in a small neighborhood

Detecting CornersDetecting Corners

• How to detect this variation?How to detect this variation?

• Not enough to check average andNot enough to check average andx

f

x

f

y

f

y

f

Detecting CornersDetecting Corners

• Claim: the following covariance matrix Claim: the following covariance matrix summarizes the statistics of the summarizes the statistics of the gradientgradient

Summations over local neighborhoodsSummations over local neighborhoods

y

ff

x

ff

fff

fffC yx

yyx

yxx

,2

2

y

ff

x

ff

fff

fffC yx

yyx

yxx

,2

2

Detecting CornersDetecting Corners

• Examine behavior of Examine behavior of CC by testing its by testing its effect in simple caseseffect in simple cases

• Case #1: Single edge in local Case #1: Single edge in local neighborhoodneighborhood

Case#1: Single EdgeCase#1: Single Edge

• Let (Let (aa,,bb) be gradient along edge) be gradient along edge

• Compute Compute CC ((aa,,bb):):

b

aff

b

aff

b

a

fff

fff

b

aC

yyx

yxx

T

2

2

b

aff

b

aff

b

a

fff

fff

b

aC

yyx

yxx

T

2

2

Case #1: Single EdgeCase #1: Single Edge

• However, in this simple case, the only However, in this simple case, the only nonzero terms are those where nonzero terms are those where ff = = ((aa,,bb))

• So, So, CC ((aa,,bb) is just some multiple of ) is just some multiple of ((aa,,bb))

Case #2: CornerCase #2: Corner

• Assume there is a corner, with Assume there is a corner, with perpendicular gradients (perpendicular gradients (aa,,bb) and () and (cc,,dd))

Case #2: CornerCase #2: Corner

• What is What is CC ((aa,,bb)?)?– Since (Since (aa,,bb) ) ( (cc,,dd) = 0, the only nonzero ) = 0, the only nonzero

terms are those where terms are those where ff = ( = (aa,,bb))

– So, So, CC ((aa,,bb) is again just a multiple of () is again just a multiple of (aa,,bb))

• What is What is CC ((cc,,dd)?)?– Since (Since (aa,,bb) ) ( (cc,,dd) = 0, the only nonzero ) = 0, the only nonzero

terms are those where terms are those where ff = ( = (cc,,dd))

– So, So, CC ((cc,,dd) is a multiple of () is a multiple of (cc,,dd))

Corner DetectionCorner Detection

• Matrix times vector = multiple of vectorMatrix times vector = multiple of vector

• Eigenvectors and eigenvalues!Eigenvectors and eigenvalues!

• In particular, if In particular, if CC has has oneone large large eigenvalue, there’s an edgeeigenvalue, there’s an edge

• If If CC has has twotwo large eigenvalues, have large eigenvalues, have cornercorner

• Tomasi-Kanade corner detectorTomasi-Kanade corner detector

Corner Detection Implementation Corner Detection Implementation

1.1. Compute image gradientCompute image gradient

2.2. For each For each mmmm neighborhood, neighborhood,compute matrix compute matrix CC

3.3. If smaller eigenvalue If smaller eigenvalue 22 is smaller than is smaller than

threshold threshold , record a corner, record a corner

4.4. Nonmaximum suppression: only keep Nonmaximum suppression: only keep strongest corner in each strongest corner in each mmmm window window

Corner Detection ResultsCorner Detection Results

• CheckerboardCheckerboardwith noisewith noise

Trucco & VerriTrucco & Verri

Corner Detection ResultsCorner Detection Results

Corner Detection ResultsCorner Detection Results

Histogram of Histogram of 22 (smaller eigenvalue (smaller eigenvalue))

Corner DetectionCorner Detection

• Application: good features for tracking, Application: good features for tracking, correspondence, etc.correspondence, etc.– Why are corners better than edges for Why are corners better than edges for

tracking?tracking?

• Other corner detectorsOther corner detectors– Look for curvature in edge detector outputLook for curvature in edge detector output

– Perform color segmentation on Perform color segmentation on neighborhoodsneighborhoods

– Others…Others…

Detecting LinesDetecting Lines

• What is the difference between line What is the difference between line detection and edge detection?detection and edge detection?– Edges = localEdges = local

– Lines = nonlocalLines = nonlocal

• Line detection usually performed on the Line detection usually performed on the output of an edge detectoroutput of an edge detector

Detecting LinesDetecting Lines

• Possible approaches:Possible approaches:– Brute force: enumerate all lines, check if Brute force: enumerate all lines, check if

presentpresent

– Hough transform: vote for lines to which Hough transform: vote for lines to which detected edges might belongdetected edges might belong

– Fitting: given guess for approximate Fitting: given guess for approximate location, refine itlocation, refine it

• Second method efficient for finding Second method efficient for finding unknown lines, but not always accurateunknown lines, but not always accurate

Hough TransformHough Transform

• General idea: transform from image General idea: transform from image coordinates to parameter space of coordinates to parameter space of featurefeature– Need parameterized model of featuresNeed parameterized model of features

– For each pixel, determine all parameter For each pixel, determine all parameter values that might have given rise to that values that might have given rise to that pixel; votepixel; vote

– At end, look for peaks in parameter spaceAt end, look for peaks in parameter space

Hough Transform for LinesHough Transform for Lines

• Generic line: Generic line: y y = = axax++bb

• Parameters: Parameters: aa and and bb

Hough Transform for LinesHough Transform for Lines

1.1. Initialize table of Initialize table of bucketsbuckets, indexed by, indexed byaa and and bb, to zero, to zero

2.2. For each detected edge pixel (For each detected edge pixel (xx,,yy):):a.a. Determine all (Determine all (aa,,bb) such that ) such that y y = = axax++bb

b.b. Increment bucket (Increment bucket (aa,,bb))

3.3. Buckets with many votes indicateBuckets with many votes indicateprobable linesprobable lines

Hough Transform for LinesHough Transform for Lines

bb

aa

Hough Transform for LinesHough Transform for Lines

bb

aa

Bucket SelectionBucket Selection

• How to select bucket size?How to select bucket size?– Too small: poor performance on noisy dataToo small: poor performance on noisy data

– Too large: poor accuracy, long running times, Too large: poor accuracy, long running times, possibility of false positivespossibility of false positives

• Large buckets + verification and refinementLarge buckets + verification and refinement– Problems distinguishing nearby linesProblems distinguishing nearby lines

• Be smarter at selecting bucketsBe smarter at selecting buckets– Use gradient information to select subset of Use gradient information to select subset of

bucketsbuckets

– More sensitive to noiseMore sensitive to noise

Difficulties withDifficulties withHough Transform for LinesHough Transform for Lines

• Slope / intercept parameterization not Slope / intercept parameterization not idealideal– Non-uniform sampling of directionsNon-uniform sampling of directions

– Can’t represent vertical linesCan’t represent vertical lines

• Angle / distance parameterizationAngle / distance parameterization– Line represented as (Line represented as (rr,,) where) where

xx cos cos + + yy sin sin = = rr rr

Angle / Distance ParameterizationAngle / Distance Parameterization

• Advantage: uniform parameterizationAdvantage: uniform parameterizationof directionsof directions

• Disadvantage: space of all linesDisadvantage: space of all linespassing through a point becomes apassing through a point becomes asinusoid in (sinusoid in (rr,,) space) space

Hough Transform ResultsHough Transform Results

Forsyth & PonceForsyth & Ponce

Hough Transform ResultsHough Transform Results

Forsyth & PonceForsyth & Ponce

Hough TransformHough Transform

• What else can be detected usingWhat else can be detected usingHough transform?Hough transform?

• Anything, but Anything, but dimensionality dimensionality is keyis key

Hough Transform for CirclesHough Transform for Circles

• Space of circles has a 3-dimensional Space of circles has a 3-dimensional parameter space: position (2-d) and parameter space: position (2-d) and radiusradius

• So, each pixel gives rise to 2-d sheet of So, each pixel gives rise to 2-d sheet of values in 3-d spacevalues in 3-d space

Hough Transform for CirclesHough Transform for Circles

• In many cases, can simplify problem byIn many cases, can simplify problem byusing more informationusing more information

• Example: using gradient informationExample: using gradient information

• Still need 3-d bucket space, but each pixel Still need 3-d bucket space, but each pixel only votes for 1-d subsetonly votes for 1-d subset

Hough Transform for CirclesHough Transform for Circles

++ + . . . =+ . . . =

Simplifying Hough TransformsSimplifying Hough Transforms

• Another trick: use prior informationAnother trick: use prior information– For example, if looking for circles of a For example, if looking for circles of a

particular size, reduce votes even furtherparticular size, reduce votes even further

FittingFitting

• Output of Hough transform often not Output of Hough transform often not accurate enoughaccurate enough

• Use as initial guess for fittingUse as initial guess for fitting

Fitting LinesFitting Lines

Initial guessInitial guess

Fitting LinesFitting Lines

Least-squaresLeast-squaresminimizationminimization

Fitting LinesFitting Lines

Fitting LinesFitting Lines

• As before, have to be careful about As before, have to be careful about parameterizationparameterization

• Simplest line fitting formulas minimize Simplest line fitting formulas minimize verticalvertical (not perpendicular) point-to-line (not perpendicular) point-to-line distancedistance

• Closed-form solution for point-to-line Closed-form solution for point-to-line distance, not necessarily true for other distance, not necessarily true for other curvescurves