computational vision

Computational Vision / Ioannis StamosComputational Vision / Ioannis Stamos

Edge DetectionEdge Detection Canny DetectorCanny Detector

Line DetectionLine Detection Hough TransformHough Transform

Trucco: Chapter 4, pp. 76 – 80Trucco: Chapter 4, pp. 76 – 80 Chapter 5, pp. 95 - 100Chapter 5, pp. 95 - 100

Computational VisionComputational Vision

CSc 83020 3-D Computer Vision – Ioannis StamosCSc 83020 3-D Computer Vision – Ioannis Stamos

Finding Corners Finding Corners


What Is a Corner?What Is a Corner?

Large gradients in more than one direction.

Edges vs. CornersEdges vs. Corners

Edges = maxima in intensity gradientEdges = maxima in intensity gradient

Edges vs. CornersEdges vs. Corners

Corners = lots of variation in direction of Corners = lots of variation in direction of gradient in a small neighborhoodgradient in a small neighborhood

Detecting CornersDetecting Corners

How to detect this variation?How to detect this variation?


How to detect this variation?How to detect this variation? Not enough to check average andNot enough to check average and

xf

yf


Claim: the following covariance matrix Claim: the following covariance matrix summarizes the statistics of the gradientsummarizes the statistics of the gradient

Summations over local neighborhoodsSummations over local neighborhoods

yff

xff

ffffff

C yxyyx

yxx

,2

2


Examine behavior of Examine behavior of CC by testing its effect by testing its effect in simple casesin simple cases

Case #1: Single edge in local Case #1: Single edge in local neighborhoodneighborhood

Case#1: Single EdgeCase#1: Single Edge

Let (Let (aa,,bb) be gradient along edge) be gradient along edge Compute Compute CC ((aa,,bb):):

ba

ff

ba

ff

ba

ffffff

ba

Cyyx

yxx

T

2

2

y

x

ff

f

Case #1: Single EdgeCase #1: Single Edge

However, in this simple case, the only However, in this simple case, the only nonzero terms are those where nonzero terms are those where ff = ( = (aa,,bb))

So, So, CC ((aa,,bb) is just some multiple of () is just some multiple of (aa,,bb))

Case #2: CornerCase #2: Corner

Assume there is a corner, with Assume there is a corner, with perpendicular gradients (perpendicular gradients (aa,,bb) and () and (cc,,dd))

Case #2: CornerCase #2: Corner

What is What is CC ((aa,,bb)?)? Since (Since (aa,,bb) ) ( (cc,,dd) = 0, the only nonzero terms ) = 0, the only nonzero terms

are those where are those where ff = ( = (aa,,bb)) So, So, CC ((aa,,bb) is again just a multiple of () is again just a multiple of (aa,,bb))

What is What is CC ((cc,,dd)?)? Since (Since (aa,,bb) ) ( (cc,,dd) = 0, the only nonzero terms ) = 0, the only nonzero terms

are those where are those where ff = ( = (cc,,dd)) So, So, CC ((cc,,dd) is a multiple of () is a multiple of (cc,,dd))

Corner DetectionCorner Detection

Matrix times vector = multiple of vectorMatrix times vector = multiple of vector Eigenvectors and eigenvalues!Eigenvectors and eigenvalues! In particular, if In particular, if CC has has oneone large large

eigenvalue, there’s an edgeeigenvalue, there’s an edge If If CC has has twotwo large eigenvalues, have large eigenvalues, have

cornercorner Tomasi-Kanade corner detectorTomasi-Kanade corner detector

Corner Detection Corner Detection Implementation Implementation

1.1. Compute image gradientCompute image gradient2.2. For each For each mmmm neighborhood, neighborhood,

compute matrix compute matrix CC3.3. If smaller eigenvalue If smaller eigenvalue 22 is larger than is larger than

threshold threshold , record a corner, record a corner4.4. Nonmaximum suppression: only keep Nonmaximum suppression: only keep

strongest corner in each strongest corner in each mmmm window window

Corner Detection ResultsCorner Detection Results

CheckerboardCheckerboardwith noisewith noise

Trucco & VerriTrucco & Verri


Histogram of Histogram of 22 (smaller eigenvalue (smaller eigenvalue))


A Simple Corner DetectorA Simple Corner Detector

2

2

yyx

yxx

IIIIII

C

21 Find eigenvalues of C

If the smaller eigenvalue is above a threshold then we have a corner.



2

2

yyx

yxx

IIIIII

C

21 Find eigenvalues of C

If the smaller eigenvalue is above a threshold then we have a corner.

0yI

000a

C

aa

C0

0

Canny Edge DetectionCanny Edge Detection Can we derive an optimal detector?Can we derive an optimal detector? Good Detection:Good Detection: minimize false positives and false negatives. minimize false positives and false negatives. Good Localization:Good Localization: close as possible to the true edges. close as possible to the true edges. Single Response Constraint:Single Response Constraint: one edge should be detected one edge should be detected

for each true edgefor each true edge


ComparisonComparison

CannySobel

Canny Edge DetectionCanny Edge Detection

3 STEPS:3 STEPS: CANNY_ENHANCERCANNY_ENHANCER NONMAX_SUPPRESIONNONMAX_SUPPRESION HYSTERESIS_THRESHHYSTERESIS_THRESH

Localization-Detection TradeoffLocalization-Detection Tradeoff

Filter’s spatial scale Filter’s spatial scale Location &Location & Detection criteria.Detection criteria.

Good tradeoff 1-D step edge detector:Good tradeoff 1-D step edge detector: (optimal)(optimal)

Approximated by the 1Approximated by the 1stst derivative of the Gaussian. derivative of the Gaussian.

Optimal 1-D step edge detectors…

Pixel (i,j)

Keep the one that gives you maximum response.

Expensive.

Solution…

Pixel (i,j)

Compute derivatives with respect to x & y directions.

Compute edge normal.

x

y

CANNY_ENHANCERCANNY_ENHANCER Compute Compute IIxx and and IIyy the gradient of the image using a derivative of the gradient of the image using a derivative of

Gaussian filter.Gaussian filter. Compute the edge strength from the magnitude of the gradient: Compute the edge strength from the magnitude of the gradient:

EsEs Compute the orientation of the edge from arctan(Compute the orientation of the edge from arctan(IIyy / / IIxx ): ): EoEo

[Canny ’86]n

• Compute image derivatives • if gradient magnitude > and the value is a local max. along gradient direction – pixel is an edge candidate• how to detect one pixel thin edges ?

gradient magnitude (Es)original image

Problem with detectorProblem with detector

NONMAX_SUPPRESSIONNONMAX_SUPPRESSION

0

4590

135

Locate local maxima from Es.

•Find direction d that best approximates Eo(i,j)•If Es(i,j) is smaller than at least one neighbor along d In(i,j)=0 else In(i,j)=Es(i,j).


Non-Maximum SupressionNon-Maximum Supression

Non-maximum suppression:Select the single maximum point across the width of an edge.


Linking to the Next Edge PointLinking to the Next Edge Point

Assume the marked point q is an edge point.

Take the normal to the gradient at that point and use this to predict continuation points (either r or p).

ThresholdingThresholding Edges are found by thresholding the output of Edges are found by thresholding the output of

NONMAX_SUPRESSIONNONMAX_SUPRESSION If the threshold is too high:If the threshold is too high:

Very few (none) edges Very few (none) edges High MISDETECTIONS, many gapsHigh MISDETECTIONS, many gaps

If the threshold is too low:If the threshold is too low: Too many (all pixels) edgesToo many (all pixels) edges

High FALSE POSITIVES, many extra edgesHigh FALSE POSITIVES, many extra edges

Edge Detection With HysteresisEdge Detection With Hysteresis

Low threshold High threshold

Hysteresis (high and low threshold)


Edge HysteresisEdge Hysteresis HysteresisHysteresis:: A lag or momentum factorA lag or momentum factor Idea: Maintain two thresholds Idea: Maintain two thresholds kkhighhigh and and kklowlow

Use Use kkhighhigh to find strong edges to start edge to find strong edges to start edge chainchain

Use Use kklowlow to find weak edges which to find weak edges which continue edge chaincontinue edge chain

Typical ratio of thresholds is roughlyTypical ratio of thresholds is roughly kkhighhigh // kklowlow == 22

Edge TrackingEdge TrackingHysteresis thresholding [Canny ’86]

Strong edge

Weak edges

Strong edges reinforce weak edges.

We call a pixel an edge if it is strong.We also call a pixel an edge if it is weak but is connected to an edge.A pixel is connected to an edge if it is in a direction perpendicular to the edge normal

Weak edge removed

edges


Canny Edge Detection (Example)Canny Edge Detection (Example)

courtesy of G. Loy

gap is gone

Originalimage

Strongedges

only

Strong +connectedweak edges

Weakedges

HYSTERESIS_THRESHHYSTERESIS_THRESH

Input: In, Eo, high threshold, low threshold.Input: In, Eo, high threshold, low threshold. Output: Lists of connected edges Output: Lists of connected edges

(contours).(contours).

Edge RelaxationEdge RelaxationParallel – iterative method to adjust edge values on the basis of neighboring edges

Crack edges:


Vertex types:

Notation:Edge to be updated

Edge

No edge

Crack edges:

(0)

(1)

(2)

(3)


Vertex types:


Edge

No edge

Crack edges:

(0)

(1)

(2)

(3)

Action table for edge type:

Decrease Increase Unchanged 0-0 1-1 0-1 0-2 1-2 2-2 0-3 1-3 2-3

3-3


Vertex types:


Edge

No edge

Crack edges:

(0)

(1)

(2)

(3)

Action table for edge type:

Decrease Increase Unchanged 0-0 1-1 0-1 0-2 1-2 2-2 0-3 1-3 2-3

3-3

Algorithm:

0. Compute initial confidence C0(e) of each edge e C0(e) =

1. k = 1

2. Compute edge type for all e

3. Modify Ck(e) based on Ck-1(e) and edge type.

4. If all Ck(e) have converged to 1 or 0 else go to step 1

)max( Ie

Computational Vision / Ioannis StamosComputational Vision / Ioannis Stamos

Canny Edge DetectorCanny Edge Detector

Original: LenaOriginal: Lena EdgesEdges


Towards Global FeaturesTowards Global Features

Local versus global


From Edges to LinesFrom Edges to Lines

Detecting LinesDetecting Lines

What is the difference between line What is the difference between line detection and edge detection?detection and edge detection? Edges = localEdges = local Lines = nonlocalLines = nonlocal

Line detection usually performed on the Line detection usually performed on the output of an edge detectoroutput of an edge detector

From Szymon Rusinkiewicz, Princeton


Possible approaches ?Possible approaches ?



Possible approaches:Possible approaches: Brute force: enumerate all lines, check if presentBrute force: enumerate all lines, check if present Hough transform: vote for lines to which detected Hough transform: vote for lines to which detected

edges might belongedges might belong Fitting: given guess for approximate location, refine itFitting: given guess for approximate location, refine it

Second method efficient for finding Second method efficient for finding unknown lines, but not always accurateunknown lines, but not always accurate


Hough TransformHough Transform

General idea: transform from image General idea: transform from image coordinates to parameter space of featurecoordinates to parameter space of feature Need parameterized model of featuresNeed parameterized model of features For each pixel, determine all parameter values that For each pixel, determine all parameter values that

might have given rise to that pixel; votemight have given rise to that pixel; vote At end, look for peaks in parameter spacAt end, look for peaks in parameter spacee


Hough Transform for LinesHough Transform for Lines

Generic line: Generic line: y y = = axax++bb Parameters: Parameters: aa and and bb



1.1. Initialize table of Initialize table of bucketsbuckets, indexed by, indexed byaa and and bb, to zero, to zero

2.2. For each detected edge pixel (For each detected edge pixel (xx,,yy):):a.a. Determine all (Determine all (aa,,bb) such that ) such that y y = = axax++bbb.b. Increment bucket (Increment bucket (aa,,bb))

3.3. Buckets with many votes indicate probable linesBuckets with many votes indicate probable lines



bb

aa


Difficulties withDifficulties withHough Transform for LinesHough Transform for Lines

Slope / intercept parameterization not idealSlope / intercept parameterization not ideal Non-uniform sampling of directionsNon-uniform sampling of directions Can’t represent vertical linesCan’t represent vertical lines

Angle / distance parameterizationAngle / distance parameterization Line represented as (Line represented as (rr,,) where) where

xx cos cos + + yy sin sin = = rr

rr



Finding Lines Using the Hough Finding Lines Using the Hough TransformTransform

sincos ii yx

ii yx ,


AlgorithmAlgorithm Discretize the parameter spaces Discretize the parameter spaces ρ and θ.ρ and θ. Create Accumulator array A(1..R,1..T).Create Accumulator array A(1..R,1..T). Set A(k,h)=0 for all k and h.Set A(k,h)=0 for all k and h. For each image edge E(i,j)=1For each image edge E(i,j)=1

For h=1…TFor h=1…T ρ =i cosθρ =i cosθdd(h)+j sinθ(h)+j sinθd d (h)(h) Find index k: ρFind index k: ρd d is closest to ρis closest to ρ Increment A(h,k) by one.Increment A(h,k) by one.

Find all local maxima (kFind all local maxima (kpp, h, hpp) such that A (k) such that A (kpp, h, hpp)>τ)>τ

Hough Transform ResultsHough Transform Results

Forsyth & PonceForsyth & Ponce



Strong local peaks correspond to lines.



Resolution Issues

…

Bucket SelectionBucket Selection

How to select bucket size?How to select bucket size?


Bucket SelectionBucket Selection

How to select bucket size?How to select bucket size? Too small: poor performance on noisy data, long Too small: poor performance on noisy data, long

running timesrunning times Too large: poor accuracy, possibility of false positivesToo large: poor accuracy, possibility of false positives

Large buckets + verification and refinementLarge buckets + verification and refinement Problems distinguishing nearby linesProblems distinguishing nearby lines

Be smarter at selecting bucketsBe smarter at selecting buckets Use gradient information to select subset of bucketsUse gradient information to select subset of buckets More sensitive to noiseMore sensitive to noise



Hough Transform: ResultsHough Transform: Results

Hough TransformImage Edge detection


Summary Hough TransformSummary Hough Transform Smart countingSmart counting

Local evidence for global featuresLocal evidence for global features Organized in a tableOrganized in a table Careful with parameterization!Careful with parameterization!

Problem: Curse of dimensionalityProblem: Curse of dimensionality Works great for simple features with 3 unknownsWorks great for simple features with 3 unknowns Will fail for complex objectsWill fail for complex objects

Problem: Not a local algorithm Problem: Not a local algorithm


What else can be detected usingWhat else can be detected usingHough transform?Hough transform?


What else can be detected usingWhat else can be detected usingHough transform?Hough transform?

Anything, but Anything, but dimensionality dimensionality is keyis key


Finding Circles by Hough TransformFinding Circles by Hough Transform

a0

b0r (xi,yi)

Equation of Circle: 220

20 )()( rbyax ii

x

y


a0

b0r (xi,yi)

Equation of Circle: 220

20 )()( rbyax ii

If radius r is known:

x

y

x

y

a

b

(xi,yi)

Circles!

Accumulator array A(a,b)



a0

b0r (xi,yi)

x

y

If r is not knownUse accumulator array A(a,b,r)

For each (xi,yi) increment A(a,b,r) such that222 )()( rbyax ii


Using Gradient InformationUsing Gradient InformationCan save lot of computations!

Given: location (xi,yi)Edge direction φi

x

y

Using Gradient InformationUsing Gradient InformationCan save lot of computations!

Given: location (xi,yi)Edge direction φi

x

y

Assume r is known:

x

y φi

(xi,yi) a=x-rcosφ b=y-rsinφNeed to incrementonly one pointin Accumulator Array.

(a,b)


Hough Transform for CurvesHough Transform for Curves

Curve y=f(x,Curve y=f(x,aa)) a=a=[a[a11, … , a, … , app] the parameters of the curve.] the parameters of the curve. Limitation: size of parameter space wrt # of Limitation: size of parameter space wrt # of

parameters.parameters. Solution: variable-resolution parameter space.Solution: variable-resolution parameter space.


Pattern Matching.Pattern Matching. More efficient than template matching.More efficient than template matching. Handles occlusion.Handles occlusion. Finds all instances of patternFinds all instances of pattern.. Handling inaccurate edge locations?Handling inaccurate edge locations? Drawbacks?Drawbacks?

FittingFitting

Output of Hough transform often not Output of Hough transform often not accurate enoughaccurate enough

Use as initial guess for fittingUse as initial guess for fitting

Fitting LinesFitting Lines

Initial guessInitial guess


Least-squaresLeast-squaresminimizationminimization


Finding LinesFinding Lines

Assume edge detectionAssume edge detection Each pixel is either edge or notEach pixel is either edge or not How do we find the line?How do we find the line?


Least Squares Fitting of LinesLeast Squares Fitting of Linescmxy

ii yx ,

cmxy ii

i

ii

NcmxyE

2

Minimize E

0

0

cEmE

Least Squares Fitting of LinesLeast Squares Fitting of Lines

i

ii

NcmxyE

2

Minimize E

0

0

cEmE

Problem: E must be formulated carefully!

Least Squares Fitting of LinesLeast Squares Fitting of Lines

ii yx ,

i

ii

NyxE

2cossin

Minimize E

0

0

E

E

0cossin yx

cossin ii yx

computational vision

Documents