edges edge detection contour segmentation hough transform edges based on a practical introduction...

EdgesEdge DetectionContour SegmentationHough Transform

EdgesBased on A Practical Introduction to Computer Vision with

OpenCV by Kenneth Dawson-Howe © Wiley & Sons Inc. 2014 Slide 1

EdgesAn approach to segmentation.The analysis of the discontinuities in an image.

No correct answer?An alternative to region based processing.



Edge Detection – Topics1st derivative edge detection2nd derivative edge detectionMultispectral edge detectionImage sharpening



Edge Detection – What is an edge?

Where brightness changes abruptly

Edges haveMagnitude (Gradient)Direction (Orientation)

Edge ProfilesStepRealNoisy



Calculus...Rate of change in two directionsVector variable:

Gradient MagnitudeOrientation (0o is East)

Edge Detection – 1st derivative definitions



Edge detection – Digital imagesDerivatives work on continuous functions

Map every point in the input image to the output

Discrete domainDifferencesOrthogonal

Need to define the partial derivates so that they:1. Cross at a single middle point2. Preferably cross at the centre of a pixel3. Evaluate points which are not too close together4. Deal with some degree of image noise

© Brooks/Cole Publishing Company



Edge detection – 1st derivative – Roberts

Convolution Masks

Sensitivity to NoiseBinary Images



Edge detection – 1st derivative – CompassCompass edge detectors:

Partial derivatives defined for a number of orientations (typically 8) Prewitt:

Use h3(i,j) and h1(i,j) to derive gradient and orientation



Edge detection – 1st derivative – CompassSobel



Edge detection – 1st derivative – Compass



Mat horizontal_derivative, vertical_derivative;

Sobel( gray_image, horizontal_derivative, CV_32F,1,0 );

Sobel( gray_image, vertical_derivative, CV_32F,0,1 );

Mat abs_gradient, l2norm_gradient, orientation;

abs_gradient = abs(horizontal_derivative) +

abs(vertical_derivative);

cartToPolar(horizontal_derivative,vertical_derivative,

l2norm_gradient,orientation);

Edge detection – 1st derivative – Thresholding Simple thresholding

Too many pointsToo few points



Edge detection – 1st derivative – Non maxima suppression Use orientation informationAlgorithm:

Quantise edge orientationsFor all points (i,j)

Look at the 2 points orthogonal to edgeif gradient(i,j) < gradient(either of these 2 points)

output(i,j) = 0else output(i,j) = gradient(i,j)

Now thresholding can be used… or hysteresis thresholding (detailed later…)



Edge detection – 1st derivative – NMS



nms_result = gradients.clone();

for (int row=1; row < gradients.rows-1; row++)

for (int column=1; column < gradients.cols-1; column++)

{

float curr_gradient = gradients.at<float>(row,column);

float curr_orientation = orientations.at<float>(row,column);

// Determine which neighbours to check

int direction = (((int) (16.0*(curr_orientation)/(2.0*PI))+15)%8)/2;

float gradient1 = 0.0, gradient2 = 0.0;

switch(direction)

{

case 0:

gradient1 = gradients.at<float>(row-1,column-1);

gradient2 = gradients.at<float>(row+1,column+1);

break;

Edge detection – 1st derivative – NMS



case 1:

gradient1 = gradients.at<float>(row-1,column);

gradient2 = gradients.at<float>(row+1,column);

break;

case 2:

gradient1 = gradients.at<float>(row-1,column+1);

gradient2 = gradients.at<float>(row+1,column-1);

break;

case 3:

gradient1 = gradients.at<float>(row,column+1);

gradient2 = gradients.at<float>(row,column-1);

break;

}

if ((gradient1 > curr_gradient) || (gradient2 > curr_gradient))

nms_result.at<float>(row,column) = 0.0;

}

Edge detection – 2nd derivative – Laplace operator

Laplace operator

Orientation independentDetermines gradient magnitudeLinear rampsNoiseZero-crossings



Edge detection – 2nd derivative – Marr-HildrethMarr-Hildreth edge detector

2nd derivative zero crossings

How to compute robustly?Smooth the image first

The smoothing filter must be Smooth and roughly band-limitedSpatial localisation

Optimal solutionGaussian filter



Edge detection – 2nd derivative – Laplacian of GaussianCombine Gaussian smoothing with the Laplacian operator

Change ordering

Convolution Mask

Mexican hat filter

Pros and ConsLarger area taken into accountToo much smoothingClosed loops of edgesGanglion Cells



Edge detection – 2nd derivative – Laplacian of Gaussian



Mat laplacian;

Mat blurred_image1_gray;

GaussianBlur( gray_image, blurred_image, Size(5,5), 1.5 );

Laplacian( blurred_image, laplacian, CV_32F, 3 );

Edge detection – 2nd derivative – Laplacian of GaussianSeparable decomposition

Difference of Gaussians

Finding zero-crossings Look for zeros?Band thresholding?Search using a 2x2 windowLimit search to significant 1st derivatives

2 I i, j G i, j G i I i, j 2

2 jG j

G j I i, j 2

2iG i



Edge detection – 2nd derivative – ScaleImage processing techniques work locally

Size of neighbourhood to use?Problem of scaleChicken before the egg…

Using different resolutionsCreate a model under each resolutionStudy changes in model to obtain meta-knowledge



Edge detection – 2nd derivative – ScaleExample: Marr’s edge detection:

Smooth image using different GaussiansLook for correspondences across scale to identify significant discontinuities



Edge detection – 2nd derivative – CannyOptimality criteria:

DetectionLocalisationOne response



Edge detection – 2nd derivative – Canny algorithmAlgorithm (Compute orientation, location, & gradient)

Convolve image with GaussianEstimate edge orientation using first derivativeLocate the edges

Non-maxima suppression using zero-crossingCompute edge magnitude using first derivativeThreshold edges with hysteresisRepeat for mulitple scalesFeature synthesis



Edge detection – 2nd derivative – Canny



Canny( gray_image, binary_edges, 100, 200 );

Multispectral edge detectionDetect edges separately in each spectral band

Use maximal value OR some linear combination

Multi-spectral edge detector

Use a different colour model )e.g. HLS space with linear combination of H & L



Image sharpeningMaking edges steeper.

Demo (Sharpening.twk)



Image sharpeningSubtract a multiple (e.g. 0.3) of the Laplacian from the image.



image.convertTo( image_32bit, CV_32F );

Laplacian( image, laplacian, CV_32F, 3 );

Mat sharp_image_32bit = image_32bit - 0.3*laplacian;

sharp_image_32bit.convertTo( sharp_image, CV_8U );

Contour Segmentation – TopicsEdge data representationsBorder DetectionLine segment extraction



Contour Segmentation – Boundary Chain Codes Each chain contains

Start pointA list of orientations

Features!Orientation & Scale dependentSlightly position dependentCan be smoothed to reduce boundary noiseDifficulties obtaining consistent representation from image.



Contour Segmentation – Boundary Chain Codes



Contour Segmentation – Boundary Chain Codes



for (int contour_number=0;

(contour_number<contours.size()); contour_number++)

{

Scalar colour( rand()&0xFF, rand()&0xFF, rand()&0xFF );

drawContours( display_image, contours, contour_number,

colour, 1, 8, hierarchy );

}

vector<vector<Point>> contours;

vector<Vec4i> hierarchy;

findContours( binary_edge_image, contours, hierarchy,

CV_RETR_CCOMP, CV_CHAIN_APPROX_NONE );

Contour Segmentation – Border detection as graph searchingGraph:

Nodes ni

Arcs (ni, nj) with associated cost and orientation

Border detectionSearch for optimal pathCost related to Gradient = s(xi) Orientation = Edge direction = (xi) ni, nj corresponding to 8-connected pixels xi, xj are connected by an arc if

(xi) and (xj) match the local border



Contour Segmentation – A-algorithm graph searchAlgorithm:

Determine successors for all nodes:For any given node ni based on (xi)

s(xi) > T, s(xj) > Txj must be one of 3 neighbours in the direction [(xi)-/4, (xi)+/4]| (xi) - (xj) | < /2

Expand(nA): Put all its successors into an OPEN list and evaluate accumulated cost f(nk) for these successors (Keep path from nA)While OPEN and nB OPEN

Choose the node in the OPEN list with the lowest f(nk) and ExpandIf nB OPEN

Optimal path foundOtherwise No path

Problems:Infinite loops



Contour Segmentation – Gradient field transformNeed to incorporate cost to the destination

ĥ(ni) = ĉ(ni , nB)h(ni) = c(ni , nB)

Use the actual cost h(ni)?

How can we estimate the cost?Better estimates…



Contour Segmentation – Estimating the costCost so far

Strength of edges(maximage s(xk)) - s(xi)Almost always gives good results

Border curvaturediff[ (xi) - (xj) ]

Remaining cost…Proximity to an approximate border

dist(xi, approximate_boundary)

Distance to the goaldist(xi, xB)



Contour Segmentation – More efficient searchPruning the solution tree

High average cost per unit length

Maximum costMax. path cost.

Depth first searchReduce candidate edges

Multi-resolution processingDetermine approximate boundary first

Incorporation of higher-level knowledgeA priori knowledge



Contour Segmentation – Heuristic search for image bordersFind all edges in an image

Algorithm:1. Search for the strongest (unused) edge in the image. If none go to step 5.2. Expand all the edges in front of the specified edge3. Expand all the edges behind of the specified edge4. If the edge chain consists of > 3 pixels store it. Go to step 1.5. Modify edge chains to fill small breaks and to remove duplicate contours.6. Repeat step 5 until stable.



Contour Segmentation – Segment sequencesBoundary

Sequence of segments

SegmentStart and end points: x1 and x2

Type: straight line, some type of curve, etc.Other parameters

Accurate polygonal representationMust define acceptable tolerancesVertices – where?



Contour Segmentation – Straight line extraction example



Contour Segmentation – Obtaining polygonal segmentsRecursive boundary splitting

Split at furthest pointKeep going until with tolerance

Divide and conquerSplit in the middleKeep going until with tolerance



Contour Segmentation – Obtaining polygonal segments



vector<vector<Point>> approx_contours(contours.size());



approxPolyDP( Mat(contours[contour_number]),

approx_contours[contour_number], 3, true );

Contour Segmentation – Obtaining polygonal segments



vector<Vec4i> line_segments;



for (int segment_num=0; (segment_num <

approx_contours[contour_number].size()-1);

segment_num++)

line_segments.push_back( Vec4i(

approx_contours[contour_number][segment_num].x,

approx_contours[contour_number][segment_num].y,

approx_contours[contour_number][ segment_num+1].x,

approx_contours[contour_number][segment_num+1].y));

Contour Segmentation – Curved segmentsCurved segments

What order of curves should we use? Curves of Constant curvatureSecond order polynomials (circles, ellipses, parabolas)

Where does one curve end and the next start?Use straight lines instead…

Determining curvatureCannot rely on orientation valuesCannot use nearest neighbour pixelsLook at well separated points…



Hough Transform – TopicsHough for linesHough for circlesGeneralised Hough



Hough transformDirect transformation from image space toprobability of the existence of some feature…

LinesCirclesGeneralised shapes



Hough Transform – Circle detectionEquation of a circle

Assume constant radius r

TransformFrom Image space (x, y)To Hough space (a, b)

AlgorithmInitialise accumulator to 0For every edge point

Increment cells in accumulator corresponding to all possible circle centers

Search for Maximums



Hough Transform – Circle detection



Hough Transform – Line detectionLine equation:

j = m.i + cLines in Hough spaceWhat about i = c

r = i.cos θ + j.sin θSinusoidal curves



Hough Transform – Line detection



Hough transform – in OpenCVHough for lines:

Probabilistic Hough for line segments:

Hough for circles:



vector<Vec2f> hough_lines;

HoughLines( binary_edge_image, hough_lines, 1, PI/200.0, 60);

vector<Vec4i> line_segments;

HoughLinesP( binary_edge_image, line_segments, 1.0,

PI/200.0, 20, 20, 5);

vector<Vec3f> circles;

HoughCircles( gray_image, circles, CV_HOUGH_GRADIENT,

2,20,300,20,5,15);

Hough Transform – Resolution of Hough SpaceCircle

(a, b) could be anywhere up to r outside the image spaceCan detect the circle centre to higher precision than the image points

Lines–dist(0,0, M,N) s + dist(0,0, M,N)- θ Duplicated space!Precision of s and θ application dependent



Hough Transform – Efficient ImplementationUse edge orientations

Restrict the mapping into space

Use half the accumulatorFor lines only

Multi-resolutionProcess at a small resolutionHigher resolution to find accurate data

ProblemWhat if the size of the circle is unknown3-D Accumulator??



Hough Transform – GeneralizedConsider an arbitrary shape

Define in terms of distance and angles from some reference xR

Distance rOrientation Orientation of line from xR through edge point



Hough Transform – Generalized – TrainingTraining phase

Build up an R-tableFor every store (r, ) pairs



Hough Transform – Generalized – RecognitionRecognition phase

Create an accumulator for xR

For every edge pointDetermine its orientation Select a list of (r, ) pairs from the R-tableDetermine the position of xR for each (, r, ) and increment the accumulator appropriately

Search for maximums



edges edge detection contour segmentation hough transform edges based on a practical introduction...

Documents

orientation slide

derivative compass edges

derivative definitions

east edge detection

edge detection topics

kenneth dawsonhowe wiley

practical introduction

computer vision