Object Detection

01 – Basic Hough TransformationJJCAO

What do you see?

“Puzzle” solved from a new view

Different view, different domain

? !

The same problem, phenomenon or data set, when viewed from a different angle, or in a new domain, may better reveal its underlying structure to facilitate the solution

10

Line and curve detection• Find lines, curves, or parts of lines or curves in an input

image. Such an image might be the output of a edge detector

• Hough Transform: Elegant method for direct object recognition– Edges need not be connected– Complete object need not be visible– Key Idea: Edges VOTE for the possible model

Detect partially occluded lines

Hough transforms

o Object detection → peaks identification in Hough images

o Hough59: P.V.C. Hough, A method for faster analysis of bubble chamber photographs, International Conference on High Energy Accelerators and Instrumentation, 1959, cited times: 33

o USE OF THE HOUGH TRASFORMTION TO DETECT LINES and Curves in Pictures_duda_CommunAcm71, cited times: 2623

o GENERALIZING THE HOUGH TRANSFORM TO DETECT ARBITRARY SHAPES_Ballard_pr81, cited times: 2476

o A survey of the hough transform_Illingworth_CVGIP88, cited times: 1339

12

Beyond lines!!!• Hough transform is a method for estimating the

parameters of a shape from its boundary points• The idea can be generalized to estimate “parameters” of

arbitrary shapes

– Ballard 1981 – Other primitives – Lowe, ICCV 1999 – Object detection– Leibe, Schiele BMVC 2003 – Object class detection– CVPR 2009: Maji& Malik, Gall& Lempitsky, Gu et al. …– CVPR 2010: Bari, Lemp & Kohli

13

HT for Lines: representation 1• straight line in image space: y = mx+ b, is plotted for

each pair of values (x, y).• What is the characteristic of the line?

– Not x or y– But its slope m and intercept b

• A line in the image corresponds to a point in Hough spaceImage space Hough parameter space

Source: S. Seitz

Parameter space representation

• What does a point (x0, y0) in the image space map to in the Hough space?

Image space Hough parameter space

Parameter space representation

• What does a point (x0, y0) in the image space map to in the Hough space?– Answer: the solutions of b = –x0m + y0

– This is a line in Hough space

Image space Hough parameter space

Parameter space representation

• Where is the line that contains both (x0, y0) and (x1, y1)?

Image space Hough parameter space

(x0, y0)

(x1, y1)

b = –x1m + y1

Parameter space representation

• Where is the line that contains both (x0, y0) and (x1, y1)?– It is the intersection of the lines b = –x0m + y0 and

b = –x1m + y1

Image space Hough parameter space

(x0, y0)

(x1, y1)

b = –x1m + y1

18

HT for Lines: representation 1

Parameter space

(b,m)

b=y-mx

Image space

y=mx+b

(x,y)

cmxy

),( ii yx

y

x

m

c

),( cm

Parameter Space

Line Detection by Hough Transform

y

x

),( cm

Parameter Space

1 1

1 1

1 1

2

1 1

1 1

1 1

),( cmA

Algorithm:

• Quantize Parameter Space

• Create Accumulator Array

• Set

• For each image edge increment:

• If lies on the line:

• Find local maxima in

),( cm

),( cmA

cmcmA ,0),(

),( ii yx

1),(),( cmAcmA

),( cm

),( cmAii ymxc

20

HT for Lines: representation 2

• Problems with the (m,b) space:– Unbounded parameter domain– Vertical lines require infinite m

• Alternative : polar representation of lines

21HT for Lines• polar representation of lines

Parameter space

(b,m)

Image space

y=mx+b

(x,y)

22

HT for Lines - Computation

• How to find the intersection point?

…

23

Algorithm HOUGH_LINES

Is any pixels a voter?

Several lines

25

HT lines: local max

& threshold

Image space votes

Horizontal axis is θ, vertical is rho.

Dealing with noise

• Try to get rid of irrelevant features – Take only edge points with significant gradient

magnitude• Choose a good grid / discretization

– Too coarse: large votes obtained when too many different lines correspond to a single bucket

– Too fine: miss lines because some points that are not exactly collinear cast votes for different buckets

• Increment neighboring bins smoothing in accumulator array

27

HT for lines: noisy example

A more complicated image

http://ostatic.com/files/images/ss_hough.jpg

29

HT lines: Problem of parameter ranges

Vertical flip

?

P

𝜃n

30

• Solution: polar representation of lines

– For image:

– For voting:

HT lines: Parameter ranges

𝑝

31

HT lines: Parameter ranges

Real World Example

Original Edge Detection Found Lines

Parameter Space

Could we extract better edges?

33

Algorithm HOUGH_line (weight)

Is any pixels a voter?

:

Gradient(i,j)

Hough_Grd

• Recall: when we detect an edge point, we also know its gradient direction

• But this means that the line is uniquely determined!

• Modified Hough transform:

• For each edge point (x,y) θ = gradient orientation at (x,y)ρ = x cos θ + y sin θA(θ, ρ) = A(θ, ρ) + 1

end Θ=[0-360] so there is a conversion

EE6358 - Computer Vision

35

Computational Load

• Image size = 512 X 512• Maximum value of • With a resolution of 1o, maximum

value of • Accumulator size = • Use of direction of gradient reduces

the computational load by 1/180

Mechanics of the Hough transform

• Difficulties– how big should the

cells be? (too big, and we merge quite different lines; too small, and noise causes lines to be missed)

• How many lines?– Count the peaks in

the Hough array– Treat adjacent peaks

as a single peak

• Which points belong to each line?– Search for points

close to the line– Solve again for line

and iterate

37

HT for high dimensional Parameter Curves

38

5.2 Parameters for analytic curves

Analytic Form Parameters Equation

Line r, q xcosq+ysinq=r

Circle x0, y0, r (x-xo)2+(y-y0)2=r2

Parabola x0, y0, , r q (y-y0)2=4r(x-xo)

Ellipse x0, y0, a, b, q (x-xo)2/a2+(y-y0)2/b2=1

39

Summary• Feature extraction technique in cv• Purpose of it is to find imperfect (partially occluded,

noisy) instances of objects within a certain class of shapes

• By a voting procedure: – Discretize parameter space into bins– For each feature point in the image, put a vote in

every bin in the parameter space that could have generated this point

– Find bins that have the most votes

• Since each point is handled independently, parallel implementations are possible

• It becomes difficult when the dimension of the parameter space is large

HT for Circles: Search with fixed R

Equation of Circle: 222 )()( rbyax ii

41

Multiple Circles with known R• Multiple circles with the same radius can be found with the same

technique. The centerpoints are represented as red cells in the parameter space drawing.

• Overlap of circles can cause spurious centers to also be found, such as at the blue cell. Spurious circles can be removed by matching to circles in the original image.

HT for Circles: Search with unknown R

Equation of Circle: 222 )()( rbyax ii

If radius is not known: 3D Hough Space!

Use Accumulator array

What is the surface in the hough space?

),,( rbaA

Hough transform for circles

),(),( yxIryx

x

y

(x,y)),(),( yxIryx

xy

r

image space Hough parameter space

44

Generalized Hough transform• We want to find a shape defined by its

boundary points and a reference point

D. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes, Pattern Recognition 13(2), 1981, pp. 111-122.

a

Generalized Hough transform• We want to find a shape defined by its boundary

points and a reference point• For every boundary point p, we can store the

displacement vector r = a – p as indexed by gradient orientation θ

D. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes, Pattern Recognition 13(2), 1981, pp. 111-122.

(xc,yc)

θiri

Pi

ai

xc = xi + ricos(ai)

yc = yi + risin(ai)

Generalizing the H.T.

(xc,yc)

Pi

firi ai

xc = xi + ricos(ai)

yc = yi + risin(ai)

• Suppose, there were m different gradient orientations: (m <= n)

f1

f2

.

.

.

fm

(r11,a1

1),(r12,a1

2),…,(r1n1,a1

n1)

(r21,a2

1),(r22,a1

2),…,(r2n2,a1

n2)

.

.

.

(rm1,am

1),(rm2,am

2),…,(rmnm,am

nm)

fj

rj

aj

R-table

Example

model shape

Example

displacement vectors for model points

Example

range of voting locations for test point

Example

range of voting locations for test point

Example

votes for points with θ =

Example

displacement vectors for model points

Example

range of voting locations for test point

votes for points with θ =

Example

Generalized Hough TransformFind Object Center given edges

Create Accumulator Array

Initialize:

For each edge point

For each entry in table, compute:

Increment Accumulator:

Find Local Maxima in

),( cc yxA

),(0),( cccc yxyxA

),,( iii yx

1),(),( cccc yxAyxA

),( cc yxA

ik

ikic

ik

ikic

ryy

rxx

sin

cos

ikr

),( cc yx ),,( iii yx

•Assumption: translation is the only transformation here, i.e., orientation and scale are fixed

Voting schemes

• Let each feature vote for all the models that are compatible with it

• Hopefully the noise features will not vote consistently for any single model

• Missing data doesn’t matter as long as there are enough features remaining to agree on a good model

59

Applications

60

Measuring local gradient saliency

Gradient => Gradient saliency

GradientShop: A Gradient-Domain Optimization Framework for Image and Video Filtering_tog09

61

Speed up – Kernel Based HT

Real-time line detection through an improved Hough transform voting scheme_pr08, cited times: 72

62

More• A new curve detection method: Randomized Hough

transform (RHT)_prl90, cited times: 555

• An improved Hough transform voting scheme utilizing surround suppression_prl_09

• An Improved Hough Transform Neighborhood Map for Straight Line Segments_tip10

• A probabilistic Hough transform_pr91• Progressive Probabilistic hough transform_98