A A RRobust obust AAlgorithm lgorithm FFor or MMeasuring easuring TTie ie PPoints oints OOn n TThe he

BBlock lock OOf f AAerialerialIImagesmages

AndreyAndrey Sechin SechinScientific DirectorScientific DirectorRACURSRACURS

AlexeyAlexey ChernyavskiyChernyavskiyAlexanderAlexander VelizhevVelizhev AntonAnton Yakubenko Yakubenko Graphics & Media LabGraphics & Media Lab, MSU, MSU

IXth International Scientific and Technical Conference From Imagery to Map: Digital Photogrammetric Technologies

October 5–8, 2009, Attica, Greece

Area based cross correlationArea based cross correlation

k

f x y g x y Nfg

f x y Nf g x y Ngx y

x yx y

( )

( , ) ( , )

( ( , ) ) ( ( , ) )( , )

/ /

( , )( , )

p

1 1

2 2 1 2 21 1

2 1 2 fN

f x y gN

g x yx yx y

1 11 1( , ) ; ( , )

( , )( , )

S f x y kg x yx y

( ( , ) ( , ))( , )

0 01 1

2

Benefits and DrawbacksBenefits and Drawbacks

+ High subpixel accuracy- Needs a good initial guess- Works on smooth surfaces- Fails on periodic structures

Zheltov S. Y., Sibiryakov A. V. 1997.Adaptive Subpixel Cross-correlation ina Point Correspondence Problem. Optical 3-D Measurement Techniques IV,Wichmann Verlag, Heidelberg, pp.86-95.

Two formulae are equivalent

V.N. Adrov, A.D.Checkurin, A.Yu.Sechin, A.N.Smirnov, J-P. Adam-Guillaume,J-A. Qussette, Program PHOTOMOD:digital photogrammetry and stereoscopic images synthesison a personal computer., Digital Photogrammetry and Remote Sensing ‘95,ISPRS Proceedings, Vol. 2646.

Segmentation/boundary correlationSegmentation/boundary correlation

Segments matching

r( )

L

R

Correlation coefficient R -

similarity function W

2005 PHOTOMOD 4.0

M. Drakin, A.Elizarov,A. Sechin, A.Zelenskiy AUTOMATIC STEREO POINTSMEASUREMENTS USINGTWO-DIMENSIONALFEATURE EXTRACTION,Optical 3-D MeasurementTechiques VIII, v I, p. 385-388,Zurich 2007.

New approachNew approach – – DetectorsDetectors, , DescriptorsDescriptors, , RANSACRANSAC

• N (N > 2) strips

• Images are ordered inside strips

• No information on strips ordering

• The problem: find tie points with subpixel accuracy

Introduction

UUniversalityniversality

• Algorithm should work with:

– Any terrain type (buildings, fields, mountains, forests, …)

– Digital, scanned, space imagenary

– arbitrary overlap

DetectorDetector

• Reduce image resolutionReduce image resolution

• DetectorDetector – – find find ««cornerscorners» (» (1D features1D features) ) on all imageson all images

– We use classic corner detectorsWe use classic corner detectors

– We selectWe select N N ((~~1000) 1000) uniformly spaced best cornersuniformly spaced best corners

DescriptorDescriptor (SIFT/(SIFT/SURFSURF//DAISYDAISY….)….)

• Calculate gradients in the neighborhood of 1D feature (corner)Calculate gradients in the neighborhood of 1D feature (corner) ((gradients are invariant to gradients are invariant to lightnesslightness shift shift) )

• Select one (or a couple) of main gradient directionsSelect one (or a couple) of main gradient directions ( (invariance to invariance to rotationsrotations))

• Calculate histograms of gradientsCalculate histograms of gradients ( (good good neighbourhoodneighbourhood desciptiondesciption))

• Normalize histogramsNormalize histograms ( (invarience to contastinvarience to contast))

Candidates for matching (1)Candidates for matching (1)

A

A’

• For all pointsFor all points A A on the first image we select the nearest on the first image we select the nearest (with respect to descriptor) point(with respect to descriptor) point A’ A’ on the second imageon the second image

Candidates for matching (2)Candidates for matching (2)

• For pointsFor points A A’’ on the second image we find the nearest on the second image we find the nearest point (with respect to descriptor) point (with respect to descriptor) AA” on the first image” on the first image

A’’

A’

Candidates for matching (3)Candidates for matching (3)

• If If A A andand A’ A’’’ coincidecoincide, , the pairthe pair (A, A’) (A, A’) fits for the nest stagefits for the nest stage

B

A, A’’

B’’B’

A’

RANdom SAmpling Consensus (RANSAC, PROSAC,…)RANdom SAmpling Consensus (RANSAC, PROSAC,…)

• N (iterations number) times repeate

– Randomly select pairs. The number of seleted paires must be enough for model calculation (homography, fundamental matrix, relative orientation

– Calculate the model for selected matches

– Calculate errors for all possible pairs for the found model

– Ellimination of “bad” matches (outliers) – that do not fit the threashould

– Calculate the number of “good” matches (inliers) and RMS

• Select the best model from all iterations

• Refinу the model using inliers

Example of found matchesExample of found matches

Speedup/Reliability increaseSpeedup/Reliability increase

Distance filtering (desriptors)Metric filteringTopological filteringReinforcement matching

Approximate overlap definitionApproximate overlap definition

Consider all candidate pairs withapproximately the same distanceon both images.Angle voting.

Shift voting with respect to x,y.

Finding matches on many imagesFinding matches on many images

1 2

3

4

Finding conflicts CR algorithm (conflict resolution)

1 2

3

4

Resolving conflicts and adding new matchesResolving conflicts and adding new matches

Finding “features” on images with initial resolution/subpixel Finding “features” on images with initial resolution/subpixel refinementrefinement

Several “features” should be found in the neighborhood.Repeat algorithm on initial resolution, take into account all found restrictions

ConclusionConclusion

The algorithm is fast and reliable on reduced resolutionsCalculation of detectors/discriptors/image overlap/RANSACneeds only several seconds of CPU time per image.CR algorithm needs some speedup(the solution is to split block into sub-blocks).

To DOTo DO

CR algorithm speedup.Speedup of the initial resolution part of the algorithm.Subpixel detectors (experiments to be performed).

Thank you for attention!Thank you for attention!