efficient variants of the icp algorithm szymon rusinkiewicz marc levoy stanford university

Efficient Variants of theEfficient Variants of theICP AlgorithmICP Algorithm

Szymon RusinkiewiczSzymon RusinkiewiczMarc LevoyMarc Levoy

Stanford UniversityStanford University

ProblemProblem

• Align two partially-Align two partially-overlapping overlapping meshesmeshesgiven initial guessgiven initial guessfor relative for relative transformtransform

Aligning 3D DataAligning 3D Data

• If correct correspondences are If correct correspondences are known,known,it is possible to find correct it is possible to find correct relative rotation/translationrelative rotation/translation


• How to find corresponding points?How to find corresponding points?

• Previous systems based on user Previous systems based on user input,input,feature matching, surface feature matching, surface signatures, etc.signatures, etc.


• Alternative: assume Alternative: assume closestclosest points points correspond to each other, correspond to each other, compute the best transform…compute the best transform…


• … … and iterate to find alignmentand iterate to find alignment– Iterated Closest Points (ICP) [Besl & Iterated Closest Points (ICP) [Besl &

McKay 92]McKay 92]

• Converges if starting position Converges if starting position “close enough““close enough“

OutlineOutline

• Enumeration and classification ofEnumeration and classification ofICP variantsICP variants

• Performance comparisonsPerformance comparisons

• High-speed ICP algorithm High-speed ICP algorithm recombines previously-introduced recombines previously-introduced variantsvariants– Suitable for real-time range scanningSuitable for real-time range scanning

ICP VariantsICP Variants

• Variants on the following stages of Variants on the following stages of ICPICPhave been proposed:have been proposed:1.1. SelectingSelecting source points (from one or both meshes) source points (from one or both meshes)

2.2. MatchingMatching to points in the other mesh to points in the other mesh3.3. WeightingWeighting the correspondences the correspondences4.4. RejectingRejecting certain (outlier) point pairs certain (outlier) point pairs5.5. Assigning an Assigning an error metricerror metric to the current transform to the current transform6.6. MinimizingMinimizing the error metric the error metric

Performance of VariantsPerformance of Variants

• Can analyze various aspects of Can analyze various aspects of performance:performance:– SpeedSpeed

– StabilityStability

– Tolerance of noise and/or outliersTolerance of noise and/or outliers

– Basin of convergence (maximum initial misalignment)Basin of convergence (maximum initial misalignment)

• Comparisons in paper focus mostly on Comparisons in paper focus mostly on speedspeed

• Today: summarize conclusions about a Today: summarize conclusions about a few categories of ICP variantsfew categories of ICP variants


1.1. SelectingSelecting source points (from one or both meshes) source points (from one or both meshes)2.2. MatchingMatching to points in the other mesh to points in the other mesh3.3. WeightingWeighting the correspondences the correspondences4.4. RejectingRejecting certain (outlier) point pairs certain (outlier) point pairs5.5. Assigning an Assigning an error metricerror metric to the current transform to the current transform6.6. MinimizingMinimizing the error metric the error metric

Selecting Source PointsSelecting Source Points

• Use all pointsUse all points

• Uniform subsamplingUniform subsampling

• Random samplingRandom sampling

• Normal-space samplingNormal-space sampling– Ensure that samples have normals Ensure that samples have normals

distributeddistributedas uniformly as possibleas uniformly as possible

Normal-Space SamplingNormal-Space Sampling

Uniform SamplingUniform Sampling Normal-Space SamplingNormal-Space Sampling

Normal-Space SamplingNormal-Space Sampling

• Conclusion: normal-space Conclusion: normal-space sampling better for mostly-smooth sampling better for mostly-smooth areas with sparse featuresareas with sparse features

Random samplingRandom sampling Normal-space samplingNormal-space sampling

Selection vs. WeightingSelection vs. Weighting

• Could achieve same effect with Could achieve same effect with weightingweighting

• Hard to ensure enough samples in Hard to ensure enough samples in features except at high sampling features except at high sampling ratesrates

• However, have to build special data However, have to build special data structurestructure

• Preprocessing / run-time cost tradeoffPreprocessing / run-time cost tradeoff

Point-to-Plane Error MetricPoint-to-Plane Error Metric

• Using point-to-plane distance Using point-to-plane distance instead of point-to-point lets flat instead of point-to-point lets flat regions slide along each other regions slide along each other [Chen & Medioni 91][Chen & Medioni 91]

MatchingMatching

• Matching strategy has greatest Matching strategy has greatest effect on convergence and speedeffect on convergence and speed

• Closest pointClosest point

• Normal shootingNormal shooting

• Closest compatible pointClosest compatible point

• ProjectionProjection

Closest-Point MatchingClosest-Point Matching

• Find closest point in other meshFind closest point in other mesh

• Closest-point matching generally Closest-point matching generally stable,stable,but slow and requires but slow and requires preprocessingpreprocessing

Normal ShootingNormal Shooting

• Project along normal, intersect Project along normal, intersect other meshother mesh

• Slightly better than closest point Slightly better than closest point for smooth meshes, worse for for smooth meshes, worse for noisy or complex meshesnoisy or complex meshes

Closest Compatible PointClosest Compatible Point

• Can improve effectiveness of both Can improve effectiveness of both of the previous variants by only of the previous variants by only matching to matching to compatiblecompatible points points

• Compatibility based on normals, Compatibility based on normals, colors, etc.colors, etc.

• At limit, degenerates to feature At limit, degenerates to feature matchingmatching

Projection to Find Projection to Find CorrespondencesCorrespondences

• Finding closest point is most expensive Finding closest point is most expensive stage ofstage ofthe ICP algorithmthe ICP algorithm

• Idea: use a simpler algorithm to find Idea: use a simpler algorithm to find correspondencescorrespondences

• For range images, can simply project point For range images, can simply project point [Blais 95][Blais 95]

Projection-Based MatchingProjection-Based Matching

• Slightly worse performance per Slightly worse performance per iterationiteration

• Each iteration is one to two orders Each iteration is one to two orders of magnitude faster than closest-of magnitude faster than closest-pointpoint

• Requires point-to-plane error Requires point-to-plane error metricmetric

High-Speed ICP AlgorithmHigh-Speed ICP Algorithm

• ICP algorithm with projection-ICP algorithm with projection-based correspondences, point-to-based correspondences, point-to-plane matchingplane matchingcan align meshes in a few tens of can align meshes in a few tens of ms.ms.(cf. over 1 sec. with closest-point)(cf. over 1 sec. with closest-point)

ApplicationApplication

• Given:Given:– A scanner that returns range images A scanner that returns range images

in real timein real time

– Fast ICPFast ICP

– Real-time merging and renderingReal-time merging and rendering

• Result: 3D model acquisitionResult: 3D model acquisition– Tight feedback loop with userTight feedback loop with user

– Can see and fill holes while scanningCan see and fill holes while scanning

Our ImplementationOur Implementation

• Real-time structured-light range Real-time structured-light range scannerscanner[Hall-Holt & Rusinkiewicz, [Hall-Holt & Rusinkiewicz, ICCV01]ICCV01]

• Off-the-shelf camera and DLP Off-the-shelf camera and DLP projectorprojector

• Range images at 60 Hz.Range images at 60 Hz.

PhotographPhotograph

Real-Time ResultReal-Time Result

ConclusionsConclusions

• Classified and compared variants of Classified and compared variants of ICPICP

• Normal-space sampling for smooth Normal-space sampling for smooth meshes with sparse featuresmeshes with sparse features

• Overall speed depends most on choice Overall speed depends most on choice of matching algorithmof matching algorithm

• Particular combination of variants can Particular combination of variants can align two range images in a few tens of align two range images in a few tens of ms.ms.– Real-time range scanningReal-time range scanning– Model-based trackingModel-based tracking

Future WorkFuture Work

• More work on examining robustness More work on examining robustness and stability of variantsand stability of variants

• Real-time global registrationReal-time global registration

• Other selection / weighting criteria?Other selection / weighting criteria?– Select in regions of high curvatureSelect in regions of high curvature

– Continuum between ICP with fancy Continuum between ICP with fancy selectionselection(or weighting) and feature matching(or weighting) and feature matching

efficient variants of the icp algorithm szymon rusinkiewicz marc levoy stanford university

Documents

icp variants variants

possible slide

error metric slide

relative transform slide

source points

best transform slide

performance of variants

corresponding points