cvpr2010 open source vision software, intro and training part viii point cloud library - rusu -...

Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion

Point Cloud Library3D Features. Registration. Surface Classification

Radu Bogdan RUSUJune 14, 2010

[Introduction] Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion

Outline

1. Introduction

2. Acquisition

3. Feature Estimation

4. (Fast) Point Feature Histograms

5. Surface Classification

6. Object Recognition

7. Registration

8. Conclusion

PCL :: 3D FeaturesRadu Bogdan RUSU


Motivation (1/2)Point-based classification

Figure out ways to “classify the world”




Two basic alternatives:1. I create a powerful, discriminative 3D feature descriptor;

I learn different classes of surface or object types;I use a machine learning classifier

(the solution efficiency depends on the method used−→ let your “neighbor” (the ML guy) solve it).

2. I use geometric reasoning techniques: segmentation, regiongrowing, robust estimators, octrees, ...

I fit linear (planes, lines) and non-linear (cylinders, spheres,etc) models to your data




Two basic alternatives:1. I create a powerful, discriminative 3D feature descriptor;

I learn different classes of surface or object types;I use a machine learning classifier

(the solution efficiency depends on the method used−→ let your “neighbor” (the ML guy) solve it).2. I use geometric reasoning techniques: segmentation, region

growing, robust estimators, octrees, ...I fit linear (planes, lines) and non-linear (cylinders, spheres,

etc) models to your data


Introduction [Acquisition] Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion

Outline

1. Introduction

2. Acquisition





7. Registration

8. Conclusion



Acquisition (1/1)How are Point Clouds acquired? Where do they come from?

Recap: Point Cloud acquisition sources:I laser/lidar sensors (2D/3D)I stereo camerasI time-of-flight (TOF) camerasI etc...

*PCL :: 3D FeaturesRadu Bogdan RUSU


Filtering (1-2/2)Statistical Gross Outlier Removal :: PCL/Filters/StatisticalOutlierRemoval

P = {pi ∈ Praw | dist(pi ,pj) > µ+ dthresh · σ}

I Distance Analysis:

I Before and after:



Filtering (1-2/2)Statistical Gross Outlier Removal :: PCL/Filters/StatisticalOutlierRemoval

P = {pi ∈ Praw | dist(pi ,pj) > µ+ dthresh · σ}

I Before and after:


Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion

Outline

1. Introduction

2. Acquisition





7. Registration

8. Conclusion



Local Features Estimation (1-2/5)Surface Normal Estimation :: PCL/Features/NormalEstimationOMP,TBB

I Given a point cloud with x,y,z 3D point coordinates

I Select each point’s k -nearest neighbors, fit a local plane,and compute the plane normal



Local Features Estimation (3/5)Surface Normal and Curvature Estimation

Cj =∑k

i=1 ξi · (pi − pj)T · (pi − pj), p = 1

k ·∑k

i=1 pi

ξi =

e− d2

iµ2 , pi outlier

1, pi inlier

σp = λ0λ0+λ1+λ2



Local Features Estimation (4-5/5)Consistent Normal Orientation

Before

I Extended Gaussian ImageI Orientation consistent for:

1. registration2. feature estimation3. surface representation

I normals on theGaussian sphere

I should be in thesame half-space



Local Features Estimation (4-5/5)Consistent Normal Orientation

Before After

(viewpoint − pi) · npi ≥ 0

or:

propagate consistencythrough an EMST



Feature Persistence (1/5)What scale to choose ?

bad scale (too small) good scale

Selecting the right scale (k -neighborhood) is problematic:



Feature Persistence (2/5)Solution: Compute the persistence of the feature over multiple scales

I Scales are independent, can be parallelizedI Where to threshold ?

I Computer Graphics: look for jumps in the feature curve

I Datasets in “robotics” have non-smooth surfaces. Solution:



Feature Persistence (3/5)Multiple radii analysis

L1-Manhattan L2-Euclidean Jeffries-Matusita



Feature Persistence (4/5)Multiple radii analysis

Bhattacharyya Chi-Square Kullback-Leibler



Feature Persistence (5/5)Example Result

RAW/intensity data Persistent Feature Histograms

*

Pf =n−1⋃i=1

[Pfi ∩ Pfi+1 ]


Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion

Outline

1. Introduction

2. Acquisition





7. Registration

8. Conclusion



Point Feature Histograms (PFH) (1-4/5)Basic Concepts :: PCL/Features/FPFHEstimation

I For every point pair〈(ps,ns); (pt ,nt)〉, letu = ns, v = (pt−ps)×u, w = u×v

f0 = 〈v ,nj〉f1 = 〈u, pj − pi〉/||pj − pi ||f2 = ||pj − pi ||f3 = atan(〈w ,nj〉, 〈u,nj〉)

ihist =∑x≤3

x=0

⌊fx ·d

fxmax −fxmin

⌋· dx




f0 = 〈v ,nj〉f1 = 〈u, pj − pi〉/||pj − pi ||f2 = ||pj − pi ||f3 = atan(〈w ,nj〉, 〈u,nj〉)

ihist =∑x≤3

x=0

⌊fx ·d

fxmax −fxmin

⌋· dx




I For every point pair〈(ps,ns); (pt ,nt)〉, letu = ns, v = (pt−ps)×u, w = u×v



Point Feature Histograms (PFH) (1-4/5)Points lying on different geometric primitives



Point Feature Histograms (PFH) (5/5)Complexity Analysis

Complexity is high: O(k2). Optimizations to the rescue!

Unordered OrderedPCL :: 3D FeaturesRadu Bogdan RUSU


Fast Point Feature Histograms (FPFH) (1/5)Basic Concepts

Re-formulate: FPFH(p) = SPF (p) + 1k∑k

i=11ωk· SPF (pk )

Point Feature Histograms (PFH) Fast Point Feature Histograms (FPFH)



Fast Point Feature Histograms (FPFH) (2/5)Theoretical formulation



Fast Point Feature Histograms (FPFH) (3/5)Noise Analysis

Synthetically noiseless Synthetically noisified



Fast Point Feature Histograms (FPFH) (4/5)Noise Analysis



Fast Point Feature Histograms (FPFH) (5/5)Persistence Similarity


Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion

Outline

1. Introduction

2. Acquisition





7. Registration

8. Conclusion



Surface Classification (1/8)Point-based classification recap

Figure out ways to “classify the world”



Surface Classification (2/8)Learning classes of surfaces

Concave vs Convex

13 classes total



Surface Classification (3/8)Noise Analysis



Surface Classification (4/8)Most Discriminating Features Selection

I How many training samples to generate ? Need tosimulate real scans.

I Solution: generate a lot, then select the mostdiscriminative ones



Surface Classification (5/8)Classification results using different methods

SVM

KNN

KMeansTable 1. Classification results

Method used Noiseless NoisySVM RBF Sublinear kernel 95.26% 89.55%KNN Bhattacharyya (µ-dist) 87.11% 83.53%

K-Means (81D) Bhattacharyya 73.63% 70.74%



Surface Classification (6/8)FPFH classification results



Surface Classification (7/8)Classification results using PFH and SVM: 89.95%

*

*



Surface Classification (8/8)Classification results using FPFH and CRF: 97.36%


Introduction Acquisition Feature Estimation F/PFH Surface Classification [Object Recognition] Registration Conclusion

Outline

1. Introduction

2. Acquisition





7. Registration

8. Conclusion



GFPFH ConceptsFrom local annotations (FPFH) to global (cluster) annotations (GFPFH)



FPFH ClassificationClassification results using FPFH and CRF: 98.27%

*



GFPFH ClassificationClassification results using GFPFH and SVM: 95.13%


Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition [Registration] Conclusion

Outline

1. Introduction

2. Acquisition





7. Registration

8. Conclusion



Registration (1/7)Multiple Scans :: PCL/Registration/IterativeClosestPoint*

points on similar surfaces

*



Registration (2/7)The classics

*



Registration (3/7)Outdoor Example: Persistence, Initial Alignment

*



Registration (4/7)Outdoor Example: Non-Linear Optimization



Registration (5/7)Outdoor Example: Non-Linear Optimization



Registration (6/7)Back to Indoor Environments



Registration (7/7)Registered 360◦ scans - 15 million points


Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration [Conclusion]

Outline

1. Introduction

2. Acquisition





7. Registration

8. Conclusion


Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration [Conclusion]

Questions?

http://www.ros.org/wiki/pclPCL :: 3D FeaturesRadu Bogdan RUSU

http://www.ros.org/wiki/pcl

cvpr2010 open source vision software, intro and training part viii point cloud library - rusu -...

Documents

surface normal estimation

d feature descriptor

surface classication6

feature estimation4

surface representationnormals

object recognition7

object types

fast point feature histograms5