object recognition in 3d scans - ros.org · bastian steder - object recognition in 3d scans -...
TRANSCRIPT
Object Recognition in 3D scans
Willow Garage Intern Presentation 09/23/2010
Bastian StederUniversity of Freiburg, Germany
Bastian Steder - Object Recognition in 3D scans - intern presentation 2
Object Recognition in 3D scans
Goal: Object detection andGoal: Object detection andlocalization in point cloudslocalization in point clouds
Bastian Steder - Object Recognition in 3D scans - intern presentation 3
Working on Range images
Main advantage:Main advantage:Encodes negative informationEncodes negative information
Bastian Steder - Object Recognition in 3D scans - intern presentation 4
• Match features between model and scene
• Estimate object candidate poses
• Score and filter candidates
General Approach
Bastian Steder - Object Recognition in 3D scans - intern presentation 5
Feature extraction
Mainly two steps:
• Interest point detection:
Where should features be extracted
• Calculation of a descriptor:
Describe the area around the interest point in a way that makes it easy to compare
Bastian Steder - Object Recognition in 3D scans - intern presentation 6
Desired properties of our Features
• Use positions where the surface is stable,
but with significant changes in surrounding (I)
• Encode the surface structure (I+D)
• Encode outer shape (I+D)
• Provide local 6DOF transformation
=> NARF: Normal aligned radial features
Bastian Steder - Object Recognition in 3D scans - intern presentation 7
Border information
• Three kinds of border points
– Obstacle borders
– Shadow borders
– Veil points
Bastian Steder - Object Recognition in 3D scans - intern presentation 8
How to detect borders?
• Possibilities:
– Impact angles of the sensor beams
– Changes in normals
– …
• What actually works well:
– Changes in distances toneighboring points
Bastian Steder - Object Recognition in 3D scans - intern presentation 9
Border extraction video
Bastian Steder - Object Recognition in 3D scans - intern presentation 10
Interest points extraction
• Calculate a score for each image point
– How much do the dominant directions of the surface changes in the vicinity vary?
(border information & principal curvature)
– How stable is the surface on the point itself?
• Find maximums above a threshold
Bastian Steder - Object Recognition in 3D scans - intern presentation 11
NARF Interest point extraction video
Bastian Steder - Object Recognition in 3D scans - intern presentation 12
NARF descriptor video
Bastian Steder - Object Recognition in 3D scans - intern presentation 13
Results
ROC curve
Bastian Steder - Object Recognition in 3D scans - intern presentation 14
Example – Step 1
Bastian Steder - Object Recognition in 3D scans - intern presentation 15
Spatial validation – Validation points
Validation points on the model views for a fast spatial validationValidation points on the model views for a fast spatial validation
Bastian Steder - Object Recognition in 3D scans - intern presentation 16
Example – Step 2
Bastian Steder - Object Recognition in 3D scans - intern presentation 17
Overlapping poses + ICP
• Removal of multiple instances of the same object in the same position, coming from different feature matches
• Refinement of the object poses using ICP
Bastian Steder - Object Recognition in 3D scans - intern presentation 18
Example – Step 3
Bastian Steder - Object Recognition in 3D scans - intern presentation 19
Spatial validation – View Simulation
Comparison of what was actually seen and Comparison of what was actually seen and what it should look like according to the modelwhat it should look like according to the model
Bastian Steder - Object Recognition in 3D scans - intern presentation 20
Example – Step 4
Bastian Steder - Object Recognition in 3D scans - intern presentation 21
Video
Bastian Steder - Object Recognition in 3D scans - intern presentation 22
Final scene
Bastian Steder - Object Recognition in 3D scans - intern presentation 23
Software and Documentation
• Currently some parts in PCL and some in point_cloud_perception_experimental
– Range image class, border extraction, NARF interest point extraction, NARF descriptor, model class, spatial verification, visualization tools …
• Some Tutorials already exists – there will be more
• Paper submitted to ICRA11
Bastian Steder - Object Recognition in 3D scans - intern presentation 24
Thank you!
• Willow Garage in general for making this possible
• Radu for always being available for (sometimes stupid) questions and discussions
• Kurt for giving valuable input
• The other interns and normal employees for making it a great time
• The people in the kitchen for protecting me from months of Fast Food
Bastian Steder - Object Recognition in 3D scans - intern presentation 25
Questions?