automated 3d model building using a robot and an rgb-depth
TRANSCRIPT
Automated 3D Model Building Using a Robot and an RGB-Depth Camera
Jacob Kneibel, Kexiang Qian, Nick Saxton, David Zoltowski, Nick ZuzowECE480 Team 2, Michigan State University
Executive SummaryNew applications of RGB-Depth cameras are in high demand due to their widening availability to industry and to consumers. 3D modeling is one such application, and while done before in various research projects, the process of using RGB-Depth cameras to model a space has yet to become an automated process [1]. The goal of this project was to combine a robotic moving platform and a commercially available RGB-Depth camera to autonomously build a 3D model of an unknown environment.
Objectives● Fully navigate room
autonomously● Avoid collisions● Complete low-level (table height)
3D model ● Integrate color and/or texture
into model
Results
Budget
Sponsor: Dr. Daniel MorrisFacilitator: Dr. Hayder Radha
Number Item Cost ($)
1 iRobot Create2 Programmable Robot
211.99
2 Microsoft Kinect Camera for Xbox v2
149.99
3 Kinect to Windows Adapter 49.99
4 Power Supplies 67.06
5 Structure 50.00
Total - 529.03
Acknowledgements and ReferencesWe would like to acknowledge our sponsor, Dr. Daniel Morris, for his guidance and advice; our facilitator, Dr. Hayder Radha, for his suggestions and comments; Matthias Nießner and Michael Zollhoefer for their Voxel Hashing code and Matthias Nießner’s correspondence; and our team member, Kexiang Qian, for loaning his laptop for the project.
[1] "BIG." Projects. Bristol Interaction and Graphics. Web. 20 Feb. 2015. <http://big.cs.bris.ac.uk/projects/mobile-kinect/>.[2] Nießner, Matthias, et al. "Real-time 3d reconstruction at scale using voxel hashing." ACM Transactions on Graphics (TOG) 32.6 (2013): 169.
Components
Hardware● iRobot Create 2 Programmable
Robot● Microsoft Kinect for Windows v2● Laptop● Assembled structure on robot● Power Supplies
Software● Depth sensing application● 3D model building software: Real-
time 3D Reconstruction at Scale using Voxel Hashing [2]
● Python script controlling robot
Design
Conclusion
No
Method Flow Chart
Device
Depth Processing: Left: The direct depth data from Kinect 2 camera, the X and Y values are the pixels. Right: depth data transformed into an occupancy grid in front of robot, the Y axis corresponds to the X axis of Left.
Above are three models created from the Voxel Hashing software, that was used as our robot navigated the room using the depth data processing to decide the path.
We have made great strides towards accomplishing the initial goal: to create a fully autonomous system capable of creating a 3D model of an unknown environment.
Successes● Combined different equipment into one functional
machine● Able to detect most objects in the robots path● Capable of constructing a 3D model
Future Work● Increase in collision avoidance functionality● Room completion detection