bryan willimon, steven hickson, ian walker, and stan birchfield clemson university iros 2012 -...
TRANSCRIPT
![Page 1: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/1.jpg)
Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield
Clemson UniversityIROS 2012 - Vilamoura, Portugal
An Energy Minimization Approach to 3D Non-Rigid Deformable Surface Estimation Using
RGBD Data
![Page 2: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/2.jpg)
We propose an algorithm that uses energy minimization to estimate the current configuration of a highly non-rigid object.
Our approach relies on a 3D nonlinear energy minimization framework to solve for the configuration using a semi-implicit scheme adapted from Fua and colleagues (Pilet et al. 2005).
Overview
![Page 3: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/3.jpg)
Previous Work on Pose Estimation for Robotics
• Elbrechter et al. (IROS 2011) use a soft-body-physics model with visual tracking to manipulate a piece of paper.
• Bersch et al. (IROS 2011) describe a method to bring a T-shirt into a desired configuration by alternately grasping the item with two hands, using a fold detection algorithm.
Both approaches require predefined fiducial markers.
![Page 4: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/4.jpg)
The purpose of this approach is to minimize the energy equation of a mesh model that involves 4 terms:
Smoothness term
Energy Minimization Approach
Correspondence term
Depth term
Boundary term
![Page 5: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/5.jpg)
The purpose of this approach is to minimize the energy equation of a mesh model that involves 4 terms:
Energy Minimization Approach
![Page 6: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/6.jpg)
Mesh Initialization
Energy Minimization Approach
![Page 7: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/7.jpg)
Energy Minimization Approach
Smoothness term
Correspondence term
Depth term
Boundary term
![Page 8: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/8.jpg)
Smoothness term
Energy Minimization Approach
![Page 9: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/9.jpg)
Correspondence term
Energy Minimization Approach
![Page 10: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/10.jpg)
Depth term
Energy Minimization Approach
Front View Top View
![Page 11: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/11.jpg)
Boundary term
Without BoundaryWith Boundary
Energy Minimization Approach
![Page 12: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/12.jpg)
Energy Minimization Approach
Minimize energy equation
![Page 13: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/13.jpg)
Experimental Results
• We captured RGBD video sequences of shirts and posters to test our proposed method’s ability to handle different non-rigid objects in a variety of scenarios.
Four experiments were conducted:
1) Illustrating the contribution of the depth term
2) Illustrating the contribution of the boundary term
3) Partial self-occlusion
4) Textureless shirt sequence
![Page 14: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/14.jpg)
Experimental Results
• Illustrating the contribution of the depth term
![Page 15: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/15.jpg)
Experimental Results
• Illustrating the contribution of the boundary term
![Page 16: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/16.jpg)
Experimental Results
• Partial self-occlusion
![Page 17: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/17.jpg)
Experimental Results
• Textureless shirt sequence
![Page 18: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/18.jpg)
Experimental Results Video
![Page 19: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/19.jpg)
ConclusionWe have presented an algorithm to estimate the 3D configuration of a highly non-rigid object through a video sequence using feature point correspondence, depth, and boundary information.
We plan to extend this research to handle a two-sided 3D triangular mesh that covers both the front and the back of the object.
![Page 20: Bryan Willimon, Steven Hickson, Ian Walker, and Stan Birchfield Clemson University IROS 2012 - Vilamoura, Portugal An Energy Minimization Approach to 3D](https://reader036.vdocument.in/reader036/viewer/2022062422/56649f125503460f94c25383/html5/thumbnails/20.jpg)
Questions?