fingertip placement planning for unknown object grasping
DESCRIPTION
Fingertip Placement Planning for Unknown Object Grasping. Natchanon Wongwilai Adviser: Nattee Niparnan , Ph.D. M.Eng . Outline. Introduction How to grasp?, Why failed to grasp?, Goal Related Works Vision-based grasping, Manipulation under uncertainty Our Problem - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/1.jpg)
1
Fingertip Placement Planning for Unknown Object GraspingNatchanon WongwilaiAdviser: Nattee Niparnan, Ph.D.M.Eng.
![Page 2: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/2.jpg)
2
Outline• Introduction• How to grasp?, Why failed to grasp?, Goal
• Related Works• Vision-based grasping, Manipulation under uncertainty
• Our Problem• Challenge, Proposed method
• Etc.• Scopes, Work plan
![Page 3: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/3.jpg)
3
Robotic Grasping
[http://spectrum.ieee.org/robotics/robotics-software/slideshow-born-bionic/0]
!?
???
??
?
Modelµ = 0.53
W = 39 g
![Page 4: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/4.jpg)
4
How to grasp?
?
![Page 5: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/5.jpg)
5
Unknown object grasping• 2D [Borst el at.,00; Chinellato el at.,05; Calli el
at.,11; ...]• 3D [Miller el at.,03; Goldfeder el at.,07; Hubner el
at.,08; ...]• 2.5D [Richtsfeld el at.,08; ...]• Others [Saxena el at.,08; ...]
![Page 6: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/6.jpg)
6
Grasping in Real World• (Video)
![Page 7: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/7.jpg)
7
Why failed to grasp?“The most common failure mode I've seen is that the closing fingers bump the object so that the fingers don't touch the intended contact points. Then the fingers knock the object completely out of the grasp. I think the causes are localization errors from the perception system and asking the robot to carry out an inherently dynamic task that was planned with static analysis tools.”
Jeff TrinkleGRSSP Workshop 2010
“The most common failure mode I've seen is that the closing fingers bump the object so that the fingers don't touch the intended contact points. Then the fingers knock the object completely out of the grasp. I think the causes are localization errors from the perception system and asking the robot to carry out an inherently dynamic task that was planned with static analysis tools.”
Jeff TrinkleGRSSP Workshop 2010
![Page 8: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/8.jpg)
8
Why failed to grasp?• Contact position error• Theory vs. Practical
• Cause of error• Sensor• Control• Computation
Uncertainty
[http://www.cs.columbia.edu/~cmatei/]
![Page 9: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/9.jpg)
9
Goal• Accuracy of fingertip placement• Planning• Using camera
![Page 10: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/10.jpg)
10
SensorSensor Price Accuracy Data typeTactile sensor Expensive High Force arrayLaser range finder
Expensive High Range
Camera Vary Moderate Image
Tactile sensor[Bekiroglu el at., 11]
Laser range finder Camera
![Page 11: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/11.jpg)
11
Related works• Vision-based grasping• Stereo camera• Eye-in-hand camera
• Manipulation under uncertainty• Independent contact region• Visual servoing• Reactive grasping• Probabilistic model
![Page 12: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/12.jpg)
12
Vision-based grasping• Stereo vision based grasping• [Popovic et al.,11; Gratal el at., 12]
![Page 13: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/13.jpg)
13
Vision-based grasping• Eye-in-hand camera• [Walck el at., 10; Lippiello el at., 11; Calli el at., 11]
![Page 14: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/14.jpg)
14
• (Video)
Eye-in-hand grasping
![Page 15: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/15.jpg)
15Manipulation under uncertainty• Independent contact region• [Nilwatchararang et al., 08; Roa et al.,09]
![Page 16: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/16.jpg)
16Manipulation under uncertainty• Visual servoing• [Gratal el at., 12; Calli el at., 11]
![Page 17: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/17.jpg)
17Manipulation under uncertainty• Reactive Grasping• [Teichmann et al.,94; Hsiao et al.,09; Hsiao et al.,10]
![Page 18: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/18.jpg)
18Manipulation under uncertainty• Probabilistic model• [Laaksonen et al.,11; Dogar et al.,11; Platt et al.,11]
![Page 19: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/19.jpg)
19
Our Problem• Propose online planning method for
accurate fingertip placement under uncertainty using eye-in-hand camera
![Page 20: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/20.jpg)
20
Challenge• ACCURACY!!!
• Insufficient information• Bearing-only data• Unknown object model and properties• Don’t have any initial information• Close-up view with featureless image
• Kinematic constraint• Unreachable position• Object out of view
• Uncertainty• Unpredictable noise
![Page 21: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/21.jpg)
21
Proposed method
![Page 22: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/22.jpg)
22
How to grasp? (revisited)Modeling
Grasp planning
Localization
Grasping
![Page 23: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/23.jpg)
23
Our method
Grasping
Loca
lizat
ion
Mod
elin
gGr
asp
plan
ning
![Page 24: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/24.jpg)
24Simultaneous Localization And Mapping (SLAM)• Robot build up a map and localize itself
simultaneously while traversing in an unknown environment
[Paul Newman, 06]
![Page 25: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/25.jpg)
25
Localization & Modeling• Robot location Hand(Fingertips)
location
• Environment map Object model
![Page 26: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/26.jpg)
26
Probabilistic map
http://www.biorobotics.org/projects/tslam/experiments/slam1experiment.html
![Page 27: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/27.jpg)
27
SLAM algorithm• Probabilistic SLAM [Smith and Cheeseman, 86]• The probability distribution of robot state and landmark locations
• The observation model
• The motion model
![Page 28: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/28.jpg)
28
SLAM algorithm• SLAM recursive algorithm• Time-update
•Measurement Update
![Page 29: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/29.jpg)
29
Observation model• Feature detection• Point features, Line features
• Feature association• How features associate with landmarks
• Feature measurements• Observation model
[http://www.sciencedirect.com/science/article/pii/S0377042711002834]
![Page 30: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/30.jpg)
30
Map representation• How to represent a map (object model)
from available features
[http://www.deskeng.com/articles/aaayex.htm]
![Page 31: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/31.jpg)
31
Planning• Exploration• How to explore for object modeling
• Strategy• Close-up strategy• Out of view strategy
![Page 32: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/32.jpg)
32
Evaluation• Fingertips placement evaluation• Using ground truth data• Contact position marking
• Modeling evaluation• Using ground truth data from structural environment• Database• Kinect
![Page 33: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/33.jpg)
33
Scopes• Develop online planning method for
accurate fingertip placement using eye-in-hand camera
• Not develop algorithm to find grasping points
• No clutter in work space• Simple & Textured object
![Page 34: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/34.jpg)
34
Work Plan• Study the works in the related fields• Develop algorithms• Test the system• Evaluate a result• Prepare and engage in a thesis defense
![Page 35: Fingertip Placement Planning for Unknown Object Grasping](https://reader035.vdocument.in/reader035/viewer/2022062310/5681610c550346895dd05c37/html5/thumbnails/35.jpg)
35
Thank you