an introduction to visual servoing in roboticstatsambo/publications/intro_visual_servoing.pdfan...
TRANSCRIPT
![Page 1: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/1.jpg)
1
Computer Vision and Robotics Research Group
Dept. of Computing Science, University of Alberta
http://webdocs.cs.ualberta.ca/~vis/research.htm
Page 1
An Introduction to Visual Servoing in Robotics
R. Tatsambon Fomena and C. Perez
![Page 2: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/2.jpg)
2
Page 2
Visual servoing: the control concept
HRI
Specification
of Goal S*
World ACTION
PERCEPTION
Robot
+
Camera(s)
S*
S
+ -
perception for action
![Page 3: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/3.jpg)
3
Page 3
Visual servoing: why visual sensing?
How to control the position of the end-effector of a robot with
respect to an object of unknown location in the robot base frame?
How to track a moving target?
A visual sensor provides relative position information
![Page 4: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/4.jpg)
4
Page 4
Visual servoing: how can you use visual data in control?
Look then move
Visual feedback control loop
ACTION
PERCEPTION
Robot
+
Camera(s)
S*
S
- +
ACTION
PERCEPTION
Robot
+
Camera(s)
S-S*
![Page 5: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/5.jpg)
5
Page 5
Quiz
Is visual servoing:
a) an open loop control
approach?
b) a closed-loop control
approach?
![Page 6: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/6.jpg)
6
Page 6
Visual servoing: Ingredients for a fully integrated system
HRI
Visual tracking method
Motion control algorithm
HRI
Specification
of Goal S*
ACTION
PERCEPTION
Robot
+
Camera(s)
S*
S
![Page 7: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/7.jpg)
7
Page 7
Visual servoing: HRI
Important for task specification
• point to point alignment for gross motions
• points to line alignment for fine motions
Should be easy and intuitive
![Page 8: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/8.jpg)
8
Page 8
Visual servoing: Visual tracking
Crucial as it provides the necessary visual feedback
• coordinates of image points or lines
Should be reliable and accurate.
Camshift color tracker
provides 2D (x,y) coordinates
of the tracked objects
PERCEPTION
Current image
Tracker searches
for the end-effector
S
S=(x,y)
Selection of the set of
Measurements to use for
control
![Page 9: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/9.jpg)
9
Page 9
Visual servoing: motion control algorithm
3 possible control methods depending on the selection of S: 2D,
3D, and 2 ½ D
(Corke, PhD, 94)
High bandwidth requires precise calibration: camera and robot-camera
3D VS
2D VS
2 ½ D VS
![Page 10: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/10.jpg)
10
Page 10
Visual servoing: motion control algorithm
Key element is the model of the system
3D VS
2D VS
2 ½ D VS
Robustness to image noise, calibration errors
Suitable for unstructured environments
(Corke, PhD, 94)
2D VS 2 ½ D VS 3D VS
Abstraction for control
![Page 11: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/11.jpg)
11
Page 11
Quiz
Which control method is more
sensitive to image noise? 1-a) 2D 2-a) 2 1/2D
1-b) 3D 2-b) 2D
1-c) 2 ½ D
![Page 12: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/12.jpg)
12
Page 12
Visual servoing: motion control algorithm
Key element is the model of the system: how does the
image measurements S change with respect to changes in robot
configuration q?
can be seen as a sensitivity matrix
![Page 13: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/13.jpg)
13
Page 13
Visual servoing: motion control algorithm
How to obtain ?
1) Machine learning technique
• Estimation using numerical methods, for example Broyen
2) Model-based approach
• Analytical expression using the robot and the camera projection model
• Example S=(x,y)
![Page 14: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/14.jpg)
14
Page 14
Visual servoing: motion control algorithm
How to move the robot knowing e = S-S* and ?
Classical approach: the control law imposes an exponential decay
of the error
Classical control
![Page 15: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/15.jpg)
15
Page 15
Visual servoing: motion control algorithm
:=VisualTracker(InitImage)
Init =
Init
While ( > T ) {
CurrentImage := GrabImage(camera)
:= VisualTracker(CurrentImage)
Compute =
Estimate
Compute
Change robot configuration with }
![Page 16: An Introduction to Visual Servoing in Roboticstatsambo/publications/Intro_visual_servoing.pdfAn Introduction to Visual Servoing in Robotics R. Tatsambon Fomena and C. Perez . 2 Page](https://reader030.vdocument.in/reader030/viewer/2022040214/5ec8e6a353da3202490ff49f/html5/thumbnails/16.jpg)
16
Page 16
Now do it yourself!