handling task prioritization as a product of gaussians · representing task prioritization as a...
TRANSCRIPT
![Page 1: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/1.jpg)
Handling task prioritization as a product of Gaussians
Sylvain CalinonSenior ResearcherIdiap Research Institute, Martigny, Switzerland
LecturerEPFL, Lausanne, Switzerland
![Page 2: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/2.jpg)
Research Groups:• Speech & Audio Processing• Natural Language Understanding• Perception & Activity Understanding• Machine Learning• Social Computing• Biometrics Security and Privacy• Biosignal Processing• Computational Bioimaging• Energy Informatics • Uncertainty Quantification and Optimal Design• Robot Learning & Interaction
Artificial Intelligence for Society
MARTIGNY
ResearchEducation
Technology transfer
![Page 3: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/3.jpg)
Learning from demonstration Observational learning
Kinesthetic teachingCorrespondence problems
![Page 4: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/4.jpg)
Superposition Fusion
t=0 t=1 t=2t=0 t=1 t=2
![Page 5: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/5.jpg)
Motivating example:
A probabilistic view on segment crossing!
Many algorithms in robotics can be re-interpreted as products of Gaussians, including task prioritization!
(superposition)
(fusion)
center of the Gaussian
covariance matrix
precision matrix
![Page 6: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/6.jpg)
Combination of primitives as a fusion problem
Scalar superposition as a product of Gaussians with :
![Page 7: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/7.jpg)
Combination of primitives as a fusion problem
The full weight matrices approach covers both scalar weights
(with isotropic diagonal matrix) and null space projection
operations!
Null space projection(hierarchy constraints)
Scalar superposition
![Page 8: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/8.jpg)
Tasks prioritization as PoGPrincipal task:track horizontal reference
Secondary task:track desired posture
Principal task
Secondary task
Principal task
Secondary task
![Page 9: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/9.jpg)
Learning tasks prioritization
Parallel organization of motion/skill primitives
![Page 10: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/10.jpg)
Task-parameterized Gaussian mixture model (TP-GMM)
[Canal, Pignat, Alenya, Calinon and Torras, ICRA’2018]
![Page 11: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/11.jpg)
Calinon, S. (2016). A Tutorial on Task-Parameterized Movement Learning and Retrieval. Intelligent Service Robotics (Springer), 9:1, 1-29.
Task-parameterized Gaussian mixture model (TP-GMM)
![Page 12: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/12.jpg)
[Calinon, Alizadeh and Caldwell, IROS’2013]
Coordinatesystem as task
parameter
Task-parameterized Gaussian mixture model (TP-GMM)
![Page 13: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/13.jpg)
Candidate hierarchy
Candidate hierarchy
Demonstration
Reproduction
Demonstration
Reproduction
[Silvério, Calinon, Rozo and Caldwell, IEEE T-RO 2019]
Priority on left hand
Learning tasks prioritization
Dr João Silvério
![Page 14: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/14.jpg)
Candidate hierarchy
Candidate hierarchy
Demonstration
Reproduction
Demonstration
Reproduction
Learning tasks prioritization
[Silvério, Calinon, Rozo and Caldwell, IEEE T-RO 2019]
Priority on right hand
Dr João Silvério
![Page 15: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/15.jpg)
Candidate hierarchy
Candidate hierarchy
Demonstration
Reproduction
Demonstration
Reproduction
Learning tasks prioritization
[Silvério, Calinon, Rozo and Caldwell, IEEE T-RO 2019]
Equal priority
Dr João Silvério
![Page 16: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/16.jpg)
Learning tasks prioritization
[Silvério, Calinon, Rozo and Caldwell, IEEE T-RO 2019]
Centaurorobot
Dr João Silvériobase position > end-effector positions > end-effector orientations
![Page 17: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/17.jpg)
Learning tasks prioritization
[Silvério, Calinon, Rozo and Caldwell, IEEE T-RO 2019]
Centaurorobot
Dr João Silvérioend-effector positions > base position > end-effector orientations
![Page 18: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/18.jpg)
Ongoing work
![Page 19: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/19.jpg)
Learning tasks prioritization
Expert A>B>CExpert A>C>BExpert B>A>CExpert B>C>AExpert C>A>BExpert C>B>A
Expert A>BExpert B>AExpert B>CExpert C>B
![Page 20: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/20.jpg)
Geodesic interpolation between nullspace structures
Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics on symmetric positive semidefinite manifolds)
Task 1 Task 2
Task 1 > Task 2 Task 2 > Task 1
Task 1 Task 1 Task 2Task 2
![Page 21: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/21.jpg)
Exploiting redundancy in planning tasks
[Girgin and Calinon, arXiv:1905.09679, 2019]Hakan Girgin
![Page 22: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/22.jpg)
Products of Uni-Gauss experts
Uni-Gauss experts are distributions combining the task constraint in the form of a Gaussianwith a uniform distribution
Intl Conf. on Artificial Neural Networks (ICANN’99)
![Page 23: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/23.jpg)
Product of Gaussians
Product of Uni-Gauss experts
with the same πi
(no hierarchy; simply go from OR to AND)
Product of Uni-Gauss experts
with different πi
(task 1 is set as being obligatory)
Products of Uni-Gauss experts
Instead of providing a solution as an exact compromise between objectives (which might fulfill none of them), Uni-Gauss experts allow to represent tasks in which we might prefer to fulfill one or another, but not both.
![Page 24: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/24.jpg)
Talos robot
Products of Uni-Gauss expertsVariational inference can be used to create distributions of good and diversifiedconfigurations.
Combined with Uni-Gauss experts, the system can sample robot configurations satisfying constraints.
[Pignat, Lembono and Calinon, arXiv:1905.09597, 2019]
Emmanuel Pignat
![Page 25: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/25.jpg)
Conclusion
A product of Gaussians can be used to represent nullspace projection operations and weighted superposition of tasks (fusion problem)
Learning task prioritization can be achieved by a statistical analysis of the data transformed with different candidate projections (TP-GMM)
Demo
Repro
Variational inference with Uni-Gauss distributions is a promising approach to learn task constraints individually, and then combine them with prioritization
Nullspace structures are present in a wide range of robotics problems, including planning problems computed with model predictive control (MPC)
![Page 26: Handling task prioritization as a product of Gaussians · Representing task prioritization as a product of Gaussians provides a way to geometrically interpolate task hierarchies (geodesics](https://reader033.vdocument.in/reader033/viewer/2022042812/5fab5e76f679fe4e3d592823/html5/thumbnails/26.jpg)
Robot Learning & Interaction Group at Idiap:
Contact:
[email protected]://calinon.ch
Source codes (Matlab/Octave, C++ and Python):
http://www.idiap.ch/software/pbdlib/
Photo: Basilio Noris
Thibaut Kulak
Emmanuel Pignat
NoémieJaquier
Dr Antonio Paolillo
HakanGirgin
MartinTroussard
TeguhLembono
?