![Page 1: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/1.jpg)
www.joanneum.at
Towards Intuitive Industrial Human-Robot Collaboration
System Design and Future Directions
Ferdinand Fuhrmann, Wolfgang Weiß, Lucas Paletta, Bernhard Reiterer, Andreas Schlotzhauer, Mathias
Brandstötter
![Page 2: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/2.jpg)
Outline
Introduction State-of-the-art Background The CollRob Project
Models, Functionalities and Use Cases
Outlook
2
![Page 3: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/3.jpg)
Introduction General
Human-Robot-Collaboration buzz-word in I4.0 Industry prepares first use cases – with certain limitation
What is Collaboration?
Combining skills and expertise to jointly achieve a goal Collaboration very high-level process involving causal and non-causal cognitive activities
3
![Page 4: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/4.jpg)
Introduction Project CollRob
4-year funded research project 8 research groups involved Aim 1: Enable H-R Collaboration
Planning Interaction Safety
Aim 2: design application scenarios
4
![Page 5: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/5.jpg)
State-of-the-art (1) Industrial point-of-view
Availability of sensitive and light weight robot arms which are safe enough to team with people Simple interactions between humans and robots Slow and therefore safe Little integration of the human aspects Implemented use cases
Collaborative assembly Service robots
5
![Page 6: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/6.jpg)
State-of-the-art (2) Consumer & Scientific point-of-view
Recent advances in artificial intelligence topics Natural language understanding, machine vision Collecting, merging and analyzing vast amount of data Machine learning and deep learning Autonomous systems …
Progress in research on Social robots, assistant robots Intelligent personal assistant, virtual assistants Chatbots, Spoken dialog systems
Availability of autonomous and intelligent consumer products Amazon Alexa, Siri, Cortana, … Pepper (SoftBank), Sony AIBO, … … and many more
6
![Page 7: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/7.jpg)
Collaboration Concepts and models for H-H collaboration extensively researched by psychology Basic characteristics of collaboration [1]
Shared activity Joint intention Common ground
Shared cooperative activity features: Mutual responsiveness Commitment to the joint activity Commitment to mutual support
7
[1] Bratman, M. E. (1992). Shared Cooperative Activity. The Philosophical Review, 101(2), 327. https://doi.org/10.2307/2185537
![Page 8: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/8.jpg)
A New Framework for Human-Robot-Collaboration High-level CollRob Model 8
Negotiation
Communication
Interaction
Coordination
Perceptiveness
Persuasion
Trusworthiness HRC
![Page 9: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/9.jpg)
Human-Robot-Interaction Intuitive Interaction
Natural Interaction is multimodal: Speech, Gaze, Gesture, ... Coupled Modalities: More natural, More robust! Context-sensitive feedback: relevant information!
Human-Factors
Situation Awarness Stress
9
![Page 10: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/10.jpg)
Modelling HRI Enabling intuitive Interaction and Communication Plan and decide what to communicate through which channel Theory-of-mind models
Is the human relaxed or stressed? Is she focused on her task? Does she know if I finished my task? Is the human aware of the goal and the steps how to reach the goal?
10
![Page 11: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/11.jpg)
Planning for Collaborative Agents Domain-independent planning towards collaborative autonomy
Knowledge is updated by sensor data Find actions for human-robot team to solve current problem
Specific challenges in collaborative systems
Dynamic environment Safety Interaction
11
https://flic.kr/p/nTPA1W
![Page 12: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/12.jpg)
Safety Aspects in Human Robot Collaboration Integrity of human health Safety in mobile manipulation and dynamic environments Ergonomics of work places
No safety without security
14
Safety verification by measuring the biomechanical load regarding ISO/TS15066
![Page 13: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/13.jpg)
Targeted Use Cases Building collaboratively tangram figures as a representative for an assembly task in the industry
One Person and one serial manipulator Sharing a common workspace Working a the same time
A sensitive mobile manipulator Delivering parts to workplaces Moving area is shared with other humans
16
![Page 14: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/14.jpg)
Evaluation of HRI: Human factors study on situation awareness
12 participants, moderated, dual task: Primary task: Read aloud (up to 6 pages at laptop), Secondary task: timed hand-over (domino brick) Robot cycle until hand-over event (cycle duration 18-26 sec.) OptiTrack mocap, eye tracking glasses 30 Hz, ABB Yumi robot, gripper, ROS synchronisation
Results: Real-time eye movement analysis Attention features correlate with SAL Classification of SAL with ≈92% accuracy Prediction of HRI performance ≈80%
18
Attention to robot: ready for hand-over ?
Primary task: reading aloud?
Secondary task: timed hand-over
![Page 15: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/15.jpg)
Next Research Questions Which granularity of communication between the human and the robot is needed to keep the human informed but not distracted from his work? How can the intention of the robot to the human be communicated to reduce (nearly) collisions between the agents and improve the performance of assembling parts? Which factors influence stress and perceived trustworthiness? How can the risk (of a collision) between the agents be described and predicted? Optimizing task planning for human robot teams when interaction results interfere the current plan
19
![Page 16: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/16.jpg)
Acknowledgements This research was funded by the Austrian Ministry for Transport, Innovation and Technology (BMVIT) within the framework of the sponsorship agreement formed for 2015 - 2018 under the project CollRob.
21
![Page 17: Towards Intuitive Industrial Human-Robot Collaborationmagazin.know-center.tugraz.at/downloads/2017/HCI Perspectives on... · Towards Intuitive Industrial Human-Robot Collaboration](https://reader030.vdocument.in/reader030/viewer/2022040313/5e0a53b2772f2249af37712c/html5/thumbnails/17.jpg)
JOANNEUM RESEARCH Forschungsgesellschaft mbH
DI Dr. Ferdinand Fuhrmann Steyrergasse 17
8010 Graz [email protected]
www.joanneum.at
22