modular components to be re-used in future years tasks and team 1. robot graphical user interface...
TRANSCRIPT
![Page 1: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/1.jpg)
Rescue Robot Project
![Page 2: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/2.jpg)
2
Project overview
Modular components to be re-used in future years
Tasks and Team1. Robot graphical user interface (Zwivhuya Tshitovha)2. Robot control interface (Jaco Colyn)3. Robot visual system (Jonathan Dowling)
8 Masters students working in the Robotics and Agents lab
![Page 3: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/3.jpg)
3
Importance of Research
Reduce risk for rescue workers Map generation Rescue tasks Detection and localisation of victims Detection of hazardous materials
Disasters that have used robots: Japan nuclear disaster Three Mile Island nuclear disaster World Trade Centre terrorist attack
![Page 4: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/4.jpg)
4
Rescue robots in disasters
Usage in Japan nuclear disaster
And in the World Trade Centre
![Page 5: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/5.jpg)
5
Rescue robots in disasters
World trade centre (continued)
![Page 6: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/6.jpg)
6
The UCT Rescue Robot
![Page 7: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/7.jpg)
7
Research Questions
How can an effective system be built to automatically detect simulated disaster zone objects? human victims hazmat signs rolling E’s
Using the normal IR cameras available on the robot and
Applying existing computer vision techniques to extract the features
![Page 8: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/8.jpg)
8
Research Questions
How can a usable and efficient GUI be built by hiding information and preventing sensory overload?
![Page 9: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/9.jpg)
9
Research Questions
How can an intuitive robot control system be created by mapping the robots functions onto a human interface device?
![Page 10: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/10.jpg)
10
RoboCup Rescue Robot overview
RoboCup – an international organisation based in Sweden
Urban search and rescue tests
![Page 11: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/11.jpg)
11
Tests and Competitions
Simulate human subjects and disaster environments
Robot vision tests Rolling E’s Hazmat signs Victim detection
Remote operation of robot through competition arena to carry out tasks such as retrieving an object
![Page 12: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/12.jpg)
12
Rolling E’s
![Page 13: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/13.jpg)
13
Hazardous Materials (Hazmat)
![Page 14: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/14.jpg)
14
Victim detection
![Page 15: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/15.jpg)
15
Deliverables
Client for software engineering project: Stephen Marais from Robotics and Agents Lab
The deliverables required for this project are: A visual system for the robot to be able to pass the
“rolling E’s” test Hazardous material signs detection Human victim detection Easy to use interface for controlling the robot via a
human interface device Intelligent graphical user interface that highlights
only important information to the operator
![Page 16: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/16.jpg)
Robot Control InterfaceGraphical User Interface
Goals and challenges:
![Page 17: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/17.jpg)
17
Fundamental Robot Control Goals
Driving the robot
Controlling the robotic arm
Adjusting the robots tracks
![Page 18: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/18.jpg)
18
Prevent Operator Mistakes No direct line of
sight to the robot
Robot is “on its own” Has to return by itself
Prevent Falling over Driving out of range
![Page 19: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/19.jpg)
19
Robot Control Challenges Efficient control
Single Teleoperator Reduce fatigue Interpret and perform commands in real time
Reduce input delay “lag” Efficient mapping of commands to the control
interface
Micire, M., Mccann, E., Desai, M., Tsui, K.M., Norton, A., and Yanco, H.A. Hand and Finger Registration for Multi-Touch Joysticks on Software-Based Operator Control Units. (2011), 88-93.
![Page 20: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/20.jpg)
20
Graphical User Interface challenges
Designing a “clean” Graphical User Interface (GUI)
Reducing sensory overload – which leads to fatigue
Relay locations of important objects that cannot be directly seen Only seen through the control interface
and GUI Interface must provide situational
awareness
![Page 21: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/21.jpg)
21
Robot Control Interface RC & GUI
Iterative and Incremental software engineering.
Human Computer Interaction (HCI) Human Robot Interaction (HRI)
Use case scenarios taken from Urban Rescue Scenarios
![Page 22: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/22.jpg)
Robot Vision(RV)
Goals and challenges:
![Page 23: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/23.jpg)
23
Fundamental RV goals
Detect Human body parts (primarily limbs)
Detect and identify Hazmat signs
Detect the “rolling-E”
![Page 24: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/24.jpg)
24
Robot vision system
Input stream of video The general procedure
Transform the image’s data into a simpler representation
Apply a machine learning algorithm onto the transformed data to detect features.
![Page 25: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/25.jpg)
Testing
![Page 26: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/26.jpg)
26
Testing
Robot control interface and GUI User centred design approach. Qualitative and Quantitative results Feedback can be generated from ▪ Test subjects ▪ Asked to complete specific tasks using the interface.
Performance measurements and quantitative results from simulation testing. Hazmat signs Human body parts Rolling “E”
![Page 27: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/27.jpg)
27
Testing
Requirements RoboCup Rescue Competition (RRC) Urban Search and Rescue
Initial testing on each sub-system
Final testing on the system as a whole Based on expected challenges in the
RRC
![Page 28: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/28.jpg)
28
Ethical, Professional and Legal Issues
Demonstrate its capabilities on dummies
No external users because robot quite expensive
Ethical clearance
![Page 29: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/29.jpg)
29
Related Work
Robot vision Mikolajczyk et al [1]
Robot control and Interface Richer et al [2] Adams [3]
![Page 30: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/30.jpg)
30
Related Work cont...
Team Description Papers 2003 Team RoBrno [4]
![Page 31: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/31.jpg)
31
Resources
Hardware 19 inch wide screen Robot Human interface device
Software OpenCV Bosch SDK ROS
![Page 32: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/32.jpg)
32
Risks
Accessing the robot
Robot hardware damage
Missing project milestones
![Page 33: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/33.jpg)
33
Anticipated Outcomes
System used for a rescue robot that is developed by the Robotics and Agents Research Laboratory.
Key success factors Robot vision -- Accurate detection in smallest
time Robot interface and control -- Usable and
efficient given by measure of time and effort
![Page 34: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/34.jpg)
Timeline
![Page 35: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/35.jpg)
35
Timeline
![Page 36: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/36.jpg)
36
Timeline
![Page 37: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/37.jpg)
37
Timeline
![Page 38: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/38.jpg)
38
References
[1] Mikolajczyk, K,Schmid, C & Zisserman ,A 2004, „Human detection based on a probabilistic assembly of robust part detectors‟ . European Conference on Computer Vision, Oxford, United Kingdom.
[2] Adams, JA 2002,‟Critical Considerations for Human-Robot Interface Developmen‟, Proceedings of 2002 AAAI Fall Symposium, 2002 - aaai.org, Rochester Institute of Technology Rochester, New York.
[3] Richer, J & Drury, JL 2006 „A Video Game-Based Framework for Analyzing Human-Robot Interaction ‟, Characterizing Interface Design in Real-Time Interactive Multimedia Applications. MITRE Corporation, United states of America
[4] Uschin, K, Inteam, S, Sae-eaw,N, Trakultongchai, A, Klairith, P, Jitviriya, W & Jenkittiyon , P,‟ RoboCup Rescue 2009 - Robot League Team iRAP_Pro (Thailand)‟,King Mongkut‟s University of Technology North Bangkok (KMUTNB) , Bangkok Thailand.
![Page 39: Modular components to be re-used in future years Tasks and Team 1. Robot graphical user interface (Zwivhuya Tshitovha) 2. Robot control interface](https://reader030.vdocument.in/reader030/viewer/2022032717/56649ced5503460f949bb0e2/html5/thumbnails/39.jpg)
39
Questions
?