an effective user-guided interface for multi-robot search
TRANSCRIPT
![Page 1: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/1.jpg)
An Effective User-Guided
Interface For Multi-Robot Search
Shahar Kosti1, Gal A. Kaminka1,2, David Sarne1
1. Computer Science Department
2. Gonda Brain Science CenterBar-Ilan University, Israel
![Page 2: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/2.jpg)
Robotic Search (Exploration)
Gain knowledge of an unknown area
Human in the loop
Variety of applications
Urban Search and Rescue
– Locate and rescue in closed spaces
– Mapping, locating victims
Source: Chuck Simmins
![Page 3: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/3.jpg)
Single operator, single robot
Camera-based teleoperation
– The robot carries camera and sensors
– Controlled by an input device (joystick)
Operator tasks
– Steering and navigation
– Watch video
Requires
– High-bandwidth, low-latency
– Operator attention
Source: Foster-Miller
![Page 4: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/4.jpg)
Multiple robots
Navigating multiple robots - can be automated
How to display imagery from multiple robots?
Operator can watch multiple video feeds
– Hard to scale with the number of robots
![Page 5: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/5.jpg)
Asynchronous video
Robots explore autonomously
Operator watches recorded imagery
– Instead of live video
How to select the most relevant image?
How to display it?
![Page 6: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/6.jpg)
Image database
Input: recorded images, sensor data
For each image we keep:
– Robot location and heading
– Field-of-view (FOV) polygon
FOVRobot location and heading
![Page 7: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/7.jpg)
Screen layout
2D Map
Selected image
Relevant images (thumbnails)
![Page 8: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/8.jpg)
Image navigation – user action
Click on POI (point of interest)
![Page 9: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/9.jpg)
Image navigation - automatic
Rank and present images that cover POI
![Page 10: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/10.jpg)
Ranking process (point p)
Find all images that cover p
Group the images in sectors
For each sector
– Compute the utility value u of all images
Output:
– Best: highest-ranked images from each sector
– Other: all other images
![Page 11: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/11.jpg)
Ranking process output
Best images
Other images
Highest rankedBest image
![Page 12: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/12.jpg)
Grouping Images
Divide to sectors by resolution r
Group by angle between robot location, POI
Images that cover p
Point of interest - p
![Page 13: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/13.jpg)
Image utility computation
Goals:
– Maximize image area
– Minimize distance from POI
– The image should be centered as possible
Linear combination of the above
– Weights determined in a pilot session
![Page 14: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/14.jpg)
Interface evaluation
Two asynchronous interfaces
Our interface – POI
State of the art – Best-First
– By Wang et al
Both assume fully-autonomous robots
![Page 15: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/15.jpg)
Best-first interface
Images are ordered by utility (unseen area)
Image navigation – “Next” button
– Displays the highest-ranked image
Thumbnails are ordered chronologically
Map is used only to locate victims
![Page 16: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/16.jpg)
Best-first example
![Page 17: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/17.jpg)
Simulated environment
Two environments for USAR
– Office environments, generated with USARSim
– 20 human characters (“victims”)
– One larger than the other
Pre-recorded data (not “live”)
Simulated robot steered manually
![Page 18: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/18.jpg)
Experiment design and participants
Experiment
– Mission: locate and mark victims
– Training session
– Two 10-minutes sessions
32 paid participants – adult students
– Balanced for gender and other conditions
– Fixed show-up fee, variable bonus for success
Between-subjects design
![Page 19: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/19.jpg)
Evaluating marks success
Set of marks for each participant
Assign marks to nearest victim
– 1m accuracy
Choose a category:
– Found
– Duplicate
– False positive (distance > 1m)
(distance ≤ 1m)
![Page 20: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/20.jpg)
Mark results - found
Map 2Map 1
![Page 21: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/21.jpg)
Correct marks / Time (Map 1)
![Page 22: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/22.jpg)
Correct marks / Time (Map 2)
![Page 23: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/23.jpg)
Conclusions
User-guided UI for robotic search missions
Improved performance under certain conditions
Allows flexible image selection
Can apply to other domains
![Page 24: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/24.jpg)
Future Work
Improve image storage and retrieval
Improve the ranking and grouping process
Support 3D mapping
Evaluate best-first and poi together
Evaluate our interface with real robots
![Page 25: An Effective User-Guided Interface For Multi-Robot Search](https://reader030.vdocument.in/reader030/viewer/2022032419/55a3442e1a28aba3038b45cd/html5/thumbnails/25.jpg)
Thank You!