enhancing geometric maps through environmental …1196889/...2016 ieee international symposium on...

Enhancing geometric maps throughenvironmental interactions

SERGIO S CACCAMO

Doctoral ThesisStockholm Sweden 2018

TRITA-EECS-AVL-201826ISBN 978-91-7729-720-8

Robotics Perception and LearningSchool of Electrical Engineering and Computer Science

KTH Royal Institute of TechnologySE-100 44 Stockholm Sweden

Copyright copy 2018 by Sergio S Caccamo except where otherwise stated

Tryck Universitetsservice US-AB 2018

iii

Abstract

The deployment of rescue robots in real operations is becoming increasingly com-mon thanks to recent advances in AI technologies and high performance hardwareRescue robots can now operate for extended period of time cover wider areas andprocess larger amounts of sensory information making them considerably more usefulduring real life threatening situations including both natural or man-made disasters

In this thesis we present results of our research which focuses on investigatingways of enhancing visual perception for Unmanned Ground Vehicles (UGVs) throughenvironmental interactions using different sensory systems such as tactile sensors andwireless receivers

We argue that a geometric representation of the robot surroundings built upon vi-sion data only may not suffice in overcoming challenging scenarios and show thatrobot interactions with the environment can provide a rich layer of new informationthat needs to be suitably represented and merged into the cognitive world model

Visual perception for mobile ground vehicles is one of the fundamental problemsin rescue robotics Phenomena such as rain fog darkness dust smoke and fire heav-ily influence the performance of visual sensors and often result in highly noisy dataleading to unreliable or incomplete mapsWe address this problem through a collection of studies and structure the thesis as fol-lowFirstly we give an overview of the Search amp Rescue (SAR) robotics field and discussscenarios hardware and related scientific questionsSecondly we focus on the problems of control and communication Mobile robotsrequire stable communication with the base station to exchange valuable informationCommunication loss often presents a significant mission risk and disconnected robotsare either abandoned or autonomously try to back-trace their way to the base stationWe show how non-visual environmental properties (eg the WiFi signal distribution)can be efficiently modeled using probabilistic active perception frameworks based onGaussian Processes and merged into geometric maps so to facilitate the SAR missionWe then show how to use tactile perception to enhance mapping Implicit environmen-tal properties such as the terrain deformability are analyzed through strategic glancesand touches and then mapped into probabilistic modelsLastly we address the problem of reconstructing objects in the environment Wepresent a technique for simultaneous 3D reconstruction of static regions and rigidlymoving objects in a scene that enables on-the-fly model generation

Although this thesis focuses mostly on rescue UGVs the concepts presented canbe applied to other mobile platforms that operates under similar circumstances Tomake sure that the suggested methods work we have put efforts into design of userinterfaces and the evaluation of those in user studies

iv

Sammanfattning

Anvaumlndandet av raumlddningsrobotar vid olyckor och katastrofer blir allt vanligaretack vare framsteg inom AI och annan informationsteknologi Dessa framsteg goumlr attraumlddningsrobotarna nu kan arbeta under laumlngre tidsperioder taumlcka stoumlrre omraringden ochbearbeta stoumlrre maumlngder sensorinformation aumln tidigare vilket aumlr anvaumlndbart i maringngaolika scenarier

I denna avhandling presenterar vi resultat som handlar om att komplettera de vi-suella sensorer som obemannade markfarkoster (Unmanned Ground Vehicle UGVer)ofta aumlr beroende av Exempel paring kompletterande sensorer aumlr taktila sensorer som tryck-sensorer laumlngst ut paring en robotarm och maumltningar av signalstyrkan i en traringdloumls foumlrbin-delse

Vi visar att en geometrisk representation av robotens omgivningar baserad paring en-dast visuell information kan vara otillraumlckligt i svaringra uppdrag och att interaktion medomgivningen kan ge ny avgoumlrande information som behoumlver representeras och integre-ras i robotens kognitiva omvaumlrldsmodell

Anvaumlndandet av visuell information foumlr skapandet av en omvaumlrldsmodell har laumlngevarit ett av de centrala problemen inom raumlddningsrobotik Foumlrutsaumlttningarna foumlr ettraumlddningsuppdrag kan dock aumlndras snabbt genom tex regn dimma moumlrker dammroumlk och eld vilket drastiskt foumlrsaumlmrar kvaliteacuten paring kamerabilder och de kartor somroboten skapar utifraringn dem

Vi analyserar dessa problem i en uppsaumlttning studier och strukturerar avhandlingenenligt foumlljande

Foumlrst presenterar vi en oumlversikt oumlver raumlddningsrobotomraringdet och diskuterar sce-narier haringrdvara och tillhoumlrande forskningfraringgor

Daumlrefter fokuserar vi paring problemen roumlrande styrning och kommunikation I deflesta tillaumlmpningar aumlr kommunikationslaumlnken mellan operatoumlr och raumlddningsrobot av-goumlrande foumlr uppdragets utfoumlrande En foumlrlorad foumlrbindelse innebaumlr antingen att robotenoumlverges eller att den paring egen hand foumlrsoumlker aringterfraring kontakten genom att ta sig till-baka den vaumlg den kom Vi visar hur icke-visuell omvaumlrldsinformation saring som WiFi-signalstyrka kan modelleras med probabilistiska ramverk som Gaussiska Processeroch infoumlrlivas med en geometrisk karta foumlr att moumljliggoumlra slutfoumlrandet av uppdraget

Sedan visar vi hur man kan anvaumlnda taktil information foumlr att foumlrbaumlttra omvaumlrldsmo-dellen Egenskaper saring som deformabilitet undersoumlks genom att roumlra omgivningen paringnoggrant utvalda staumlllen och informationen aggregeras i en probabilistisk modell

Till sist visar vi hur man kan rekonstruera objekt i omgivningen med hjaumllp av enmetod som fungerar paring baringde statiska objekt och roumlrliga icke-deformerbara kroppar

Trots att denna avhandling aumlr uppbyggd kring scenarier och behov foumlr raumlddnings-robotar kan de flesta metoder aumlven tillaumlmpas paring de maringnga andra typer av robotar somhar liknande problem i form av ostrukturerade och foumlraumlnderliga miljoumler Foumlr att saumlker-staumllla att de foumlreslagna metoderna verkligen fungerar har vi vidare lagt stor vikt vid valav interface och utvaumlrderat dessa i anvaumlndarstudier

v

Acknowledgments

Thanks to my supervisors Dani and PetterDani thank you for your precious advice which have helped me in countless situationsYou have taught me the art of being constructively self-critical a not-so-obvious skill totake myself through the academic life and become a researcherPetter your knowledge and your enthusiasm have been an inspiration in both my profes-sional and my personal life I have enjoyed our conversations you have the power ofmaking any subject fun easing it up even in the toughest of timesThanks to all of the RPL professors for their useful insights and for making the lab such apleasant fair and stimulating environmentThank you John for your feedback and assistance in improving this thesisMy sincere gratitude goes to all of the great people at RPL and CVAP with whom I haveshared amazing trips fruitful conversations and countless laughsFredrik your generosity and genuineness helped me a lot during these years I am happyto have shared the office with youThank you to all my colleagues and friends in ETH CTU Fraunhofer DFKI TNOTU Delft University La Sapienza Ascendic Technologies and fire departments of Italy(VVF) Dortmund (FDDO) and Rozenburg (GB) for making TRADR such a fun wonder-ful and successful experienceI am grateful to MERL and to all of the wonderful and generous people I have met inBoston in special account to Esra and Yuichi for recruiting and welcoming me into theirlab in CambridgeThank you dear innebandy fellows for all of those awesome and very much needed gamesMy friends thank you all for being there when I needed you mostMy family Giovanni Maria and Daniele and my grandmas Grazie dal profondo delcuore Con il vostro amore incondizionato siete la mia forza e il mio rifugio in tempi diffi-ciliFinally I want to express my deepest gratitude to Carlotta Non sarei mai arrivato dovesono senza il tuo constante supporto i tuoi consigli il tuo affetto la tua forza e queldono meraviglioso che hai di riuscire sempre a sorridere e vedere il buono nelle cose inqualunque circostanza Ti ammiro e ti sarograve per sempre riconoscenteI gratefully acknowledge funding under the European Unionrsquos seventh framework program(FP7) under grant agreements FP7-ICT-609763 TRADR

This thesis is dedicated to my grandpas Salvo and Vincenzo I miss you dearly

Sergio CaccamoStockholm March 2018

vi

List of Papers

The thesis is based on the following papers

[A] Fredrik Baringberg Sergio Caccamo Nanjia Smets Mark Neerincx and PetterOumlgren Free Look UGV Teleoperation Control Tested in Game Environ-ment Enhanced Performance and Reduced Workload In Proceedings of the2016 IEEE International Symposium on Safety Security and Rescue Robotics(SSRRrsquo16) Lausanne Switzerland October 2016

[B] Sergio Caccamo Ramviyas Parasuraman Fredrik Baringberg and Petter OumlgrenExtending a ugv teleoperation flc interface with wireless network connectivityinformation In Proceedings of the 2015 IEEERSJ International Conferenceon Intelligent Robots and Systems (IROSrsquo15) Hamburg Germany September2015

[C] Ramviyas Parasuraman Sergio Caccamo Fredrik Baringberg Petter Oumlgren andMark Neerincx A New UGV Teleoperation Interface for Improved Aware-ness of Network Connectivity and Physical Surroundings In Journal ofHuman-Robot Interaction (JHRI) December 2017

[D] Sergio Caccamo Ramviyas Parasuraman Luigi Freda Mario Gianni and Pet-ter Oumlgren RCAMP Resilient Communication-Aware Motion Planner andAutonomous Repair of Wireless Connectivity in Mobile Robots In Proceed-ings of the 2017 IEEERSJ International Conference on Intelligent Robots andSystems (IROSrsquo17) Vancouver Canada September 2017

[E] Sergio Caccamo Yasemine Bekiroglu Carl Henrik Ek and Danica KragicActive Exploration Using Gaussian Random Fields and Gaussian Process Im-plicit Surfaces In Proceedings of the 2016 IEEERSJ International Confer-ence on Intelligent Robots and Systems (IROSrsquo16) Daejeon Korea October2016

[F] Sergio Caccamo Puumlren Guler Hedvig Kjellstroumlm and Danica Kragic ActivePerception and Modeling of Deformable Surfaces using Gaussian Processesand Position-based Dynamics In Proceedings of the 2016 IEEERAS Interna-tional Conference on Humanoid Robots (HUMANOIDSrsquo16) Cancun MexicoNovember 2016

[G] Sergio Caccamo Esra Ataer-Cansizoglu Yuichi Taguchi Joint 3D Recon-struction of a Static Scene and Moving Objects In Proceedings of the 2017International Conference on 3D Vision (3DVrsquo17) Qingdao China October2017

vii

Other relevant publications not included in the thesis

[A] Wim Abbeloos Esra Ataer-Cansizoglu Sergio Caccamo Yuichi TaguchiYukiyasu Domae 3D Object Discovery and Modeling Using Single RGB-D Images Containing Multiple Object Instances In Proceedings of the 2017International Conference on 3D Vision (3DVrsquo17) Qingdao China October2017

[B] Wim Abbeloos and Sergio Caccamo 1 E Ataer-Cansizoglu Y TaguchiC Feng Teng-Yok Lee Detecting and Grouping Identical Objects for RegionProposal and Classification In Proceedings of the 2017 IEEE The Conferenceon Computer Vision and Pattern Recognition (CVPRrsquo17) Workshop - DeepLearning for Robotic Vision Honolulu Hawaii July 2017

[C] Ramviyas Parasuraman Sergio Caccamo Petter Oumlgren Byung-Cheol MinAn Approach to Retrieve from Communication Loss in Field Robots In IEEERobotics Science and Systems (RSSrsquo17) Workshop - Robot Communicationin the Wild Meeting the Challenges of Real-World Systems Cambridge Mas-sachusetts USA July 2017

[D] D Almeida R Ambrus S Caccamo X Chen S Cruciani J F Pinto BDe Carvalho J Haustein A Marzinotto F E Vintildea B Y Karayiannidis POumlgren P Jensfelt and D Kragic Team KTHs Picking Solution for the Ama-zon Picking Challenge 2016 In The 2017 IEEE International Conferenceon Robotics and Automation (ICRArsquo17) Workshop - Warehouse Picking Au-tomation Workshop 2017 Solutions Experience Learnings and Outlook ofthe Amazon Picking Challenge Singapore China May 2017

[E] Fredrik Baringberg Yaquan Wang Sergio Caccamo Petter Oumlgren Adaptive ob-ject centered teleoperation control of a mobile manipulator In The 2016 IEEEInternational Conference on Robotics and Automation (ICRArsquo16) StockholmSweden May 2016

1 indicates equal contribution to this work

viii

List of Acronyms

AP Wireless Access Point Active PerceptionAUV Autonomous Underwater VehicleCAMP Communication Aware Motion PlannerCNN Convolutional Neural NetworkDARPA Defense Advance Research Project AgencyDNN Deep Neural NetworkDoA Direction of Arrival of a Radio SignalFLC Free Look ControlFEM Finite Element MethodGP Gaussian ProcessGPR Gaussian Process for RegressionGPIS Gaussian Process Implicit SurfaceGRF Gaussian Random FieldHMI Human Machine InterfaceIP Interactive PerceptionMSM Meshless Shape MatchingOCU Operator Control UnitPBD Position-based DynamicsRCAMP Resilient Communication Aware Motion PlannerROV Remote Operated VehicleRSME Rating Scale Mental EffortRSS Radio Signal StrengthSampR Search And RescueSA Situation AwarenessSAR Search And RescueSCE Situated Cognitive EngineeringSLAM Simultaneous Localization And MappingTRADR The EU project Long Term Human Robot Teaming for Disaster ResponseTC Tank ControlUAV Unmanned Aerial VehicleUGV Unmanned Ground VehicleUSV Unmanned Surface VehicleUSAR Urban Search And RescueUUV Unmanned Underwater or Undersea VehicleWMG Wireless Map Generator

Contents

Contents ix

I Introduction 1

1 Introduction 311 Robotic systems that save lives 412 Vision in SAR robotics 513 Contributions and thesis outline 6

2 Disaster Robotics 1121 Overview 1122 Historical Perspective 1323 Types of Rescue Robots 1524 The Userrsquos Perspective 1725 Notable SAR Research Projects Groups and Challenges 1826 SAR Problems Addressed in This Thesis 19

261 Control 20262 Communication 21263 Human-robot interaction 21264 Mapping 22

3 Enhancing perception capabilities 2331 Visual Perception System 23

311 RGBD cameras and LiDARs 23312 Limitation of vision sensors 24

32 Benefits of Active and Interactive Perception 2533 Geometric Representations 2634 A Probabilistic Approach to Interactive Mapping 27

341 Gaussian Random Fields 28342 Gaussian Processes Implicit Surfaces 28

35 Strategies for training and interacting 2936 Mapping the environment and moving objects 31

ix

x CONTENTS

4 Conclusions 3541 Future Work 36

5 Summary of Papers 39A Free Look UGV Teleoperation Control Tested in Game Environment En-

hanced Performance and Reduced Workload 40B Extending a ugv teleoperation flc interface with wireless network connec-

tivity information 41C A New UGV Teleoperation Interface for Improved Awareness of Network

Connectivity and Physical Surroundings 42D RCAMP Resilient Communication-Aware Motion Planner and Autonomous

Repair of Wireless Connectivity in Mobile Robots 43E Active Exploration Using Gaussian Random Fields and Gaussian Process

Implicit Surfaces 44F Active Perception and Modeling of Deformable Surfaces using Gaussian

Processes and Position-based Dynamics 45G Joint 3D Reconstruction of a Static Scene and Moving Objects 46

Bibliography 47

Part I

Introduction

Chapter 1

Introduction

For most living beings the ability to sense and understanding the surrounding is of imper-ative importance for both surviving and evolution

Perception of the environment is a core concept in the field of robotics too A robotmust be able to sense process and organize sensory information in order to acquire newknowledge comprehend the surrounding and represent it A good perception system al-lows a robot to refine its own cognitive model and make more accurate behavioral predic-tions for a specific task or tool

Computer Vision is the science that studies visual perception for machines For themost part computer vision scientists have been focusing on visual recognition image en-hancement or mapping problems that exploit passive observations of the environment (iepictures or videos) In contrast to a computer though a robot is an embodied agent ableto perform actions and interact with the environment

In this thesis we argue that mobile robotic systems especially the ones involved in dis-aster response should create and use a world representation that is not built upon merelyvision data but on a collection of rich sensory signals result from the interaction of therobot with the environment We present a series of studies that revolve around the idea thatenhancing a geometric map using non visual sensor data leads to a more exhaustive worldrepresentation that greatly helps overcoming difficult scenariosTo achieve this we will initially give a small introduction to the field of Search and RescueRobotics highlighting the open problems and difficulties that arise when using complexrobotic systems in disaster scenarios and what solutions we propose to mitigate them Wethen discuss more in detail the science and technology at the base of our work and con-clude with a presentation of our scientific contributions

In this thesis we use the terms disaster robots rescue robots and SAR (Search andRescue) robots interchangeably to refer to all kinds of robots designed to assist humansin disaster response efforts although someone may argue that rescue robots should refersolely to those robotics systems used during rescue operations

3

4 CHAPTER 1 INTRODUCTION

11 Robotic systems that save lives

The size complexity and dangerousness of many man-made and natural disasters havemotivated the development of more intelligent and robust robotics systems able to increasethe chances of finding and saving victims and facilitate the post-disaster recovery of theaffected areas The specific nature of the disaster motivates the use of a particular rescuerobot over another Large natural disasters require Unmanned Aerial Vehicles (UAVs) to flyover the affected area and generate high resolution top-view maps whereas bomb disposaloperations require arm equipped remotely operated vehicles (ROVs) to manipulate theexplosive from a safe distance Collapsed buildings are usually hard to traverse for wheeledrobots and tracked or legged platforms are largely preferred floods require UnmannedSurface Vehicles (USVs) to navigate and inspect the disaster zone

A rescue robot can assist a human rescue team by exploring areas difficult to reachmanipulate dangerous substances provide medical support and remove rubble in what isoften a race against time to find survivors

Figure 1 A squad of unmanned ground and aerial vehicles used in the European Project TRADR

EM-DAT (the international database of natural disaster) reports that between 1994 and2013 6873 natural disasters were registered worldwide which claimed 134 million livesFrom a broader perspective approximately 218 million people per annum were affected bynatural disasters during this period [72] In 2016 the number of people affected by naturaldisasters was higher than average reaching 5644 million [29]

Rescue robots can greatly mitigate the effects of such catastrophic events by supportingrescue teams in the first hours of the crisis (usually victims are most likely to be found alivein the the first 72 hours) and in the aftermath (days or weeks after the event)

Disasters have an enormous impact not only in terms of loss of human lives but alsoon the long term economy of the affected area which can take up to 30 years to recoverresulting in billions of dollars in economic losses [84] The Centre for Research on theEpidemiology of Disasters (CRED) reports that in 2016 alone worldwide disasters madeUS $ 154 billions in economic damages [29] Slightly reducing the initial disaster response

12 VISION IN SAR ROBOTICS 5

time can effectively take down the overall recovery time of the affected region savingbillions of dollars This further motivates governments to invest generous resources in thedevelopment of technologies to assist in disaster response In Section 25 we describe someof the many international research projects and robotics challenges that have been fundedin the past two decades

12 Vision in SAR robotics

Many open issues are still present in the field of SAR robotics spanning from ensuring aneasy and effective control mode to improve tools for human-robot interactions (HRI) toreduce communication losses and power consumption UGVs must satisfy a large numberof strict requirements to be able to operate in harsh environments They must be sturdywater resistant and have a communication interface (either tethered or wireless) to ex-change vital information with the base station Very importantly they must be equippedwith a robust possibly redundant visual perception system This represents a particularlychallenging problem in SAR scenarios

Figure 2 Two examples of problematic environmental conditions that challenge the vision systemof mobile robots In (A) the robot must traverse an obstacle that is occluded by dense smoke In (B)darkness makes object detection and navigation difficult

The vast majority of UGVs autonomous or teleoperated build their world represen-tation by relying on sensors using visible or infra red light (which we refer to as visionsensors in this thesis) LiDAR (Light Detection And Ranging) RGB cameras Stereo cam-eras RGB-D cameras Omnidirectional cameras and Infrared cameras are excellent sensorscapable of feeding the robotic system with a rich flow of information useful for detectingobjects building maps and localizing the robot (SLAM Simultaneous Localization andMapping)

To build its 3D representation of the surroundings the SAR robot uses its vision sys-tem to collect multiple observations and combine them into a consistent world map Suchpassive observations are obtained over the duration of one or multiple missions [33] Some-


times the generated maps are the result of a joint effort of multiple robotics system explor-ing different areas of the same disaster scenario [42]

Following the recent trend of the robotics field SAR robots are becoming more intelli-gent and reaching higher levels of autonomy Teleoperation remains by far the most com-monly adopted operational modality but in order to reduce the cognitive workload of theoperator and increase hisher Situational Awareness [35] the rescue robot must show a cer-tain level of autonomy and feature high level functionalities As an example an unmannedground vehicle can autonomously patrol and map a target area allowing the operator to fo-cus on other aspects of the mission rather than manually controlling the movements of therobotrsquos chassis Autonomous actions are planned and executed based on the inner worldrepresentation that the robot creates based on the analysis and understanding of the envi-ronment and therefore highly dependent on the perception strategy Basing the decisionsmaking process of the robot on the analysis of an uncontrolled stream of pictures or video(ie passive observations) is a common situation in SAR robotics systemsWith passive observations we refer to the process of perceiving the environment withoutpurposely change the intrinsic or extrinsic camera parameters In this thesis we highlighttwo major limitations of this approach

Firstly environmental phenomena such as rain fog smoke darkness dust etc thatlargely influence the performance of many vision sensors may generate highly noisy dataand potentially lead the robot to take wrong assumptions on the nature of the surroundingsThese phenomena are frequently observed in all kinds of rescue scenarios and represent aserious problem to many modern vision algorithm For example an algorithm that relieson thermovision for victims detection fails in presence of fire giving false positives orfalse negatives Grey dust which generally covers large part of the interiors of the rubblehides objects shapes textures and colors making object detection and mapping difficultIn Figure 2 we show two of the limitations of visual perception in presence of smokeand darkness encountered during a field exercise done in the context of the EU projectTRADR [62]

Secondly implicit geometric properties such as the deformability of the terrain cannotbe deduced from vision sensor data Neither can non-visual environmental properties suchas the wireless distribution of a radio signal which requires a different sensor systems (iewireless receivers) to be perceivedIn our works we use the term geometric representation of the environment to refer to thepoint cloud representation of the robot surroundings constructed using a mapping algo-rithm such as the ICP (Iterative Closest Point) [9] on multiple scans [22] In Section 33we discuss other kinds of world representation

13 Contributions and thesis outline

This thesis focuses primarily on mobile ground vehicles and the enhancement of theirvisual perception system We claim and will demonstrate that an enhanced multi sensoryperception system greatly helps overcoming major open problems of the rescue roboticsfield Such assertion is the result of a slow and gradual analysis of the field that moves from

13 CONTRIBUTIONS AND THESIS OUTLINE 7

the study of control methodologies to the development of interactive perception strategiesfor better communication and navigation

The contributions listed at the end of the thesis target four main open issues related toSAR robotics Control HRI Communication and 3D mapping of objects and environmentThe diagram in Figure 3 summarizes the listed contributions with respect to the problemthey address

Figure 3 Conceptual organization of our scientific contributions into four categories

Paper A In this work we compare through a user study two control modes the clas-sic TANK control and a computer gaming inspired control mode called Free Look Control(FLC) The control of a multi degree of freedom system is a tedious task that heavily affectsthe mental workload of an operator [106] The gaming community has put considerable ef-forts in designing control modes that enhance performance of players in platforms (screenand gamepad) that share many similarities with classic Operator Control Units (OCUs)of rescue robots We want to investigate if the use of FLC a control mode used in FirstPerson Shooter games (FPS) that decouples camera translations and rotations can increasethe mission performance and reduce the operator stress In order to do so we create sev-eral virtually simulated urban search and rescue (USAR) scenarios and design use cases tosystematically compare the two control modes in typical USAR tasks such as explorationand navigation We then collect quantitative and qualitative data from the simulator andevaluation questionnaires given to volunteers The results show that FLC has a number ofadvantages compared to TANK control

Paper B Wireless communication represents one of the most problematic aspects ofany USAR operation In spite of its obvious advantages compared to tethered communi-cation wireless connection suffers from continuous drops in signal quality that can resultin the robot losing contact with the base station The signal loss interrupts the informationexchange of the robot with its operator and can seriously compromise the outcome of themission Regardless of its importance a very small portion of a typical OCU interface isdedicated to representing the quality of the signal which is often reported merely in termsof radio signal strength (RSS) perceived at the robot location as shown in Figure 4

In this work we address the problem above by extending an FLC operator interfacewith a system capable of estimating and representing the direction of arrival of a radio


signal (DoA) In our approach the DoA signal which represents the RSS gradient at therobot location is robustly extracted using commercial-off-the-shelf directional antennasThis information allows the operator to drive the robot away from areas with low wirelessconnectivity reducing the risk of signal drops The graphical interface that shows the DoAis carefully designed to exploit the peripheral vision of the operator which can in this wayfocus solely on the video stream therefore improving the mission performance

Figure 4 Classic teleoperation interface showing limited information regarding the robot connec-tivity

Paper C The effectiveness of the aforementioned approach in comparison to the clas-sic representation of the radio signal strength (ie intensity bar) is evaluated in this workthrough a user study Participants were asked to operate a real mobile robot on a simulatedUSAR scenario and perform tasks such as look for symbols and explore the map whileavoiding connection loss The evaluation showed that users were able to significantly in-crease their performance using the new DoA interface and hardware

Paper D Despite the operatorrsquos skill in driving the UGV in connection-safe areas un-predictable events such as hardware failures and stochastic elements in radio signal propa-gation remain a concrete risk for communication lossUSAR scenarios are characterized by concrete walls and metallic segments that influencethe wireless signal propagation leading to delays and sudden drops in the quality of thesignal It is not uncommon for disconnected USAR robots to be abandoned when this hap-pens see eg the UGV SOLEM in Figure 5 deployed after the world trade center (WTC)terrorist attack in September 2001 [19]

In less dramatic circumstances different rescue strategies can be adopted to retrievethe disconnected robot For instance the platform may attempt to back track its way tothe base station or it can be pulled using cables Both these solutions are unfruitful if theenvironment changes making it impossible for the robot to travel the same path backwardsor causing the cable to break (eg a fire that propagates into a room a wall that collapsesor a water leakage) Furthermore the wireless access point can be relocated many timesduring a mission thwarting the attempt to reestablish connection [61]

In this work we introduce an active perception framework capable of estimating and


Figure 5 (A) The Foster Miller robot SOLEM that was teleoperated (wirelessly) into the WTC onthe 911 terrorist attack (B) View from the robot(Courtesy of the Center for Robot-Assisted Search and Rescue (CRASAR))

registering the radio signal distribution onto the geometric map and propose a ResilientCommunication-Aware Motion Planner (RCAMP) that integrates the wireless distributionwith a motion planner RCAMP enables a self-repair strategy by considering the environ-ment the physical constraints of the robot the connectivity and the goal position whendriving to a connection-safe position in the event of a communication lossWe demonstrate the effectiveness of the planner in a set of simulations in single or multi-channel communication scenarios

Paper E The use of different sensory signals not only permits obtaining a more feature-rich world representation but also improving the geometric map built from visual analysisIn this work we propose an online active perception framework for surface explorationthat allows building a compact 3D representation of the environment surrounding a robotThe use of tactile signals enables the robot to refine the geometric map (point cloud) ofthe environment which may contain occlusions or incomplete areas A mobile robot au-tonomously identifies regions of interest in its proximity and uses its robotic arm equippedwith force sensors to touch the surface and obtain tactile sensory data The new informa-tion is used to train probabilistic models and reconstruct the missing information in thegeometric map A challenging aspect of any interactive perception framework is the diffi-culty of defining a proper perceptive strategy that in our case translates to how where andif ever when to perform the tactile exploration In SAR operations reducing the numberof physical interactions with the environment may determine the successful outcome ofthe mission Too many interactions may take too long time and consume too much energyThe novelty of the method is its ability to quickly detect incomplete regions in the pointcloud and limit the number of environmental interactions needed while still ensuring asatisfactory reconstruction of the target area An example of usage of the proposed sys-tem on a real scenario is shown in Figure 2(A) where the robot may activate its mobilearm to investigate the obstacle occluded by the thick smoke and add its estimated shapeto the world map We then proceed to demonstrate how to apply this online probabilisticframework to object detection and terrain classification problems


Paper F The ability to model heterogeneous elastic surfaces is of great importancefor both navigation and manipulation A robot that is able to perceive the deformabilitydistribution of a surface can decide whether to traverse a terrain or not and whether tomanipulate objects on a surface or not Exploring and modeling heterogeneous elastic sur-faces requires multiple interactions with the environment and a complicated selection ofphysical material parameters which is commonly done through a set of passive observa-tions using computationally expensive force-based simulatorsIn this work we present an online probabilistic framework for estimation of heterogeneousdeformability distribution maps that allows a mobile robot to autonomously model the elas-tic behavior of a portion of the environment from few interactions This work demonstrateshow to extract useful implicit environmental properties and merge them into a geometricmap We show experimental results using a robotic arm equipped with tactile sensorsinvestigating different deformable surfaces

Paper G Strategies of interactive perception are not only useful for map enhancementbut also for object reconstruction Mapping and object reconstruction methods commonlycoexist into two separated pipelines or processes The robot explores the area and col-lects observations that are merged together to create the geometric map Object discoveryand reconstruction is usually done offline after the map has been created Following thisapproach objects of interest may appear as part of the static world map and an offlinepruning operations has to be done in order to separate them and clean the map Even moreproblematic is the ghost effect created by undetected moving objects In this work wepropose a technique for simultaneous 3D reconstruction of static regions and rigidly mov-ing objects in a scene The system is based on a multi-group RANSAC-based registrationapproach [39] to separate input features into two maps (object map and static map) thatwe grow separately The system is designed to overcome situations where large scenesare dominated by static regions making object tracking more difficult and where movingobjects have larger pose variation between frames compared to the static regionsThe approach allows simultaneous mapping and on-the-fly object model generation en-abling an arm equipped UGV to create models of objects of interests (eg samples valvesetc) through physical interaction while mapping the environment

Chapter 2

Disaster Robotics

In this chapter we provide a brief overview of the field of Search amp Rescue RoboticsSeveral surveys and books have been written that cover the topic extensively [74 84 85107] It is outside the scope of this thesis to provide a similar level of detail instead weaim to give the reader a basic understanding of the scenarios hardware and challengesinvolved in this broad and fascinating scientific field that strongly motivated our researchefforts We will put particular emphasis on the analysis of ground vehicles although someof the technologies introduced in this thesis can be applied to other robotics systems too

Figure 1 (A) A Thermite RS1-T3 Robotic Fire Fighter used during USAR operations (Courtesy ofHowe amp Howe Technologies) (B) Italian Rescuers (Vigili del Fuoco - VVF) (C) An Italian caninerescue unit

21 Overview

Search amp Rescue robotics is still a fairly new area of research which has made significantprogresses since its first steps in the late rsquo90s

A common misconception likely enforced by modern sci-fi pop culture pictures therescue robot as a fully autonomous multi-functional system capable of substituting human

11

12 CHAPTER 2 DISASTER ROBOTICS

and canine rescue teams in any circumstance The reality is quite different A disasterrobot is not meant to replace human or canine rescuers but to complement their set ofskills and help them to deal with complex and perilous scenarios Rescue robots do nottire and operate uninterruptedly for a long time bearing high heat radiation and toxicatmospheres operating in areas difficult to reach for humans or dogs Most importantlythey are expendableThe range of applications where rescue robots can be used is very broad Typically disasterscenarios can be subsumed under two broad categories

Natural disasters such as earthquakes tsunamis hurricanes or floods are major anddevastating consequences of natural planetary processes The impact of climate change andirresponsible urbanization is alarmingly amplifying the magnitude of these catastrophicevents [55] making the intervention of human rescue teams more difficult and sometimesprohibitive In 2017 alone three major hurricanes (Harvey Irma and Maria in South-ern US and Caribbean) over 1500 earthquakes above magnitude 50 (Italy Iraq MexicoCosta Rica etc) multiple floodings and landslides (Philippines Sierra Leone ColombiaBangladesh etc) have caused billions of dollars in damages millions of evacuees and tensof thousands of deaths

Man-made disasters contrarily to natural disasters usually occurs in smaller scalesTypical examples are terrorist attacks mining accidents bomb disarming operations leak-ages of toxic or radioactive substances explosions or any other kind of industrial acci-dent Man-made disasters are more common on urban areas leading to the creation of theacronym USAR (Urban Search amp Rescue) Collapsed structures increase the complexity ofthe environment making human or canine intervention arduous ( Figure 1(B-C)) In USARscenarios the presence of close and potentially live power lines facilitates the set-up ofa base camp which is essential for prompt logistics Figure 2 illustrates an example of aGerman USAR simulation area where firefighters recreate a small scale man-made disaster(industrial accident) to be used for rescue robotics research

Figure 2 Example of urban industrial scenario used for training (old furnace in Dortmund Ger-many) The complexity of the structure makes mapping difficult and the metallic components of theold furnace attenuate the wireless signal propagation

22 HISTORICAL PERSPECTIVE 13

22 Historical Perspective

In the rsquo80s the scientific robotics community moved its focus towards the study of morecomplex systems capable of performing intelligent tasks in seemingly unconstrained en-vironments This new paradigm extends the classic definition of industrial robot designedto help humans on the so-called dirty dull and dangerous jobs The possibility of havingrobotic systems capable of working outside the ad-hoc designed assembly line of a factorybroaden the spectrum of applications the robot could be used in In the same years re-searchers started considering the use of mobile platforms for emergency situations includ-ing remote surveillance and firefighting One of the first recorded cases of robots used fordisaster response dates back to the 1979 where three robots built by researchers at CarnegieMellon University (CMU) led by Professor William L Whittaker helped to clean up andsurvey a meltdown at the Three Mile Island (TMI) nuclear power plant in PennsylvaniaThe robots built at CMU required multiple operators they were big heavy and outfittedwith lights cameras and tools for handling radioactive materials Later models kept thisdistinctive bulky design which made them unable to explore small voids and navigate therubbles without the risk of causing more collapse [84]It was only in 1995 when research into search and rescue robotics started at Kobe Univer-sity in Japan Experience from successive real deployments [70] made it clear that robotsneeded to be redesigned to avoid them aggravating the situation causing progressive col-lapse of unstable structures or triggering explosions on gas saturated spaces The idealrescue robot needed to be smaller easier to operate more agile and have a certain degreeof autonomy [81]

The first time a SAR robot was used in a real operation was 2001 in response to theterrorist attack at the World Trade Center in NY US Two civil planes were crashed into theNorth and South towers causing severe structural damages to the buildings Within twohours the floors of the towers begun to pancake down on one another creating extremelydifficult operational conditions for the rescuers and killing thousandsA team of researcher from the Center for Robot-Assisted Search and Rescue (CRASAR)led by Professor Robin Murphy deployed ground robots to assist rescuers in detectingsurvivors Although no survivors were found the mission was considered a success asthe robot extended the senses of the responders far beyond the reach of the classic searchcameras and managed to find 10 sets of human remains [19]

ldquoThe effectiveness of robots for man-made disaster stems from their potential to extendthe senses of the responders into the interior of the rubblerdquo [84]

The evolution of SAR robots not only touches the mechanical design of the robot and itssensors but also its operational architecture Initial deployment of rescue robots includedone or multiple operators piloting a single UGV in order to gather information from a safedistance A team leader would assign to team members one or multiple micro tasks (egcollect a radioactive sample)

As of 2018 disaster robotics is following the modern trend of robotics in moving froma task-specific perspective to a human-centric perspective where one or multiple robots


collaborate with human teams to gradually develop their understanding of the disaster sce-nario [60]

Figure 3 UGVs and UAVs inspect an old church in Amatrice (Italy) severely damaged after a 63magnitude earthquake in 2016

Example of real use of this operational architecture is the recent deployment of rescueUGVs and UAVs in August 2016 when a strong earthquake of Magnitued 62 struck theItalian city of Amatrice causing severe damage to the old town and the death of 299 peo-ple Two ground robots and an aerial vehicle were sent to assist the Italian Vigili Del Fuoco(VVFF) generating 3D models of the Basilica of Saint Francis of Assisi and the Churchof SantrsquoAgostino [61] The 3D models were then used to asses the structural stability ofthe old buildings and several cracks were found During the mission the robots sufferedfrom communication issues The access point used to exchange information between thebase station and the robots had to be relocated multiple times around the church to ensuregood connectivity due to the thick stone walls of the structureThis more intricate conceptualization is possible only thanks to recent advances in commu-nication technologies AI and to the increased computational power which allows robot toperform more complex tasks in a unsupervised or semi unsupervised fashion

The advance in AI raises numerous moral questions and concerns [31] naturally con-textualized in SAR robotics The ethical aspect is in fact becoming an important aspect ofthis field In this regard Winfield et al in [119] investigated the consequences of introduc-ing ethical action selection mechanisms in robots In their experiments the authors usedhockey-puck-sized mobile robots two of them representing humans in danger (the robotswere dangerously wandering on a surface filled with holes) and one being the rescue robot(the ethical machine) in charge of saving them The rescuer could perceive the environment(eg position of the holes) and predict the actions of other dynamic actors The robot wasalso forced to follow a minimalist ethic role analogous to Asimovrsquos first law After manyruns the experiments showed that the rescue robot often successfully managed to save oneor both human agents This oversimplified cognitive design though made the robot wan-der in confusion 50 of the times when both robots were simultaneously in danger leadingto the questions ldquoWho should the robot prioritize And in what circumstancesrdquo

23 TYPES OF RESCUE ROBOTS 15

23 Types of Rescue Robots

Several taxonomies have been proposed to categorize disaster robots It is possible toclassify rescue robots according to their size (eg packable robots) operational mode(eg aerial robots) or specifics of the task (eg explorers) A widely accepted nomen-clature distinguishes between Unmanned Ground Vehicles (UGVs) Unmanned AerialVehicles (UAVs) Unmanned Undersea or Underwater Vehicles (UUVs) and UnmannedSurface Vehicles (USVs) Sometimes UUVs are also called Autonomous Underwater Ve-hicles (AUVs) Sub classes exist within each category for instance underwater gliders areparticular kinds of UUVs that change their buoyancy to navigate the water and RemoteOperate Vehicles (ROVs) are the teleoperated-only subclass of UGVs (not to be confusedwith the Remotely Operated Underwater Vehicle ROUV sometimes also called ROV)Inspired by nature rescue robots can have peculiar shapes which facilitate specific taskssuch as the snake robot [120] in Figure 4 capable of infiltrating rubble and providing animmediate video feed to the operator

Figure 4 CMU Snake robot sent to assist rescuers days after a 71-magnitude earthquake struckMexico City in 2017 (Courtesy of CMU Robotics photo by Evan Ackerman)

Unmanned Ground Vehicles (UGVs) are commonly used to navigate the environmentin search of victims or hazardous goods They can have different sizes be packable or verylarge and are used both in the first intervention phase (eg extinguish a fire search forsurvivors sample collection) and on the post-disaster phase (eg structural inspections ora victimrsquos body recovery) A vision sensor such as a thermo camera a laser scanner or anRGB camera is always present on-board to allow the robot to navigate the environmentand provide visual feedback to the operator UGVs can be autonomous or teleoperatedwork alone or in squads [8] and their operational time can extend over single or multiplemissions [62] Connection with the users be it tethered or wireless is often a problematicaspect of any UGV rescue operation as we discuss in the papers B C D A large numberof UGVs as the one in Figure 5 are equipped with a robotic manipulator that allows themto perform physical tasks such as collecting samples close valves or remove debris


Figure 5 A PackBot UGV inspects a barrel containing dangerous chemicals (Courtesy of EndeavorRobotics formerly known as the Defense amp Security unit at iRobot)

Unmanned Aerial Vehicles (UAVs) are revolutionizing SAR operational strategiesby providing the rescuer units the possibility to consult updated top-view maps of thehot zone The operator can manually set way points or specify a geographical regionthat the unmanned aerial vehicle as the one in Figure 6 will visit and map UAVs canalso be operated close to unstable structures so to obtain a close look to areas difficultto access otherwise Larger categories of UAVs are used to study severe meteorologicalphenomena such as hurricanes Their sophisticated sensory system collects atmosphericdata which permits generating reliable mathematical models that provide precious hintsabout the future trajectory and severity of the storm This technology enables authoritiesto prepare for large scale evacuations or find the best option of disaster response

Figure 6 AscTec Falcon UAV used in the EU project TRADR

Unmanned Underwater Vehicles (UUVs) are able to substantially speed up under-water search operations Their operational time and reach are considerably higher than theone of a scuba diver [121] Visibility in turbid water and underwater darkness remain oneof the major challenges for this kind of rescue robots Remotely operated UUVs have been

24 THE USERrsquoS PERSPECTIVE 17

enormously useful in containing environmental disasters such as the 2010 oil spill in theGulf of Mexico [7] They were used to turn valves and connect pipes in extremely deepwaters where no scuba diver could ever venture

Figure 7 A Double Eagle SAROV AUV (Courtesy of SAAB)

Unmanned Surface Vehicles (USVs) have an important role in assisting rescue teamson locations situated in proximity to water and during floods They are particularly usefulin the inspection of damaged bridges or piers as they allow structural engineers to quicklyobtain a detailed look of the affected structure without endangering human rescuers [82]Their use can however be difficult in the presence of swift waters

24 The Userrsquos Perspective

The cognitive workload of the operator defines the mental effort needed to operate andsupervise the UGVs during the lifetime of the mission [106] Studies have shown that agood portion of the rescue mission is used by the operator to navigate the enviroment [14]The design of proper USAR HRI strategies [102] including interfaces and control systemsallows an operator to focus more on the resolution of the specific situation rather thanthe robot navigation Reduction of the operatorrsquos cognitive workload can be achieved indifferent ways and reach different level of complexity

Low level control of the robot (eg direct control of the manipulator or tracks) can beimproved by introducing new control paradigms Controlling a multi degree of freedomsystem is in fact a complex task which requires dexterity and concentration [53] [102]In response authors in [128] propose to use a reinforcement learning framework to traina system to autonomously control parts of a UGV reducing its teleoperation to mere di-rectional control A relatively new trend in SAR robotics is to use computer game controlmodes adapted to SAR scenarios [75] [26] This approach brings several benefits suchas allowing an operator to familiarize faster with the robot movements Gaming controlmodes are in fact designed for the general public and are less cumbersome to use comparedto classic control modes as we discuss in paper A


Design of proper user interfaces (UI) facilitates control of the UGV and simplifies theway information is presented to the user In paper B and C we put great emphasis in theuser evaluation and in the design of an interface that conveys readable information to therobot operator

Figure 8 Teleoperation in line of sight (LOS) with a Thermite RS1-T3 Robotic Fire Fighter duringa USAR operations (Courtesy of Howe amp Howe Technologies)

High level functionalities such as automatic victim detection or autonomous patrollingand mapping of target areas lower the number of simultaneous tasks the user needs toperform and therefore hisher cognitive workload

Full autonomy is the ultimate condition for a system that enables an operator to entirelyfocus on the mission rather than the robot The increasing level of autonomy brings to thetable new challenges and ethical questions For instance the design of a proper AI thatinstead of using a first-come first-served role makes decision based on the specifics of thescenario to prioritize victims to be saved

25 Notable SAR Research Projects Groups and Challenges

A large variety of research projects have been funded in the past years focusing on differ-ent aspects of rescue robotics A notable example is the EU project TRADR (Long-TermHuman-Robot Teaming for Disaster Response) [62] Started in 2014 as a continuation ofthe EU project NIFTi TRADR aims to develop science and technologies for teams of hu-mans and robots that collaborate over multiple sorties in urban rescue scenarios Missionsin TRADR have a time span of days or weeks and reach a high level of complexity Thenovelty of this project is its human oriented perspective where teams of robots (UAVsand UGVs) jointly build interactive maps that can be easily consulted by human rescueteams Similarly the EU project ICARUS [50] focuses on building tools for detection andlocalization of humans in disaster scenarios aiming to fill the gap between technologies de-veloped in research labs and usage of such technologies during emergency situations andcrisis Project SHERPA [71] contrarily to the previous ones focuses mostly on UAVs in

26 SAR PROBLEMS ADDRESSED IN THIS THESIS 19

alpine rescuing scenarios In SHERPA the human and the robot work in the same team ex-ploiting each others individual features and capabilities towards a common goal Differentin its concept is the project CENTAURO [20] which aims to develop a Centaur-like robottele-operated by a human operator This human-robot symbiotic system should be able toperform dexterous manipulation (eg as closing valves) in austere conditions

An historical and important contributor to the the development of SAR technologiesin the past decades is the Center for Robot-Assisted Search and Rescue (CRASAR) [21]in Texas US CRASAR has been actively supporting international rescue teams on a mul-titude of real operations which include mining accidents terrorist attacks hurricanes andfloods CRASAR was also part of the Rescue Robot for Research and Response Program(R4) sponsored by the National Science Foundationrsquos Computer and Information Scienceand Engineering (CISE) that aimed to facilitate research in SAR robotics by making avail-able to researchers expensive equipment and realistic datasets [83]Other notable projects linked to SAR robotics are GUARDIANS [45] SmokeBot [105]and COCORO [23] In 2002 The Ministry of Education Culture Sport Science and Tech-nology in Japan promoted the Special Project for Earthquake Disaster Mitigation in UrbanAreas that aimed to significantly mitigate the impact that a strong earthquake could haveon big cities such as the Tokyo metropolitan area

The notorious DARPA Robotic Challange (DRC) [27] was an international RoboticsChallenge aimed to accelerate RampD of robots that could help humans in disaster responsemissions It was funded after the accident at the Fukushima Daiichi nuclear disaster in2011 as a wake up call for the robotics community Robots scored points by performing anintrinsic sequence of tasks in a simulated industrial disaster scenarioAnother important SAR robotics challenge is the ERL (European Robotics League) Emer-gency Robots previously known as euRathlon [36] This challenge was also inspired bythe Fukushima accident in 2011 and focuses mostly on cooperative teams of UAVs UGVsand UUVs to survey the simulation area collect data and identify critical hazards

Finally the RoboCup Rescue League [99] is one of the oldest rescue robots challengesthat aims to provide researchers with standardized environments and benchmarks for urbanrescue robots It was created in the aftermath of the disastrous Kobe earthquake that hitJapan in 2001 and consists of two main subcategories the Rescue Robot League for mobilerobots that search and map a physically simulated rescue environment and the RescueRobot Simulation League that tackles the problem of multi-robot collaboration on largevirtually simulated disaster scenariosThere is still a need for proper benchmark protocols in this field but this issue will not beaddressed in this thesis

26 SAR Problems Addressed in This Thesis

The lack of proper evaluation methodologies and the limited number of real deploymentshave been making it difficult to identify the foremost causes of failure in SAR missionsAuthors in [19 61 107] highlighted some of the open issues encountered during realdeployment of robots in rescue operations


Rescue robots in general are very complex systems that require careful design [81]The spectrum of problems that follows the creation and adoption of SAR robots into realapplications is very broad and touching many interconnected fields For instance from themechanical point of view the robot must ensure great reliability and be sturdy enough tobear adverse environmental conditions and abrasive substances At the same time the robotmust be sufficiently small and agile to improve its mobility in constrained areas and smallvoids in the rubble Depending on the mission requirements the robot should also be waterresistant and shielded from radiation or very high temperatures Its sensory system mustbe durable and able to face extreme weather conditions extreme temperatures water dustand dirt

Figure 9 (A) USAR Training facility in Rotterdam NL (B) OCUs for multi-robot USAR missions(C) Robot collaborating on a simulation scenario

In this thesis we tackle mainly the problems of Control Communication HRI andMapping Compelling discussions regarding other fundamental problems are carefullyinvestigated in [81] [85]

261 Control

The teleoperation strategy of a rescue robot has deep implications on the outcome on theSAR mission The design of a proper control mode which is frequently dictated by themission requirements and the robot mechanical complexity (eg tracked versus wheeledplatforms or the presence of a manipulator) must ensure reactive maneuverability of theUGV with minimum operatorrsquos cognitive effort This means that when the mission de-mands rapid intervention and maneuverability in contained areas the operator must intu-itively perform the correct sequence of commands that leads the robot to overcome thesituation The effectiveness of the control layer relies on a closed loop architecture that in-cludes the robot sensory system and actuators the communication means and the operatorcontrol unit (OCU)

Teleoperation is also heavily affected by other environmental phenomena such as thewireless connectivity that causes strong delays on the video feed hampering the manuvra-bility of the robot and stressing the operator [122]


262 Communication

The robot Solem depicted in Figure 5 is a good example that shows how important it is toensure good connectivity during a SAR mission The UGV lost wireless communicationwhile exploring the dense rubble of the WTC terrorist attack The communication lossinterrupted the video feed preventing the possibility of remote control Rescuers unsuc-cessfully tried to retrieved the robot from the rubble through a safety rope that broke Amore recent example of the importance of communication happened during the FukushimaDaichii nuclear disaster inspection in 2011In order to protect the nuclear power plant personnel from radioactive rays the buildingwas shielded with thick walls which made it difficult to guarantee a robust propagation ofradio waves for transmissionThe problem of communication was then extensively examined by a team of engineersfrom TEPCO (Tokyo Electric Power Company Holdings) which evaluated the feasibilityof the use of wireless connectivity for their Quince rescue robot [87] In the experiments24 GHz transmitters and receivers where placed at different location in order to assess thesignal quality Results of the studies suggested the adoption of a hybrid theteredwirelesscommunication architecture Nevertheless the communication cable broke during one ofthe missions and the robot Quince was abandoned on the third floor of the Fukushima reac-tor building [123] In many cases the use of wireless mesh networks [100] for multi-robotteleoperation is considered to be a promising direction to ensure reliable remote control inGPS and wireless denied environments In these architectures the robots behave as mobilebeacons and extend the signal propagation in poorly reached areas

Continuous relocation of the access point represents another important issue relatedto communication as experienced in the USAR robots deployment in Amatrice (IT) de-scribed in Section 22

263 Human-robot interaction

Human-robot interaction (HRI) in SAR robotics aims to reduce the workload and stressthe robot operator is subjected to during a mission [102] The strict time constraints thedifficulty in operating the robot the struggle of the life-saving urgency and the chaoticinherent characteristic of the hot zone heavily impact the performance of the operatorwho is likely to make poor choices and errors A proper user interface (UI) must coher-ently and intuitively show world information to the operator Some of the robots broughtby CRASAR at the WTC were not used because their interfaces were too complex [19]The UI should be easy to learn so to speed up the operator training time and facilitatean immidiate deployment of the UGV A TEPCO spokesman reported that it took severalweeks for crews to learn how to operate the complex devices and this delayed the missionstart considerably [109] The design of user interfaces for single or multi robot operationas the one showed in Figure 10(C) and the study of user-robot interactions are thereforeimportant problems that we consider and discuss in our works


Figure 10 (A) TRADR UGVs patroling and mapping an industrial disaster zone (B) TRADR UAVproviding indoor 3rd person view to the UGV (C) Collaborative Mapping and Patrolling interfacefor human supervision

264 Mapping

The complexity of a typical SAR scenario raises numerous questions regarding the effec-tiveness of common mapping algorithms Highly complex structures as the one showed inFigure 9(A) or in Figure 2 require careful navigation and high resolution mapping whichis achieved through continuous scanning using high performance sensory systems suchas LiDAR Mapping unstructured environments can be even harder and proper mappingstrategies must be adopted Several solutions have been proposed to speed up the map-ping process during SAR operations such as collaborative mapping [77] (shown in Fig-ure 10(A)) or aerial mapping [78] [40]The problem of enriching geometrical maps integrating other sensor data which we coverin this thesis tackles the mapping problem from a transversal perspective as we will discussin the next chapter

Chapter 3

Enhancing perception capabilities

31 Visual Perception System

Most of the robots designed to interact with the environment are equipped with a visionsystem capable of generating a descriptive representation of the surrounding The need ofadvanced visual perception systems became imperative when robots started moving awayfrom the controlled environment of an industrial production line

As of 2018 vision remains by far the most studied and effective method of perceptionfor a robot that wants to map and investigate the world

More recently researches have started investigating ways of enhancing vision throughdifferent sensory signals For instance sound has been demostrated to enhance visualperception [115] in humans as well as in robots [114] Authors in [104] proposed a methodto enhance object recognition by letting a robot observe the changes in its proprioceptiveand auditory sensory streams while performing five exploratory behaviors (namely liftshake drop crush and push objects) on several different objects

311 RGBD cameras and LiDARs

A sensor capable of estimating the true 3D geometry of the environment opens the doorsto a variety or robot applications such as navigation mapping and object reconstructionLaser scanners have been the de-facto standard solution for 3D sensing in mobile robots inthe last decades

A LiDAR sensor is generally composed of three main parts a laser a photodetectorand optics [57] The sensor emits a pulsing laser beam directed towards a moving mir-ror that steers the beam in the direction of the scene Measurements on the laser returntime and wavelength permits the sensor to calculate the distance of the hit points [116]and consequentially obtain the objectrsquos point cloud representation Modern LiDARs canreach sub-millimeter precision at long range making them widely used for building high-resolution maps They are however rather expensive and do not capture the colors of thereconstructed surfaces Their high scanning frequency (30-100 Hz) and the density of the

23

24 CHAPTER 3 ENHANCING PERCEPTION CAPABILITIES

output data (30000-180000 pointsscan) make them ideal for modern autonomous driv-ing applications [126] [65] [13]

The increasing availability of cheap time of flight (ToF) and structured lights cam-eras [125] in the last years made possible the use of 3D sensors in virtually any roboticapplication Initially developed as a gaming interface the Kinect [59] is one of the mostsuccessful examples A structured light camera uses an infrared (IR) projector to producea pattern of light (usually parallel stripes) that are shot onto a 3D surface which distortsthe pattern depending on the geometry of the scene and the perceptive perspective Amonochrome CMOS sensor conveniently placed on the side of the projector analyzes thedistortions of the light pattern and uses geometric feature matching to compute the pointcloud representation of the objectrsquos surface or landscape This method is very versatileand fast and permits capturing the color of the 3D surfaces if an RGB camera pointingat the same scene is properly integrated The accuracy of Kinect-like cameras for indoormapping applications are evaluated in [56]

312 Limitation of vision sensors

Sensors that rely on light propagation and reflection to compute the underlying 3D surfaceof a scene are affected by several phenomena that hamper their capability to extrapolatedistances Figure 1 illustrates some of the most common illumination problems that affectthe performance of structured-lights based 3D sensors Volumetric scattering is generallycaused by phenomenons such as thin fog or smoke The scattered rays produce shafts oflight and volumetric shadows cast from geometric objects in the scene These effects areobserved by the sensor and generate artifacts in the 3D reconstruction The sub-surfacescattering result of the characteristics of the material is another factor that yields sys-tematic distortions in the depth map Translucent objects severely degrade the vision sen-sor performance as observed by authors in [66] who proposed a novel 3D reconstructionmethod based on the high dynamic range imaging techniques [97] [30] The same problemwas studied by the authors in [103] who developed a distance representation that capturesthe depth distortion induced as a result of translucency for time of flight cameras

To deal with global illumination caused errors researchers in [46] proposed novel re-silient structured light patterns that use simple logical operations and tools from combina-torial mathematics

Reflective surfaces such as mirror-type objects are particularly difficult to detect andreconstruct as they do not possess their own appearance and the reflections from the envi-ronment are view-dependent Authors in [112] proposed a technique for recovering smallto medium scale mirror-type objects The technique adopts a novel ray coding scheme thatuses a two-layer liquid crystal display (LCD) setup to encode the illumination directionsDifferently authors in [67] propose a way to reconstruct specular or transparent objects byusing a frequency-based 3D reconstruction approach which incorporates the frequency-based matting method [127] The same problem was investigated in [47] and [95] Otherphenomena such as the vignetting (radial falloff) and exposure (gain) variations causedby the RGB camera contributes to limiting the quality of the 3D reconstruction and theirremoval was investigated in [43] and [2]

32 BENEFITS OF ACTIVE AND INTERACTIVE PERCEPTION 25

Figure 1 Unexpected illuminations are source of error for structured lights camera

In this thesis we take a transversal approach to deal with the limitation of these vi-sion systems and introduce active and interactive perception frameworks to investigate theregions of the scene affected by distortion and occlusion

32 Benefits of Active and Interactive Perception

Computer vision applied to disaster robotics mostly focuses on processing and interpretingstatic images or video streams to feed the cognitive model of the robot A cognitive modelas defined in behavioral science and neuroscience describes how a personrsquos thoughts andperceptions influence his or her decision making process and consequently his or her life[92] Humans and Animals however build their understanding of the world through non-static observations of the surrounding Researchers in [37] show compelling evidencethat the exploratory process at the very base of the perception mechanisms of all livingorganisms including the simplest ones generates a rich informative set of sensory signalsfundamental to the development of a functional and responsive cognitive model

Robots can mimic these mechanisms and use their body to physically interact withthe environment in order to generate a more insightful collection of visual sensory signalsInteractive Perception (IP) is the field of robotics that studies ways of purposely interactingwith (and possibly modifying) the environment in order to have a better understanding ofthe robotrsquos surrounding The difference between Active Perception (AP) [5] and IP is thatAP can be seen as a special case of IP that aims to change only the sensor intrinsics andextrinsics without any physical interaction with the environmentAn intuitive example of Interactive Perception is when a subject uses hisher hands tomove an object from occluding a second one hence revealing new unobserved featuresActive Perception instead is when the subject moves hisher head to change viewpointand observe the occluded element without altering the state of the scene

When interacting with an object the agent can obtain information not easily or evenfeasibly achieved through passive observation such as the objectrsquos weight material shape


rigidity and degrees of freedom [88] In order to be an Interactive or Active Perceiver anagent must know what to perceive and how when and where to do the perception actions[6]Authors in [11] present a thorough investigation of IP and its applications In the workspresented in this thesis we show several applications of AP and IP to help robots in SARtasks

We use an arm equipped mobile robot to haptically explore the terrain surrounding amobile platform A probabilistic model based on Gaussian Process Regression (GPR)is used to detect regions in the map that require a physical investigation Tactile signalswhich are able to enrich the robotrsquos capabilities in many applications such as manipulation[10] are used to refine the geometric representation of the world built using vision sensorsalone (Paper E) A similar approach is used to estimate implicit environmental propertiessuch as the deformability of the terrain (Paper F)

In our application AP is not restricted to vision We use an active perception frame-work to generate wireless distribution maps and guiding a robot inside a disaster scenario(Paper D) The robot generates a new layer of information that is added to the geometricworld representation In Section 33 we discuss how to remove ambiguity in the geometricworld representation (the map) inferring an expressive probabilistic model continuouslytrained on active observations

33 Geometric Representations

Object representation in all its flavors (Figure 2) seeks to provide an informative visualdescription of the object that preserves its underlying structures and properties A reason-able reaction is to wonder what is the best representation for a specific problem that makesit more tractable

Ideally a 3D representation suitable for an interactive perception framework should be

minus Well suited for learningminus Flexible so that it can model a wide variety of shapesminus Geometrically manipulable therefore deformable interpolable and convenient to

impose structural constraints on

Different representations such as the ones illustrated in Figure 2 bring different ad-vantages and disadvantages For example volumetric occupancy is optimal to model alarge variety of shapes but is unsuitable for learning (expensive to compute) and unfit toflexible geometrical manipulation In certain research fields such as robotic mapping theproblem of finding the proper representation becomes less tricky for the 2D case and stan-dards have been released that include a limited number of metric representation for planarenvironments representation [1]

Recent years have witnessed breathtaking advances in the field of Deep Learning (DL)for analysis of 2D images audio streams videos and also 3D geometric shapes [38]

A 3D representation for DL applications must be easily formulated as the output of theneural network ensuring fast forwardbackward propagation [94] Similarly to the work

34 A PROBABILISTIC APPROACH TO INTERACTIVE MAPPING 27

in [94] in our frameworks we use point clouds (PC) as geometric representation A pointcloud although wasteful in terms of memory is geometrically flexible can be obtaineddirectly from a variety of sensory systems (eg RGB-D camera LiDARs Stereo Cameras)and it is compatible with the vast majority of 3D mapping algorithms Most importantlyPC representation is well suited for learning as it adapts well to the probabilistic modelsused in our methods A point in a PC is a feature vector whose elements are the coordinatesof the point on a certain reference system Our frameworks expand the feature vector withproperties of the environment under analysis as we discuss in the next section

Figure 2 Different representations for objects grouped into regular and irregulars grids(Courtesyof Hao Su Stanford University)

34 A Probabilistic Approach to Interactive Mapping

Particularly suitable for our problem are Gaussian Processes for Regression (GPR) [96]which have been wideley used on surface reconstruction problems [58] [113]

The problem of reconstructing a surface which we consider as a compact connectedorientable twodimensional manifold possibly with boundary embedded in R3 (cf OrsquoNeill[91]) has been thoroughly studied in [49] GPRs enable us to extend the objectrsquos or terrainrsquossurface definition to include other environmental properties

In our work we merge multiple sensory signals into probabilistic active and interac-tive perception frameworks using two and three dimensional GPR known respectively


as Gaussian Random Fields (GRF) and Gaussian Process Implicit Surfaces (GPIS) Wedemonstrate how to exploit GPRs for modeling a surface appearance its deformability orother environmental properties such as the wireless distribution We also demonstrate howto obtain compact representations of the environment from sparse point clouds and tactileinformation

341 Gaussian Random Fields

A 25D surface can be described with a function f R2 rarr R where each vector of xy-coordinates generates a single height A Gaussian Process Regression shaped over a bi-dimensional Euclidean set is commonly referred as Gaussian Random Field (GRF) GRFare powerful statistical tools able to smoothly fit values ensuring the best linear unbiasedprediction (BLPU) as opposed to a common piecewise-polynomial spline This interpola-tion process is widely used in spatial analysis and geostatistics where it is more frequentlyknown as Kriging [90] In its simplest form Krigingrsquos prediction in a location s0 canbe formulated as a weighted sum of N measurements Z (si) at spatial locations si whereweights φi are computed depending on a fitting model of the measurements

Z (s0) =N

sumi=1

φiZ (si) (31)

In our work we make use of GRF for terrain reconstruction (Paper E) terrain deforma-bility (Paper F) and wifi distribution (Paper D) estimation The methodology used to ex-ploit GRF varies sensibly between different papers We leave details about the mathemati-cal formulation of GRF to the attached papers and describe here only the data sets used todefine our perception strategiesGiven a set of observation points (ie point cloud) PS = p1p2 pNpi isin R3 we candefine a training set DRF = xiyiN

i=1 where xi isin X sub R2 are the xy-coordinates of thepoints in PS

1Depending on the application yi contains the environmental property of interest measuredor estimated at the location xi namely

In paper E yi contains the z-coordinate (height)In paper F yi contains the deformability parameter (β )In paper D yi contains the radio signal strength (RSS)

Note that the formulation of f does not allow us to model complex terrains shapeswhich require multiple heights for a single xy-coordinate such as in the example shown inFigure 3GRF allow predicting the behavior of a 25D surface on unobserved regions of the space

342 Gaussian Processes Implicit Surfaces

Many applications in computer vision and computer graphics require the definition ofcurves and surfaces [32] Implicit surfaces are a popular choice for this because they

1 Axis are described considering the frame represented in Figure 3

35 STRATEGIES FOR TRAINING AND INTERACTING 29

Figure 3 (A) Planar cut of a 25D surface that has a single height for a coordinate point xi (B)Planar cut of a 25D surface that has multiple heights for a coordinate point xi

are smooth can be appropriately constrained by known geometry and require no specialtreatment for topology changes

In this section we briefly describe Gaussian Process Implicit Surfaces (GPIS) [118] andtheir ability to model highly complex shapes GPIS allow to model a function f R3rarr Rwhich supporting points allows defining an implicit surface Whereas their mathematicalformulation mantains the same form as for GRF the training and test sets change sensiblyDIS = xiyiN

i=1 where xi isin Xsube PS and yi isin R are defined as in [118]

yi

=minus1 if xi is below surface= 0 if xi is on the surface= 1 if xi is above the surface

(32)

For clarity Figure 4 shows example training points laying on a curve traversing animplicit surface colored according to Equation 32GPIS do not allow us to directly obtain the shape the surface of interest Whereas theoutput of the GRF maps the explicit behavior the 25D surface z = f (xy) GPIS maps thetarget set of the implicit function F (xyz) not solvable for xy or z In order to obtain thesurface we needed to define a large set of 3D test points eg a dense cubic set of pointscentered on a region of interest and then find the isosurface of value 0 on the output meanfunction This operation is computationally expensive depending on the size of the test setXlowast GPIS however brings several benefits compared to GRF

1 It allows us to model very complex surfaces (eg objects)

2 Training points do not need to lay on the iso surface

Point (2) in particular plays a significant role in defining the training strategy of ouractive perception frameworks as described in Paper E

35 Strategies for training and interacting

The problem we addressed in this thesis required the implementation of online interactiveperception frameworks with basic mechanisms outlined in this sectionThe challenges that arise when using GPR in a close-loop online perception framework(Figure 5) are (1) to define the correct training set which must have a small size (2) tomake it possible to feed and retrain the model with data coming from new environmental


Figure 4 Illustrative representation of above-surface (blue) on-surface (green) and below-surface(red) training points along a trajectory

interactions and (3) identify unexplored regions on the map that need to be investigatedThe problems of power consumption strict time constraints and risk of causing furthercollapses discussed in Chapter 2 force us to limit environmental interactions as much aspossible

Defining the most appropriate training data set X is crucial for a correct understand-ing of the environmental properties Using the entire sensory input as training set is oftenprohibitive in terms of time and computational power needed and subsets are usually pre-ferred As for GP specifically a large set X leads to a large covariance matrix which needsto be inverted [96] Standard methods for matrix inversion of a ntimesn positive-definite sym-metric matrix require time O

(n3)

making GP unsuitable for online tasks The applicationitself helps defining the strategies of training and data collection For example modelingan objectrsquos surface with high level of details requires a dense set of training points andmillimetric precision whereas modeling the radio signal distribution over a large area is amore forgiving application in terms of quantity and quality of training data neededThe probabilistic model used plays a big role too GRF usually require smaller amounts oftraining data as opposed to GPIS which need a large number of on and off-surface pointsIn our work we define different perception strategies that follow the general system outlineillustrated in Figure 5

The robot uses vision clues to built the initial geometry of the environment which worksas foundation for other sensor data Looking at the variance of the GPR model the robot isable to isolate areas on the map that require further investigation Geometric analysis doneusing Delaunay triangulation (the dual of the Voronoi diagram) [41] can also give precioushints regarding the sparsity of the data as shown in our work and in [86] Measurementsare then sequentially collected on the isolated regions by either touching the surface orcollecting RSS data 2 After each interaction the training set is shaped so to accommodatethe new measurements while preserving the old ones

Alternatively to the closed-loop perception diagram of Figure 5 described above it ispossible to define an open-loop perception strategy where multiple environmental inter-

2A RSS dataset collected during a real UGV deployment is available in [93]

36 MAPPING THE ENVIRONMENT AND MOVING OBJECTS 31

Figure 5 Generic Active-Interactive perception framework

actions are triggered all at once after all the region of interest are detected and actionsselected Observations made during the interactions are stored into a separated training setwhich is ultimately fed to the model This approach allows one to save processing time asthe model does not need repeated updating steps after each interaction but it is less reac-tive to environmental changes and discoveries that could reduce the number of explorationsteps required For instance a USAR UGV that tries to traverse a rough terrain may decideto poke the terrain surface to estimate its stability Given the geometric shape of the terrainthe robot could decide to touch the environment multiple times After the first touch a largechunk of terrain moves giving evident hints regarding the precarious stability condition ofthe area to be traversed The robot will continue touching the surface if its cognitive modelis not updated after the first interaction

Figure 6 shows results of an active exploration process obtained using one of our frame-works Here we let the robot autonomously explore the environment In this case the actionselection and region detection steps are triggered and executed by a Communication AwareMotion Planner [79] that would drive the robot in the map according to the predicted qual-ity of the wireless signal (represented as the height and color of the bell overlapping thegeometric map) and its uncertainty This perceptive paradigm shares similarities with amapping techinque called active SLAM [64] that we will briefly mention in the next sec-tion

36 Mapping the environment and moving objects

Robotic mapping [111] is one of the core concepts of this thesis More specifically we willbriefly discuss Simultaneous Localization and Mapping (SLAM) defined as the problemof combining sensor observations for jointly estimating the map and the robot pose Theterm SLAM which was originally affiliated with a family of mapping algorithms based onKalman filters [76] now describes a large variety of different mapping techniques and it


Figure 6 A virtually simulated UGV explores and maps a disaster area (grey points indicatetraversable areas red points are buildings or obstacles) Simultaneously the robot estimates thedistribution of the wireless signal on the environment and overlaps this information onto the geo-metric map (blue-green points in the wireless bell indicate good signal orange-red points are poorlyconnected areas)

is often used to refer to robotic mapping in general Depending on the application andthe kind of sensor available the mapping process can take advantage of different ways tomodel geometry

Dense Representations attempt to explicitly model the appearance of the surface withhigh quality and large amount of data Approaches such as Kinect Fusion [89] or ElasticFusion [117] have pioneered the field of low-level raw dense mapping using Kinect-likesensors This techniques generally use point clouds polygons or others low level geo-metric primitives such as surfels (ie small disks) to encode the scenersquos geometry Otherdense approaches tend to use function descriptors to generate compact representation ofthe environment The aforementioned implicit surface is a popular choice of representa-tions among these techniques Notable works use distinctive implicit surface descriptivefunctions such as the signed-distance function [25] the radial-basis functions [16] and themost recently introduced truncated signed-distance function (TSDF) [124]

A more tractable approach to mapping represents the environment as a collection ofsparse 3D features in the environment in what is commonly referred to as Landmark-BasedSparse Representations Often the large number of landmarks in an environment lowersthe performance of the mapping process due to the high computational cost of Kalman


filter-based algorithms To address this problem authors in [80] propose a Bayesian ap-proach named FastSLAM that scales logarithmically with the number of landmarks in themapFaster and more robust sparse mapping approaches for 3D cameras were proposed in [108]and [4] where in contrast to classic approaches that mainly use 3D points for registra-tion authors also used planes and 2D projected points (ie SIFT [69] features computedin the RGB camera frame and projected onto the scene using the camerarsquos intrinsics andextrinsics) as minimal set of primitives in a RANSAC [39] framework to robustly computecorrespondences and estimate the sensor pose

One interesting approach is the object oriented method named SLAM++ proposedby researchers in [101] Their framework takes advantage of the prior knowledge thatobservations in an environment present repeated domain-specific objects and structuresthat authors exploit to create robust camera-object constraints The constraints are storedin a graph of objects constantly refined by an efficient pose-graph optimization

More closely related to our work is the set of robotic mapping techniques denoted asactive SLAM [64] Active SLAM investigates how to leverage a robotrsquos motion to improvethe mapping result [110] The ideas at the base of this mapping problem shares commonancestors with our active and interactive perception frameworks [6] whose decision mak-ing process is based on the predicted map uncertainty [17] [18] Other notable approachesto active SLAM include the use of Bayesian Optimization [73] and Partially ObservableMarkov Decision Processes [54]Researchers have also been focusing on studying the mapping problem from a temporalperspective For instance works done in the context of the EU project STRANDS [48]such as [12] and [3] focused on observing the temporal behavior of an environment overtime trying to isolate the static parts of an environment and studying objectsrsquo dynamicsand contextualization (ie the meaning behind the presence of an object in a specific loca-tion at a specific time)More compelling discussions on robotic SLAM are found in [15] and [34] At the time ofthis writing the robotics mapping literature shows large gaps in aspects related to SLAMfor mobile vehicles and novel research questions are arising [15] Among many are theproblems of SLAM in Resource-Constrained Platforms Robust Distributed Mapping andMap Representation that closely touch the disaster robotics field

In this regard part of this thesis focused in developing a features-based mappingmethod for simultaneous reconstruction of static and moving parts of a scene The mainmotivation behind this work is to enable a mobile platform (eg a UGV with limited pro-cessing power on-board) to map an area while tracking and modeling rigidly moving ob-jects in an online and unsupervised fashion

Differently from other works [3] [98] [52] we avoid using observations of the envi-ronment taken at different times (ie long-term operations) as these may be unavailable inmany USAR operations and grow two different maps for the static scene and the rigidlymoving object We designed our SLAM framework to be well suited for interactive per-ception thus enabling a mobile manipulator to model objects in the environment (egsamples) by interacting with them (ie poking and rotating the object) while limiting therisk of contamination of the static scene

Chapter 4

Conclusions

We have investigated major problems in the search and rescue robotics field and proposedways of enhancing vision perception through the use of non visual sensory systems andenvironmental interactions The ideas at the base of the papers were gradually developedas a result of experience gained during the deployment of robots in real scenarios and as aconsequence of several fruitful conversations with the brilliant colleagues and researcherswhom I have shared this journey with

The explored fundamental questions are

bull How can a rescue robot cope with the lack of visual sensor information when map-ping an environment

bull What are the benefits of the interaction with the environment for a rescue robotbull How do we define the proper interactive perception strategy and how do we represent

and exploit the new information

We started by comparing two different control modes Free Look Control (FLC) andTank control and showed that FLC could substantially improve the mission performancereducing the operatorrsquos cognitive load The outcome of this study led us to implement FLCon the UGV showed in Figure 10 and to use it during real robot deployments We thenproposed a method to estimate the direction of arrival (DoA) of a radio signal and combineit with an FLC interface to reduce the risk of signal loss during remote control of a mobilerobot The effectiveness of this new interface compared to the classic signal strength bar-indicator was later demonstrated through a comparative user study This line of study gaverise to the introduction of a Resilient Communication Aware Motion Planner that takesinto account the environment geometry and a novel signal mapping method to increasethe awareness of the robot to the radio signal propagation allowing it to autonomouslynavigate in connection-safe position

The decision to focus on the development of frameworks for enhancing geometric mapsby means of interactions with the environment was initially inspired by observing the oper-ational mechanisms human responders use to explore the disaster area when poor visibilitycaused by smoke or darkness hinders mobility In limited vision conditions rescuers use

35

36 CHAPTER 4 CONCLUSIONS

their hands to touch and feel the geometry of the environment and speak to communi-cate their location Similarly a rescue robot could use its manipulator to touch and betterperceive the surrounding when environmental conditions weakens its vision system Forthis reason we introduced a tactile probabilistic perception framework based on GaussianProcess Regression that enables a rescue robot to cope with the lack of visual informationwhen mapping an environmentUsing a fast Position-based Dynamics simulator we redesigned and extended the frame-work to estimate implicit environmental properties such as the terrain deformabilityThe last contribution of this thesis is a SLAM algorithm that allows a mobile robot tosimultaneously map the environment and rigidly moving objects

The algorithms and methods presented define perception systems that

bull enable a mobile robot to map and exploit non-visual information in order to improvethe mission outcome

bull continuously trade off different sensor modalities limiting the number of environ-mental interactions as much as possible in favor of fast exploration processes

bull efficiently identify regions of interest in the robotrsquos surrounding that needs to beinvestigated

bull track and model rigidly moving objects while generating a static map of the environ-ment

With this thesis we aim to raise awareness on the many gaps found in the disasterrobotics literature and hope to inspire the reader in proposing novel solutions to addressthe open problems that still affect this field

41 Future Work

The diversity of the presented contributions gives rise to several roads of development onecan pursue We hereby highlight possible directions of research

Need of benchmarking and datasets - Although some of the robotics challenges andprojects discussed in Section 25 try to define benchmark protocols for specific rescuescenarios (eg a complex sequence of operations that define a rescue mission) the lack ofdata sets for active perception frameworks in rescue robots makes a fair comparisons withother methods difficult

Deep Learning - The impact of Deep Learning (DL) is revolutionizing the approachestaken towards many Robotics and Computer vision problems and increasingly raising thebar in many low and high level perceptual supervised learning problems such as visualrecognition tasks or speech and video analysis [63] [44] The use of DL has been extendedto reinforcement learning (RL) problems that involve visual perception and to more tradi-tional robotics problem such as SLAM In this regard [24] shows how to improve visualodometry by using deep networks to directly estimate the frame-to-frame motion sensed bya moving robot thus bypassing traditional geometric approaches for camera pose estima-tion Also interesting is the work in [68] that shows how to use deep networks to estimatethe depth map from single monocular images More recently Deep Learning is making

41 FUTURE WORK 37

significant inroads into active perception applications [28] For instance [51] shows howit is possible to perform end-to-end learning of motion policies using a Recurrent NeuralNetwork (RNN) that is suited for active recognition tasksOn the other hand most of these advances require large datasets to be effective and tend tobe restricted to the supervised learning setting which provides clear error signals Thesefactors effectively limit their usefulness in active perception frameworks for USAR appli-cations Nevertheless we strongly believe that this is a promising direction of developmentfor the framework presented in this thesis DL could be adopted to learn to forecast theeffects of the agentrsquos actions on its internal representation of the environment conditionalon all past motions and observations so to define the most effective exploration strategy

Use of different sensors - Other sensor modalities can be adopted to either complementvision or extract further useful environmental properties to enhance the geometric worldrepresentation

Multi-robot collaboration - Team of robots can be used to minimize the overall per-ceptual exploration time or to perform more complex interactions which are able to unveilspecific environmental properties The key problem of collaborative multi-robot perceptionis the robot task allocation or in simpler terms to choose which robot that has to performa certain action in a designed location For instance multiple UGVs can be used to quicklygenerate the wireless signal distribution on a large map In case of signal loss the robotscan elaborate sophisticated self re-connection strategies which see certain robots driv-ing back to designed connection safe locations and working as repeaters for the operatingagents

Use of more complex high level actions for tactile exploration - To facilitate the ex-ploration process we used simple actions such as poking and pushing the surface Aricher set of actions such as slide on surface rotate the object or any in-hand ma-nipulation may generate more informative tactile signals that could potentially reduce thenumber of interactions even further A larger action set however complicates the actionselection process raising the questions Which sequence of actions should be selectedWhen and Where

Chapter 5

Summary of Papers

39

40 CHAPTER 5 SUMMARY OF PAPERS

A Free Look UGV Teleoperation Control Tested in GameEnvironment Enhanced Performance and Reduced Workload

Abstract Concurrent telecontrol of the chassis and camera of an Unmanned Ground Vehi-cle (UGV) is a demanding task for Urban Search and Rescue (USAR) teams The standardway of controlling UGVs is called Tank Control (TC) but there is reason to believe thatFree Look Control (FLC) a control mode used in games could reduce this load substan-tially by decoupling and providing separate controls for camera translation and rotationThe general hypothesis is that FLC (1) reduces robot operatorsrsquo workload and (2) en-hances their performance for dynamic and time-critical USAR scenarios A game-basedenvironment was set-up to systematically compare FLC with TC in two typical search andrescue tasks navigation and exploration The results show that FLC improves missionperformance in both exploration (search) and path following (navigation) scenarios In theformer more objects were found and in the latter shorter navigation times were achievedFLC also caused lower workload and stress levels in both scenarios without inducing asignificant difference in the number of collisions Finally FLC was preferred by 75 ofthe subjects for exploration and 56 for path following

Figure 1 The virtually simulated UGV is controlled using two control modes (FLC and Tankcontrol) on exploration and navigation USAR scenarios

B Extending a ugv teleoperation flc interface with wireless networkconnectivity information

Abstract Teleoperated Unmanned Ground Vehicles (UGVs) are expected to play an im-portant role in future search and rescue operations In such tasks two factors are crucialfor a successful mission completion operator situational awareness and robust networkconnectivity between operator and UGV In this paper we address both these factors byextending a new Free Look Control (FLC) operator interface with a graphical representa-tion of the Radio Signal Strength (RSS) gradient at the UGV location We also provide anew way of estimating this gradient using multiple receivers with directional antennas Theproposed approach allows the operator to stay focused on the video stream providing thecrucial situational awareness while controlling the UGV to complete the mission withoutmoving into areas with dangerously low wireless connectivity

The approach is implemented on a KUKA youBot using commercial-off-the-shelfcomponents We provide experimental results showing how the proposed RSS gradientestimation method performs better than a difference approximation using omnidirectionalantennas and verify that it is indeed useful for predicting the RSS development along aUGV trajectory We also evaluate the proposed combined approach in terms of accuracyprecision sensitivity and specificity

Figure 2 The mobile robot can perceive the Direction of Arrival (DoA) or a radio signal The userinterface allows to show the DoA in the video feed consistently with the orientation of the camerawrt the world frame


C A New UGV Teleoperation Interface for Improved Awareness ofNetwork Connectivity and Physical Surroundings

Abstract A reliable wireless connection between the operator and the teleoperated un-manned ground vehicle (UGV) is critical in many urban search and rescue (USAR) mis-sions Unfortunately as was seen in for example the Fukushima nuclear disaster thenetworks available in areas where USAR missions take place are often severely limited inrange and coverage Therefore during mission execution the operator needs to keep trackof not only the physical parts of the mission such as navigating through an area or search-ing for victims but also the variations in network connectivity across the environment

In this paper we propose and evaluate a new teleoperation user interface (UI) that in-cludes a way of estimating the direction of arrival (DoA) of the radio signal strength (RSS)and integrating the DoA information in the interface The evaluation shows that usingthe interface results in more objects found and less aborted missions due to connectivityproblems as compared to a standard interface

The proposed interface is an extension to an existing interface centered on the videostream captured by the UGV But instead of just showing the network signal strength interms of percent and a set of bars the additional information of DoA is added in termsof a color bar surrounding the video feed With this information the operator knowswhat movement directions are safe even when moving in regions close to the connectivitythreshold

Figure 3 The YouBot is teleoperated into a small map partially covered by a wireless radio signalParticipants of a user study must found symbols and explore the environment without losing signalusing different user interfaces

D RCAMP Resilient Communication-Aware Motion Planner andAutonomous Repair of Wireless Connectivity in Mobile Robots

Abstract Mobile robots be it autonomous or teleoperated require stable communicationwith the base station to exchange valuable information Given the stochastic elements in ra-dio signal propagation such as shadowing and fading and the possibilities of unpredictableevents or hardware failures communication loss often presents a significant mission riskboth in terms of probability and impact especially in Urban Search and Rescue (USAR)operations Depending on the circumstances disconnected robots are either abandoned orattempt to autonomously back-trace their way to the base station Although recent resultsin Communication-Aware Motion Planning can be used to effectively manage connectiv-ity with robots there are no results focusing on autonomously re-establishing the wirelessconnectivity of a mobile robot without back-tracing or using detailed a priori informationof the network

In this paper we present a robust and online radio signal mapping method usingGaussian Random Fields and propose a Resilient Communication-Aware Motion Plan-ner (RCAMP) that integrates the above signal mapping framework with a motion plan-ner RCAMP considers both the environment and the physical constraints of the robotbased on the available sensory information We also propose a self-repair strategy us-ing RCMAP that takes both connectivity and the goal position into account when drivingto a connection-safe position in the event of a communication loss We demonstrate theproposed planner in a set of realistic simulations of an exploration task in single or multi-channel communication scenarios

Figure 4 A virtually simulated UGV explores a disaster area and builds its geometric representationSimultaneously the system estimates the distribution of the wireless signal on the environment andoverlaps this information onto the geometric map


E Active Exploration Using Gaussian Random Fields and GaussianProcess Implicit Surfaces

Abstract In this work we study the problem of exploring surfaces and building compact3D representations of the environment surrounding a robot through active perception Wepropose an online probabilistic framework that merges visual and tactile measurementsusing Gaussian Random Field and Gaussian Process Implicit Surfaces The system inves-tigates incomplete point clouds in order to find a small set of regions of interest whichare then physically explored with a robotic arm equipped with tactile sensors We showexperimental results obtained using a PrimeSense camera a Kinova Jaco2 robotic armand Optoforce sensors on different scenarios We then demonstrate how to use the onlineframework for object detection and terrain classification

Figure 5 The geometric representation of the scenario based on vision alone can lead to ambiguityin case of occlusions (A) or photo metric effects Physical interactions with the environment allowto refine the representation and remove ambiguity (B)

F Active Perception and Modeling of Deformable Surfaces usingGaussian Processes and Position-based Dynamics

Abstract Exploring and modeling heterogeneous elastic surfaces requires multiple inter-actions with the environment and a complex selection of physical material parameters Themost common approaches model deformable properties from sets of offline observationsusing computationally expensive force-based simulators In this work we present an onlineprobabilistic framework for autonomous estimation of a deformability distribution map ofheterogeneous elastic surfaces from few physical interactions The method takes advantageof Gaussian Processes for constructing a model of the environment geometry surrounding arobot A fast Position-based Dynamics simulator uses focused environmental observationsin order to model the elastic behavior of portions of the environment Gaussian ProcessRegression maps the local deformability on the whole environment in order to generate adeformability distribution map We show experimental results using a PrimeSense cameraa Kinova Jaco2 robotic arm and an Optoforce sensor on different deformable surfaces

Figure 6 Conceptual representation of the deformability analysis of a surface through physicalinteractions


G Joint 3D Reconstruction of a Static Scene and Moving Objects

Abstract We present a technique for simultaneous 3D reconstruction of static regionsand rigidly moving objects in a scene An RGB-D frame is represented as a collectionof features which are points and planes We classify the features into static and dynamicregions and grow separate maps static and object maps for each of them To robustlyclassify the features in each frame we fuse multiple RANSAC-based registration resultsobtained by registering different groups of the features to different maps including (1) allthe features to the static map (2) all the features to each object map and (3) subsets ofthe features each forming a segment to each object map This multi-group registrationapproach is designed to overcome the following challenges scenes can be dominated bystatic regions making object tracking more difficult and moving object might have largerpose variation between frames compared to the static regions We show qualitative resultsfrom indoor scenes with objects in various shapes The technique enables on-the-fly objectmodel generation to be used for robotic manipulation

Figure 7 Example of on-line object reconstruction (C) from environmental observations Theobjects is tracked (A) and a feature map is created (B) Simultaneously the algorithm allows to mapthe environment and localize both the robot and the object

47

References

[1] Ieee standard for robot map data representation for navigation 1873-2015 IEEEStandard for Robot Map Data Representation for Navigation pages 1ndash54 Oct 2015

[2] S V Alexandrov J Prankl M Zillich and M Vincze Calibration and correctionof vignetting effects with an application to 3d mapping In 2016 IEEERSJ Interna-tional Conference on Intelligent Robots and Systems (IROS) pages 4217ndash4223 Oct2016

[3] R Ambrus J Ekekrantz J Folkesson and P Jensfelt Unsupervised learning ofspatial-temporal models of objects in a long-term autonomy scenario In 2015IEEERSJ International Conference on Intelligent Robots and Systems (IROS)pages 5678ndash5685 Sept 2015

[4] E Ataer-Cansizoglu Y Taguchi and S Ramalingam Pinpoint slam A hybrid of2d and 3d simultaneous localization and mapping for rgb-d sensors In 2016 IEEEInternational Conference on Robotics and Automation (ICRA) pages 1300ndash1307May 2016

[5] R Bajcsy Active perception Proceedings of the IEEE 76(8)966ndash1005 Aug 1988

[6] Ruzena Bajcsy Yiannis Aloimonos and John K Tsotsos Revisiting active per-ception Autonomous Robots Feb 2017 URL httpsdoiorg101007s10514-017-9615-3

[7] Robot vessels used to cap gulf of mexico oil leak 2018 URL httpnewsbbccouk2hiamericas8643782stm [Online accessed 23-January-2018]

[8] Zoltan Beck Collaborative search and rescue by autonomous robots PhD thesisUniversity of Southampton December 2016 URL httpseprintssotonacuk411031

[9] Paul J Besl and Neil D McKay Method for registration of 3-d shapes In SensorFusion IV Control Paradigms and Data Structures volume 1611 pages 586ndash607International Society for Optics and Photonics 1992

[10] A Bicchi J K Salisbury and P Dario Augmentation of grasp robustness usingintrinsic tactile sensing In Proceedings 1989 International Conference on Roboticsand Automation pages 302ndash307 vol1 May 1989

[11] J Bohg K Hausman B Sankaran O Brock D Kragic S Schaal and G SSukhatme Interactive perception Leveraging action in perception and perceptionin action IEEE Transactions on Robotics 33(6)1273ndash1291 Dec 2017

[12] Nils Bore Johan Ekekrantz Patric Jensfelt and John Folkesson Detectionand tracking of general movable objects in large 3d maps arXiv preprintarXiv171208409 2017

48 BIBLIOGRAPHY

[13] Michael Bosse and Robert Zlot Continuous 3d scan-matching with a spinning 2dlaser In Robotics and Automation 2009 ICRArsquo09 IEEE International Conferenceon pages 4312ndash4319 IEEE 2009

[14] Jennifer L Burke Robin R Murphy Michael D Coovert and Dawn L Rid-dle Moonlight in miami A field study of human-robot interaction in the con-text of an urban search and rescue disaster response training exercise Hum-Comput Interact 19(1)85ndash116 June 2004 URL httpdxdoiorg101207s15327051hci1901amp2_5

[15] C Cadena L Carlone H Carrillo Y Latif D Scaramuzza J Neira I Reid andJ J Leonard Past present and future of simultaneous localization and mappingToward the robust-perception age IEEE Transactions on Robotics 32(6)1309ndash1332 Dec 2016

[16] Jonathan C Carr Richard K Beatson Jon B Cherrie Tim J Mitchell W RichardFright Bruce C McCallum and Tim R Evans Reconstruction and representation of3d objects with radial basis functions In Proceedings of the 28th annual conferenceon Computer graphics and interactive techniques pages 67ndash76 ACM 2001

[17] H Carrillo Y Latif J Neira and J A Castellanos Fast minimum uncertaintysearch on a graph map representation In 2012 IEEERSJ International Conferenceon Intelligent Robots and Systems pages 2504ndash2511 Oct 2012

[18] H Carrillo I Reid and J A Castellanos On the comparison of uncertainty criteriafor active slam In 2012 IEEE International Conference on Robotics and Automa-tion pages 2080ndash2087 May 2012

[19] J Casper and R R Murphy Human-robot interactions during the robot-assistedurban search and rescue response at the world trade center IEEE Transactions onSystems Man and Cybernetics Part B (Cybernetics) 33(3)367ndash385 June 2003

[20] Centauro Robust mobility and dexterous manipulation in disaster responseby fullbody telepresence in a centaur-like robot 2018 URL httpwwwcentauro-projecteu [Online accessed 18-January-2018]

[21] Center for robot-assisted search and rescue (crasar) 2018 URL httpcrasarorg [Online accessed 18-January-2018]

[22] Yang Chen and Geacuterard Medioni Object modelling by registration of multiple rangeimages Image and Vision Computing 10(3)145 ndash 155 1992 URL httpwwwsciencedirectcomsciencearticlepii026288569290066C Range Im-age Understanding

[23] Cocoro robot swarms use collective cognition to perform tasks 2018 URL httpcordiseuropaeuprojectrcn97473_enhtml [Online accessed 18-January-2018]

49

[24] G Costante M Mancini P Valigi and T A Ciarfuglia Exploring representationlearning with cnns for frame-to-frame ego-motion estimation IEEE Robotics andAutomation Letters 1(1)18ndash25 Jan 2016

[25] Brian Curless and Marc Levoy A volumetric method for building complex modelsfrom range images In Proceedings of the 23rd annual conference on Computergraphics and interactive techniques pages 303ndash312 ACM 1996

[26] J d Leon M Garzon D Garzon E Narvaez J d Cerro and A Barrientos Fromvideo games multiple cameras to multi-robot teleoperation in disaster scenarios In2016 International Conference on Autonomous Robot Systems and Competitions(ICARSC) pages 323ndash328 May 2016

[27] Darpa robotics challenge (drc) 2018 URL httpswwwdarpamilprogramdarpa-robotics-challenge [Online accessed 18-January-2018]

[28] Emmanuel Dauceacute Toward predictive machine learning for active vision CoRRabs171010460 2017 URL httparxivorgabs171010460

[29] Pascaline Wallemacq Debarati Guha-Sapir Philippe Hoyois and Regina BelowAnnual disaster statistical review 2016 - the numbers and trends Technical re-port Universiteacute catholique de Louvain Brussels Belgium 01 2017 URL httpemdatbesitesdefaultfilesadsr_2016pdf

[30] Paul E Debevec and Jitendra Malik Recovering high dynamic range radiance mapsfrom photographs In Proceedings of the 24th Annual Conference on ComputerGraphics and Interactive Techniques SIGGRAPH rsquo97 pages 369ndash378 New YorkNY USA 1997 ACM PressAddison-Wesley Publishing Co ISBN 0-89791-896-7URL httpdxdoiorg101145258734258884

[31] B Deng Machine ethics The robotrsquos dilemma Nature 52324ndash26 jul 2015 URLhttpadsabsharvardeduabs2015Natur52324D

[32] Manfredo P Do Carmo Differential Geometry of Curves and Surfaces Revised andUpdated Second Edition Courier Dover Publications 2016

[33] R Dubeacute A Gawel C Cadena R Siegwart L Freda and M Gianni 3d localiza-tion mapping and path planning for search and rescue operations In 2016 IEEEInternational Symposium on Safety Security and Rescue Robotics (SSRR) pages272ndash273 Oct 2016

[34] H Durrant-Whyte and T Bailey Simultaneous localization and mapping part iIEEE Robotics Automation Magazine 13(2)99ndash110 June 2006

[35] Mica R Endsley Measurement of situation awareness in dynamic sys-tems Human Factors 37(1)65ndash84 1995 URL httpsdoiorg101518001872095779049499

50 BIBLIOGRAPHY

[36] European robotics league Emergency robots 2018 URL httpswwweu-roboticsnetrobotics_league [Online accessed 18-January-2018]

[37] Marc O Ernst and Heinrich H Buumllthoff Merging the senses into a robust perceptTrends in cognitive sciences 8(4)162ndash169 2004

[38] Haoqiang Fan Hao Su and Leonidas Guibas A point set generation network for 3dobject reconstruction from a single image In Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition pages 652ndash660 2017

[39] Martin A Fischler and Robert C Bolles Random sample consensus a paradigmfor model fitting with applications to image analysis and automated cartography InReadings in computer vision pages 726ndash740 Elsevier 1987

[40] C Forster M Pizzoli and D Scaramuzza Air-ground localization and map aug-mentation using monocular dense reconstruction In 2013 IEEERSJ InternationalConference on Intelligent Robots and Systems pages 3971ndash3978 Nov 2013

[41] Steven Fortune Handbook of discrete and computational geometry chapterVoronoi Diagrams and Delaunay Triangulations pages 377ndash388 CRC Press IncBoca Raton FL USA 1997 ISBN 0-8493-8524-5 URL httpdlacmorgcitationcfmid=285869285891

[42] D Fox J Ko K Konolige B Limketkai D Schulz and B Stewart Distributedmultirobot exploration and mapping Proceedings of the IEEE 94(7)1325ndash1339July 2006

[43] D B Goldman and Jiun-Hung Chen Vignette and exposure calibration and com-pensation In Tenth IEEE International Conference on Computer Vision (ICCVrsquo05)Volume 1 volume 1 pages 899ndash906 Vol 1 Oct 2005

[44] Ian Goodfellow Yoshua Bengio and Aaron Courville Deep Learning MIT Press2016 httpwwwdeeplearningbookorg

[45] Guardians Group of unmanned assistant robots deployed in aggregative navigationsupported by scent detection 2018 URL httpcordiseuropaeuprojectrcn80210_enhtml [Online accessed 18-January-2018]

[46] M Gupta A Agrawal A Veeraraghavan and S G Narasimhan Structured light3d scanning in the presence of global illumination In CVPR 2011 pages 713ndash720June 2011

[47] K Han K Y K Wong and M Liu A fixed viewpoint approach for dense recon-struction of transparent objects In 2015 IEEE Conference on Computer Vision andPattern Recognition (CVPR) pages 4001ndash4008 June 2015

51

[48] N Hawes C Burbridge F Jovan L Kunze B Lacerda L Mudrova J YoungJ Wyatt D Hebesberger T Kortner R Ambrus N Bore J Folkesson P JensfeltL Beyer A Hermans B Leibe A Aldoma T Faulhammer M Zillich M VinczeE Chinellato M Al-Omari P Duckworth Y Gatsoulis D C Hogg A G CohnC Dondrup J Pulido Fentanes T Krajnik J M Santos T Duckett and M Han-heide The strands project Long-term autonomy in everyday environments IEEERobotics Automation Magazine 24(3)146ndash156 Sept 2017

[49] Hugues Hoppe Tony DeRose Tom Duchamp John McDonald and Werner Stuet-zle Surface reconstruction from unorganized points In Proceedings of the 19thAnnual Conference on Computer Graphics and Interactive Techniques SIGGRAPHrsquo92 pages 71ndash78 New York NY USA 1992 ACM ISBN 0-89791-479-1 URLhttpdoiacmorg101145133994134011

[50] Icarus - unmanned search and rescue 2018 URL httpwwwfp7-icaruseuproject-overview [Online accessed 20-January-2018]

[51] Dinesh Jayaraman and Kristen Grauman Look-ahead before you leap End-to-endactive recognition by forecasting the effect of motion In Bastian Leibe Jiri MatasNicu Sebe and Max Welling editors Computer Vision ndash ECCV 2016 pages 489ndash505 Cham 2016 Springer International Publishing ISBN 978-3-319-46454-1

[52] C Jiang D P Paudel Y Fougerolle D Fofi and C Demonceaux Static-map anddynamic object reconstruction in outdoor scenes using 3-d motion segmentationIEEE Robotics and Automation Letters 1(1)324ndash331 Jan 2016

[53] M W Kadous R K M Sheh and C Sammut Controlling heterogeneous semi-autonomous rescue robot teams In 2006 IEEE International Conference on Sys-tems Man and Cybernetics volume 4 pages 3204ndash3209 Oct 2006

[54] Leslie Pack Kaelbling Michael L Littman and Anthony R Cassandra Planning andacting in partially observable stochastic domains Artificial intelligence 101(1-2)99ndash134 1998

[55] Eugenia Kalnay and Ming Cai Impact of urbanization and land-use change onclimate Nature 423528 EP ndash May 2003 URL httpdxdoiorg101038nature01675

[56] Kourosh Khoshelham and Sander Oude Elberink Accuracy and resolution of kinectdepth data for indoor mapping applications Sensors 12(2)1437ndash1454 2012 URLhttpwwwmdpicom1424-82201221437

[57] Dennis K Killinger and Aram Mooradian Optical and laser remote sensing vol-ume 39 Springer 2013

[58] S Kim and J Kim Occupancy mapping and surface reconstruction using localgaussian processes with kinect sensors IEEE Transactions on Cybernetics 43(5)1335ndash1346 Oct 2013

52 BIBLIOGRAPHY

[59] Kinect for microsoft windows 2018 URL httpsdevelopermicrosoftcomen-uswindowskinect [Online accessed 18-January-2018]

[60] G J M Kruijff M Janiacutecek S Keshavdas B Larochelle H Zender N J J MSmets T Mioch M A Neerincx J V Diggelen F Colas M Liu F PomerleauR Siegwart V Hlavaacutec T Svoboda T Petriacutecek M Reinstein K ZimmermannF Pirri M Gianni P Papadakis A Sinha P Balmer N Tomatis R Worst T Lin-der H Surmann V Tretyakov S Corrao S Pratzler-Wanczura and M Sulk Ex-perience in System Design for Human-Robot Teaming in Urban Search and Rescuepages 111ndash125 Springer Berlin Heidelberg Berlin Heidelberg 2014 ISBN 978-3-642-40686-7 URL httpsdoiorg101007978-3-642-40686-7_8

[61] I Kruijff-Korbayovaacute L Freda M Gianni V Ntouskos V Hlavaacutec V KubelkaE Zimmermann H Surmann K Dulic W Rottner and E Gissi Deploymentof ground and aerial robots in earthquake-struck amatrice in italy (brief report)In 2016 IEEE International Symposium on Safety Security and Rescue Robotics(SSRR) pages 278ndash279 Oct 2016

[62] Ivana Kruijff-Korbayovaacute Francis Colas Mario Gianni Fiora Pirri Joachim de Gre-eff Koen Hindriks Mark Neerincx Petter Oumlgren Tomaacuteš Svoboda and RainerWorst Tradr project Long-term human-robot teaming for robot assisted dis-aster response KI - Kuumlnstliche Intelligenz 29(2)193ndash201 Jun 2015 URLhttpsdoiorg101007s13218-015-0352-5

[63] Yann LeCun Yoshua Bengio and Geoffrey Hinton Deep learning nature 521(7553)436 2015

[64] C Leung S Huang and G Dissanayake Active slam using model predictive con-trol and attractor based exploration In 2006 IEEERSJ International Conference onIntelligent Robots and Systems pages 5026ndash5031 Oct 2006

[65] Jesse Levinson Jake Askeland Jan Becker Jennifer Dolson David Held SoerenKammel J Zico Kolter Dirk Langer Oliver Pink Vaughan Pratt et al Towardsfully autonomous driving Systems and algorithms In Intelligent Vehicles Sympo-sium (IV) 2011 IEEE pages 163ndash168 IEEE 2011

[66] H Lin and Z Song 3d reconstruction of specular surface via a novel structured lightapproach In 2015 IEEE International Conference on Information and Automationpages 530ndash534 Aug 2015

[67] D Liu X Chen and Y H Yang Frequency-based 3d reconstruction of transparentand specular objects In 2014 IEEE Conference on Computer Vision and PatternRecognition pages 660ndash667 June 2014

[68] Fayao Liu Chunhua Shen and Guosheng Lin Deep convolutional neural fields fordepth estimation from a single image In Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition pages 5162ndash5170 2015

53

[69] David G Lowe Distinctive image features from scale-invariant keypoints In-ternational Journal of Computer Vision 60(2)91ndash110 Nov 2004 URL httpsdoiorg101023BVISI00000296649961594

[70] Catherine Manzi Michael J Powers and Kristina Zetterlund Critical informationflows in the Alfred P Murrah Building bombing a case study Chemical and Bio-logical Arms Control Institute 2002

[71] L Marconi C Melchiorri M Beetz D Pangercic R Siegwart S LeuteneggerR Carloni S Stramigioli H Bruyninckx P Doherty A Kleiner V LippielloA Finzi B Siciliano A Sala and N Tomatis The sherpa project Smart collab-oration between humans and ground-aerial robots for improving rescuing activitiesin alpine environments In 2012 IEEE International Symposium on Safety Securityand Rescue Robotics (SSRR) pages 1ndash4 Nov 2018

[72] Debarati Guha-Sapir Margareta Wahlstrom The human cost of natural disasters2015 a global perspective Technical report Universiteacute catholique de LouvainBrussels Belgium 01 2015 URL httpsreliefwebintreportworldhuman-cost-natural-disasters-2015-global-perspective

[73] Ruben Martinez-Cantin Nando de Freitas Eric Brochu Joseacute Castellanos and Ar-naud Doucet A bayesian exploration-exploitation approach for optimal online sens-ing and planning with a visually guided mobile robot Autonomous Robots 27(2)93ndash103 2009

[74] F Matsuno and S Tadokoro Rescue robots and systems in japan In 2004 IEEEInternational Conference on Robotics and Biomimetics pages 12ndash20 Aug 2004

[75] Bruce A Maxwell Nicolas Ward and Frederick Heckel Game-based design ofhuman-robot interfaces for urban search and rescue In In Computer-Human Inter-face Fringe 2004

[76] Peter S Maybeck Stochastic models estimation and control volume 3 Academicpress 1982

[77] Nathan Michael Shaojie Shen Kartik Mohta Yash Mulgaonkar Vijay KumarKeiji Nagatani Yoshito Okada Seiga Kiribayashi Kazuki Otake Kazuya YoshidaKazunori Ohno Eijiro Takeuchi and Satoshi Tadokoro Collaborative mappingof an earthquake-damaged building via ground and aerial robots Journal ofField Robotics 29(5)832ndash841 2012 URL httpdxdoiorg101002rob21436


54 BIBLIOGRAPHY

[79] Magnus Minnema Lindheacute Communication-aware motion planning for mobilerobots PhD thesis KTH Royal Institute of Technology 2012

[80] Michael Montemerlo Sebastian Thrun Daphne Koller Ben Wegbreit et al Fast-slam A factored solution to the simultaneous localization and mapping problemAaaiiaai 593598 2002

[81] S A A Moosavian H Semsarilar and A Kalantari Design and manufacturing ofa mobile rescue robot In 2006 IEEERSJ International Conference on IntelligentRobots and Systems pages 3982ndash3987 Oct 2006

[82] R R Murphy E Steimle M Hall M Lindemuth D Trejo S HurlebausZ Medina-Cetina and D Slocum Robot-assisted bridge inspection after hurri-cane ike In 2009 IEEE International Workshop on Safety Security Rescue Robotics(SSRR 2009) pages 1ndash5 Nov 2009

[83] Robin R Murphy National science foundation summer field institute for rescuerobots for research and response (r4) AI Magazine 25(2)133 2004

[84] Robin R Murphy Disaster Robotics The MIT Press 2014 ISBN 02620273569780262027359

[85] Robin R Murphy Satoshi Tadokoro and Alexander Kleiner Disaster Roboticspages 1577ndash1604 Springer International Publishing Cham 2016 ISBN 978-3-319-32552-1 URL httpsdoiorg101007978-3-319-32552-1_60

[86] K Nagatani and H Choset Toward robust sensor based exploration by constructingreduced generalized voronoi graph In Proceedings 1999 IEEERSJ InternationalConference on Intelligent Robots and Systems Human and Environment FriendlyRobots with High Intelligence and Emotional Quotients (Cat No99CH36289) vol-ume 3 pages 1687ndash1692 vol3 1999

[87] Keiji Nagatani Seiga Kiribayashi Yoshito Okada Kazuki Otake Kazuya YoshidaSatoshi Tadokoro Takeshi Nishimura Tomoaki Yoshida Eiji Koyanagi MineoFukushima et al Emergency response to the nuclear accident at the fukushimadaiichi nuclear power plants using mobile rescue robots Journal of Field Robotics30(1)44ndash63 2013

[88] Lorenzo Natale Giorgio Metta and Giulio Sandini Learning haptic representationof objects In International Conference on Intelligent Manipulation and GraspingJuly 2004

[89] R A Newcombe S Izadi O Hilliges D Molyneaux D Kim A J DavisonP Kohi J Shotton S Hodges and A Fitzgibbon Kinectfusion Real-time densesurface mapping and tracking In 2011 10th IEEE International Symposium onMixed and Augmented Reality pages 127ndash136 Oct 2011

55

[90] M A OLIVER and R WEBSTER Kriging a method of interpolation forgeographical information systems International Journal of Geographical In-formation Systems 4(3)313ndash332 1990 URL httpsdoiorg10108002693799008941549

[91] Barrett Orsquoneill Elementary differential geometry Academic press 2006

[92] Thomas J Palmeri Bradley C Love and Brandon M Turner Model-based cognitive neuroscience Journal of Mathematical Psychology 7659 ndash64 2017 URL httpwwwsciencedirectcomsciencearticlepiiS002224961630116X Model-based Cognitive Neuroscience

[93] Ramviyas Parasuraman Sergio Caccamo Fredrik Baberg and Petter Ogren Craw-dad dataset kthrss (v 2016-01-05) 2016

[94] Charles R Qi Hao Su Kaichun Mo and Leonidas J Guibas Pointnet Deep learningon point sets for 3d classification and segmentation In Proceedings of the IEEEConference on Computer Vision and Pattern Recognition pages 652ndash660 2017

[95] Y Qian M Gong and Y H Yang 3d reconstruction of transparent objects withposition-normal consistency In 2016 IEEE Conference on Computer Vision andPattern Recognition (CVPR) pages 4369ndash4377 June 2016

[96] Carl Edward Rasmussen and Christopher K I Williams Gaussian Processes forMachine Learning The MIT Press 2006 ISBN 13978-0-262-18253-9 URL httpwwwgaussianprocessorggpmlchapters

[97] Erik Reinhard Greg Ward Sumanta Pattanaik and Paul Debevec High DynamicRange Imaging Acquisition Display and Image-Based Lighting (The MorganKaufmann Series in Computer Graphics) Morgan Kaufmann Publishers Inc SanFrancisco CA USA 2005 ISBN 0125852630

[98] C Y Ren V Prisacariu D Murray and I Reid Star3d Simultaneous tracking andreconstruction of 3d objects using rgb-d data In 2013 IEEE International Confer-ence on Computer Vision pages 1561ndash1568 Dec 2013

[99] The robocup rescue league 2018 URL httpwwwrobocup2016orgenleaguesrobocup-rescue [Online accessed 18-January-2018]

[100] David Rodenas-Herraiz Antonio-Javier Garcia-Sanchez Felipe Garcia-Sanchezand Joan Garcia-Haro Current trends in wireless mesh sensor networks A re-view of competing approaches Sensors 13(12)5958ndash5995 May 2013 URLhttpdxdoiorg103390s130505958

[101] Renato F Salas-Moreno Richard A Newcombe Hauke Strasdat Paul H J Kellyand Andrew J Davison Slam++ Simultaneous localisation and mapping at thelevel of objects In Proceedings of the 2013 IEEE Conference on Computer Visionand Pattern Recognition CVPR rsquo13 pages 1352ndash1359 Washington DC USA

56 BIBLIOGRAPHY

2013 IEEE Computer Society ISBN 978-0-7695-4989-7 URL httpdxdoiorg101109CVPR2013178

[102] J Scholtz J Young J L Drury and H A Yanco Evaluation of human-robotinteraction awareness in search and rescue In Robotics and Automation 2004Proceedings ICRA rsquo04 2004 IEEE International Conference on volume 3 pages2327ndash2332 Vol3 April 2004

[103] H Shim and S Lee Recovering translucent objects using a single time-of-flightdepth camera IEEE Transactions on Circuits and Systems for Video Technology 26(5)841ndash854 May 2016

[104] Jivko Sinapov Taylor Bergquist Connor Schenck Ugonna Ohiri Shane Griffithand Alexander Stoytchev Interactive object recognition using proprioceptive andauditory feedback The International Journal of Robotics Research 30(10)1250ndash1262 2011

[105] Smokebot Mobile robots with novel environmental sensors for inspection of disas-ter sites with low visibility 2018 URL httpcordiseuropaeuprojectrcn194282_enhtml [Online accessed 18-January-2018]

[106] Aaron Steinfeld Terrence Fong David Kaber Michael Lewis Jean Scholtz AlanSchultz and Michael Goodrich Common metrics for human-robot interaction InProceedings of the 1st ACM SIGCHISIGART Conference on Human-robot Interac-tion HRI rsquo06 pages 33ndash40 New York NY USA 2006 ACM ISBN 1-59593-294-1 URL httpdoiacmorg10114511212411121249

[107] Satoshi Tadokoro Summary of DDT Project Unsolved Problems and FutureRoadmap pages 175ndash189 Springer London London 2009 ISBN 978-1-84882-474-4 URL httpsdoiorg101007978-1-84882-474-4_10

[108] Yuichi Taguchi Yong-Dian Jian Srikumar Ramalingam and Chen Feng Point-plane slam for hand-held 3d sensors In Robotics and Automation (ICRA) 2013IEEE International Conference on pages 5182ndash5189 IEEE 2013

[109] Ieee spectrum Robots enter fukushima reactors detect high radiation 2012 URLhttpsspectrumieeeorgautomatonroboticsindustrial-robotsrobots-enter-fukushima-reactors-detect-high-radiation [Onlineaccessed 18-January-2018]

[110] Sebastian Thrun Probabilistic robotics Communications of the ACM 45(3)52ndash572002

[111] Sebastian Thrun et al Robotic mapping A survey Exploring artificial intelligencein the new millennium 11ndash35

57

[112] S K Tin J Ye M Nezamabadi and C Chen 3d reconstruction of mirror-typeobjects using efficient ray coding In 2016 IEEE International Conference on Com-putational Photography (ICCP) pages 1ndash11 May 2016

[113] Shrihari Vasudevan Fabio Ramos Eric Nettleton and Hugh Durrant-Whyte Gaus-sian process modeling of large-scale terrain Journal of Field Robotics 26(10)812ndash840 2009 URL httpdxdoiorg101002rob20309

[114] Raquel Viciana-Abad Rebeca Marfil Jose M Perez-Lorenzo Juan P BanderaAdrian Romero-Garces and Pedro Reche-Lopez Audio-visual perception sys-tem for a humanoid robotic head Sensors 14(6)9522ndash9545 2014 URL httpwwwmdpicom1424-82201469522

[115] Jean Vroomen and Beatrice de Gelder Sound enhances visual perception cross-modal effects of auditory organization on vision Journal of experimental psychol-ogy Human perception and performance 26(5)1583 2000

[116] Claus Weitkamp Lidar range-resolved optical remote sensing of the atmospherevolume 102 Springer Science amp Business 2006

[117] Thomas Whelan Renato F Salas-Moreno Ben Glocker Andrew J Davison and Ste-fan Leutenegger Elasticfusion Real-time dense slam and light source estimationThe International Journal of Robotics Research 35(14)1697ndash1716 2016 URLhttpsdoiorg1011770278364916669237

[118] O Williams and A Fitzgibbon Gaussian process implicit surfaces Gaussian Procin Practice 2007

[119] Alan F T Winfield Christian Blum and Wenguo Liu Towards an ethical robotInternal models consequences and ethical action selection In Michael Mistry AlešLeonardis Mark Witkowski and Chris Melhuish editors Advances in AutonomousRobotics Systems pages 85ndash96 Cham 2014 Springer International PublishingISBN 978-3-319-10401-0

[120] C Wright A Johnson A Peck Z McCord A Naaktgeboren P GianfortoniM Gonzalez-Rivero R Hatton and H Choset Design of a modular snake robot In2007 IEEERSJ International Conference on Intelligent Robots and Systems pages2609ndash2614 Oct 2007

[121] Russell B Wynn Veerle AI Huvenne Timothy P Le Bas Bramley J MurtonDouglas P Connelly Brian J Bett Henry A Ruhl Kirsty J Morris Jeffrey PeakallDaniel R Parsons Esther J Sumner Stephen E Darby Robert M Dorrell andJames E Hunt Autonomous underwater vehicles (auvs) Their past present and fu-ture contributions to the advancement of marine geoscience Marine Geology 352451 ndash 468 2014 URL httpwwwsciencedirectcomsciencearticlepiiS0025322714000747 50th Anniversary Special Issue

58 BIBLIOGRAPHY

[122] Euijung Yang and Michael C Dorneich The emotional cognitive physiologi-cal and performance effects of variable time delay in robotic teleoperation In-ternational Journal of Social Robotics 9(4)491ndash508 Sep 2017 URL httpsdoiorg101007s12369-017-0407-x

[123] Tomoaki Yoshida Keiji Nagatani Satoshi Tadokoro Takeshi Nishimura and EijiKoyanagi Improvements to the Rescue Robot Quince Toward Future Indoor Surveil-lance Missions in the Fukushima Daiichi Nuclear Power Plant pages 19ndash32Springer Berlin Heidelberg Berlin Heidelberg 2014 ISBN 978-3-642-40686-7URL httpsdoiorg101007978-3-642-40686-7_2

[124] Christopher Zach Thomas Pock and Horst Bischof A globally optimal algorithmfor robust tv-l 1 range image integration In Computer Vision 2007 ICCV 2007IEEE 11th International Conference on pages 1ndash8 IEEE 2007

[125] Pietro Zanuttigh Giulio Marin Carlo Dal Mutto Fabio Dominio Ludovico Mintoand Guido Maria Cortelazzo Time-of-flight and structured light depth camerasSpringer 2016 URL httpsdoiorg101007978-3-319-30973-6

[126] Wende Zhang Lidar-based road and road-edge detection In Intelligent VehiclesSymposium (IV) 2010 IEEE pages 845ndash848 IEEE 2010

[127] Jiayuan Zhu and Yee-Hong Yang Frequency-based environment matting In 12thPacific Conference on Computer Graphics and Applications 2004 PG 2004 Pro-ceedings pages 402ndash410 Oct 2004

[128] K Zimmermann P Zuzanek M Reinstein and V Hlavac Adaptive traversabilityof unknown complex terrain with obstacles for mobile robots In 2014 IEEE Inter-national Conference on Robotics and Automation (ICRA) pages 5177ndash5182 May2014

Contents

Introduction

Introduction

Robotic systems that save lives

Vision in SAR robotics

Contributions and thesis outline

Disaster Robotics

Overview

Historical Perspective

Types of Rescue Robots

The Users Perspective

Notable SAR Research Projects Groups and Challenges

SAR Problems Addressed in This Thesis

Control

Communication

Human-robot interaction

Mapping


Visual Perception System

RGBD cameras and LiDARs

Limitation of vision sensors

Benefits of Active and Interactive Perception

Geometric Representations

A Probabilistic Approach to Interactive Mapping

Gaussian Random Fields

Gaussian Processes Implicit Surfaces

Strategies for training and interacting

Mapping the environment and moving objects

Conclusions

Future Work

Summary of Papers

Free Look UGV Teleoperation Control Tested in Game Environment Enhanced Performance and Reduced Workload

Extending a ugv teleoperation flc interface with wireless network connectivity information

A New UGV Teleoperation Interface for Improved Awareness of Network Connectivity and Physical Surroundings

RCAMP Resilient Communication-Aware Motion Planner and Autonomous Repair of Wireless Connectivity in Mobile Robots

Active Exploration Using Gaussian Random Fields and Gaussian Process Implicit Surfaces

Active Perception and Modeling of Deformable Surfaces using Gaussian Processes and Position-based Dynamics

Joint 3D Reconstruction of a Static Scene and Moving Objects

Bibliography

TRITA-EECS-AVL-201826ISBN 978-91-7729-720-8

Robotics Perception and LearningSchool of Electrical Engineering and Computer Science

KTH Royal Institute of TechnologySE-100 44 Stockholm Sweden

Copyright copy 2018 by Sergio S Caccamo except where otherwise stated

Tryck Universitetsservice US-AB 2018

iii

Abstract






iv

Sammanfattning











v

Acknowledgments




vi

List of Papers









vii








viii

List of Acronyms


Contents

Contents ix

I Introduction 1









ix

x CONTENTS









Bibliography 47

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

iii

Abstract






iv

Sammanfattning











v

Acknowledgments




vi

List of Papers









vii








viii

List of Acronyms


Contents

Contents ix

I Introduction 1









ix

x CONTENTS









Bibliography 47

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

iv

Sammanfattning











v

Acknowledgments




vi

List of Papers









vii








viii

List of Acronyms


Contents

Contents ix

I Introduction 1









ix

x CONTENTS









Bibliography 47

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

v

Acknowledgments




vi

List of Papers









vii








viii

List of Acronyms


Contents

Contents ix

I Introduction 1









ix

x CONTENTS









Bibliography 47

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

vi

List of Papers









vii








viii

List of Acronyms


Contents

Contents ix

I Introduction 1









ix

x CONTENTS









Bibliography 47

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

vii








viii

List of Acronyms


Contents

Contents ix

I Introduction 1









ix

x CONTENTS









Bibliography 47

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

viii

List of Acronyms


Contents

Contents ix

I Introduction 1









ix

x CONTENTS









Bibliography 47

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

Contents

Contents ix

I Introduction 1









ix

x CONTENTS









Bibliography 47

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

x CONTENTS









Bibliography 47

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

Part I

Introduction

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

Chapter 1

Introduction






3












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography












































Chapter 2

Disaster Robotics



21 Overview



11





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography





















































261 Control




262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography


262 Communication







264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography



264 Mapping


Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

Chapter 3









23






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography






































Z (s0) =N

sumi=1

φiZ (si) (31)














yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography






yi


(32)











(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography





(n3)






















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography


















Chapter 4

Conclusions








35









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography









41 Future Work




41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

41 FUTURE WORK 37





Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

Chapter 5

Summary of Papers

39






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography






























47

References













48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

48 BIBLIOGRAPHY












49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

49













50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

50 BIBLIOGRAPHY













51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

51












52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

52 BIBLIOGRAPHY











53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

53











54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

54 BIBLIOGRAPHY












55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

55













56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

56 BIBLIOGRAPHY












57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

57











58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

58 BIBLIOGRAPHY








Contents

Introduction

Introduction




Disaster Robotics

Overview






Control

Communication


Mapping












Conclusions

Future Work

Summary of Papers








Bibliography

enhancing geometric maps through environmental …1196889/...2016 ieee international symposium on...

Documents