spatial allocation in swarm robotics

Spatial Allocation in Swarm Robotics

Thesis submitted in partial fulfillment of the requirements for the degree ofMaster of Science in Engineering of Computing SystemsSchool of Industrial and Information Engineering (Ing. Ind-Inf)

Jacopo De Stefani - Student ID 782688

SupervisorProf. Andrea Bonarini

Co-supervisorProf. Mauro Birattari

LaboratoryAIR

Academic Year2013 - 2014

Jacopo De Stefani: Spatial Allocation in Swarm Robotics, Master thesisin Engineering of Computing Systems, c© March 2015

"My family is my strength and my weakness."

— Aishwarya Rai Bachchan

“A friend is one that knows you as you are, understands where you havebeen, accepts what you have become, and still, gently allows you to grow.”

— William Shakespeare

Grazie di tutto, Nonno.Dedicated to the loving memory of Mario Enrico De Stefani.

α 1931 – Ω 2013

We have seen that computer programming is an art,because it applies accumulated knowledge to the world,

because it requires skill and ingenuity, and especiallybecause it produces objects of beauty.

— Donald E. Knuth [Knuth, 1974]

A C K N O W L E D G E M E N T S

This thesis is the outcome of a long journey, started in Brussels, inSeptember 2012.

I would like to thank prof. Mauro Birattari, for giving me the possi-bility to stay at IRIDIA, and for the continuous and fruitful feedbackhe gave me through the preparation of the first version of this work.However, the completion of this manuscript wouldn’t have been pos-sible without the supervision of prof. Andrea Bonarini, which guidedme to the definition of the final shape of this document.

Thanks also to Giovanni, Carlo, Manuele and Giampiero, especiallyfor their help on technical matters.

Special thanks to all the people that accompanied me during theBrussels part of my journey.

To my colleagues, Nadine, Dimitri and Robin, who have welcomedme since the beginning, gave me helpful advices concerning the uni-versity and helped improving my French skills.

To all the people I met throughout these two years that supportedme and helped me grow as a person: Michele, Simona, Chiara, Laura,Thomas & Francesca among all.

A special mention for my "second family" in Brussels, Fabrizio,Giorgio, Laura, Eleonora, Blanca and Sinan, which is now spreadall across the world.

Thanks to Gianluca, Marco, Davide G., Davide & Anissa, Stefanoand all my friends, and teammates in Castelleone, who always mademe feel at home, no matter how much time I spent away and howmany times I ditched them to complete this work.

Ultimo ma non meno importante, un grazie di cuore ai miei geni-tori. Grazie per avermi dato la possibilità di partire e per non avermimai fatto mancare il vostro supporto e la vostra presenza, indipen-dentemente dalla distanza.

v

C O N T E N T S

1 introduction 1

2 background & related work 4

2.1 Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1.1 Self-organization . . . . . . . . . . . . . . . . . . 4

2.1.2 Swarm intelligence . . . . . . . . . . . . . . . . . 6

2.1.3 Swarm robotics . . . . . . . . . . . . . . . . . . . 9

2.2 Spatial allocation . . . . . . . . . . . . . . . . . . . . . . 18

2.2.1 Related work . . . . . . . . . . . . . . . . . . . . 19

2.3 Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.3.1 e-puck . . . . . . . . . . . . . . . . . . . . . . . . 33

2.3.2 IRIDIA Task Abstraction Module . . . . . . . . . 37

2.3.3 Autonomous Robot Go Swarming . . . . . . . . 39

3 methodology 43

3.1 Problem statement . . . . . . . . . . . . . . . . . . . . . 43

3.2 Experimental setup . . . . . . . . . . . . . . . . . . . . . 46

3.2.1 Formal modelling . . . . . . . . . . . . . . . . . . 47

3.2.2 Multi-robot task abstraction . . . . . . . . . . . . 50

3.2.3 Single-robot task abstraction . . . . . . . . . . . 51

3.2.4 Robot placement . . . . . . . . . . . . . . . . . . 52

3.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . 53

3.3.2 Naive . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.3.3 Probabilistic . . . . . . . . . . . . . . . . . . . . . 57

3.3.4 Informed . . . . . . . . . . . . . . . . . . . . . . . 59

3.3.5 Summary . . . . . . . . . . . . . . . . . . . . . . 61

4 results 63

4.1 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4.1.1 Uniform . . . . . . . . . . . . . . . . . . . . . . . 63

4.1.2 Biased . . . . . . . . . . . . . . . . . . . . . . . . 63

4.1.3 Corridor . . . . . . . . . . . . . . . . . . . . . . . 65

4.2 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

4.2.1 Allocated robots . . . . . . . . . . . . . . . . . . 66

4.2.2 Maximum error . . . . . . . . . . . . . . . . . . . 67

4.2.3 Allocation levels . . . . . . . . . . . . . . . . . . 67

4.2.4 Cluster views . . . . . . . . . . . . . . . . . . . . 68

4.3 Allocation uniformity . . . . . . . . . . . . . . . . . . . . 69

4.3.1 Maximum error . . . . . . . . . . . . . . . . . . . 70

4.3.2 Allocated robots . . . . . . . . . . . . . . . . . . 71

4.3.3 Allocation levels . . . . . . . . . . . . . . . . . . 72

vii

viii contents

4.4 Allocation speed . . . . . . . . . . . . . . . . . . . . . . 73

4.4.1 Scenario Uniform . . . . . . . . . . . . . . . . . . 74

4.4.2 Scenario Biased . . . . . . . . . . . . . . . . . . . 74

4.4.3 Scenario Corridor . . . . . . . . . . . . . . . . . . 75

4.5 Scenario difficulty . . . . . . . . . . . . . . . . . . . . . . 82

4.5.1 Scenario Uniform . . . . . . . . . . . . . . . . . . 82

4.5.2 Scenario Biased . . . . . . . . . . . . . . . . . . . 82

4.5.3 Scenario Corridor . . . . . . . . . . . . . . . . . . 83

5 conclusion & future work 87

5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 87

5.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . 89

a additional plots 90

a.1 Maximum error . . . . . . . . . . . . . . . . . . . . . . . 91

a.2 Allocated robots . . . . . . . . . . . . . . . . . . . . . . . 94

bibliography 97

L I S T O F F I G U R E S

Figure 2.1 Examples of self-organizing patterns emergingfrom morphogenesis . . . . . . . . . . . . . . . 5

Figure 2.2 Examples of swarm intelligence . . . . . . . . . 9

Figure 2.3 Overview of the typical development processin swarm engineering . . . . . . . . . . . . . . . 12

Figure 2.4 Overview of the main challenges in the devel-opment of a swarm robotics system . . . . . . 17

Figure 2.5 Overview of the approaches used in the sci-entific literature to tackle problems related tospatial allocation . . . . . . . . . . . . . . . . . . 19

Figure 2.6 Finite state machine representing a high leveldecomposition of the basic foraging task . . . 20

Figure 2.7 Overview of the approaches used in the scien-tific literature to tackle multi-robot task alloca-tion in a centralized fashion . . . . . . . . . . . 25

Figure 2.8 Overview of the approaches used in the scien-tific literature to tackle multi-robot task alloca-tion in a distributed fashion . . . . . . . . . . . 27

Figure 2.9 Annotated picture of the e-puck robot . . . . . 33

Figure 2.10 Sensor placement on the e-puck . . . . . . . . . 34

Figure 2.11 Picture of the TAM with an e-puck with the rangeand bearing board and the omni-directional cam-era extension . . . . . . . . . . . . . . . . . . . . 37

Figure 2.12 Architecture of ARGoS . . . . . . . . . . . . . . . 40

Figure 2.13 Simplified pseudo-code of the main simulationloop of ARGoS . . . . . . . . . . . . . . . . . . . 41

Figure 2.14 Screenshot of the ARGoS simulator for the sce-nario Biased environment configuration . . . . 42

Figure 3.1 UML Activity Diagram representing the ND SR-ST-IA approach to the multi-robot task alloca-tion problem . . . . . . . . . . . . . . . . . . . . 48

Figure 3.2 UML Activity Diagram representing the XD SR-MT-IA approach to the multi-robot task alloca-tion problem . . . . . . . . . . . . . . . . . . . . 49

Figure 3.3 Schematic representation of the TAM disposi-tion in a cluster . . . . . . . . . . . . . . . . . . 50

Figure 3.4 Finite state machine representing the TAM states 51

Figure 3.5 Finite state machine representing the high levelbehavior of a single agent . . . . . . . . . . . . 53

ix

x List of Figures

Figure 3.6 Deterministic finite state machine correspond-ing to the implemented individual robot con-troller for the Naive method . . . . . . . . . . . 55

Figure 3.7 Probabilistic finite state machine correspond-ing to the implemented individual robot con-troller for the Probabilistic method . . . . . . . . 57

Figure 3.8 Probabilistic finite state machine correspond-ing to the implemented individual robot con-troller for the Informed method . . . . . . . . . 59

Figure 3.9 Schematic representation of the Informed odometry-based cluster avoidance procedure . . . . . . . 61

Figure 4.1 Representation of the cluster disposition in sce-nario Uniform . . . . . . . . . . . . . . . . . . . 64

Figure 4.2 Representation of the cluster disposition in sce-nario Biased . . . . . . . . . . . . . . . . . . . . . 64

Figure 4.3 Representation of the cluster disposition in sce-nario Corridor . . . . . . . . . . . . . . . . . . . 65

Figure 4.4 Median values for the maximum error acrossclusters emax(t) and the number of allocatedrobots R(t) on the scenario Uniform . . . . . . . 76

Figure 4.5 Empirical cumulative density functions for theallocation levels to(l), (l ∈ 0.25, 0.50) distribu-tions of the three methods on the scenario Uni-form . . . . . . . . . . . . . . . . . . . . . . . . . 77

Figure 4.6 Median values for the maximum error acrossclusters emax(t) and the number of allocatedrobots R(t) on the scenario Biased . . . . . . . . 78

Figure 4.7 Empirical cumulative density functions for theallocation levels to(l), (l ∈ 0.25, 0.50) distribu-tions of the three methods on the scenario Biased 79

Figure 4.8 Median values for the maximum error acrossclusters emax(t) and the number of allocatedrobots R(t) on the scenario Corridor . . . . . . . 80

Figure 4.9 Empirical cumulative density functions for theallocation levels to(l), (l ∈ 0.25, 0.50) distribu-tions of the three methods on the scenario Cor-ridor . . . . . . . . . . . . . . . . . . . . . . . . . 81

Figure 4.10 Median value of vi(t) for i ∈ 1, · · · , 4 on sce-nario Uniform . . . . . . . . . . . . . . . . . . . 84

Figure 4.11 Median value of vi(t) for i ∈ 1, · · · , 4 on sce-nario Biased . . . . . . . . . . . . . . . . . . . . . 85

Figure 4.12 Median value of vi(t) for i ∈ 1, · · · , 4 on sce-nario Corridor . . . . . . . . . . . . . . . . . . . 86

Figure A.1 Comparison of the maximum allocation erroremax of the three methods on the scenario Uni-form . . . . . . . . . . . . . . . . . . . . . . . . . 91

List of Figures xi

Figure A.2 Comparison of the maximum allocation erroremax of the three methods on the scenario Biased 92

Figure A.3 Comparison of the maximum allocation erroremax of the three methods on the scenario Cor-ridor . . . . . . . . . . . . . . . . . . . . . . . . . 93

Figure A.4 Comparison of number of allocated robots R(t)of the three methods on the scenario Uniform . 94

Figure A.5 Comparison of number of allocated robots R(t)of the three methods on the scenario Biased . . 95

Figure A.6 Comparison of number of allocated robots R(t)of the three methods on the scenario Corridor . 96

L I S T O F TA B L E S

Table 2.1 Summary of the applications of swarm robotics 15

Table 2.2 Overview of the technical specifications of thee-puck robot . . . . . . . . . . . . . . . . . . . . . 36

Table 2.3 Overview of the technical specifications of theTAM device . . . . . . . . . . . . . . . . . . . . . 39

Table 3.1 Clusters requests in Uniform, Biased and Corri-dor experiment setups . . . . . . . . . . . . . . . 51

Table 3.2 Overview of the developed methods . . . . . . 62

Table 4.1 Summary of the parameters describing the sce-narios used in the experiment . . . . . . . . . . 66

Table 4.2 Summary of the values of the maximum allo-cation error emax(t) at t = 10000. . . . . . . . . 70

Table 4.3 Summary of the values of the number of allo-cated robots R(t) at t = 10000. . . . . . . . . . . 71

Table 4.4 Summary of the cumulative probabilities to reachallocation levels 0.25 and 0.5 across 50 trialswithin 10000 time steps . . . . . . . . . . . . . . 72

Table 4.5 Summary of the median values of cluster viewsv(t) at t = 10000 in scenario Uniform . . . . . . 84

Table 4.6 Summary of the median values of cluster viewsv(t) at t = 10000 in scenario Biased . . . . . . . 85

Table 4.7 Summary of the median values of cluster viewsv(t) at t = 10000 in scenario Corridor . . . . . . 86

xii

A C R O N Y M S

AI Artificial Intelligence

ARGoS Autonomous Robot Go Swarming

CMOS Complementary Metal–Oxide–Semiconductor

CPU Central Processing Unit

DCSP Distributed Constraint Satisfaction Problem

ER Evolutionary Robotics

FSA Finite State Automaton

IR Infrared

LED Light Emitting Diode

MAS Multi-Agent Systems

OAP Optimal Assignment Problem

RAB Range and Bearing

RL Reinforcement Learning

SI Swarm Intelligence

SO Self-organization

SR Swarm Robotics

TAM Task Abstraction Module

UML Unified Modeling Language

USAR Urban Search And Rescue

XML eXtensible Markup Language

xiii

A B S T R A C T

This thesis proposes three distributed methods to achieve the al-location of a homogeneous swarm of robots to spatially distributedcollective activities. Such tasks require the simultaneous presence ofmultiple robots to be successfully performed.

Several approaches to solve the problem have been proposed in thescientific literature, both centralized and decentralized (relying eitheron implicit or explicit coordination), but, to the best of our knowledge,none of these works has focused on an instance of the problem with ascarce number of agents (i.e. inferior to the number of available task),using robots and tasks both embodied.

The problem has first been formally analyzed and specified, usinga novel approach based on UML diagrams.

Then, the methods have been developed in the form of probabilis-tic finite state machines (PFSM), a microscopic level, behavior-basedapproach in swarm robotics.

The first method (Naive) simply consists of a greedy allocation ofthe robots to available tasks in the environment, as soon as they havebeen localized.

The second one (Probabilistic) improves the Naive one with proba-bilistic rules to avoid allocation conflicts and achieve a more uniformallocation.

The third (Informed) is built upon the Probabilistic one, using odom-etry to avoid the redundant exploration of already visited clusters.

The methods have been simulated on three different scenarios: Uni-form, Biased and Corridor. We characterize the methods according totheir allocation uniformity and allocation speed.

We show that there exists a trade-off between uniformity and speedfor the developed methods. We also find that the positioning of theclusters has a strong impact on the performances of the methods. Weconclude that the Informed method has the best performances on theproposed scenarios.

keywords : Swarm robotics, Task allocation, Decentralized con-trol, Coalition formation.

mots clés : Robotique en essaim, Répartition des tâches, Contrôledécentralisé, Formation des coalitions.

xv

A B S T R A C T

Lo scopo di questo studio é lo sviluppo di un metodo distribuitoper l’allocazione di uno sciame di robot omogenei ad un’insieme diattivitá collettive aventi una precisa distribuzione nello spazio. Tali at-tività richiedono la presenza simultanea di molteplici robot per poteressere eseguite con successo.

Nella letteratura scientifica sono stati proposti diversi approcci, siacentralizzati che decentralizzati (basati in parte sulla coordinazioneimplicita, in parte su quella esplicita), ma, sulla base delle infor-mazioni in nostro possesso, nessuno di questi lavori ha affrontatoun’istanza del problema in cui il numero degli agenti é scarso (ossiainferiore al numero delle attivitá disponibili), utilizzando sia robotche attività rappresentati concretamente.

Il primo passo del nostro lavoro é stata l’analisi e la produzione diuna specifica formale del problema, utilizzando un approccio innova-tivo basato sui diagrammi UML.

Successivamente, sono stati sviluppati tre metodi, sotto forma diautomi a stati finiti probabilistici (PFSM), un approccio utilizzatonella swarm robotics al fine di definire il comportamento dei singoliagenti (i.e. a livello microscopico).

Il primo metodo (Naive), consiste semplicemente nell’allocazionediretta dei robot alle attività disponibili nell’ambiente, non appenaesse vengano localizzate.

Il secondo (Probabilistic) migliora il metodo Naive introducendo al-cune regole probabilistiche al fine di evitare conflitti in fase di allo-cazione ed ottenere una distribuzione uniforme dei robot.

Il terzo (Informed) costituisce un’estensione del metodo Probabilis-tic, introducendo l’uso dell’odometria al fine di evitare l’esplorazioneridondante di cluster giá visitati.

Gli algoritmi sopraccitati sono stati simulati su tre scenari dallecaratteristiche differenti: Uniform, Biased and Corridor, al fine di carat-terizzare i metodi secondo la velocità nell’allocazione e l’uniformitànella distribuzione dei robot sui diversi cluster.

Attraverso gli esperimenti, siamo riusciti a dimostrare l’esistenzadi un trade-off tra l’uniformità della distribuzione e la velocità diallocazione dei diversi metodi. Abbiamo inoltre constatato che ilposizionamento dei cluster influenza in modo non trascurabile leprestazioni del metodo. Considerati tutti i risultati, possiamo con-cludere che il metodo Informed é in grado di ottenere le miglioriprestazioni su tutti gli scenari proposti.

parole chiave : Swarm robotics, Allocazione di attivitá, Con-trollo decentralizzato, Creazione di coalizioni.

xvi

1I N T R O D U C T I O N

In nature, many animals societies exhibit forms of collective behav-ior. Such kind of behavior emerges from interactions occurring atanimal level. One of the reasons for those interactions is that agentsare often faced with demanding activities, whose complexity is gen-erally beyond the capabilities of the individual. Hence, the only wayto perform such kind of tasks is to require the intervention of otherentities and join forces with them. When cooperation is triggered, theproblem of dividing the global duty into more, simpler componentsand assign (i.e. allocate) them to the different agents arises.

Indeed, task allocation is a common problem that can be observed indifferent collaborative societies. The way the allocation is performed,however, varies across different species.

The division oflabour process inhuman societies hasraised manyphilosophical andsociological issues,discussed by Plato,Adam Smith andKarl Marx, amongothers, whosediscussion is beyondthe scope of thisthesis.

In human societies, the most prominent example of this problemis the division of labour. This process consists of the decomposition awork activity into sub-tasks and their allocation to different individu-als, in order to profit of the skills and capabilities possessed by each ofthem. The allocation is generally performed in a centralized manner,with a single entity that assigns tasks and/or determines the degreeof specialization of each agent. The presence of such entity, with aglobal knowledge of the agents and the tasks, allows, in most cases,for a quick and coherent allocation. However, a similar approach hassome major drawbacks. First of all, the presence of a single unit withenhanced capabilities, introduces a single potential point of failure inthe system. In fact, the allocation, hence the possibility to work ofall the other agents depends on the "allocator". As soon as it will beunable to perform the allocation, the whole system will stop function-ing. Furthermore, in many contexts it is impossible or impractical togather a global knowledge of the system, hence limiting the effective-ness of the centralized solution.

Surprisingly, successful examples of task allocation and task parti-tioning can be found in the animal kingdom. For instance, eusocialorganisms (mostly insects, with some exceptions) are characterizedby a reproductive division of labour, a specialization among repro-ductive and non-reproductive activities (caring for other individu-als, nest building and defense, among others) that occurs at insectlevel. Moreover, some species of ants and bees are able to respondto changes in demand for particular activities by redistributing theavailable workforce. Here, task allocation is a product of the coop-eration or, in other words, a consequence of the self-organization ofthe individuals. Even though each agent is a relatively simple entity,

1

2 introduction

with limited sensing capabilities, the task allocation for the complexactivity can be achieved through processes and interactions locallyoccurring among them.

The study of collective behavior of decentralized, self-organizedsystems, natural or artificial, composed by a relevant number of ho-mogeneous organisms, is the main focus of swarm intelligence, a re-search field in Artificial Intelligence (AI). The goal of this field is toformulate models to explain the emergence of such behavior in or-der to be able to design effective, distributed algorithms for problemsolving. In this thesis, we apply this paradigm to the task allocationproblem. To be more precise, we focus on the problem of allocatingagents to spatially distributed tasks.

Our study will be performed in the framework of swarm robotics,hence the agents will be a group of simple and (quasi-) identicalrobots, with decentralized control and lack of synchronicity [Beni,2005]. Given their characteristics, we decided to adopt the e-puck[Mondada et al., 2009], a small mobile robot, as the robotic platformfor our experience. Tasks, on the other hand, are represented byIRIDIA Task Abstraction Modules, devices specifically designed toallow the e-puck to abstract tasks that are beyond its capabilities. Inour implementation of the problem, these devices are arranged toform a limited number of clusters in space, representing activities tobe tackled simultaneously by multiple robots.

Our purpose is to understand whether it is possible to achieve auniform allocation of the robots across the clusters by relying only onself-organization, without the need for any centralized solution.

In order to do so, we developed three methods. The first one (Naive)simply consists of a greedy allocation of the robots, as soon as thetasks are discovered, while the second one (Probabilistic) introducesa probabilistic mechanism to better distribute the robots across dif-ferent clusters. The third (Informed) is built by adding an odometry-based memory of the last visited cluster to the Probabilistic one.

The performance of these methods are compared on three differentscenarios: Uniform, Biased, Corridor, each one of them characterizedby a precise disposition in space of the clusters and the initial deploy-ment area of the robots.

thesis layout

The thesis is structured in 5 chapters.In Chapter 2 we start by presenting the reader the relevant notions

related to the Swarm Intelligence (SI) field. Then, we discuss theproblem of spatial allocation and we provide a summary of the state-of-the-art approaches to its solution (Section 2.2). We conclude thechapter by providing the readers some technical specifications con-cerning the hardware (e-puck, IRIDIA Task Abstraction Module) and

introduction 3

software (Autonomous Robot Go Swarming) components used in thedevelopment of the thesis.

In Chapter 3, we first provide a formal problem statement (Sec-tion 3.1), we describe our experimental setup (Section 3.2), and weillustrate the three developed methods, the Naive, Probabilistic andInformed one.

Chapter 4 is dedicated to a detailed analysis of the results and prop-erties of the methods. First, we present the scenarios (Section 4.1) thatwe used to test our methods and the metrics (Section 4.2) we mea-sured in our experiments. Then, we focus on two relevant propertiesof our methods: the uniformity of the speed of the allocation, analyzedrespectively in Sections 4.3 and 4.4. The chapter is concluded by anevaluation of the difficulty of the different scenarios (Section 4.5).

Finally, Chapter 5 concludes the thesis with a discussion on theobtained results and suggestions for future research directions to ex-plore.

2B A C K G R O U N D & R E L AT E D W O R K

The main purpose of this chapter is to present all the relevantknowledge required to understand the work described in Chapter3. In Section 2.1 we discuss the basic concepts upon which the thesishas been developed with a top-down approach. We begin by givinga quick overview of the self-organization framework (Section 2.1.1).Then, we proceed by illustrating the swarm intelligence field (Sec-tion 2.1.2), focusing especially on the presentation of swarm robotics(Section 2.1.3).

Once the context have been clarified, we state the objective of ourwork, the motivation behind it, its specific literature and related workin Section 2.2, highlighting the different approaches to the problemwe are interested in.

In Section 2.3, we conclude by giving a brief overview on the hard-ware and software components used in the development of the thesis.

2.1 concepts

2.1.1 Self-organization

A first, general notion that is required to fully understand the pro-posed methods is self-organization. Self-organization occurs natu-rally in a variety of biological, chemical, and social systems. Due tothe heterogeneity and strong diversification among these categories,it is difficult to define precisely. For our scope, we borrow a workingdefinition from [Camazine et al., 2001]:

"Self-organization is a process in which a pattern at the globallevel of a system emerges solely from numerous interactionsamong the lower-level components of the system.Moreover, the rules specifying interactions among the system’scomponents are executed using only local information, withoutreference to the global pattern."

A pattern can be seen as an organized disposition of the system com-ponents in space (or time).

Many processes in chemistry, for instance, are examples of howinteractions at a microscopic level have a great influence on the finalproperties of a material at the macroscopic level.

One of them, crystallization, is the process of forming solid crys-tals by precipitation from a (generally) liquid solution. It takes placewhen a solute (chemical compound) is mixed with a solvent, heatedat high temperature. This causes the molecules of the compound to

4

2.1 concepts 5

disaggregate and start floating around. By gradually reducing thetemperature of the compound, the free solute molecules, which can-not be held anymore in the solvent, start to aggregate. Not all the clus-ters of molecules become crystals, but only those who have reacheda critical size (determined by the operating conditions of the process,namely temperature and pressure). The remaining molecules thenjoin the "surviving" aggregates, arranging themselves in a defined pe-riodical manner that gives the crystal its characteristic structure. Itis indeed an example of self-assembling process, analogous to the pro-cess that binds the DNA molecules to form the characteristic doublehelical structure.

Without restraining ourselves to chemistry, similar activities canbe observed in biology as well. Notable illustrations of the self-organizing phenomena are, indeed, morphogenesis, protein foldingand homeostasis. Morphogenesis is a term used to identify the processthrough which a biological organism develops its final shape. Manyparticular patterns in nature, such as pigmentation on some speciesof shells (Figure 2.1a) and coat patterns in giraffes and zebras (Figure2.1b) are examples of morphogenesis products.

(a) Pigmentation pattern on a coneshell. (Courtesy of Stephan Wol-fram)

(b) Pigmentation patter of a zebra coat.(Courtesy of The Technium)

Figure 2.1: Examples of self-organizing patterns emerging from morphogen-esis.

Alan Turing explained the occurrence of similar structures by meansof the diffusion in the cells of chemical substances (morphogens) (cf .[Turing, 1952]). By modeling this phenomenon with systems of differ-ential equations, he discovered that a particular pattern is the resultof the interaction between different types of morphogenes, some pro-moting and other preventing cell growth. On the other hand, home-

"Positive feedbackisn’t alwaysnegative"–M.Resnick,Learning aboutLife

ostasis is the property of system to self-regulate its internal environ-ment to maintain a stable, relatively constant state. This process, firststudied by Bernard in the 18th Century (cf. [Cooper, 2008]), has beenexplained in the framework of dynamical systems (feedback cycles).

Even though, at a first glance, the aforementioned processes mayseem really different among them, they can all be explained throughthe combination of positive feedback, that iteratively amplifies the re-

http://www.stephenwolfram.com/publications/articles/ca/83-cellular/2/text.html

http://www.stephenwolfram.com/publications/articles/ca/83-cellular/2/text.html

http://www.kk.org/thetechnium/archives/2009/08/ratcheting_up_a.php

6 background & related work

sponse of the system and negative feedback, which compensates bylimiting it, leading the system towards stable equilibrium states.

From a different point of view, another relevant definition of self-organization has been given by W. Ashby in [Ashby, 1962]:

"[...] the system that starts with its parts separate (so that thebehavior of each is independent of the others’ states) and whoseparts then act so that they change towards forming connectionsof some type."

Examples of self-organizing systems in this sense, are human neu-ral networks, whose connections and capabilities evolve across timethanks to the interconnection with similar cells. It should be notedthe latter definition does not address the question of the usefulness ofthe emerging organization and that both the definitions focus on theinteraction (and consequent cooperation) among the parts of the sys-tem. In both cases, no constraints are imposed on the basic buildingblocks of the system, which could be either living or inanimate enti-ties. Moreover, the definitions do not specify whether information isexchanged among the agents during their interaction and how suchknowledge is used in the process.

The research in swarm intelligence, on the other hand, is focusedon the study of self-organizing behaviors of a precise typology ofentity: agents (i.e. entities capable of observing and modifying theirenvironment by means of, respectively, sensors and actuators).

2.1.2 Swarm intelligence

What is, then, an intelligent swarm? We can find a good answer in[Beni, 2005]:

"[...] a group of “machines” capable of forming “ordered”material patterns “unpredictably”."

This definition allows us to immediately clarify the most impor-tant aspects concerning this discipline. First of all, the fact of dealingwith systems composed by multiple individuals which are able to orga-nize themselves (i.e. forming a material pattern, in time and space)autonomously.

Here, the concept of intelligence of the swarm is expressed by com-bining the ability to create order with the unpredictability of the process.The ability to create order is related to the capability of the group of rec-ognizing, analzying and exploiting patterns (as in swarm optimiza-tion techniques such as Particle Swarm Optimization by Eberhart andKennedy [1995] and Ant Colony Optimization by Dorigo et al. [1996])or to synthesize both physical (e.g. buildings) and abstract (e.g. orga-nized behaviors) ordered structures (as in swarm robotics, see Section2.1.3). The unpredictability is expressed as the fact that an intelligentswarm can be seen as a parallel computational system which is as

2.1 concepts 7

powerful as any other universal computational model (e.g. TuringMachine, Von Neumann Architecture). As a consequence, there is nomodel able to predict the outcome of the given swarm in a reasonableamount of time.

Moreover, the notion of machine (as "[...] an entity capable of pro-cessing matter/energy", from [Beni, 2005]) does not define the nature ofthe entity itself which could be either a living creature (natural swarmintelligence) or a human artifact (artificial swarm intelligence).

Natural swarm intelligence deals with the study of the collective be-haviors emerging from the interactions of biological organisms (gen-erally animals or insects). The most fascinating examples in this cate-gory, under a visual point of view, are surely the flocking (schooling)behavior that can be observed in groups of birds (fish). While dis-playing a similar behavior (Figures 2.2a, 2.2b), the group of animalsmoves as if it were a single, fluid entity. Furthermore, even thoughrapid changes of direction are triggered as a reaction to events inthe environment (e.g. presence of a predator/obstacle), accidentalcollisions among the individuals are rare. Surprisingly, local sensingcapability and a limited set of rules (three, to be precise, cf. [Reynolds,1987]) describing the inter-agent interactions are sufficient to achievethis complex global behavior.

Insects are an interesting example of how organization can be achie-ved when the number of individuals scales up from tens to thousandsof units in the group. For instance, ants of the Pheidole genus are di-vided two distinct categories of individuals: minors and majors. In-sects from the former category carry out the majority of the tasks re-quired in the colony (grooming, brood care, foraging) while those inthe latter focus on nest defence and seed milling. As shown by Wilson[1984], when the ratio minors-majors drops below a certain threshold,the repartition of the activities among the agents is modified in a waythat majors compensate for the lack of minors by performing theirtasks. If the ratio is increased again, the ants recover the original taskallocation. An explanation of this adaptive unsupervised division oflabor, has been given several years later with the response-thresholdmodel (cf. [Theraulaz et al., 1998]), based only on the features of eachagent.

Bees, on the other hand, show us how the global structure of theirhive can emerge from the independent, but self-organized work ofa large number of individuals (Figure 2.2d). A honey bee colonypresents in fact a characteristic pattern, made by three concentriczones containing respectively brood, pollen and honey (from the cen-ter to the exterior area). One of the first hypothesis that has been for-mulated to describe this phenomenon was that bees possessed someinnate knowledge about how to arrange cells (e.g. blueprint or pat-tern). What Camazine [1991] has shown, is that brood is initiallydeposed in a compact way by the queen, while pollen and honey


are dispersed randomly in the structure or in cells where the samesubstance is already present. The structure then emerges from a sim-ple nourishment 1 displacement behavior performed by all the agents,based on brood quantity in the neighborhood, along with the differ-ent rates at which new substances are brought in the colony.

Evidence of the emergence of a collective intelligence through sim-ple interactions in a large group of individuals have been observedin humans as well. In [Krause et al., 2010] we find the application ofthis principle to different everyday situations such as the guess of theexact quantity of marbles in a jar, the management of a football team,the design of a new product. The main benefit of a similar coopera-tion in humans is the collective information processing leading to anew cognitive solution to a problem (i.e. the wisdom of the crowds).

Why are these behaviors widely observed in nature? Why there isan interest in their study from an engineering standpoint? An answercan be found in the characteristics that these systems possess.

First of all, they are completely distributed, in the sense that thereare no leading individuals in the group that impose a certain be-havior, neither locally nor globally. Furthermore, all the agents arerelatively homogeneous (i.e., they present only limited variability interms of morphology or capabilities) with only local sensing capabil-ities (that is, they could only have insights on what is happening intheir surroundings, but not at a global level). The combination ofthese features implies that no individual is essential to the survivalof the group, thus ensuring the ability to cope with the loss of indi-viduals (due to death or temporary/permanent inability), hence therobustness of the system. Moreover, through simple behavioral rulesconcerning the interaction with other agents and basic informationtransfer, both direct or indirect (e.g. stigmergy), a collective, intel-ligent organization emerges. This locality allows the system to beresilient with respect to both the fluctuations in the environment andin the swarm. As we may notice, all the aforementioned featuresbring a set of advantageous properties to the system.

The research in swarm intelligence is mainly twofold. On one hand,the aim is to understand the mechanisms underlying such kind of sys-tems and provide a model which is able to explain the emergence ofa collective behavior (scientific swarm intelligence). Since swarm intel-ligence is a bio-inspired discipline, the largest part of this research isconducted by life scientists. On the other hand, the focus is on how toapply the discovered knowledge to solve problems having a practicalrelevance (engineering swarm intelligence). In this case, researchers incomputer science, mostly in artificial intelligence and robotics, are us-ing the knowledge gathered in other fields to create algorithms ableto solve optimization problems, to perform realistic simulations ofthousands of agents or develop efficient robotic systems. The rest of

1. Pollen and honey.

2.1 concepts 9

(a) Flock of auklets (seabirds). (Cour-tesy of D. Dibenski)

(b) School of fishes swimming togetherin a spherical formation.

(c) Major and minor ants of the Phei-dole genus taking care of the colonybrood. (Courtesy of AlexanderWild)

(d) Bees deposing honey in theircolony. (Courtesy of KamilloKluth)

(e) Exemplification of the "Wisdom ofcrowds" process.

Figure 2.2: Examples of swarm intelligence.

this thesis will be concerning artificial swarm intelligence, the branchof swarm intelligence related to the study of the outcomes of self-organization in human artifacts. To be more precise, we will be focus-ing on a precise category of artifacts: physically embodied agents orrobots, as they are commonly called.

2.1.3 Swarm robotics

In the early nineties, while life scientists [Camazine, 1991] were ob-serving and modeling the behavior of social insects, the first examples

http://www.alexanderwild.com/

http://www.alexanderwild.com/

https://secure.flickr.com/photos/27330306@N08

https://secure.flickr.com/photos/27330306@N08


of multi-agent systems were being developed by computer scientists(cf. [Singh, 1991]).

These artificial systems are composed by a group of autonomousentities interacting among them and with an environment, which canbe either physical (e.g robots in a real environment) or virtual (soft-ware agents, for example). The nature (physical/virtual, human/arti-ficial) and the complexity of the agents (active/passive, cognitive/re-flexive) could be highly variable, yielding to heterogeneous sets ofagents. Over the years, similar systems have been mainly used toperform distributed problem-solving (cf. [Colorni et al., 1991]) ormulti-agent simulation of real world processes (cf. [Benenson, 1998]).

In the years 2000, the biological and artificial research paths havejoined in the Swarm Robotics field.

A more precise definition can be found in [Sahin, 2005]:

"Swarm robotics is the study of how large number of rela-tively simple physically embodied agents can be designed suchthat a desired collective behavior emerges from the local inter-actions among agents and between the agents and the environ-ment."

A physically embodied agent is an entity whose behavior is af-fected by its morphological features and the environment it is situ-ated in (cf. [Pfeifer et al., 2007]). In order for the agent to be embod-ied, it must be able to transfer and process matter, energy (through itsactuators) and information (by means of its sensors and its internalarchitecture), hence it must be, according to [Beni, 2005], a robot.

As a consequence, we can observe from the definition that swarmrobotics is a branch of collective robotics where a global behavioris achieved through the self-organization of the individuals. The au-tonomous agents are relatively simple in the sense that their equip-ment consists of a minimal set of sensors and actuators, that preventseach agent from having a global knowledge of the system and the envi-ronment.

It is important to clarify that, while the definition deals with a largenumber of robots (from hundred to thousands of robots), the empiricalexperiments in swarm robotics are performed with groups of robotswhose size does not exceed a hundred of units (while often beingjust a few tens). This discrepancy is mainly due to practical andeconomical reasons. In fact, even though the technological evolutionallowed to reduce the costs of the microelectronic components neededto build a robot, a single assembled agent still remains expensive(ranging from tens to several thousands euros). Furthermore, mostof the robots used in the experiments are battery-powered, whichconstrains the autonomy of the robots to that of their power supply.

Despite of these limitations, why there is still an interest in swarmrobotics? We can answer the question by analyzing how self-organiza-tion can solve some common issues in collective robotics. For in-

2.1 concepts 11

stance, in many application of collective robotics robot coordinationis achieved through a shared, centralized communication medium,which is accessed concurrently by all the agents. A similar solutionhas some major drawbacks: if the medium is unavailable or unreach-able by the agents, the system cannot operate correctly. Moreover, thequality of the communication decays as the number of robots in thesystem increases, due to concurrency. Conversely, when coordinationis achieved through local interactions of the robots, the robustness ofthe system with respect to faults and disturbances can be achieved.The points of failure will then become distributed among the agents,lowering their impact on the overall performance of the system. Also,if the agents are homogeneous and relatively simple, the swarm be-comes even more dependable by means of redundancy. In addition,the lack of a central communication mechanism favors the scalabilityof the system with respect to changes in the number of individu-als. Another advantage of self-organization is that robots could op-erate autonomously, hence performing parallel activities which aredistributed in space and in time, improving the global efficiency ofthe system.

It is important to stress the fact that the aforementioned propertiesare not guaranteed as a consequence of self-organization; rather theyshould be achieved through a specific design of the system.

2.1.3.1 Development methodology

Indeed, the main research question in swarm robotics is how to ex-ploit self-organization in order to develop multi-robot systems robustto failures, scalable and resilient, that could be employed in activitiesrequiring an high degree of parallelism (cf. [Dorigo et al., 2014]).

From an engineering standpoint, it would be interesting to elab-orate a formal development methodology to achieve the aforemen-tioned properties. However, the main obstacle to such frameworkis represented by the emergent nature of the collective behavior ofthe system. In fact, the global behavior stems from the presence ofnon-linear interactions at the individual level. This lack of linearitysystem makes the establishment of a relationship between the localand global dynamics of the system difficult. As a matter of fact, de-spite the research activity performed in the last decade, a commonlyaccepted theoretical framework for swarm systems still remains unar-ticulated (cf. [Schweitzer, 2003]).

Nevertheless, researchers in swarm robotics are studying how toperform the systematic application of scientific and technical knowl-edge in a structured way in the development process, i.e. swarm en-gineering (cf. [Brambilla et al., 2013]). According to Brambilla [2014]:

"Swarm engineering is still in a very early stage and itsdevelopment is not homogeneous. On the one hand, some top-ics, such as design and analysis, have already received atten-


tion from the swarm robotics community and several approacheshave been proposed. [...] On the other hand, other topics, such asrequirements analysis, maintenance and performance measure-ment, have received almost no attention."

Requirement specification

Design

Implementation

Test and Analysis

Validation

Operation and maintenance

ManualAutomatic

Microscopic levelMacroscopic level

SimulationReal robots

Figure 2.3: Overview of the typical development process in swarm engineer-ing. Feedback loops are omitted for clarity.The elements having a dashed border are not generally per-formed in the context of swarm robotics.

An overview of the state-of-the-art development process in swarmengineering is depicted in Figure 2.3. It should be noted that, to thebest of our knowledge, at the present time (March 2015), the appli-cations of swarm robotics are confined to academic research, with-out real-world uses. As a consequence, rigorous methodologies forrequirements elicitation and formal specification languages are still aresearch subject.

The design methods can be classified into manual and automatic. Themanual ones are processes that consist in developing by hand a speci-fication of the agents’ individual behavior which will cause the emer-gence of a global behavior at the swarm level. These approaches canbe further divided into top-down and bottom-up. In the top-down ap-The interested reader

could find all thedetails and relevant

articles describingtheir application in

[Brambilla et al.,2013]

proach, the developer starts by designing the collective behavior ofthe swarm, and derives from it the individual behavior of the agents.However, due the lack of a framework for requirements specificationand the difficulty to relate microscopic and macroscopic models forthe swarm because of the non-linear dynamic of the system, similarapproaches have been proven successful only for specific problems(cf. [Brambilla, 2014]). The bottom-up approach, on the other hand,

2.1 concepts 13

is a trial and error process at the microscopic level, guided by theexpertise and the ingenuity of the developer. In fact, it consists ofan iterative refinement of the individual behavior until the emergentbehavior of the system satisfies the desired properties. According toBrambilla et al. [2013] the most used methods in this category are thefinite state machine and virtual physics one. The finite state machine oneis based on the formalization of the robot behavior in the form of aFinite State Automaton (FSA) whose states correspond to the robots’actions and whose transitions are triggered by the inputs from thesensors and by the robot current state. Virtual physics (cf. [Spearset al., 2004]) treats each agent as a particle subject to virtual forces.Those forces are the result of the interaction of the robot with a vir-tual potential field, that can be perceived through its sensors.

As for automatic methods, the idea is to apply artificial intelligencetechniques to the design process of the robot, without relying onhuman-based development. The most frequently used techniques areReinforcement Learning (RL) and Evolutionary Robotics (ER).

With Reinforcement Learning (cf. [Sutton and Barto, 1998]) theagent learns a desired individual behavior by means of iterated inter-actions with the environment, receiving a feedback on its actions. Thepolarity (positive or negative) and the intensity of the feedback, deter-mine whether the agent should retain or forget a certain componentof the behavior.

In Evolutionary Robotics, evolutionary computation techniques (cf.[Goldberg and Holland, 1988]) are applied to the swarm of robots.The individual behavior of the robots (identical for all the agents) isthe basic component that is evolved through different iterations of themethod. At the beginning, this behavior is generated randomly andthe collective behavior produced by the simultaneous execution of iton all the robots is evaluated by means of a fitness function. At eachiteration, the individual behaviors are modified by means of selection,recombination and mutation operations. The process is stopped oncethe desired collective behavior has been achieved.

After the design phase has been completed, the emerging behaviormust be thoroughly analyzed in order to assess whether the desiredproperties hold or not, before actually validating it with tests on thereal robots. The analysis is performed mainly by means of simulationsand mathematical models.

Simulations offer a swarm representation which is as close as pos-sible to the reality, by modeling each robot component at a micro-scopic level. Unfortunately, this accuracy comes at the price of com-putational complexity, which dramatically increases as the number ofagents becomes larger, making technically unfeasible the simulationof numerous swarms.

On the other hand, mathematical models based on the theory of dy-namical systems (e.g. rate, differential, Langevin and Fokker-Plank


equations) or on the control and stability theory can be used to de-scribe the system at a macroscopic level. They can be studied regard-less of the group size, but their extent is often restricted to simplifiedrepresentations of the systems to study, limiting their practical utility.

The final step of this engineering process is the execution of theindividual behavior by a swarm of real agents. Surprisingly, as Bram-billa [2014] shows, the validation on real robots is performed on lessthan half of the papers that the authors have reviewed.

The validation through simulation of the results presents someclear advantages: it prevents damages to the robot in case of a behav-ior fault, it can be easily parallelized thus executed faster and it doesnot require the actual deployment and maintenance of the robots.

Nevertheless, the simulation does not test whether the system isable to bridge the "reality gap". In the study by Jakobi et al. [1995],the authors proved that, whenever the noise on sensors and actua-tors was not properly modeled in the simulations, the behavior of therobot displayed relevant differences between the simulated and realenvironment. It should also be noted that, in all the reviewed papersthat performed tests on real robots, the simulation was made in envi-ronments whose settings (e.g. environmental noise in terms of light,radio transmission, sounds) could be controlled by the experimenter.The main advantage of this method is the possibility to validate therobustness of the developed method with respect to realistic noisemodels on sensors and actuators. However, it does not give any guar-antees on the feasibility of the methods and on the validity of theunderlying assumptions in a real world scenario, thus preventing aneffective application and operation of a swarm-based system.

2.1.3.2 Applications of swarm robotics

In addition to the development complications, as we discussedin Section 2.1.3, the affordability and the maintenance issues of aswarm of real robots are the main problems that hinder the creationof swarm-based solutions that can operate in everyday life.

Nevertheless, the research in swarm robotics has provided valuableinsights on the functioning of self-organized biological systems andproofs-of-concept regarding how agent cooperation can successfullytackle complex problems. Hence, according to the taxonomy definedby Dorigo and Birattari [2007], these results can be classified as artifi-cial swarm intelligence with both scientific and engineering purposes.

Under a scientific point of view, artificial systems are used to un-derstand the mechanisms that allow a biological system to behave asa single coordinated unit as a result of local interactions among in-dividuals and with the environment, to model them and to validatesuch model through artificial embodied agents.

In this field, the most fruitful applications are related to spatially-organizing behaviors, i.e. collective behaviors that cause the emergence

2.1 concepts 15

scientific engineering

— Aggregation— Coordinated motion— Collective transport— Pattern formation— Morphogenesis

— Area coverage— Collective exploration— Collective transport— Consensus achievement— Fault detection— Group size regulation— Human-swarm interaction— Object clustering— Self-assembled motion— Task allocation

Table 2.1: Summary of the applications of swarm robotics. The task typolo-gies are those presented in Brambilla et al. [2013].

of patterns with a precise connotation in space. For instance, Garnieret al. [2005] have successfully designed a robot controller that causesthe formation of group of agents in defined regions of the space (i.e.their aggregation), replicating the behavior of cockroaches. Also, thedeployment of robots with a regular and repetitive structure in space(i.e. the formation of a pattern), similar to the crystals’ morphology,have been extensively studied, both from a theoretical [Flocchini et al.,2008] and an applicative [Varghese and McKee, 2009] standpoint.

Moreover, the behavior of ants has been a successful source of in-spiration for studies concerning morphogenesis, i.e. the capability ofthe entities to physically connect to each other in order to form an ap-propriate configuration to tackle a given task. The work by Mondadaet al. [2005], for example, shows how self-assembled robots couldovercome a ditch which is too large for a single robot to overcome,in the same manner as ants connects their bodies together to formbridges to move to otherwise unreachable destinations (cf. [Ander-son et al., 2002]). In a similar fashion, the chaining behavior of Argen-tine ants have been modeled and implemented on artificial agents,without physical connection among the agents (cf. [Nouyan et al.,2008]).

Other relevant applications have been found through the analysisof the coordinated motion of social animals. The most visually appeal-ing of such behaviors is flocking, depicted in Figures 2.2a and 2.2b,which has been successfully achieved with a group of robots with-out explicit communication (cf. [Turgut et al., 2008]), by simply im-plementing the simple rules devised by Reynolds [1987] . Paytonet al. [2001] studied the use of stigmergy (i.e. local, indirect, nonsymbolic form of communication mediated by the environment) toperform swarm-based navigation using robots as veichles for virtual


pheromones, sensed by other robots and exploited to reach a giventarget, in the same fashion as Argentine ants (cf. [Deneubourg et al.,1990]). Last but not least, the analysis of the technique employedby Aphaenogaster cockerelli ants to perform collective transport of theretrieved food without prior knowledge about the payload has beenperformed by Berman et al. [2011], yielding to the development of amathematical model which could be ported on robots.

Studies in swarm robotics have not only proven to be useful in ex-plaining the underlying mechanisms that trigger the emergence ofswarm intelligence in animals and insects, but also in developing so-lutions to problems having a practical relevance (i.e. from a engineer-ing standpoint). It should be clarified that the two branches are notcompletely disjoint from one another, but rather the knowledge gath-ered through the scientific studies is exploited and refined throughthe engineering applications.

Here, the most interesting results have been obtained while dealingwith environment exploration, collective transport, object clustering, swarmawareness and decision making problems.

Within the exploration framework, solutions have been found by de-ploying robots in a way that they maximize the area coverage of thegiven environment, sometimes by having a connectivity constraintamong the agents (cf. [Howard et al., 2002]) or by using an heteroge-neous swarm, in order to take advantage of the different characteris-tics of the individuals (cf. [Ducatelle et al., 2011]).

In addition, the studies on coordinated motion allowed to devise amethod based uniquely on local sensing to let the robot move in aspecified formation and performing obstacle avoidance while main-taining the chosen pattern (cf. [Balch and Hybinette, 2000]).

Collective object transport has been successfully tackled by a groupof robots, simply by relying on the force, position and orientationsensing of each agent (cf. [Donald et al., 1997]).

Concerning object clustering, Werfel et al. [2011] have developed athree-dimensional multi-robot construction system that employs sim-ple agents with local sensing capabilities. Through the stacking ofsquared, specifically designed construction blocks, the robots are ableto build both articulate structures and the scaffolds needed to furtherextend them.

Some methods have also been developed to make the swarm awareof its current state, for example, by being able to detect the presenceof faulty robots (cf. [Christensen et al., 2009]) or estimate and regulateits group size (cf. [Brambilla et al., 2009]).

Moreover, the problem of having the swarm reach a consensus con-cerning a choice (e.g. moving direction, aggregation point) that willaffect the performance of the whole group, have been solved usingboth direct and indirect communication (cf. [Gutiérrez et al., 2010]and [Garnier et al., 2005], respectively).

2.1 concepts 17

A recent trend in swarm robotics is concerned with human-swarminteraction. As a matter of fact, even though the swarm is designedto be autonomous by construction, it might be useful to take controlover the swarm (e.g. to avoid situations where the activity of theswarm might harm some humans, on even the robots themselves)or to receive some feedback regarding the state of the swarm. Forinstance, the work by Giusti et al. [2012] illustrates how it is possibleto control a swarm through gestures despite a noisy perception bythe agents, by achieving a consensus through a distributed votingprocess.

Last but not least, the problem of allocating robots to different tasks(foraging against resting) in order to maximize the throughput of thesystem have been successfully addressed with an individual decisionmechanism (cf. [Krieger and Billeter, 2000]).

Challenges inSwarm Robotics

Hybrid cen-tralized/de-centralized

controlscheme

Abstractlevel

supervision

Scalabilityand

RobustnessEnergy

efficient tasksupportiveformations

Distributedpath

planningand obstacle

avoidance

Objecttransportand ma-

nipulationMapping

andlocalization

Energydepletion

Figure 2.4: Overview of the main challenges in the development of a swarmrobotics system. The issues are illustrated in [Barca and Seker-cioglu, 2013]. The color of the node represents the degree of inter-est that the corresponding challenge has received in the literature(from darker: more interesting, to brighter: less interesting).

We are aware that the overview we presented might be incomplete,but its main purpose is to illustrate the most prominent results ob-tained in swarm robotics. For a more detalied review, analyzing morethan 140 articles across 30 years, we refer the interested reader to


[Barca and Sekercioglu, 2013]. As summarized in Figure 2.4, the maincontribution of the aforementioned researchers is a snapshot of thestate-of-the-art concerning swarm robotics, with a particular focus onthe still unresolved issues that hinder the realization of a real-worldswarm robotics system. The authors further investigated which arti-cles tackled a particular issue and discovered that some issues (e.g.which control scheme to apply for a certain problem), received muchgreater attention than others (such as the influence of energy deple-tion on the methods’ performance), even though the impact of suchissues on the feasibility of an application in a real environment is com-parable. Moreover, to the best of the authors’ knowledge, no studyhas addressed all the challenges simultaneously, confirming the needfor further research before producing a practical swarm robotics sys-tem.

2.2 spatial allocation

In this thesis, we investigate the problem of allocating embodiedagents to physical tasks distributed in space. A brief example mighthelp to put such abstract problem statement in an applicative context.

Consider the following situation: A natural catastrophe (e.g. earth-quake, hurricane, tsunami) has severely damaged public facilities asschools, offices, hospitals. As a consequence, some people have po-tentially been dispersed while there might be wounded persons aswell.

Indeed, this is a typical Urban Search And Rescue (USAR) scenario:the locations of some subjects in an hazardous environment is un-known and assistance should be provided to them in the shortestpossible time. The cluttered nature of the environment generally hin-ders a global exchange of information among the rescuers, making acentralized coordination of the process infeasible. Moreover, the infor-mation concerning the topology of the environment and the knowl-edge of the number of people in distress is generally unavailable orunreliable.

In a similar scenario, there are several reasons to prefer a swarmof artificial agents to a group of human rescuers. According to Liuand Nejat [2013], robots could reduce the personal risk of human andcanine agents by substituting them while entering unstable structuresor inaccessible voids, avoiding the occurrence of physical injuries (e.g.broken bones, respiratory issues due to hazardous and toxic materi-als). Furthermore, the performance of the agents will not be affectedby psychological factors as stress and fatigue, with the only limits be-ing their power supply and their actuators capabilities. Lastly, robotsare expendables: when damaged, they can be repaired, or, in theworst case, substituted. In addition, once replaced they only need to

2.2 spatial allocation 19

be properly programmed, without requiring the same training effort(time and costs) than their human counterparts.

Given these properties of the agents and the aforementioned fea-tures of the environment, a promising solution for the problem wouldinvolve the self-coordination of robots during the rescue process.

With this in mind, how is it possible to evaluate the quality of theallocation provided by such method? The optimal allocation, in a sim-ilar scenario, is the assignment of robots to tasks that allows to rescueall the people in the shortest possible time. Indeed, the fundamen-tal aspects that characterize the performance of the method are theuniformity of the distribution of agents across the rescue activities inthe environment and the time needed to discover and allocate to suchtasks. While this informal definition is provided here to complete thecontextualization our problem, a more rigorous and thorough analy-sis of these metrics will be provided in Chapter 3.

2.2.1 Related work

Spatial Allocation

Multi-robot taskallocation

Distri-buted

DPS

MAS

Centra-lized

Optimiza-tion

Coalitiontheory

Collectiveforaging

Figure 2.5: Overview of the approaches used in the scientific literature totackle problems related to spatial allocation. In the following,we will mainly focus on a branch of multi-robot task allocation:multi-robot coalition formation. Concerning this problem, theinformation in this overview is adapted from [Vig, 2006].

2.2.1.1 Collective foraging

To get back to the point, the collective behavior that should emergefrom the robot’s coordination is in between the task allocation (coalitionformation, to be more precise) and the foraging activity (cf. Figure 2.5).


On one hand, foraging is the process of collectively searching forresources scattered in the environment, harvesting them and deposit-ing them at predefined collection points. It is a well observed phe-nomenon in social insects, where the collected resources are bothbuilding blocks for the colony and food to be stored in the colony’sreserves. Due to its complex structure, involving four different sub-tasks (cf. Figure 2.6) and the cooperation required to successfullytackle it, it has become a benchmark problem for multi-robot systems.

Environmentexploration

(Task localization)start

Physicalcollection

(Task allocation)Deposition

Navigation

Object (Task) found

Object grabbedCollection

point reached

Object deposited

Figure 2.6: Finite state machine representing a high level decomposition ofthe basic foraging task. The dashed elements are not studiedwithin the scope of this thesis. (Adapted from [Winfield, 2009]).

The first sub-task consists of the localization of the target resourcein space. Due to their limited perception capabilities, the robots needto adopt a search strategy (e.g., uninformed random walk, system-atic horizontal/vertical sweeping) in order to reach their objective.Once the object has been found, the agents have to physically collectit. Then, they must apply a navigation strategy, eventually employingthe information gathered in the localization phase, to head back tothe collection point. Finally, the robots deposit the gathered object andrestart the process. A detailed analysis of the process (including ataxonomy) can be found in [Winfield, 2009].

In our vision of the problem, the only common subtasks with theforaging activity are the localization and the collection one. To be moreprecise, we consider the tasks as resources scattered across the envi-ronment, outside the sensing range of the agents. To be successfully


performed, such tasks need to be discovered first, and then allocatedto the robots. Once the allocation has been done, the robots do notmove from the location where the task has been found any more (i.e.no reallocation is performed). As a consequence, the navigation anddeposit operations (cf. Figure 2.6) are outside of the scope of our meth-ods.

To solve the task localization problem, we could potentially employtechniques for collective environment exploration. For instance, a coordi-nated motion (cf. [Balch and Hybinette, 2000]) of the swarm throughthe environment allows for a systematic discovery and a shared knowl-edge of the environment by every robot in the group. Alternatively,area coverage could be achieved using an approach based on virtualphysics (cf. [Howard et al., 2002]) and an adequate number of robotsto form a connected mesh of agents. Additionally, swarm-based navi-gation can be used to direct the robots towards the tasks, either usinga part of the swarm as navigation beacons (e.g. navigation chain) forthe other robots (cf. [Nouyan et al., 2008]) or using dynamically re-distributed flying agents (cf. [Stirling and Floreano, 2010]). The maindrawback of all the aforementioned methods is that a reasonably highnumber of robots is required in order to have an effective application.

By trying to devise a method whose performance is independentof the group size, we chose instead to implement a simpler and unbi-ased exploration technique: random walk.

A random walk is a displacement process in which the direction ofthe agent which is performing the exploration is chosen at randomat every time step, according to a given probability distribution. Inthe most simple form, isotropic random walk, such probability distri-bution is the uniform one (i.e. at each step, every direction is equallyprobable). More complex forms of this technique exhibit a short termcorrelation concerning the directions of movement (correlated randomwalk), a general preference towards a certain orientation (biased ran-dom walk) or the possibility to dynamically adapt the probability dis-tribution in a learning fashion (reinforced random walk). A thoroughmathematical analysis of this process can be found in [Codling et al.,2008].

To be more precise, we chose to implement a correlated random walk,in which the agents move in a straight line (i.e. the direction of therobot in two sequential time steps is identical) until some event oc-curs (e.g. task location detected, obstacle detected). Our choice hasbeen mainly driven by the fact that correlated random walk is widelyobserved in nature and has been successfully used to model animalpaths (cf. [Codling et al., 2008]).

Concerning the collection operation, we decided to model the tasksas if they were resources scattered through the environment.

As a consequence, these activities will have a precise, fixed positionin space and will require a certain amount of time to be performed.


Moreover, every task can be performed by single robot at a time (i.e.single-robot stationary tasks). Following the approach of Brutschy et al.[2014], we decided to abstract these activities by means of a specificdevice (TAM - Section 2.3.2).

Our choice is motivated by the fact that the robotic agents thatwe chose (e-puck - Section 2.3.1) have limited capabilities in termsof object manipulation and performance of specific activities, dueto the simple technical equipment and their general purpose nature,thus requiring the abstraction of such tasks. Even though the de-tails of these activities have not been implemented as a consequenceof the abstraction, we decided to preserve the spatial connotation ofthe tasks, through their representation by means of a specifically de-signed physical device.

2.2.1.2 Multi-robot task allocation & coalition formation

Having clarified the relevant aspects related to the foraging process,we mainly focus our related literature analysis on task allocation andcoalition formation.

existing taxonomies In the context of Swarm Robotics, the taskallocation problem is usually related to the choice among differentalternatives, which are known a priori, concerning the role of theagents in the swarm (cf. [Krieger and Billeter, 2000; Agassounon andMartinoli, 2002; Pini et al., 2009]).

In order to classify our problem, we refer to the taxonomy definedby Korsah et al. [2013]. This classification, based on the article byGerkey and Mataric [2004], proposes to analyze the articles using 4

dimensions: utility interdependencies (α), robot type (β), task type(γ), allocation type (δ), to express the article classification in the formα [β− γ− δ] and to relate every category defined within this frame-work with a corresponding optimization problem.

The optimization theoretical framework adopted in the taxonomyrequires the definition of an objective function, representing the qual-ity of the allocation, that should be optimized by the agent choices.Such objective function is called utility. It is defined for every robot-task pair and it measures (on a standardized scale) the estimatedeffectiveness of the performance of the given robot, on the given task.According to the dependencies among the utilities of different agents,α can take the values ND (No Dependencies) when the utilities are inde-pendent of each other, ID (In-Schedule Dependencies) when the utilityof a task depends also on the other tasks that the robot is currentlyperforming, XD (Cross-Schedule Dependencies) when the utility of atask for a given agent depends also on the activity schedule of otherrobots or CD (Complex Dependencies), when the inter-schedule depen-dencies exists also among complex (i.e. hierarchies of) subtasks.


Concerning the robot type (β), the taxonomy distinguishes betweenagents that can tackle only one task at a time (ST: Single task agent) andagents able to perform multiple tasks at the same time (MT: Multi taskagent).

As for the task type (γ), we have a similar decomposition in taskthat must be performed by a single robot (SR: Single robot task) andactivities that require the joint effort of multiple agents (MR: Multirobot task).

Finally, the allocation type (δ) could be: (IA: Instantaneous Assign-ment) when the available information only allows for a single assign-ment of robots to tasks or (TA: Time-extended Assignment) where thereis the possibility to plan for future allocations.

As a matter of fact, our problem deals with agents that could per-form one task at a time (β = SR) with limited knowledge of the en-vironment, only allowing for an instantaneous assignment (δ = IA).Concerning task type and utility interdependencies, the most intu-itive modelling choice for our problem would be to have tasks withindependent utilities (α = ND), that only require a single robot tobe executed (γ = ST ). In fact, we analyze a problem in which homo-geneous agents should localize homogeneous tasks (that is, any robotcould potentially allocate itself to any activity) in space and then de-cide whether their allocation to the task is required or not.

However, according to Korsah et al. [2013] taxonomy, a ND [ST-SR-IA] problem is considered to be equivalent to the linear assignmentproblem. In the classical formulation of the problem (cf. [Votaw,1952]) there are n agents that have to be assigned to m jobs. Eachagent i is characterized by a utility estimate value uij that predicts itsperformance on task j. The problem can be formally stated as follows:

max

n∑i=0

m∑j=0

uijxij (2.1)

Subject to:

n∑i=0

xij = 1 ∀i (2.2)

m∑j=0

xij = 1 ∀j (2.3)

xij ∈ 0, 1 (2.4)

The optimal solution of the problem consists in the allocation ofagents to jobs, in a way that each agent is assigned to only one job andevery job is undertaken by a single agent that maximizes the systemperformance (utility). Using a centralized algorithm (i.e. the Hun-garian method by Kuhn [1955]), it is possible to solve the problem inpolynomial time.


Nevertheless, the major drawbacks of this formulation are two.First, in order to obtain a feasible solution for the problem, the num-ber of agents n must match the number of jobs m, which is not thecase for our problem, in which we would have in imbalance betweenagents and tasks. Second, the assumption of no utility interdepen-dencies among tasks is too simplistic and does not allow to capturethe real nature of our study.

For these reasons, we chose to model our problem as CD [SR-MT-IA]. In addition, instead of considering each task as independent ofeach other, both in terms of utility determination and inter-task con-straints (e.g. temporal ordering), we decided to group our tasks asto form clusters (i.e. aggregates) with a precise spatial arrangement.In this manner, we combine multiple simple tasks to form a single,complex activity that requires multiple robots to be tackled, or, inother words, a multi robot task (γ = MT ). The complex dependen-cies (α = CD) arise from the fact that, under the assumption of aperfect knowledge of the environment, the utility of a given complexactivity will depend on the number of robots assigned to it, but alsoon the allocation of the rest of the swarm to the other multi-robottasks.

According to Korsah et al. [2013], there is no well-known optimiza-tion problem that represents such class of problems.

By taking into account the fact that, given the limited sensing ca-pabilities of the agents, it is not possible to have a global knowledgeof the environment, we can reduce the utility dependencies to cross-schedule (α = XD), since the robots are only able to get local infor-mation (i.e. the schedules of neighbouring robots) concerning theallocation. In this case, we can reduce our multi robot task allocationproblem to the set partitioning problem. The classical formulation ofthe problem (cf. [Balas and Padberg, 1976]), involves a finite set S ofelements s ∈ S and a set of feasible sets F ⊆ P(S). Every F ∈ F is,indeed, a set of elements of S. A solution for the problem is repre-sented by a choice of sets F∗ ⊂ F such that that every element of sbelongs to a single set in F∗ and the sets in F∗ are disjoint from oneanother. In more formal terms, the problem becomes:

min∑F∈F

cFxF (2.5)

Subject to:∑F∈F

δsFxF = 1 ∀s ∈ S (2.6)

xF ∈ 0, 1 ∀F ∈ F (2.7)

cF ∈N (2.8)

δsF =

0 s /∈ F

1 s ∈ F(2.9)


In the context of multi-robot systems, such problem is called coali-tion formation, where S consists of the set of available robots in thesystem, and a feasible solution involves the division of the set in mul-tiple, non overlapping, robot coalitions. The optimality of a solutionis determined with respect to the cost cF, that is assigned to everypartition of S and that has to be minimized. A detailed formulationof the problem we are going to tackle is presented in Section 3.1.

As depicted in Figure 2.5, in the scientific literature, the multi-robotcoalition formation problem is tackled using two different approaches:centralized and distributed.

Centralizedapproaches

Optimizationtheory

Set par-titioning

Job shopschedul-

ing

Coalitiontheory

Gametheoreticalapproach

Figure 2.7: Overview of the approaches used in the scientific literature totackle multi-robot task allocation in a centralized fashion.

centralized approaches Centralized approaches are based onthe modelling of the allocation activity as an abstract problem, inwhich the majority of the aspects related to the embodiment of theagent are abstracted out. The prominent approaches in the literatureare related to coalition theory and optimization theory.

Coalition theory is a sub-field of game theory that analyzes the coali-tion formation problem as an n-person game, where the game’s pay-off is determined by the choices of the agents involved, concerningwhether or not to join a given coalition. Although the research inthis field has brought interesting results concerning the prediction ofthe outcomes and the stability of the payoffs of the aforementionedgames, there are several assumptions that limit the applicability ofthis theoretical framework to our problem. In fact, these approachesgenerally require a complete knowledge of the strategies of the otherplayers (not feasible due to the limited sensing and communicationcapabilities of our agents) and a statically, predetermined payoff (con-


trary to the nature of our problem, where the allocation of agents totasks is performed using dynamically changing information). More-over, the specific constraints of a multi-agent environment, such asthe agents’ distribution, communication and computational costs andlimits, are not taken into account in the model of the problem.

Optimization theory, on the other hand, maps the coalition formationproblem onto well-known optimization problems, for which effectivecentralized solution methods have already been found. For instance,coalition formation can be cast into an instance of a linear program-ming problem, which can be solved through the simplex method,with a polynomial average case complexity. Another possibility isto view the problem as a particular instance of the job shop schedul-ing problem, a problem that has been successfully solved using sev-eral approaches (constraint satisfaction search, integer and dynamicprogramming) and for which effective heuristics based on swarm ap-proaches have been developed (cf. [Colorni et al., 1994]). Another in-teresting hybrid optimization-swarm approach is presented by Qianand Cheng [2014], where ant colony optimization and genetic algo-rithms are used to found the optimal solution to a specifically definedmathematical problem. We refer the interested reader to [Vig, 2006],where a more detailed analysis of the problems can be found. Lastbut not least, the problem can be also formulated as an instance ofthe set partitioning problem (as proposed by Korsah et al. [2013]), a wellstudied NP-Hard problem, for which different centralized heuristicshave been proposed (e.g. [Balas and Padberg, 1976]).

The main drawback of this family of approaches is that they requireto gather all the information concerning the agents in a single entity(since distributed, parallel methods are either infeasible or still subjectof research), which can be either a robot in the group or an externalcomputation device, run the algorithm to obtain a solution and thanredistribute the solution among the agents. A similar approach re-quires complex communication capabilities and a global knowledgeof the state of the agents in order to be effective, requirements that aregenerally not met on real swarm robotics applications. Moreover, thecentralized collection of information in a single allocator introducesa bottleneck and a single point of failure in the system. In case ofits malfunction or unavailability the whole system ceases to function.In addition, the majority of the algorithms proposed in the literaturehave been developed for multi agent systems, dealing with disem-bodied agents. As Vig and Adams [2005] pointed out, there are otherissues, related to the computational and communicational complex-ity of the methods and the task representation, that further hinderthe feasibility of these methods on real multi-robot systems.

We can conclude that these approaches lack of robustness and scal-ability, thus being unsuitable for efficiently and effectively solvingour problem.


Distributedapproaches

Multi-agentsystems(MAS)

Explicitcoordi-nation

Marketbased

Implicitcoordi-nation

Responsethreshold

Swarmrobotics

Distributedproblemsolving(DPS)

DCSP

Figure 2.8: Overview of the approaches used in the scientific literature totackle multi-robot task allocation in a distributed fashion.

distributed approaches In the distributed approaches, the ef-fort (i.e. the algorithm) required to find a suitable solution for theproblem is distributed across different entities, instead of being cen-tralized in a single one. To be more precise, we can distinguish twosub-categories: distributed (cooperative) problem solving and multi agentsystems.

Distributed (cooperative) problem solving (DPS), focuses on the paral-lelization of centralized algorithms. In other words, these algorithmsare studied and adapted in order to be effectively run in parallel andbe synchronized across different computational entities. By exploit-ing the parallelism of the algorithm, it is possible to reduce the com-putation time required to get a solution and thus be able to tacklebigger instances of the problem in the same amount of time.

For example, the multi-robot task allocation problem can also be seenas an instance of the Constraint Satisfaction Problem, a well-knownoptimization problem, widely analzyed in the scientific literature, of-ten with centralized algorithms. The problem consists in finding anassignment of values to a set of variables V from the correspond-ing domains set D that respect the set of constraints C. Informallyspeaking, if we consider V as the set of tasks, D as the set of robotsand we introduce the constraint that each robot must be assignedto exactly one task we obtain the single robot, single task allocationproblem. The work of Shen and Salemi [2002] approaches this prob-lem in a distributed fashion (Distributed Constraint Satisfaction Prob-lem (DCSP)), by allowing the robots to compute parts of the whole


solution, and trading those parts when the capabilities of a singleagents are not sufficient to obtain a result. The proposed solutionhowever, is particularly demanding in terms of task modelling andinformation exchange among robots, thus difficult to apply on largegroups of robots.

Multi agent systems (MAS), on the other hand, deal with entitieswhich are able to perform computations (as those in distributed prob-lem solving) but also to observe the environment they are situated inthrough sensors and modify it by means of actuators.

Multi-agent approaches can be classified according to the type of co-ordination they employ. On one hand, we have intentional coordinationamong the agents, as in market-based approaches, for whose a detailedsurvey of can be found in [Dias et al., 2006]. The underlying principleis similar to the one used in the optimization-based approaches.

Each agent possesses, in fact, an individual utility function thatquantifies its preference for the allocation to a certain task. In addi-tion, the agent can estimate the cost (e.g. the distance, the energyconsumption) related to the task it is seeking allocation for.

Whenever more than one agent is interested in a certain task, anauction process is started. On basis of the information it possesses, itmakes a certain bid for the task. The agent with the highest bid winsthe auction process and gets the task. Examples of approaches in thiscategory are [Lin and Zheng, 2005; Guerrero and Oliver, 2003].

The advantages of this method are that the auction process doesnot necessarily require global or perfect information and that theagents perform their decisions autonomously and in a distributedfashion. Unfortunately, the bidding process is particularly demand-ing in terms of communication resources, since it requires an iteratedexchange of messages by all the parties involved in the transactionand does not scale well as the number of robots increases.

A special mention should be made for the article by Gunn and An-derson [2013]. To the best of our knowledge, it is the only work whichemploys a specifically designed approach based on explicit coordina-tion within groups of heterogeneous physically embodied agents ina Urban Search And Rescue scenario. The authors also showed therobustness of their method with respect to robot failure and resiliencewith respect to changes in the environment by dynamically adaptingthe structure of the coalition.

On the other hand, coordination emerges from simple interactionsamong the agents, without explicit agreement. These bio-inspiredapproaches can be classified in two different categories: response-threshold based and swarm robotics.

Response-threshold based approaches, are based on the homonymmodel, developed by Theraulaz et al. [1998] to model the task allo-cation process in ants, which assumes that each task in the environ-ment has an associated stimulus. Each agent has a different response


threshold for the available tasks and its probability to engage in theactivity is a function of the threshold and the value of the stimulus.The stronger the stimulus, the higher the likelihood that an agent willallocate itself to a certain task. The threshold can be either fixed (asin the original model) or adaptive (in a Reinforcement Learning fash-ion). In the latter case, the decrease of the threshold value for a certainactivity can make an individual more sensible to the correspondingstimulus, resulting in more frequent allocations. Conversely, an agentwill be more reticent to allocate to a certain task when the correspond-ing threshold is raised.

A similar approach has been devised by Arcaute et al. [2009] tomodel the foraging behavior of ants, in which, instead of having astimulus, the probability of a given insect to engage in a certain taskdepends on the spatial (e.g. location) and temporal (e.g. periodicity)characteristics of the task, but also on the number of agents currentlyperforming the task and the amount of learning performed.

Since the allocation decision is performed without direct commu-nication among the agents, these approaches scale well as the num-ber of agents increases. Moreover, they are robust with respect tofaults on the robots and flexible with respect to variations in the de-mand. The main drawback of these approaches is the difficulty topredict the emergent behavior of the system, making the design ofa system for a specific purpose problematic. A comparative analysisbetween intentional and emergent coordination approaches has beendone by Kalra and Martinoli [2006], showing a better performance ofthe response-threshold approach (i.e. the emergent one) with respectto the market-based one when the information is inaccurate. More-over, the work by de Lope et al. [2013] compares the performanceof a response-threshold approach with a probabilistic learning au-tomata as decision mechanisms (within an agent) for task allocation.The authors tested both a greedy (choosing the action with the high-est probability) and a random selection on a simulated environment,showing that both the approaches are able to efficiently achieve taskallocation in a self-organized way.

The aforementioned theoretical models have also been successfullyimplemented using real robots. The response-threshold model ofTheraulaz et al. [1998] has been adapted by Low et al. [2004] to per-form adaptive coalition formation in order to achieve sensor cover-age in a closed environment. The work of Sarker et al. [2014], onthe other hand, implements the model by Arcaute et al. [2009] onthe e-puck robotic platform in a simplified manufacturing shop-floorscenario, both in a centralized and a distributed fashion, in order tocompare the performances of the two approaches. Although both theapproaches have been able to produce the desired task allocation be-havior, the experiments showed that the distributed method, employ-ing only local sensing and communication capabilities, achieved bet-


ter performances in terms of task throughput and energy efficiency,by reducing the time spent by the agents travelling from one task toanother.

While the other categories of approaches are suitable for differ-ent types of agents (both abstract and physical), swarm robotics ap-proaches are explicitly devised for simple physically embodied agents(i.e. robots). The aim of these approaches is to achieve coalition for-mation relying only on the self-organization of agents with limitedsensing and communication capabilities.

For instance, the work of Pinciroli et al. [2013] is focused on theparallel formation of robot coalitions of different sizes, using a com-pletely decentralized approach based on the interaction among het-erogeneous robots. In fact, their approach mimics the aggregationbehavior of cockroaches, which has been successfully studied for asingle aggregation location (cf. [Amé et al., 2006]), and extends itto multiple locations, each one requiring a precise number of robotsto be grouped. They successfully achieve coalition formation usingmobile robots (foot-bots) employing a probabilistic approach to aggre-gation, where the probabilities to join/leave an aggregate are locallydetermined and broadcast by flying robots (eye-bots) representing ac-tive aggregation locations. In addition, the method has been provento be robust with respect to sudden changes in the environment andscalable when the number of robots and shelters is increased.

This overview is concluded by two articles that we deemed to beparticularly interesting to better understand our formulation of theproblem. Despite having a completely different approach, the workof Hsieh et al. [2008] addresses a problem closely resembling ours.Their problem involves N agents to be distributed among M sites inan environment whose topology can be expressed as a graph G(V,E),in terms of sites (V) and one-way connections among them (E). Thestate of the system is represented by the relative number of agentsat each site i (xi(t)). The authors propose two strategies for redistri-bution: baseline and quorum-based. In the first one, the distributionof the agents occurs by means of transition probabilities per unit oftime (kij) between nodes, defined on every edge in E. In the secondone, each site has an associated quorum value qi. Whenever the cur-rent occupation xi(t) is above the quorum then one of the outgoingtransitions rates from that node is multiplied by a coefficient α un-til the occupation descends again below the quorum value. In bothcases, the entries of K (i.e. kij∀i, j) are determined using Metropolisoptimization (cf. [Landau and Binder, 2009]).

Even though the model allows to obtain promising results, success-fully achieving the redistribution of a swarm of 20000 robots, thereare some strong assumptions that differentiate this approach fromours. First of all, the agents do possess a global knowledge of thetopology of the environment (G(V,E)) and thus are capable of lo-


calizing themselves and navigating easily from one site to another.Moreover, the robots are supposed to be able to estimate the currentoccupation xi(t) of the site they are in. Also, the results are obtainedonly with simulations where real-world non-linear effects on the sen-sors and actuators along with realistic deployment dynamics for therobot have not been modeled. Although the stochastic transition po-lices are defined at an agent-level and do not require any wirelesscommunication, the actual values of the transition probabilities arecomputed off-line and then distributed to all the agents, making theproposed solution de facto centralized. In contrast to [Hsieh et al.,2008], our purpose is to distribute uniformly agents across spatiallydistributed tasks, in real-time, without global information concerningthe topology of the environment or the transition rates.

Finally, the work from Jevtic et al. [2012] is probably the closestwith respect to ours. In fact, the article deals with a SR-MT-IA task al-location problem in which N e-puck robots must be distributed acrossM different target locations (modelling the multi-robot tasks) scat-tered in the environment. Every location has an associated qualityvalue Qi and a desired fraction of the robot population fdi , whichcorresponds to the normalized quality qi, with i ∈ 1, · · · ,M. Theagents choose the target destination for the allocation according to aprobabilistic decision mechanism, based on roulette wheel selection,where the selection probability pi is a function of the quality of thelocation (as a measure of its utility) and the distance of the agent(located in (xr, yr)) from the target location i (in (xi, yi)):

qi =Qi∑Mj=1Qj

(2.10)

ηi =1√

(xi − xr)2 + (yi − yr)2(2.11)

pi =qαi η

βi∑N

j=1 qαj ηβj

α,β > 0;α,β ∈ R (2.12)

M∑j=1

pj = 1 (2.13)

The quality of the allocation is then determined by computing themean absolute error between the actual fraction of allocated robots fiand the desired allocation fdi :

MAE =1

M

M∑j=1

|fj − fdj | (2.14)

The method is then tested on three scenarios, differing from one an-other on the positioning and the quality values of the different targetlocations, on which it is able to reach low levels of mean absoluteerror (< 5%). In addition, the scalability of the approach has been


tested by replicating the experiments with an increasing number ofrobots (i.e. 10, 20, 40, 60 and 100), showing that the average and max-imum values of MAE across the experiments decrease as the numberof agents scales up. Nevertheless, some assumptions of the methoddiffer completely from our work and hold only for specific cases. Infact, the algorithm assumes that the robots communication range cov-ers the whole arena and that the number of tasks in the environment,as well as their qualities, are coded into the robots. Moreover, thedecision is made only when the agents have perfect knowledge of therelevant information about the environment (i.e. when all the taskshave been discovered).

The majority of the studies we reviewed deals with abstract agentsor high-level abstractions of the tasks. Up to now, to the best of ourknowledge, there are no studies that explicitly focus on the allocationof physically embodied agents to physically situated activities, repre-sented by active devices, with a self-organizing approach and withoutsimplistic assumptions. Our aim is to reduce the level of abstraction,while keeping in mind the intrinsic limitations of the agents, and todevise a method which could be easily ported to real robots.

2.3 materials 33

2.3 materials

Since we are going to develop a method in the context of swarmrobotics, we must clarify two fundamental aspects: the characteristicsof the embodied agents and the development methodology we are go-ing to adopt.

Our embodied agents are mobile robots, the e-pucks, presented indetail in Section 2.3.1.

A novel point in our approach is that tasks are abstracted but em-bodied as well, by means of a specifically designed device, the TAM,whose specifications will be discussed in Section 2.3.2.

As for the development methodology, we are going to adopt a behav-ior based approach (explained in Chapter 3) with a microscopic-levelsimulation-based analysis. All the relevant information concerningthe multi-robot simulator can be found in Section 2.3.3.

2.3.1 e-puck

Figure 2.9: Annotated picture of the e-puck robot. (Courtesy of RoadNar-rows)

Only the relevantaspects for the thesishave been discussedhere. For moredetails concerningthe robot, visithttp:

//www.e-puck.org

The e-puck (Figure 2.9) is an open-source educational desktop mo-bile robot developed at the EPFL (Ecole Politechnique Federale deLausanne) (cf. [Mondada et al., 2009]), in order to provide a commonrobot platform to be used in a broad range of university courses. Theeducational purpose of the robot has influenced its design process.

Its compact and modular shape is thought to be robust with respectto the student use but easy to repair at the same time. Moreover, itssize has been conceived to allow the usage of the robot in relativelysmall environments, like a desk.

In addition, the robot is equipped with several different sensorsand actuators, as well as with an expansion board, making the robot

http://www.roadnarrows-store.com/

http://www.roadnarrows-store.com/

http://www.e-puck.org

http://www.e-puck.org


flexible and employable in a wide range of contexts (from signal pro-cessing to automatic control, for example).

In order to favour the diffusion of knowledge and the improve-ment of the robot, the designers have given open access to its hard-ware specifications and distributed the related software with an opensource license (cf. Figure 2.10). The publication of all the informationconcerning the robot has made the development of several extensionboards for the e-puck possible.

(a) Placement of the omnidi-rectional microphones onthe e-puck board. (Cour-tesy of EPFL)

(b) Placement of the proxim-ity sensors on the e-puckboard. (Courtesy of EPFL)

(c) Placement of the RAB modules on the range and bearing board.(Courtesy of RBZ Robot Design)

Figure 2.10: Sensor placement on the e-puck.

For more detailsconcerning the

omnidirectionalcamera, visit http:

//www.e-puck.

org/index.php?

option=com_

content&view=

article&id=

26&Itemid=21

Among the different available extensions, we focus only on thoserelevant for the development of our methods, namely the omni-directionalcamera and the Range and Bearing (RAB) board.

The omni-directional camera is built by encapsulating the standarde-puck camera in a glass cylinder, with an hyperbolic mirror on theopposite side. By pointing the camera towards the mirror it is pos-sible to obtain a 360 degrees viewing range around the robot. TheCMOS camera, along with a FIFO buffer and an additional microcon-troller, is mounted on a board on top of the robot and connected to itby means of a card-edge connector, leaving the possibility to connectother extensions to the e-puck.

The Range and Bearing board is an extension developed to providelocal communication capabilities to the robot (cf. [Gutiérrez et al.,2008]). By means of 12 IR emission/reception modules, nearly uni-

http://http://www.e-puck.org/index.php?option=com_content&view=article&id=19&Itemid=11

http://www.e-puck.org/index.php?option=com_content&view=article&id=22&Itemid=13

http://www.rbzrobotdesign.com/en/products/e-puck








2.3 materials 35

formly distributed on the perimeter of the board (see Figure 2.10c),the e-puck becomes capable of broadcasting up to 16 bit of informa-tion to the nearby robots. The board takes its name from the fact that, All the relevant

details concerningthe range andbearing board(e-RandB) can befound athttp://www.

rbzrobotdesign.

com/en/products/

e-puck

given the power of the received signal and the placement of the sen-sors/actuators, the embedded microcontroller on the board is able tocompute the range (distance) and bearing (angle with respect to the re-ceiver) of the sender robot. Figure 2.11 shows an e-puck with both theaforementioned extensions installed.

We decided to choose the e-puck as the robotic platform for thedevelopment of our methods since it is a simple yet powerful devicewhich best represents the notion of swarm agent.

If we look at the specifications (Table 2.2), we may notice the limitedsensing capability, especially with respect to the sensors we are goingto use in our method: the proximity sensors and the camera.

The affordability and reduced size (with respect to similar devices)makes also the creation of a large swarm of robots feasible. Phototaxis is a kind

of taxis (i.e.locomotorymovement), thatoccurs when a wholeorganism moves inresponse to thestimulus of light.

Although the e-puck possesses a wide set of sensors and actua-tors, we are only going to use those required to perform displace-ments (wheels), obstacle avoidance (proximity sensors) and odometry(encoders), plus the functionalities offered by the extension boards toperform phototaxis (omni-directional camera) and local communication(range and bearing system).






features technical information

size and weight 70 mm (diameter) x 55 mm (height), 150

g

processor (robot) dsPIC 30F6014A @ 60 MHz ( 15 MIPS)16 bit microcontroller with DSP core

controller (omnidirectional camera) dsPIC33FJ256 GP506 16 bit microcon-troller with (Averlogic AL440B-24-PBF)(FIFO) frame buffer

controller (rab board) dsPIC 33FJ256 16 bit microcontroller

motors 2 miniature stepper motors (20 steps perrevolution) with a 50:1 reduction gear

wheels Diameter: 41 mm, Interwheel distance:53 mm, Maximum speed : one wheelrevolution per second (1000 steps/s)

encoders One per wheel, pulses resolution: 0.13

mm

proximity sensors 8 IR sensors unevenly distributed belowthe LED ring (cf. Figure 2.10b)

camera 2 (front and omni-directional) CMOSVGA color camera, resolution: 480x640

pixels (4 fps at 40x40)

rab emission/reception module 12 IR emitting diodes with IR modulatedreceivers and IR photodiodes (cf. Figure2.10c)

microphones 3 omni-directional microphones forsound localization (cf. Figure 2.10a)

accelerometer 3D accelerometer along the X, Y and Zaxis

leds 8 red LEDs (ring), green LEDs (body),one beacon red LED (front)

speaker On-board speaker (WAV and tonesound playback)

connectivity Serial port (RS232), Bluetooth, IR Re-mote control

battery autonomy 5Wh Li-ION rechargeable and remov-able battery (∼ 3 hours autonomy)

programming languages api C (General purpose), ASM (DSP)

Table 2.2: Overview of the technical specifications of the e-puck robot.

2.3 materials 37

2.3.2 IRIDIA Task Abstraction Module

Figure 2.11: Picture of the TAM with an e-puck with range and bearing boardand the omni-directional camera extension. (Courtesy of ArneBrutschy)

The simple nature of the e-puck robot entails several advantages:reduced cost, easiness of use and high maintainability among others.Nevertheless, it also poses important limitations on the type of activi-ties that the robot could actually perform. In fact, due to its physicalstructure, the e-puck cannot be extended by means of an object manip-ulation device (e.g. a gripper).

To overcome such limit, we decided to employ task abstraction inthe development of our methods. According to Brutschy [2014], taskabstraction is the process of considering a task independently of thoseof its aspects that are not relevant for the study at hand. In otherwords, we decided to abstract all the aspects of the tasks that cannotbe physically performed by the robots and those that are not consid-ered essential with respect to the problem we are analyzing.

For our research, abstraction has two main advantages. From anengineering standpoint, our goal is to develop a method to solve aproblem having a practical relevance (i.e. a real-world application).Abstraction allows to filter out all the aspects of the real-world tasksthat are not relevant to our vision of the problem, allowing us to focuson its core features. Under a scientific point of view, generalizationis a direct consequence of abstraction. As a matter of fact, if the ab-straction is sufficiently generic, the method developed for the givenproblem can be applied also to all the instances of different problems

http://iridia.ulb.ac.be/~abrutschy/index.php

http://iridia.ulb.ac.be/~abrutschy/index.php


whose tasks have the same essential aspects, thus extending the sci-entific interest of the result.

In the scientific literature, task abstraction has been performed us-ing virtual (i.e. simulated/disembodied) tasks and activities repre-sented by both passive (inanimate) and active (possessing some formof sensing/actuation) objects. Tasks are usually virtualized in for-aging tasks when the employed robotic platform has limited manip-ulation capabilities, while passive objects are usually employed inbucket-brigading and collective transport tasks (cf. [Brutschy, 2014]).It should be noted that all the relevant studies mentioned in 2.2.1.2,[Jevtic et al., 2012; Hsieh et al., 2008; Pinciroli et al., 2013] employ taskabstraction by virtualization, with disembodied tasks.

Instead, we decided to use the IRIDIA TAM, an active device for taskabstraction specifically designed for the e-puck robot (cf. [Brutschyet al., 2010; Brutschy, 2014]). The TAM comes from the idea that thetasks that are beyond the capabilities of the robot could be insteadsimulated, while preserving the essential aspects of the activity: itsspatial and temporal localization (i.e. the place in space where/themoments in time when the task is available). It has been designed asan U-shaped booth (Figure 2.11) that could welcome a single robot,whose presence can be detected by means of the light barrier. Theabstraction of the task occurs through the interaction between the de-vice and the robot. In fact, the TAM is a programmable device whichcan exchange information with the e-puck in two ways: using the col-ored RGB LEDs to be detected and to display its internal state, andusing the IR transceiver to send data to one of the robot’s proximitysensors. In addition, the 802.15.4 XBee DigiMesh module can be usedto perform real-time communication of the experimental data withan external system for reporting and further analysis.

The choice of limiting the TAM capacity to a single robot has beena precise design choice. In this way, the device represents a stationarysingle-robot task, in other words, an activity that requires a robot tobe busy in a precise location in space for a specified period of time.By representing every atomic task with a separate device it is easyto model complex tasks by composition of single tasks and task in-terrelationships by specifically programmed interactions among thedevices. Finally, it is a duty of the researcher to find a suitable rep-resentation of the task it is modelling, given the functionalities ofthe aforementioned device. For our problem, a thorough discussionabout the modelling is done in Chapter 3. An example of a successfulapplication of this device is Pini et al. [2011], where a group of TAM

is used as a cache site to simulate the deposit and pick up of objects.In the context of our work, we take advantage of the fact that a TAM

could be seen as a physical, embodied representation of an activity,thus having a precise collocation in space and time. Indeed, TAMs will

2.3 materials 39

model the spatially distributed tasks that will require the allocationof a robot.

features technical information

sizes 120 mm x 120 mm x 108.3 mm (height x width xdepth)

processor Atmel ATmega1284p @ 16 MHz ( 20 MIPS) 8-bit mi-crocontroller with 16 KiB main memory and 128 KiBflash memory

battery 5Wh LiION rechargeable and removable (∼10 hoursautonomy in a typical experiment setting (cf.[Brutschy, 2014]))

light barrier IR LED (emitter, left side) with IR transistor (receiver,right side)

leds 4 RGB LEDs (3 on the main board, one on top) con-trolled through PWM

connectivity 802.15.4 XBee DigiMesh (Wireless communicationwith coordinator pc), IR transciver (Infrared commu-nication with the e-puck using IRcom protocol)

programming languages api Processing, C, C++

Table 2.3: Overview of the technical specifications of the TAM device.

2.3.3 Autonomous Robot Go Swarming

ARGoS 2.0development hasbeen discontinued,the new version ofthe simulator, still inbeta testing phase, isavailable at https://github.com/

ilpincy/argos3

ARGoS 2 is a multi-robot simulator, written in C++, and releasedunder GPL-3 license.

It has been developed by Pinciroli et al. [2012] in the frameworkof the EU-Funded project Swarmanoid 3 which is also the officialsimulator of the EU-Funded projects ASCENS 4, H2SWARM 5, andE-SWARM 6.

By being designed to deal with potentially large swarms of robots,the simulator needs to couple the accuracy of the simulation (i.e. thecorrespondence between results obtained in simulation and in realworld experiments), with an efficient use of the resources, in order toavoid a performance degradation as the number of simulated agentsscales up.

Another desirable feature of the simulator is flexibility, which canbe expressed as the ability of the simulator to cover a wide range ofuser cases. As a consequence, a flexible simulator should allow theuser to implement and integrate new features (e.g. sensors or robotmodels). In ARGoS, flexibility is achieved through a modular archi-

2. http://iridia.ulb.ac.be/argos/home.php

3. http://www.swarmanoid.org

4. http://ascens-ist.eu

5. http://www.esf.org/activities/eurocores/running-programmes/

eurobiosas/collaborative-research-projectscrps/h2swarm.html

6. http://www.e-swarm.org/

https://github.com/ilpincy/argos3



http://iridia.ulb.ac.be/argos/home.php

http://www.swarmanoid.org

http://ascens-ist.eu

http://www.esf.org/activities/eurocores/running-programmes/eurobiosas/collaborative-research-projectscrps/h2swarm.html

http://www.esf.org/activities/eurocores/running-programmes/eurobiosas/collaborative-research-projectscrps/h2swarm.html

http://www.e-swarm.org/


tecture on every level. Every simulated entity, from the sensors tothe physics engine, is implemented as a plugin, which can be easilymodified or extended by the end-user. A similar architecture sup-ports the existence of different versions of the same components (e.g.actuators/sensors with/without noise models, fine/coarse grainedphysical simulations, graphical/textual visualization).

As a consequence. the efficiency of the simulation can be achievedby exploiting the plugin nature of the components (see Figure 2.12)(i.e. loading only the required ones) and by precisely tuning the levelof accuracy of the simulation through the selection of the most ap-propriate components for the experiment at hand. Efficiency is alsoattained through the support for the simultaneous (i.e. parallel) sim-ulation of different physical engines, each one of them assigned tonon overlapping regions of the simulation space, resulting in an im-proved allocation of computational resources.276 Swarm Intell (2012) 6:271–295

Fig. 1 The architecture of ARGoS. The white boxes correspond to user-definable plug-ins

3.1.1 The simulated 3D space

The simulated 3D space, depicted at the center of Fig. 1, is a collection of data structures thatcontains the complete state of the simulation. This state information includes the positionand the orientation of each object such as obstacles or robots. The state of objects composedof different parts or equipped with special devices, such as sets of colored LEDs, is alsostored in this space.

The data is organized into basic items referred to as entities. ARGoS natively offersseveral entity types, and the user can customize them or add new ones if necessary. Eachtype of entity stores information about a specific aspect of the simulation.

For instance, to store the complete state of a wheeled robot, a composable entity is used.Composable entities are logical containers that are used to group other entities. Compos-able entities can be nested to form trees of arbitrary complexity. The controllable entity isa component that stores a reference to the user-defined control code and to the robot’s sen-sors and actuators. The embodied entity component stores the position, orientation and 3Dbounding box of the robot. The current wheel speed is stored into the wheeled entity compo-nent. If the robot is equipped with colored LEDs, their state is stored in a component calledLED-equipped entity.

Entity types are organized in hierarchies. For instance, the embodied entity is an extensionof the simpler positional entity, which contains just the position and orientation of an object,but not its bounding box. These design choices (entity composition and extension) ensureflexibility, enhance code reuse and diminish information redundancy.

Entity types are indexed in efficient data structures optimized for access speed. In thisway, the performance of the plug-ins that access the simulated 3D space is enhanced. Forexample, positional entities and their extensions are indexed in several type-specific spacehashes (Teschner et al. 2003).

3.1.2 Sensors and actuators

Sensors and actuators are plug-ins that access the state of the simulated 3D space. Sensorsare granted read-only access to the simulated 3D space, while actuators are allowed to mod-ify it. As explained in Sect. 3.1.1, information about the simulation state is stored in a num-ber of specialized entities. Sensors and actuators are designed to only access the necessary

Figure 2.12: Architecture of ARGoS. The white boxes correspond to user-definable plug-ins.From Pinciroli et al. [2012] in Swarm Intelligence, Volume6, Issue 4, December 2012. Reprinted with permission fromSpringer Science and Business Media. All rights reserved.

We decided to use ARGoS since, in addition to the aforementionedproperties, it offers full support for the e-puck robot and the Task Ab-straction Module (i.e. all their sensors/actuators are implemented inthe simulator).

In order to launch a working simulation, ARGoS requires two com-ponents to be provided: the controllers and the configuration file.

The controllers are plugins which make use of all the other compo-nents (i.e. sensors/actuators) provided by the simulator to implementthe individual behavior of an entity (TAM and e-puck, in our case).They consist of C++ source files (.cpp), with the respective headers(.h), which must be compiled before being dynamically loaded atruntime. The control interface against which write entity controllers

2.3 materials 41

in ARGoS is common to both simulated and real devices. This meansthat the transition between simulated and real entities (robots andTAM) is seamless, requiring only a recompilation for a different targetarchitecture (e.g. PIC for e-puck, Atmel ATmega1284p (Arduino) forTAM).

The configuration file is an XML file which describes the structure ofthe simulated environment (employed physics engines, mobile andstatic objects displacement, cameras placement) and which is used tomap each simulated entity with the corresponding controller, givingalso the possibility to pass parameters to it.

1: Initialization2: Initialization loop function code3: Simulated 3D space visualization4: while Experiment not finished do5: for all robots do . Sense+control phase6: Sensor readings update7: Control step execution8: end for9: for all robots do . Act phase

10: Robot status update11: end for12: Pre-update loop function code13: for all physics engines do . Physics update phase14: Physics update15: end for16: Post-update loop function code17: Simulated 3D space visualization18: end while19: Cleanup

Figure 2.13: Simplified pseudo-code of the main simulation loop of ARGoS.Each ‘for all’ loop corresponds to a phase of the main simulationloop. The lines in italics are optional and corresponds to theloop function hooks.

Another important functionality offered by ARGoS is the possibilityto create loop functions. The simulation loop in ARGoS involves threephases: sense and control, act, update (cf. Figure 2.13). In the first phase,the values of the sensors are read and passed to the code of the userdefined controller, which is then executed, modifying the state of theactuators. Then, in the act phase, the actions stored in the actuatorsare executed. Lastly, in the update phase, the physics engine updatesthe state of the entities under its control. Each of the aforementionedphases can be executed in parallel by a number of threads defined bythe user in the configuration file, further improving the efficiency ofthe simulation by exploiting the presence of multiple core CPUs.


Loop functions are user-defined functions hooks that are placed inprecise points in the simulation loop (i.e. at initialization time, beforeand after the execution of the update phase). Through this feature,the user is able to query the physics engine and to modify its stateat run time, by means of custom code. We took advantage of thisopportunity to collect data at each simulation step for further analysisand visualization.

Figure 2.14: Screenshot of the ARGoS simulator for the scenario Biased envi-ronment configuration.

3M E T H O D O L O G Y

Chapter 3 contains an accurate description of the methodology thatwe adopted for development of our methods.

We begin by giving a formal definition to the allocation problemwe are going to tackle (Section 3.1).

In Section 3.2, we describe how we have translated the problemstatement into a physical implementation of the problem.

Then, in Section 3.3, we present our contribution using a top-downapproach. We start by presenting an overview (Section 3.3.1) of un-derlying idea of the methods. Lastly, we conclude by characterizingour methods: Naive, Probabilistic and Informed.

3.1 problem statement

"Normally, in thepresence of radiation,communicationlinks fail. But withautonomous robots,you don’t needcommunications."–Vijay Kumar

The objective of the thesis is to devise an algorithm to perform theallocation of a set of robots (more precisely, a swarm, as it is called inthe context of swarm robotics), to spatially distributed activities.

As we discussed in Section 2.2.1.2, such problem can be seen as aninstance of the multi-robot task allocation class of problems. This classhas been extensively studied in the scientific literature of multi-robotsystems and several taxonomies have been proposed to classify thedifferent works in the field.

We decided to classify our work using the taxonomy by Korsahet al. [2013], the most recent and comprehensive at the time of writing.To be more precise, according to the taxonomy, the problem we areinterested in is of type XD [SR-MT-IA].

In other words, the problem involves the allocation of robots capa-ble to tackle a single task at a time (SR: Single task robot), to activitiesrequiring the simultaneous presence of multiple agents (MT: Multi-robot task). Moreover, the limited information available in the envi-ronment only allows an instantaneous assignment (IA) of the robot tothe activities, preventing the scheduling of future assignments. An-other complication is introduced by the fact that the utility (i.e. themeasure used by a single robot to decide whether to allocate or notto a given activity) also depends on the decisions took by the otherrobots (XD: Cross-schedule dependencies).

According to the aforementioned taxonomy, a XD [SR-MT-IA] prob-lem is equivalent to the coalition formation/set partitioning combina-torial optimization problem, where the set of robots must be parti-tioned in different coalitions in order to tackle concurrently the dif-ferent multi-robot tasks available in the environment. However, as

43

44 methodology

discussed by Vig [2006], such problem is computationally intractable,since the computation of an optimal solution requires the explorationof the combinatorial space of coalitions, whose size is exponential inthe number of robots in the swarm (O(2n), with n being the swarmcardinality), even for static coalitions. In addition, some heuristics (cf.[Shehory and Kraus, 1998; Sandhlom and Lesser, 1997]) have beenproposed to obtain an approximate solution to the coalition formationproblem with dynamic coalitions. Unfortunately, such algorithmshave still not been demonstrated in practice in the robotic domain,and have strict assumptions concerning the agents’ knowledge of theenvironment and their communication capabilities, which could notbe met, given the environment and the robotic platform we are inter-ested in.

For these reasons, we decided to have a novel approach to the prob-lem, following the principles of swarm intelligence: instead of requir-ing a complete knowledge of the environment and strong commu-nication requirements, we decided to employ simple agents that donot perform explicit utility computation and which do not directlyexchange information with each other. This kind of approach, inturn, does not need the same degree of formalization required whileperforming combinatorial optimization, yielding to a simplified state-ment of the problem.

Generally speaking, our problem can be described as:

Problem statement 1. Given n robots and c multi-robot tasks (decompos-able in m single-robot activities having a precise spatial distribution), deter-mine a mapping of the robots to the tasks which optimizes a given allocationmetric.

The parameters of this general definition are: the number of robots,the number of single-robot tasks, the distribution of tasks and the alloca-tion metric.

Moreover, the relation between the number of robots and single-robottasks is a key factor in the problem statement, strictly related to theallocation metric.

For example, if we consider a scenario in which the number ofrobots is greater or equal than the number of tasks (n > m), if the ex-periment duration is sufficiently long, the allocation of all the agentswill be eventually reached. In such case, it could be interesting toanalyze the speed with which the complete allocation is achieved.

On the other hand, if the number of tasks is greater than the num-ber of agents (n < m), the allocation metric could measure howevenly are the robots spread over the task in space or how fast theswarm could reach some allocation "milestones" (e.g. 25%, 50%, 75%of the tasks).

Regarding the spatial distribution (in a two-dimensional space) wecould foresee two possible categories of problems: random and deter-ministic distribution.

3.1 problem statement 45

With a random distribution, the positions of the tasks are deter-mined by a bivariate random distribution (e.g. normal or uniform).

Conversely, a deterministic distribution consists of a precise disposi-tion of the tasks in space, potentially as to form a grid or clusters.

In the light of this classification, we can give a more precise defini-tion of our problem:

Problem statement 2. Given n robots and m tasks (n < m) clustered inspace (as to form c multi-robot tasks), determine a mapping of the robots tothe tasks which is as uniform as possible across multi-robot tasks.

We assume that the values of n, m and c remain constant for thewhole experiment duration. Furthermore, robots are assumed to beidentical to each other and individual tasks are considered homoge-neous (i.e. there are no differences among tasks in terms of requiredskills to be performed) and independent from each other (i.e. there areno interdependencies among the tasks).

The allocation of a robot to an individual task automatically pre-vents the allocation of other robots to the same task and reallocationto another activity is not allowed. As a consequence, any free robotcould perform any available task.

In addition, the m tasks to be performed are distributed across cclusters in space. Each cluster i represent a complex task requiringmultiple robots at the same time to be performed, having a fixedrequest of robots ri and a time varying occupation oi(t). Henceforth,we will use cluster and multi-robot task as synonyms to represent thesame aforementioned concept.

The request ri consists of the number of robots that are required toperform the collective activity represented by cluster i.

The occupation oi(t) corresponds to the number of tasks of the clus-ter i that are being performed by a robot at time t.

Consequently, we have m > ri > oi(t), ∀ i, t.Given the two measures, we can compute a third one, the error,

which will serve as a measure for the quality of the allocation.

ei(t) = ri − oi(t) (3.1)

Indeed, if a cluster request will be completely satisfied, the error willbe null. A thorough discussion of the measures of allocation qualityis made in Chapter 4.

In summary, we are proposing an approach based on self-organiza-tion (as opposed to the traditional combinatorial optimization frame-work) to tackle the problem of coalition formation for task allocationin a multi robot system (also known as Multi-Robot Task Allocation(MRTA)). To be more precise, with respect to the taxonomy by Dorigoand Birattari [2007], it can be classified as an artificial swarm intelligenceapproach with engineering purposes.

The main difference among our vision of the problem and a largepart of the works proposed in the literature is that both robots and

46 methodology

tasks are embodied through the use of specific devices (i.e. e-puck - cf.2.3.1 and TAM - cf. 2.3.2).

The implications of this embodiment choice are twofold. On onehand, we have the opportunity to test our methods with realisticconditions, using existing, completely developed devices. On theother hand, these conditions entails some representational assump-tions which we have to deal with during the development process.For instance, by having limited sensing and communication capabil-ities, the agents are not able to construct a global map or a globalcoordinate system to represent the surrounding environment. As aresult, a global knowledge concerning the displacement and the avail-ability of the tasks cannot be obtained (in contrast to what is requiredby the algorithms proposed by [Hsieh et al., 2008] and [Jevtic et al.,2012]).

3.2 experimental setup

Our novel approach, based on embodiment, requires in turn a spe-cific experimental setup to represents both agents and tasks properly.Before getting into the details of the setup, we must remind that, atthe present time, swarm robotics is pratically a fundamental researchfield (i.e. no swarm robotic system has been successfully applied toa real-world problem, cf. [Brutschy, 2014]) due to both technologicaland economical constraints.

In this context, the use of abstraction is a powerful tool that en-tails several advantages. First of all, it allows to isolate the essentialfeatures of the practical application at hand, while removing all theaspects that are not relevant (with respect to our vision of the prob-lem) and potentially overcoming some of the aforementioned limita-tions. Moreover, the practical application can be seen as a particularinstance of a generic abstract problem. By approaching the more gen-eral case, we might be able to devise a solution viable for differentinstances of the analyzed problem. The quality of the generalizationis highly dependent on the level of abstraction, an important param-eter that must be finely tuned by the researchers.

Abstraction is a key feature of our approach. In fact, in our prob-lem there are three main aspects that have undergone the process ofabstraction: robots, tasks and the environment.

Concerning robots, we chose a platform with a minimal equipment(wheels, proximity sensors, LEDs, camera) retaining only the essen-tial features for the task localization and allocation, in order to max-imize the possibility to generalize the obtained results. As a matterof fact, the aforementioned equipment constitutes the minimal setof sensors/actuators available on the majority of the wheeled robots.For this reason, the e-puck is a de facto standard for Swarm Roboticsapplications (cf. [Brutschy, 2014]). In other words, the minimal ca-

3.2 experimental setup 47

pabilities of this robot are a form of abstraction by simplification (i.e.all the activities that could not be performed due to technologicallimitations are abstracted).

The abstraction of the environment, on the other hand, is a trade-off concerning the choice of relevant aspects for the simulation and asuitable environment for empirical experiments using the aforemen-tioned platform. We decided to focus on the feasibility of the experi-ments, by building a simple, bounded environment only containing aplain, blank floor and the representations of the multi-robot tasks weare interested in. This setup guarantees that the robots’ performancewould not be altered by the presence of irregularities of the terrainand allows us to better observe the dynamics of allocation, thanks tothe absence of irrelevant features of the environment.

Lastly, task abstraction is the core point of the experimental setup.The essential features of a task, in the domain of swarm robotics, are:the capabilities required to perform it, the time at which the activityis spawn and its duration. If we introduce the constraint that the taskwill occupy a fixed position in space (i.e a stationary task), its positionbecomes another relevant aspect. As we saw in Section 2.3.2, a single-robot stationary task can be abstracted by means of a specific device (i.e.TAM), which actually performs task emulation, an intermediate formof abstraction between task simulation (maximum abstraction level)and real robots experiments (minimum abstraction level). It shouldbe noted that the TAM does not consist in an ad hoc abstraction forour problem, but rather is part of a systematic framework for taskabstraction that allows to study complex tasks with simple robotswithout specific capabilities and offers a practical infrastructure tocollect empirical data (cf. [Brutschy, 2014]).

3.2.1 Formal modelling

How can the single robot stationary tasks abstracted through theTAMs can be combined to form multi-robot tasks, as described in Prob-lem Statement 2? Our solution is based on task modelling and taskdecomposition using the high level modelling approach proposed byBrutschy [2014].

The approach is based on Unified Modeling Language (UML) 2.x,a well-known graphical general purpose modeling language, com-monly applied in the domain of object-oriented software engineering.To be more precise, the visual model of the tasks is provided usingUML Activity Diagrams (we refer to [Rumbaugh et al., 2004] for a de-tailed explanation of the semantic of the diagram), representing theworkflow of the activities composing the task.

In these diagrams, one can find oval nodes, representing elemen-tary actions in the workflow, and activities, hierarchical containersfor elementary actions, depicted as rectangular nodes with rounded

48 methodology

corners. Activities and elementary actions can be directly connectedby arrows, representing a sequential interdependence among them or en-closed between black bars, representing a concurrent interdependence (i.e.the parallel execution of the activities).

In this contest, task decomposition is implemented through a hier-archical representation of the activities. To be more precise, the activ-ity to model is recursively decomposed into subtasks until a desiredlevel of accuracy is reached.

Our first approach to model the problem of multi-robot task alloca-tion is depicted in Figure 3.1.

start end

Tlocalization1 Tallocation1

Tlocalizationm Tallocationm

m tasks tobe tackled byn < m robots

Figure 3.1: UML Activity Diagram representing the ND [SR-ST-IA] (cf. [Kor-sah et al., 2013]) approach to the multi-robot task allocation prob-lem. The problem is seen as a collection of single-robot stationarytasks to be performed in parallel. Each task has to be localizedfirst, and then executed.

Here, the multi-robot task allocation problem presented in ProblemStatement 1 is decomposed in m independent single-robot tasks. Eachtask must be first localized and then allocated ( τlocalizationi andτallocationi , respectively) to the robot. This sequential interdepen-dence is represented through the arrow connecting the two subtasks.In addition, all these pairs of activities must be performed in parallelby the robots, hence the concurrent interdependence represented bythe black bars. We chose to stop the decomposition at this level, with-out going further into the details of each subtasks, since we believethat such level of detail is unnecessary with respect to the extent ofour model.

By representing each task with the corresponding TAM, the exper-imental setup can be naturally developed from this high-level speci-fication, simply by scattering the different task abstraction devices inthe environment. With this configuration, the robots need to first ex-plore the environment to localize the tasks, and then decide whetherto allocate or not to the activity.


However, such modelling does not take into account the multi-robotnature of the activities, a key point of our problem. For this reason,we refined our initial model (cf. Figure 3.1) to obtain the model de-picted in Figure 3.2.

start end

Tlocalization1

Tlocalizationr1

Tcollective1

Tlocalizationrc

Tlocalizationj

Tcollectivec

m individualtasks, grouped inc collective tasks,to be tackled byn < m robots

Figure 3.2: UML Activity Diagram representing the XD [SR-MT-IA] (cf. [Kor-sah et al., 2013]) approach to the multi-robot task allocation prob-lem. The problem is seen as a collection of multi-robot tasks tobe performed in parallel. Each multi-robot task can be executedonly when a sufficient number of robots has localized the single-robot tasks it is composed of.

In this case, the localization of the activities is still performed inparallel, but a synchronization point is introduced before the execu-tion of the collective task. Although it is not explicitly modelled inthe diagram, the synchronization point has a capacity equal to therobot request ri of the corresponding collective activity τcollectivei .With this structure, we model the fact that a certain number of robotis required at the same time to tackle a given collective activity.

We should mention that the proposed approach presents some lim-itations, namely the fact that the semantics of the UML Activity Dia-grams does only allow to produce an high-level specification of themodel, without giving the possibility of performing further formalanalysis or simulations. Nevertheless, our main purpose is twofold.On one hand, with a descriptive approach, we aim to develop a for-mal specification that will capture the essential aspects of our prob-lem. On the other hand, our model has a prescriptive purpose, serv-ing as a starting point for a structured development process for ourmethods (as done in swarm engineering, cf. Section 2.1.3.1), support-ing the design of a coherent experimental setup.

50 methodology

3.2.2 Multi-robot task abstraction

In our experimental setup, a stationary collective activity (depictedas τcollectivei , cf. Figure 3.2) is represented by a circular arrange-ment of task abstraction devices (i.e. a cluster, cf. Figure 3.3). Eachcluster is able to broadcast data concerning its id i, the number of re-quests ri and its current occupation oi, in a limited surrounding area.In order to test our methods under realistic assumptions, in the simu-lation, this information transfer occurs only when the robot is withina circular range (radius: 51cm, empirically determined) around thecluster. With the real devices and within the given range, this opera-tion could be easily implemented by means of a local communicationdevice such as the Range and Bearing board by Gutiérrez et al. [2008].With the current equipment of the TAM, such cluster-to-robot com-munication can be also performed via IR communication (cf. Section2.3.2), with the constraint that the robot must be inside a TAM be-longing to the cluster. The shape of the cluster has been specificallydesigned to allow the robot navigation around it, without the pos-sibility of getting stuck, either to assess the cluster occupation or todirect to an available task.

Figure 3.3: Schematic representation of the TAM disposition in a cluster.Each square box represent a single TAM entity. The grey circle cor-responds to the circular area (r=51cm, empirically determined,in our experiments) within which the robots are able to senseinformation concerning the cluster.

In our experimental setup we are going to use 4 clusters, each onecomposed by 8 TAMs. The requests ri of the clusters will be dis-tributed as follows:


cluster tam request

1 8 7 (0.28)

2 8 5 (0.20)

3 8 8 (0.32)

4 8 5 (0.20)

total 25 (1.00)

Table 3.1: Clusters request in Uniform, Biased, Corridor experiment setups(cf. Section 4.1). The values between round brackets representeach cluster’s relative request (i.e. the request of the cluster di-vided by the total number of robots, ri/m).

3.2.3 Single-robot task abstraction

The limited sensing capabilities of the e-puck robot, however, donot allow to sense a collective activity as a whole. The only way togather information concerning the cluster is to detect one of its sta-tionary single-robot sub-task (i.e. TAM) and move close enough toit in order to trigger the information exchange. Given the technicalequipment of the TAM (Table 2.3), the simplest way to make its inter-nal state detectable by the robots is to display it through the colors ofits RGB LEDs. Figure 3.4 shows the possible internal states that can bereached by the TAM, along with the associated colors and the allowedtransitions.

AvailableDisabledstart Occupied

SenseRobotEnabled

Figure 3.4: Finite state machine representing the TAM states. The color of thestates corresponds to the actual color displayed by the TAM LEDs(Orange, Green, Red, from left to right).

At the beginning of the experiment all the TAMs are initialized inthe Disabled state. Then, a number of TAMs corresponding to therequests ri defined in Table 3.1 is enabled in each cluster, making theselected devices Available. Finally, when a robot is sensed in the TAM

by means of the light barrier, the device becomes Occupied. Clearly,in each cluster i, 8− ri cluster will remain disabled, thus inaccessibleby the robots.

We decided to represent the tasks as sporadic (i.e. not occurring pe-riodically), atomic (i.e. they cannot be suspended and later resumed)in order to test the capability of the swarm to dynamically adapt to

52 methodology

changes in configurations instead of learning periodic allocation pat-terns.

3.2.4 Robot placement

As we need to have a number of tasks greater than the number of robots(i.e. n < m), we will use 20 e-pucks randomly deployed in a prede-fined area of the environment. The initial position of the robots willbe determined by drawing the x and y coordinates of the robot from auniform distribution within the aforementioned deployment area. Itshould be noted that, given their technical specifications, the chosennumber of robots is not sufficient to perform a complete coverage ofthe environment, thus requiring the e-pucks to explore it.

3.3 methods

Unlike many methods in swarm robotics, ours are not inspired byany natural phenomenon. Instead, simplicity was the principle thatguided our development.

We started by focusing ourselves on the less complex (i.e. Naive)method that achieved a reasonable allocation of the robots.

Once that has been found, we incrementally built a second method(Probabilistic) upon the basic one, trying to devise specific measuresto overcome its limitations.

Lastly, we tried to further improve the second method by takinginto account navigation information (Informed).

All the methods are based on the sense, think, act paradigm. At eachsimulation step, the robot first collects the data gathered through thesensors, then, according to its internal controller, determine the valuesto transmit to the actuators. Here, for the sake of clarity, we focus onlyon the think phase, giving a brief description of the robot controllers,assuming that the sense and act steps will be automatically performedby each agent, in the aforementioned order.

Among the different behavior-based developing approaches describedin Section 2.1.3.1, we decided to use the probabilistic finite state ma-chine one. Finite state machines allow to decouple a complex prob-lem in several easier sub-problems, which can be tackled separatelyand independently of each other. Furthermore, they provide an el-egant and clear representation of the robot controller, in terms ofinternal states of the robot and transitions, based on both the stateand the currently sensed values of the robot.

In order to operate correctly, all the methods require the use of aminimal set of sensors and actuators: the wheels to make the robotsmove in the environment, the omni-directional camera to localize thetasks, the proximity sensors to perform obstacle avoidance and therange and bearing board to receive the informations transmitted by

3.3 methods 53

the cluster. In addition, the Informed method requires the use of theencoder sensors on the wheels, as described in Section 3.3.4.

3.3.1 Overview

Explorationstart

Assessingcluster

Allocation

Decision

Sense task

Within informationsensing range

Req

uest

sati

sfied

∨N

otal

loca

te

Allocate

Figure 3.5: Finite state machine representing the high level behavior of asingle agent. This high-level behavior constitutes the backboneupon which the Naive, Improved and Informed methods are built.

“The function ofgood software is tomake the complexappear to be simple.”–Grady Booch

Figure 3.5 summarizes the high-level behavior implemented in ourmethods. Such finite state machine is implemented in each robot, thatacts independently of any other agent.

All the robots are initially placed in a common deployment area,where they start their exploration of the environment. As soon asone of them visually detects (i.e sense) a collective task by means ofits camera, the robot directs itself towards the corresponding clusterin order to collect information, thus performing a cluster assessment.When the information has been gathered, the robot enters the decisionphase. In the case of a positive outcome of the decision phase, therobot decides to allocate itself to the task, thus completing its duty.Otherwise, the robot goes back to the exploration phase to find anotheravailable activity to perform.

As we discussed in Section 2.2, our problem can be decoupled intotwo sub-problems: task localization and task allocation.

54 methodology

The task localization problem is addressed in the Exploration state,whereas the task allocation is performed in the Assessing cluster, Deci-sion and Allocation states.

Instead of implementing collective navigation techniques, such asarea coverage or chain formation, we chose to perform the Explorationphase with a random walk.

Our choice was made taking into account the advantages offeredby such a method: its simplicity, its minimal requirements in termsof sensors and computational resources and the absence of bias to-wards any preferred directions. Also, according to Oyekan and Hu[2013], a random walk model for robot displacement is sufficient toachieve environmental exploration and static target detection in anenvironment. The Naive and Probabilistic methods implement an un-informed version of a correlated random walk, while the Informedone makes use of odometric information to guide the exploration.

The decision mechanism for the allocation is the key feature thatdistinguishes the different methods.

The Naive method applies a greedy allocation rule: as soon as anavailable task has been detected, the robot tries to allocate to it.

On the other hand, the Probabilistic and Informed methods intro-duce an actual probabilistic decision phase, prior to the allocation.

Every time, during the assessment phase, that a change in the cur-rent cluster occupation is sensed, a stochastic decision mechanism istriggered. The robot decides whether to leave or not the cluster witha probability equal to the cluster’s current relative occupation (i.e.oi(t)ri

). Through this simple decision rule, we would like to prevent aconcentration of the robots on a single cluster and stimulate a moreuniform allocation across the other collective activities.

In addition to this probabilistic rule, in every method, the decisionto leave the cluster is taken anytime a robot detects that the clustercurrent occupation equals the request (i.e. oi(t) = ri for the clus-ter i being assessed) or, in other words, when the cluster request issatisfied.

Whenever the decision to leave is taken, the robot enters a tempo-rary blind state (not depicted on the state machine in Figure 3.5). Therobot remains in this state for 100 time steps, during which it ignoresthe readings coming from the camera. This simple mechanism hasbeen developed to prevent a robot from being attracted by a clusterthat it has just decided to leave, without resorting to more sophisti-cated solutions (e.g. odometry).

3.3 methods 55

3.3.2 Naive

LinearWalk

start RandomTurn

Directingto TAM

AssessCluster

In TAM

PerformTask

Obstacle

Available ∨

OccupiedTAM

Turn end

LostTAM

db < din ∧

AvailableTAM

db < das∧ Occu-pied TAM

Available TAM

oi = riOccupiedTAM

Figure 3.6: Deterministic finite state machine corresponding to the imple-mented individual robot controller for the Naive method. Thered circles (i.e In TAM, Perform Task) mark the states in which therobot has lit up the red LEDs on its body.

Figure 3.6 provides a detailed description of the robot controller forthe Naive state, based on the high level structure depicted in Figure3.5.

Here, the uninformed random walk is performed by combining twosimple behaviors: Linear walk and Random turn. Linear walk consistsof letting the robot move in a straight line, i.e. actuating the samespeed on both the wheels. As soon as the proximity sensors detectthe presence of an obstacle (within a range of 15 cm), the robot entersthe Random turn state. A random turn is done by first choosing a rota-tion direction (clockwise or counterclockwise) and then performing apivot turn for a random number of time steps.

The robot is initialized in the Linear walk state, but whenever asingle-robot task (either available or occupied) is detected, the robotenters the Directing to TAM state and starts moving in the directionof the corresponding TAM. As explained in Section 3.2.3, the robotis able to determine the state of a task by perceiving the color of thecorresponding TAM with its omni-directional camera within a rangeof 50 cm. In our simulation, the resulting readings from the omni-directional camera are colored points, characterized by their color,their distance and angle with respect to the direction where the robotis heading. Thanks to this information, the robot can easily rotate andhead in the same direction as the sensed TAM which could be eitheravailable or already occupied.

Since the Naive method has a greedy allocation policy, whenever anabstracted task is available, the robot directs towards it, avoiding anyfurther exploration. When the distance of the color point db it per-

56 methodology

ceives is smaller than the TAM depth (din=10.83 cm), the e-puck lightsup the red LEDs on its body to prevent other robots from allocating tothe same task and enters the In TAM state, stopping inside the TAM.Once the TAM has detected the presence of the e-puck by means of itslight barrier, it signals the change in its internal state (from Availableto Occupied) by changing the color of its RGB LEDs to red. This oper-ation consists indeed, in the abstraction of the activity that the e-puckshould perform. In response to this state transition, the robot movesfrom the In TAM state to the the final state Perform Task. From thee-puck point of view, in our experimental setup, the abstraction of atask consists of remaining idle inside the TAM.

On the other hand, if the sensed TAM is occupied, while being inthe Directing to TAM state, the e-puck moves in the direction of thecorresponding cluster to be able to assess its occupation. When therobot arrives closer to the TAM than the assessing distance threshold(i.e. when the distance of the perceived color point db is smaller thandas = 25cm), the robot’s internal state changes to Assessing Cluster.The assessment phase has two possible outcomes: either there are stilltasks available in the cluster or the cluster request has been satisfied.

In the case of the presence of available tasks, the robot starts nav-igating around the cluster, performing a circular motion, remaininginside the assessing range, until it perceives a green color point, corre-sponding to the available task. Then it moves to the Directing to TAMstate, eventually entering the TAM as described previously.

Otherwise, if the cluster request have been satisfied (i.e. oi(t) = ri),the e-puck performs a random turn and restarts the exploration of theenvironment, by moving back to the Linear Walk state.

3.3 methods 57

3.3.3 Probabilistic

LinearWalk

start RandomTurn

Directingto TAM

AssessCluster

In TAM

PerformTask

Obstacle

Available ∨

OccupiedTAM

Turn end

Stalemate

∨Lost

TAM

db < din ∧

AvailableTAM


Allocate

Not al-locate ∨

Stalemate∨ oi = ri

OccupiedTAM

Figure 3.7: Probabilistic finite state machine corresponding to the imple-mented individual robot controller for the Probabilistic method.The red circles (i.e In TAM, Perform Task) mark the states in whichthe robot has lit up the red LEDs on its body. Allocate and NotAllocate transitions represent the two possible outcomes of theprobabilistic decision.

Since the Probabilistic method (Figure 3.7) is incrementally builtupon the Naive one, the general dynamics of the controller is simi-lar to what has been explained in the Naive section. The behaviorof the robot is identical to that of the Naive method in the In TAM,Perform Task, Linear walk and Random turn states.

The exploration technique is indeed an uninformed random walk.The allocation rule, on the other hand, is modified in order to ad-

dress the two main issues of the Naive approach: the occurrence of astalemate and the lack of an allocation rule to obtain a more uniformtask allocation.

A stalemate may occur whenever two robots decide to allocate them-selves to the same task. In that case, both the robots will start movingtowards the task. If the robots would arrive simultaneously closeenough (around 30cm) to the TAM, they will start trying to avoid eachother, instead of entering the task abstraction device. Since there isno direct communication among the robots, there is no possibility ofan explicit agreement on which robot should perform the task.

Our solution for this issue has been developed by implementing acounter which is started once the robot enters the Directing to TAMor Assessing Cluster states and incremented each simulation time step.When the counter surpasses a predefined threshold (100 time steps,in our implementation), the robots decide stochastically whether towait or leave the cluster. The probability to leave (pl), by performinga random turn and starting a blind exploration, is equal to 0.01. Thedecision process is then repeated at each time step, until either one of

58 methodology

the robots decides to leave before the other, thus allowing the remain-ing one to allocate itself to the task, or both will leave the cluster.

On the other hand, the purely greedy allocation rule of the Naivemethod is integrated with a probabilistic decision mechanism.

In the Probabilistic method, a robot in the Linear walk state still headstowards a cluster as soon as it has sensed a TAM belonging to it,regardless of its state, but behaves differently during the assessmentphase.

In fact, if the perceived task is directly available, the robot will im-mediately try to allocate itself to it.

Otherwise, the robot will enter the Assessing Cluster state and theprobabilistic decision will be triggered. Here, the robot will eitherdecide to allocate or to not allocate.

In the first case, in the same way as in the Naive method, the robotwill turn around the cluster until the first available task is found.

In the second one, the robot will move to the Random turn state,perform a random turn and start a temporary blind exploration.

Moreover, the decision phase will occur whenever a change in theoccupation of the currently assessed cluster is sensed.

The idea of introducing a stochastic component in the decision rulearises from the objective of achieving a uniform allocation of therobots across the clusters. As a matter of fact, allowing the robotsto move from already crowded clusters to those that are still almostempty would balance the occupation among them. With this idea inmind, we tried to devise a new allocation rule.

A deterministic rule has been immediately discarded since it lacksof flexibility with respect to differences in the size of the cluster or thenumber of robots. If the rule would have been based on an absoluteoccupation threshold (e.g. leave the cluster if oi(t) > Xi), it wouldhave required a global knowledge of the environment, in order to de-termine a priori the optimal values to achieve a uniform allocation.With a deterministic rule based on relative occupation (e.g. leave thecluster if oi(t)ri

> Xi), in addition to the global knowledge require-ment, once the threshold would have been met in all the clusters, itwould not have been possible to allocate the remaining robots.

Thus, we decided to implement a probabilistic rule based on a sim-ple intuition: limiting the allocation of robots to cluster whose occupa-tion is already high. In order to do so, we propose an abandon proba-bility, computed every time that a robot enters the decision phase. Theabandon probability ai for a certain robot assessing cluster i, at timet is defined as:

ai(t) =oi(t)

ri(3.2)

The occupation oi is normalized by the cluster request in order toobtain a value in the [0, 1] range.

3.3 methods 59

It should be noted that: the higher the number of robots currentlybeing allocated to tasks belonging to the cluster, the higher the likeli-hood of leaving the cluster.

The advantages of this allocation rule is that it is completely dis-tributed, flexible with respect to the different requests values of theclusters and with minimal requirements in terms of communication(i.e. only the information on the occupation oi(t) must be transferredto the robot) and computational capabilities.

3.3.4 Informed

LinearWalk

start RandomTurn

Directingto TAM

AssessCluster

In TAM

PerformTask

Obstacle ∨

Odometry

Available ∨

OccupiedTAM

Turn end

Stalemate

∨Lost

TAM

db < din ∧

AvailableTAM


Allocate

Not al-locate ∨

Stalemate∨ oi = ri

OccupiedTAM

Figure 3.8: Probabilistic finite state machine corresponding to the imple-mented individual robot controller for the Informed method.Thered circles (i.e In TAM, Perform Task) mark the states in which therobot has lit up the red LEDs on its body.Allocate and Not Allocatetransitions represent the two possible outcomes of the probabilis-tic decision.

The Informed method is built upon the Probabilistic one, hence thebehavior of the robots in all the states is the same as described inSection 3.3.3, including the mechanism to solve the stalemate issueand the probabilistic allocation rule. With the uniform allocation prob-lem being tackled with the probabilistic rule introduced in Equation3.2, the only difference with the previous method is how the explo-ration is performed. While visualizing the first experiments with theProbabilistic method, we discovered a minor issue in the explorationphase. In fact, every time a robot decides to leave a cluster, after hav-ing completed the blind exploration phase, there is no guarantee thatit will actually direct towards another cluster without coming back tothe previously left one. Indeed, a robot may assess several times thecluster it has just left due to the lack of available tasks, thus havinga redundant and inefficient behavior. This could happen as a conse-quence of the obstacle avoidance with respect to other robots or thearena walls. Given the sensory equipment of the e-puck, we decided

60 methodology

to propose a solution to this issue using odometry. Odometry is atechnique to estimate the change of position of the robot with respectto a known position, using moving sensors. In our solution, as soonas a robot decides to leave a cluster, it starts keeping track of the po-sition of the cluster p. This relative localization with respect to therobot is then updated at each time step with the information comingfrom the sensors.

A detailedexplanation of the

model can be foundin Lucas [2001]

Odometry is required since the displacement and the rotation of therobot cause its reference frame to move and rotate accordingly. Thus,if a position of a fixed point is not translated into the new referenceframe, its exact position could not be tracked anymore. Through thereadings of the encoder sensors mounted on the wheels it is possibleto determine the distance traveled by each wheel (dl and dr, for theleft and right wheel, respectively) of the robot during each simulationstep. Since the development of our methods has been performed insimulation, there is no error in the readings coming from the sensors.However, the readings from the sensors on real robots are perturbedby noise. Here, we propose to model the noise as an additive gaussiannoise with mean 0 and standard deviation 0.2, where the values ofthe parameters of the distribution have been empirically determined.Thanks to these values, knowing the inter-wheel distance of the robotdiw, it is possible to estimate the displacement dd and rotation ϑd ofthe robots:

dd =dl + dr2

ϑd =dl − drdiw

(3.3)

The displacement magnitude and angle can be combined to form adisplacement vector:The vectors are

expressed in polarcoordinates, in the

form(magnitude,angle)

or, equally (r, ϑ).d =

[dd

ϑd

](3.4)

The vector d is then used to perform the roto-translation requiredto correctly update the stored position of the previous cluster. Let pi

the estimate of the position of the previous cluster at time step i:

pt+1 =

[cos(ϑd) − sin(ϑd)

sin(ϑd) cos(ϑd)

]· (pt + d) =

[dp

ϑp

](3.5)

By performing the sequence of operations described in Equations3.3, 3.4, 3.5 at each time step it is possible to maintain a reasonableestimate of the position of the most recently left cluster. We speak of

3.3 methods 61

"a reasonable estimate" since the presence of the error in the readingsdoes not allow to precisely track the position p of the cluster.

This information is used in the Linear walk state to perform an in-formed random walk. In fact, whenever the cluster estimated orien-tation ϑp is included in the range (h − π

6 , h + π6 ), with h being the

direction in which the robot is currently heading (i.e. the directionorthogonal to the interwheel axis), and the cluster estimated distancedp is smaller than two times the omnidirectional camera range (i.e.1m), the robot performs a change of direction (as in Figure 3.9a).Conversely, the robot keeps moving in a straight line if the currentestimate pt is outside the aforementioned range (as in Figure 3.9b).

h

2cr

h

π6

π6

pt

(a) Change of direction triggered

h

2cr

h

π6

π6

pt

(b) Change of direction not triggered

Figure 3.9: Schematic representation of the Informed odometry-based clusteravoidance procedure. The dashed arrow h represents the direc-tion in which the robot camera is heading, while pt is the cur-rent estimate of the cluster position. The grey area represents therange within which the change of direction is triggered, with crbeing the omnidirectional camera range.

The change of direction (marked with Odometry in Figure 3.8) isimplemented by changing the state of the robot to Random Turn.

We believe that, even though our solution could not completelyprofit from the advantages brought by the use of an exact odometry,the increased occurrence of direction changes due to this mechanismwill result in a better exploration of the environment and possibly, ina more even distribution of the robots across clusters.

3.3.5 Summary

Our starting point was the problem of uniformly allocating robotsto spatially distributed collective tasks (cf. Problem Statement 2)

We clarified this statement by fixing constraints on the number ofagents (20), the number of tasks (25) and the arrangement of the tasks(4 clusters).

From this statement, we devised a formal specification (i.e. UML Ac-tivity Diagram) of the problem, that led us to the design of a physicalimplementation of the experiment by means of the TAMs and theirarrangement to form collective activities (Figure 3.3).

62 methodology

After having defined a concrete setup for the problem, we beganthe development of our methods by means of an iterative manual,bottom-up process based on simulations.

We developed three robot controllers, presented here as finite statemachines, each of which tackles separately the two sub-problems thatcharacterizes our definition of the problem : exploration and allocation.A summary of the relevant features of the solutions we implementedin our methods to tackle these problem is presented in Table 3.2.

In Chapter 4 we present the results we obtained by implementingthe same controller on all the robots in the swarm and launchingsimulations of 1000 s (10000 time steps) each.

Probabilisticstalemate

prevention rule:After the 100th time

step spent indirecting state,

decide every timestep whether to leave

with probabilityp = 0.01.

method exploration decision rule (at time t∗ )

Naive Uninformedrandom walk

Greedy.Leave if at time t∗, ri(t∗) =oi(t

∗).

Probabilistic Uninformedrandom walk

Probabilistic withabandon probabilityai(t

∗) =oi(t

∗)ri(t∗)

.Probabilistic stalemate pre-vention rule.Leave if at time t∗, ri(t∗) =oi(t

∗).

Informed Informed ran-dom walkusing odome-try

Probabilistic withabandon probabilityai(t

∗) =oi(t

∗)ri(t∗)

.Probabilistic stalemate pre-vention rule.Leave if at time t∗, ri(t∗) =oi(t

∗)

Table 3.2: Overview of the developed methods.

4R E S U LT S

Chapter 4 is dedicated to the discussion of the results we obtainedthrough the application of our methods to the experimental setuppresented in Section 3.2.

We start by describing the different scenarios (Section 4.1) that char-acterizes our experimental setup: scenario Uniform (Section 4.1.1), Bi-ased (Section 4.1.2), Corridor (Section 4.1.3).

Then, we present the metrics (Section 4.2) that we have devised toevaluate the two most important properties of our methods: allocationuniformity, allocation speed.

The core of the chapter is represented by Sections 4.3 and 4.4, de-voted to a thorough analysis of the aforementioned properties.

We conclude by evaluating the difficulty of the proposed scenarios(Section 4.5).

4.1 scenarios

The problem statement (cf. Problem Statement 2) gives us severaldegrees of freedom in the definition of an experimental setup. In Sec-tion 3.2, we clarified the number of robots, the number of tasks andthe presence of collective activities (i.e. clusters). However, the dis-tribution of such task aggregates in space and the initial distributionof robots was not defined. By varying these two aspects, we devisedthree different scenarios. The purpose of this choice is to test theperformances of the methods in different environments, in order todetermine the weaknesses and the strong points of each method.

4.1.1 Uniform

The notion ofdifficulty of ascenario will beclarified in Section4.5.

The scenario Uniform (Figure 4.1) is characterized by having thedeployment area in the center of the arena, thus being equally distantfrom all the clusters. This is the first scenario we developed and thesimplest one, which will serve as a baseline for comparison with theother ones.

4.1.2 Biased

In the scenario Biased (Figure 4.2), the deployment area is movedin the bottom left corner of the arena and cluster 2 is placed nearthe opposite corner of the environment. This results in having threeclusters closer to the deployment site and the fourth one far away. We

63

64 results

12

3 4

Figure 4.1: Representation of the cluster disposition in scenario Uniform.Clusters are represented by circles, while the dashed square in-dicates the robot deployment area.The arena size is 4m x 4m, while the deployment area is 1m x1m.

1

2

3

4

Figure 4.2: Representation of the cluster disposition in scenario Biased.Clusters are represented by circles, while the dashed square in-dicates the robot deployment area.The arena size is 4m x 4m, while the deployment area is 1m x1m.

4.1 scenarios 65

1

2

3

4

Figure 4.3: Representation of the cluster disposition in scenario Corridor.Clusters are represented by circles, while the dashed square in-dicates the robot deployment area.The arena size is 2m x 5m, while the deployment area is 1m x1m.

introduced this bias in the environment in order to test the influenceon the methods’ performance of a non-uniform positioning of theclusters.

4.1.3 Corridor

In scenario Corridor (Figure 4.3) we placed the clusters in a nar-row rectangular arena, closely resembling a corridor. The clustersare equally spaced among them and the deployment area is locatedat one edge of the corridor. The motivation of the scenario Corridoris to test the impact of inter-robot interference on the performanceof the methods. In fact, the environment presents a limited spacearound the clusters, due to the proximity of the walls and the clusterthemselves, which will cause the robots to aggregate and potentiallyinterfere with each other.

66 results

scenario

uniform biased corridor

arena dimensions [m2] 4×4 4×4 2×5

number of robots 20 20 20

experiment duration [s] 1000 1000 1000

experiment repetitions 50 50 50

number of clusters 4 4 4

deployment area size [m2] 1×1 1×1 1×1

cluster 1 location (x, y) [m] (-1,1) (1,1) (0, 2)

cluster 2 location (x, y) [m] (1,1) (-1.3,1.3) (0, 1)

cluster 3 location (x, y) [m] (-1,-1) (-1,1) (0, 0)

cluster 4 location (x, y) [m] (1,-1) (0,0) (0, -1)

Table 4.1: Summary of the parameters describing the scenarios used in theexperiment. The origin of the x-y coordinate system is in the cen-ter of the arena. Each second of the simulation corresponds to 10

time steps.

4.2 metrics

4.2.1 Allocated robots

Since we are dealing with methods to allocate robots to tasks, themost intuitive aspects we can analyze are, respectively: the numberof robots that could be actually allocated and the time required toachieve such allocation. By indicating with si(t) the internal state ofrobot i at time t, we can define the number of allocated robot at eachtime step t as:

R(t) =

n=20∑i=1

I[si(t) = Perform Task] (4.1)

where

I[si(t) = Perform Task] =

1 si(t) = Perform Task

0 si(t) 6= Perform Task(4.2)

We analyze this metric under two distinct points of view. On onehand, we look at the number of robots allocated at each time step, an-alyzing, for instance, the maximum and minimum number of robotsallocated at a given time step t across a set of trials. On the otherhand, by looking at the evolution in time of this allocation function,we can easily compare the performances of the methods.

4.2 metrics 67

4.2.2 Maximum error

In Section 3.1, we defined the allocation error ei(t) for each clusteras the difference between the desired number of robots in the cluster(i.e. the request ri) and the current number of robots in the cluster(i.e. the occupation oi(t)). This measure easily allow to understandand visualize how well the robots are allocated in a certain cluster.

Our goal, on the other hand, is to distribute the robots uniformlyacross the clusters, hence this value is of a little interest to us. Never-theless, by computing an aggregate measure from all the clusters, wecould have a better view on the overall allocation.

Hence, we compute the maximum error across clusters:

emax(t) = maxi∈1,··· ,c=4

ei(t) = maxi∈1,··· ,c=4

(ri − oi(t)) (4.3)

Indeed, the value of emax(t) corresponds to the greatest differencebetween request and occupation in all the clusters. Moreover, a uni-form allocation of the robots is the one that better distributes therobot across the clusters. This entails that, in case of a uniform allo-cation, the error ei(t) in each cluster will not be null (due to the factthat the number of tasks is greater than the number of robots), butthe maximum error will be reduced, since the fair distribution of therobot will reduce the differences ri − oi(t) in each cluster.

The notion of fair distribution is applied to determine the lowerbound for this metric, the optimal allocation error eopt:

eopt = maxi∈1,··· ,c=4

(ri − round

( rim·n))

(4.4)

where

round(x) =

⌊x+

1

2

⌋(4.5)

Here, m corresponds to the global number of single-robot tasks, nis the number of robots and ri is the robot request for the collectiveactivity i. The optimal allocation error eopt corresponds to the valueof emax(t) that would be obtained by allocating to each collective ac-tivity a fraction of the total number of robots n corresponding to thatactivity’s relative request ri/m. The round(x) function is requiredin order to obtain an integer result, since a decimal one would bemeaningless while dealing with physical robots.

4.2.3 Allocation levels

Another way to measure how evenly are the robots distributedacross clusters is to determine if certain allocation levels are reached.The concept of allocation level is defined with respect to the relative

68 results

occupation oi(t)ri

of each cluster. As a matter of fact, if the relative oc-cupation of all the clusters is greater than a certain value l, we couldsay that the allocation level l has been reached. Whenever this occurs,we are also able to determine the time step to(l) at which the level lis attained:∧

i corresponds tothe logical AND

operation extendedto every cluster i.

to(l) = arg mint

(c=4∧i=1

[oi(t)

ri> l

])l ∈ 0.25, 0.50 (4.6)

to(l) corresponds to the first time step at which the relative allocationof all the clusters is above the desired threshold l.

The values of l are selected in order to have representative andfeasible measures for the cluster allocation. It should be noted that,given the previous definition of fair allocation, it is required to attainat least a relative occupation of all the clusters greater or equal than0.5, in order to achieve the optimal allocation error emax. Consideringthe distribution of requests ri and the optimal allocation error eopt inour experimental setup, the value of 0.75 has been discarded since therelative occupations are guaranteed to be above the aforementionedthreshold in every cluster only in the case of an optimal allocation,which is not reached in our experiments (cf. Table 4.2).

For every batch of k experiments we run, we measured the valuesof tio(l) on each single trial i, and then aggregated the values to buildan empirical cumulative distribution function. To be more precise:

Fk(x, l) = Pk[to(l) 6 x] =1

k

k∑i=1

I[tio(l) 6 x] l ∈ 0.25, 0.50 (4.7)

where

I[tio(l) 6 x] =

1 tio(l) 6 x

0 tio(l) > x(4.8)

Basically, over the batch of k experiments, we count how many timesour methods are able to achieve the allocation level lwithin x timestepsand normalize with respect to the total number of trials k. Our goalis to assess how strong is the impact of the stochastic components ofour methods on their performance and how well our approaches areable to compensate these random differences.

4.2.4 Cluster views

In Section 4.1 we presented the developed scenarios and the moti-vations behind their structure.

What we actually did, at design time, was formulating hypothesison how the structure of the environment could potentially affect theperformances of the methods.

4.3 allocation uniformity 69

In order to test our assumptions, we decided to measure the num-ber of times that cluster i has been seen at time t:

vi(t) =

t∑s=0

n=20∑k=1

I[dik(s) 6 cr] (4.9)

where

I[dik(s) 6 cr] =

1 dik(s) 6 cr

0 dik(s) > cr

(4.10)

Here, dik(s) represents the distance of robot k from cluster i at time The cluster-robotdistance is measuredfrom the robot to theclosest TAMbelonging to clusteri.

s, while cr is the range of the omni-directional camera (50 cm in ourexperiments). Through the analysis of the number of visits, we expectto assess whether the number of visits to a certain cluster is biased bythe nature of the scenario and whether the differences in the methodshave an impact on the exploration of the environment.

4.3 allocation uniformity

The main goal of our methods is to spread the robots evenly acrossthe spatially distributed collective activities.

In order to achieve this result, we improved the Naive method by in-troducing probabilistic mechanisms (Probabilistic) and informed deci-sions (Informed) to favor the redistribution of the robots from crowdedclusters to the empty ones.

Here, we analyze the performances of the three methods, by focus-ing on the final allocation achieved by each of them on a set of 50simulations. Each simulation is characterized by a seed value, whichwill be used to initialize the simulator’s internal random number gen-erator, thus influencing the stochastic behavior of the method. As amatter of fact, the seed will influence the initial robot placement, theirorientation and all the probabilistic decisions made by the methods.We run each simulation for 1000 s, corresponding to 10000 simulatedtime steps, using the same set of 50 seeds for all the different methods,on the same scenario.

In order to evaluate the final allocation, we compute the maximumerror across clusters emax(t) and the number of allocated robots R(t)for t = 10000. Instead of presenting the whole distribution of values,we decided to aggregate the values using non-parametric statistics,namely median value and interquartile range. Moreover, since weare dealing with integer values (e.g. number of tasks or robots) thecomputation of some statistics (e.g. mean) could result in decimalvalues having no real, physical meaning.

The results are shown in Tables 4.2 and 4.3.

70 results

4.3.1 Maximum error

method scenario


Median (q25, q75) Median (q25, q75) Median (q25, q75)

naive 3 (3,4) 4 (4,5) 6 (6,6)

probabilistic 3 (3,3) 3 (3,4) 5 (4,5)

informed 3 (3,3) 3 (3,3) 5 (4,5)

Table 4.2: Summary of the values of the maximum allocation error emax(t)

at t = 10000 for a swarm of 20 e-pucks with 25 available tasks.Median values and corresponding inter-quartile ranges are com-puted across 50 trials of 10000 time steps each.The optimal allocation error eopt is equal to 2 for all the scenarios.

Table 4.2 reports the median maximum error across the clusters atthe end of the experiment.

The inter-quartile range is included in order to add informationconcerning the dispersion of the values (by definition, half of thevalues of the sample are included in the inter-quartile range).

At a first glance we can observe that no method is able to achievethe optimal allocation error eopt of 2 tasks, but on scenario Uniformand scenario Biased the Probabilistic and Informed methods are able toreach an error of 3 tasks, with no variability.

Furthermore, by comparing the methods we are able to see thatthe enhanced methods (i.e. Probabilistic and Informed) have similarperformances and both clearly outperform the Naive one.

Also, by looking at the different scenarios (in the order they appearin Table 4.2), we can see an increase in the values of the medianmaximum error and in its variability, indicating a growing complexityof the environments.

4.3 allocation uniformity 71

4.3.2 Allocated robots

method scenario


Median Range Median Range Median Range

naive 20 (12,20) 20 (9,20) 19 (6,20)

probabilistic 20 (20,20) 20 (20,20) 20 (18,20)

informed 20 (20,20) 20 (20,20) 20 (18,20)

Table 4.3: Summary of the values of the number of allocated robots R(t) att = 10000 for a swarm of 20 e-pucks with 25 available tasks.Median values and corresponding ranges (min,max) are com-puted across 50 trials of 10000 time steps each.

Table 4.3 offers a complementary vision on the final allocation, dis-playing the median number of allocated robots.

Here, instead of focusing on the inter-quartile range, the wholerange of the 50 sampled values is presented, in order to verify whetherall the methods are able to allocate all the robots or some problemsarise.

As for the median values, all the methods are able to achieve thecomplete allocation of the robot, except the Naive method on scenarioCorridor.

The same trend described in Section 4.3.1 can be observed here:there is a clear difference between the Probabilistic and Informed meth-ods’ performance and the Naive one.

The remarkably low minimum number of allocated robots of theNaive method can be explained through the absence of the stalematecheck. In fact, since no scenario presents obstacles or considerablynarrow passages that could cause the robots to get stuck in them, wecould safely suppose that after a sufficiently long time all the robotswill eventually be allocated. Moreover, the same figures for the min-imum numbers of allocated robots are not present in the Probabilisticand Informed method, where the stalemate condition is used.

Another view of the data concerning the number of allocated robotsR(t) can be found in the annex, Section A.2.

72 results

4.3.3 Allocation levels

method scenario


level 0.25 0.50 0.25 0.50 0.25 0.50

naive 0.900 0.660 0.240 0.020 0.200 0.020

probabilistic 1.000 0.800 0.800 0.440 0.800 0.040

informed 1.000 0.740 0.900 0.680 0.800 0.060

Table 4.4: Summary of the cumulative probabilities to reach allocation levels0.25 and 0.5 across 50 trials within 10000 time steps for a swarmof 20 e-pucks with 25 available tasks.

Table 4.4 summarizes the cumulative probabilities (cf. Section 4.2.3)to reach the allocation levels 25% and 50% within the chosen experi-ment duration (10000 time steps). The analysis of these results allowsus to highlight the limitations of the proposed methods. In fact, nomethod is able to achieve high cumulative probabilities for the allo-cation level 50% (necessary condition for an optimal allocation), onall the scenarios. Nevertheless, the Informed and Probabilistic methodsare able to successfully achieve (within 10000 time steps) the 25% al-location level on all the trials, and the 50% level in a relevant numberof simulations.

The Naive method also has fairly good performances on the sce-nario Uniform, but the cumulative probabilities have a significant dropon scenarios Biased and Corridor. We suppose that this variation is de-termined by the combination of the biased placement of the clustersacross the scenario Biased and the greedy allocation rule of the Naiveexperiment, which hinder the possibility of having a uniform alloca-tion. Another possible explanation for the performances of the Naivemethod on scenario Corridor, especially for the allocation level 25%,can be given by looking at Table 4.3. In case Naive-scenario Corridorwe can observe a high variability on the number of allocated robots,reaching a minimal value of 6 units. By referring to Table 3.1, wecan observe that this value is smaller than the number of requests ofa single cluster, thus making the likelihood of a uniform allocationacross the clusters really low, using a greedy approach.

It should be noted that the performances of the other two methodsare also affected by the change in scenarios. However, the impactof this change is less dramatic than the Naive one, with the Corridorstill being the most difficult to tackle (i.e. the one having the smallestvalues for F50(10000, 0.25) and F50(10000, 0.5) respectively, the cumu-lative probabilities of to(0.25) and to(0.5) within 10000 time steps).

4.4 allocation speed 73

Again, by comparing the performances of the different methods,we notice the similarity between the Probabilistic and Informed meth-ods’ results. The only remarkable difference is the higher value forthe cumulative probability of reaching 50% allocation in scenario Bi-ased. This result can be related to the use of the odometry in a rea-sonably large environment, as the one depicted in Figure 4.2. Thewider nature of this environment limits the number of encountersamong the robots, after the initial deployment phase. Consideringour implementation of the random walk in both the Probabilistic andInformed methods, less encounters between the robots implies a re-duced number of random turns, which in turn limits the explorationof the environment. Due to the odometry-based change of direction,to avoid returning to already visited clusters, the Informed methodpresents a higher probability to make random turns, thus it shouldbetter explore the environment and potentially achieve a more uni-form distribution.

4.4 allocation speed

Section 4.3 has been devoted to the evaluation of the uniformity ofthe allocation across clusters, mainly through the analysis of the finalresults achieved by the methods.

Another interesting point-of-view on our data can be given by an-alyzing how the methods have achieved these results, instead of re-straining ourselves to the final values of the proposed metrics.

The simulation setup is the same as the previous section, 50 simula-tions of 10000 time steps (1000 s) of all the methods on each scenario,using the same set of seeds on each scenario to maintain the sameinitial conditions for all the methods.

We try to gather information on the speed of the allocation by look-ing at the evolution of the maximum error across clusters emax(t) andthe number of allocated robots R(t) for t ∈ 1, · · · , 10000. The twometrics allow us to focus on both the uniformity aspect and the alloca-tion velocity one. On one hand, by analyzing the evolution of emax(t)

we can understand how much time do the methods require to bringthe overall error across clusters below a certain threshold, explainingalso how uniformly are the robots distributed across clusters at a cer-tain moment in time. On the other hand, R(t) captures how manyrobots are allocated to a task at a certain time step, making no dis-tinction among clusters. Through the comparison of the number ofallocated robots for the different methods, we are able to evaluatehow fast is each method in assigning the robots to the tasks. In ad-dition, we analyze the empirical cumulative distribution functions ofthe times needed to reach the allocation levels 25% and 50% (to(0.25)and to(0.5), respectively) in order to have a different standpoint onthe uniformity of the allocation.

74 results

As for the emax(t) and R(t), side by side comparisons of the plotsincluding all the quantiles and the mean values can be found in theannex, Chapter A.

4.4.1 Scenario Uniform

By looking at Figure 4.4, we can see that, considering the mediannumber of allocated robots, Naive method is able to achieve the fastestallocation in scenario Uniform (Figure 4.4b), with all the 20 robotssuccessfully performing a task in less than 1500 time steps. Up to500 time steps, the Probabilistic and Informed curves are paired buteventually, the Probabilistic method achieves a faster allocation thanthe Informed one.

However, the fastest allocation is not necessarily the fairest. As wesee in Figure 4.4a, the Naive method is also the one presenting thehighest median maximum error across clusters, while the plots of theProbabilistic and Improved methods are superposed. The analysis ofthe empirical cumulative distribution functions (Figure 4.5) confirmsthis trade-off between speed and quality of the allocation.

A greedy method (i.e. Naive) is the one ensuring the fastest al-location of the robots to the tasks, since the robots spend less timeexploring the environment. Nevertheless, given the structure of sce-nario Uniform (Figure 4.1), with a central deployment area, equallydistant from all the clusters and the uniform distribution of the robotat the beginning of the experiment, relatively high probabilities ofreaching at least the 50% of the occupation of all the clusters (i.e.P50[to(0.5) 6 10000]) can still be obtained. It should be noted that,by looking at the uniformity of the allocation, even on the simplestscenario, methods Probabilistic and Informed attain a smaller value ofemax(t) faster than the Naive one, but finally, all the curves convergeto the value of 3.

4.4.2 Scenario Biased

Figure 4.6 gives additional evidence to support the hypothesis ofthe existence of a trade-off between speed and allocation quality.

Again, the Naive method represents the upper bound in terms ofallocation velocity while being the lower bound in terms of maximummedian allocation error across cluster.

By looking at Figure 4.6a and 4.6b, no remarkable differences canbe seen concerning the performances of Probabilistic and Informedmethods.

With respect to scenario Uniform (Figure 4.4), we can observe that,in scenario Biased, more time is required to reach the same levelsof robot allocation error. We explain this shift in the curves withthe differences in the nature of the environment, more precisely the


placement of cluster 2. While in scenario Uniform it is as distant as allthe other clusters from the deployment area, in scenario Biased it isthe farthest one. As a consequence, more time is required to navigatethrough the arena to actually reach tasks belonging to the secondcluster, before eventually deciding to allocate to them.

In addition, we suppose that the bias in the positioning of the clus-ters could be a potential cause of the highest median maximum erroracross clusters emax(t) reached by the Naive method at the end of theexperiment.

In Figure 4.7, we can observe that the Naive method is able toreach the 25% and 50% allocation levels in a smaller number of trialswith respect to the other methods. Moreover, whenever those levelsare reached, the time required to reach these allocation thresholds ishigher than both the Probabilistic and Informed methods.

Thus, we can conclude that the performances of the Naive methodare definitely worse than those of the enhanced methods on scenarioBiased.

4.4.3 Scenario Corridor

Figure 4.8 highlights the relative improvements that the introduc-tion of probabilistic mechanisms and the use of odometry have broughtto the Naive method.

In Figure 4.8b we can see that the Naive method is slower than theProbabilistic and Informed one and it is not able to reach the completeallocation of the robots to the tasks. On the contrary, the enhancedmethods are able to attain this objective, with the Informed one beingthe fastest.

Although there is an improvement in the velocity of the alloca-tion, it should be noted that the median maximum error across clus-ters, even for the Informed method, converges to a value considerablyhigher with respect to the optimal one.

Furthermore, the time needed to achieve the complete allocationof the robots is more than four times higher than the one of scenarioBiased. Since this delay is observed in all the methods, we take it as anempirical evidence supporting our hypothesis of increasing difficultyof the scenarios.

Figure 4.9 confirms the better overall performances of the Informedmethod. In Figure 4.9a the steeper increase of the empirical cumula-tive density function shows that the 25% allocation level is reachedmore often and in less time than the other methods. While the samefinal probability value is reached by the Probabilistic method as well,here the Naive methods displays its limits. Nevertheless, Figure 4.9battests that the 50% allocation level is rarely reached by all the meth-ods, confirming the impossibility to achieve a uniform allocation inthis scenario.

76 results

0 500 1000 1500 2000 2500 3000

02

46

8

Time steps

Task

s

Naive Probabilistic Informed Optimal

(a) Median emax(t). The dashed grey line represents the optimalallocation error eopt. The differences among the curves are sta-tistically significant. (Wilcoxon signed-rank test, p « 0.05).

0 500 1000 1500 2000 2500 3000

05

1015

20

Time steps

Robo

ts

Naive Probabilistic Informed Complete

(b) Median R(t). The dashed grey line represents the total numberof robots in the experiment. The differences among the curvesare statistically significant. (Wilcoxon signed-rank test, p « 0.05)

Figure 4.4: Median values for the maximum error across clusters emax(t)

and the number of allocated robots R(t) for each of the threemethods on the scenario Uniform on 50 trials of 1000 s (10000

simulation steps) each. Each trial is performed with 20 robotsand 25 available tasks.


0 2000 4000 6000 8000 10000

0.0

0.2

0.4

0.6

0.8

1.0

Time steps

P(x)

Method

NaiveProbabilisticInformed

(a) Allocation level: 25%.The differences among the curves are statis-tically significant. (Mann-Whitney test, p « 0.05)

0 2000 4000 6000 8000 10000

0.0

0.2

0.4

0.6

0.8

1.0

Time steps

P(x)

Method


(b) Allocation level: 50%. The differences among the curves arestatistically significant. (Mann-Whitney test, p « 0.05)

Figure 4.5: Empirical cumulative density functions for the allocation levelsto(l), (l ∈ 0.25, 0.50) distributions of the three methods on thescenario Uniform on 50 trials of 1000 s (10000 simulation steps)each. Each trial is performed with 20 robots and 25 availabletasks.

78 results

0 2000 4000 6000 8000 10000

02

46

8

Time steps

Robo

ts


(a) Median emax(t). Only the differences between the Naive and Prob-abilistic and Naive and Informed curves are statistically significant.(Wilcoxon signed-rank test, p « 0.05)

0 2000 4000 6000 8000 10000

05

1015

20

Time steps

Robo

ts


(b) Median R(t). Only the differences between the Naive and Proba-bilistic and Naive and Informed curves are statistically significant.(Wilcoxon signed-rank test, p « 0.05)


and the number of allocated robots R(t) for each of the threemethods on the scenario Biased on 50 trials of 1000 s (10000 sim-ulation steps) each. Each trial is performed with 20 robots and25 available tasks.


0 2000 4000 6000 8000 10000

0.0

0.2

0.4

0.6

0.8

1.0

Time steps

P(x)

Method


(a) Allocation level: 25%. The differences among the curves arestatistically significant. (Mann-Whitney test, p « 0.05)

0 2000 4000 6000 8000 10000

0.0

0.2

0.4

0.6

0.8

1.0

Time steps

P(x)

Method


(b) Allocation level: 50%. The differences among the curves arestatistically significant. (Mann-Whitney test, p « 0.05)

Figure 4.7: Empirical cumulative density functions for the allocation levelsto(l), (l ∈ 0.25, 0.50) distributions of the three methods on thescenario Biased on 50 trials of 1000 s (10000 simulation steps)each. Each trial is performed with 20 robots and 25 availabletasks.

80 results

0 2000 4000 6000 8000 10000

02

46

8

Time steps

Robo

ts


(a) Median emax(t). The differences among the curves are statisti-cally significant. (Wilcoxon signed-rank test, p « 0.05)

0 2000 4000 6000 8000 10000

05

1015

20

Time steps

Robo

ts


(b) Median R(t). The differences among the curves are statisticallysignificant. (Wilcoxon signed-rank test, p « 0.05)


and the number of allocated robots R(t) for each of three meth-ods on the scenario Corridor on 50 trials of 1000 s (10000 simula-tion steps) each. Each trial is performed with 20 robots and 25available tasks.


0 2000 4000 6000 8000 10000

0.0

0.2

0.4

0.6

0.8

1.0

Time steps

P(x)

Method


(a) Allocation level: 25%. The differences among the curves are sta-tistically significant. (Mann-Whitney test, p « 0.05)

0 2000 4000 6000 8000 10000

0.0

0.2

0.4

0.6

0.8

1.0

Time steps

P(x)

Method


(b) Allocation level: 50%. The differences among the curves are sta-tistically significant. (Mann-Whitney test, p « 0.05)

Figure 4.9: Empirical cumulative density functions for the allocation levelsto(l), (l ∈ 0.25, 0.50) distributions of the three methods on thescenario Corridor on 50 trials of 1000 s (10000 simulation steps)each. Each trial is performed with 20 robots and 25 availabletasks.

82 results

4.5 scenario difficulty

After having discussed the properties of the different methods, wenow focus on those of the scenarios, by trying to characterize theirdifficulty with respect to the achievement of a uniform allocation.

We choose to measure this difficulty in terms of the number oftimes that the different clusters are seen by the robots, following asimple intuition: the higher the number of times a cluster is viewed,the higher the likelihood of a robot to decide to allocate itself to thecorresponding collective activity. This naturally implies that, when-ever there is a remarkable difference in the number of views of theclusters, it would be difficult to achieve a uniform allocation acrossall the activities, since there exists a bias towards one or more of theclusters.

Moreover, the distribution of the views across the clusters dependson the exploration strategy adopted by the different methods. Sincethe vi(t) metric consists of a cumulative sum of the number of viewsof the cluster, we can assess the effectiveness of the exploration tech-nique of the different methods by evaluating the magnitude of vi(t).

4.5.1 Scenario Uniform

The assumption of the simplicity of the scenario Uniform (cf. Fig-ure 4.1) we made during the design phase is confirmed by bothFigure 4.10 and Table 4.5. For all the methods, we can observe aquasi-uniform distribution of the median values of the views at timet = 10000 to the clusters. Regarding the number of times the clusterhave been seen, the Naive method is the one having the most limitedvariability across clusters. Concerning the magnitude of the numberof views, we can observe a non-negligible difference among the en-hanced methods (Probabilistic and Informed) with respect to the Naiveone.

4.5.2 Scenario Biased

The scenario Biased (cf. Figure 4.2) is characterized by having clus-ters 1, 3, 4 closer to the deployment area than cluster 2. This evidentcluster distribution bias is confirmed by the magnitude of the mediannumber of views, as shown in Table 4.6, which, for all the methods,is similar for the three closer clusters and completely different fromthat of the isolated one. The difference is clearly displayed in Figure4.11. Moreover, the Naive method has a median number of views ofthe cluster 2 which is ten times smaller than those of the Probabilis-tic and Informed ones. We believe that the systematic ignorance ofthe isolated cluster arises from the greedy allocation strategy withoutprobabilistic redistribution mechanisms adopted by the first method.

4.5 scenario difficulty 83

Given the nature of the Naive method, in fact, the robots will try togo past clusters 1, 3, 4 only when their requests will be completelysatisfied.

4.5.3 Scenario Corridor

The distinctive feature of scenario Corridor (cf. Figure 4.3) is therectangular shape of the arena, with a slightly smaller area than theother two. The clusters are arranged linearly in descending orderfrom the deployment area: the cluster having the greater id is theclosest one.

The narrower profile of the arena has a strong impact on the magni-tude of the median number of views of the clusters (Table 4.7), which,for example, for the Naive method, on cluster 4, becomes ten timesbigger than the corresponding value in scenario Uniform.

At a first glance, the fact that cluster 3 has a greater number ofviews than cluster 4 for all the methods seems surprising. A possi-ble justification for this behavior is that the closer cluster is the onehaving the higher likelihood to be seen and also the higher likelihoodto become fully occupied in a short time. As shown in Figure 4.12,once its request has been satisfied, the cluster rebounds the robotselsewhere in the environment, thus favoring the exploration of otherclusters.

Also, the method having the higher number of views on the clus-ters closer to the deployment area is the Naive one, while the Prob-abilistic and Informed methods displays a reduced number of visits.This can be explained by the fact that the Naive method do not pos-sess any mechanism to perform an efficient exploration of the envi-ronment. For this reason, especially in a narrow environment likescenario Corridor, it is likely that a robot will keep viewing the clus-ters that it has already decided to leave, thus increasing the numberof views without actually allocating.

84 results

0 2000 4000 6000 8000 10000

020

0040

0060

0080

0010

000

Time steps

Clus

ter

obse

rvat

ions

Clusters

1 2 3 4

Figure 4.10: Median value of vi(t) for i ∈ 1, · · · , 4 on scenario Uniform,using method Informed, across 50 trials of 10000 time steps each.Each trial is performed with 20 robots and 25 available tasks.The cluster disposition can be seen in Figure 4.1. The differencesamong the curves are statistically significant. (Wilcoxon signed-rank test, p « 0.05)

method cluster

1 2 3 4

naive 5650 4673 5464 5070

probabilistic 7866 5942 7420 6716

informed 7412 7198 8734 6481

Table 4.5: Summary of the median values of cluster views v(t) at t = 10000

in scenario Uniform. Median values are computed across 50 trialsof 10000 time steps each. Each trial is performed with 20 robotsand 25 available tasks.

4.5 scenario difficulty 85

0 2000 4000 6000 8000 10000

050

0010

000

1500

0

Time steps

Clus

ter

obse

rvat

ions

Clusters

1 2 3 4

Figure 4.11: Median value of vi(t) for i ∈ 1, · · · , 4 on scenario Biased, usingmethod Informed, across 50 trials of 10000 time steps each. Eachtrial is performed with 20 robots and 25 available tasks. Thecluster disposition can be seen in Figure 4.2. The differencesamong the curves are statistically significant. (Wilcoxon signed-rank test, p « 0.05)

method cluster

1 2 3 4

naive 13805 498 13231 11854

probabilistic 13390 4118 14405 12924

informed 12299 4726 11242 10457


in scenario Biased. Median values are computed across 50 trials of10000 time steps each. Each trial is performed with 20 robots and25 available tasks.

86 results

0 2000 4000 6000 8000 10000

010

000

2000

030

000

4000

050

000

Time steps

Clus

ter

obse

rvat

ions

Clusters

1 2 3 4

Figure 4.12: Median value of vi(t) for i ∈ 1, · · · , 4 on scenario Corridor,using method Informed, across 50 trials of 10000 time steps each.Each trial is performed with 20 robots and 25 available tasks.The cluster disposition can be seen in Figure 4.3. The differencesamong the curves are statistically significant. (Wilcoxon signed-rank test, p « 0.05)

method cluster

1 2 3 4

naive 4363 28015 73242 57410

probabilistic 7247 23554 62802 49276

informed 7516 18148 47106 39893


in scenario Corridor. Median values are computed across 50 trialsof 10000 time steps each. Each trial is performed with 20 robotsand 25 available tasks.

5C O N C L U S I O N & F U T U R E W O R K

5.1 conclusion

The problem of allocating agents to tasks having a precise colloca-tion is space is a recurrent problem in collective robotics.

In environmental monitoring, for instance, we may observe theproblem of having robots deployed in an unknown environment inorder to check for the presence of dangerous substances. Generally,the size and the morphology of the environment prevents the robotsfrom having a global communication and from sharing knowledgeabout the environment. Hence, each robot needs to rely only onits own capabilities or eventually, communicate with the neighbor-ing robots. Clearly, a centralized solution is technically unfeasibleand presents several disadvantages: single point of failure and lack ofscalability among all. Thus, devising a distributed method, possiblyemerging from the local coordination among the agents, becomes theonly possible solution.

Our research question mainly arises from the need of finding effi-cient solutions to such problems.

In this thesis, we consider a specific instance of the problem of allo-cating robots to spatially distributed tasks, by fixing some constraints.In our vision of the problem, we have a finite number of robots (n)and a finite number of tasks (m), with n < m. The tasks are dis-tributed in space according to precise circular patterns, representingcollective activities (or clusters), requiring the simultaneous presenceof multiple robots to be executed. Each cluster is characterized by thenumber of tasks it is composed of (i.e. the robot request for the col-lective activity) and the number of robots currently being allocatedto tasks belonging to the cluster (i.e. the cluster occupation). Therobots, on the other hand, are homogeneous agents, with a minimalset of sensors, endowed with local sensing and local communicationcapabilities.

To the best of our knowledge, we are not aware of other studies inthe scientific literature that try to solve a similar problem, using bothembodied agents and embodied tasks. For this reason, we developedthe Naive method to serve as a baseline for comparison with our maincontributions: the Probabilistic and Informed one. We have decom-posed the spatial allocation problem into two distinct sub-problems,to be tackled independently and sequentially: task localization andtask allocation. Each method is characterized by a different strategy tosolve the aforementioned subproblems.

87

88 conclusion & future work

The Naive method simply consists of a random exploration of theenvironment to find the tasks and a greedy allocation to them as soonas they are detected.

The Probabilistic method maintains the uninformed random walkfrom the Naive method as exploration technique, but substitutes thepurely deterministic allocation rule with a probabilistic decision mech-anism. Here, the probability of leaving a cluster to further explore theenvironment is proportional to the cluster occupation. The use of thisstochastic decision mechanism should in principle, favor the redistri-bution of the robots across the clusters, yielding to a more uniformdistribution. Moreover, a check on the occurrence of a stalemate, a sit-uation that may occur when two robots decide to allocate to the sametask, has been introduced, along with a probabilistic decision mech-anism, to overcome this issue and increase the number of allocatedrobots.

The Informed method is built upon the Probabilistic one, preserv-ing the probabilistic allocation mechanism but upgrading the unin-formed random walk to an informed one through the use of odom-etry. Odometry is introduced to prevent multiple visits to clustersnot requiring additional robots to be allocated (i.e. whose occupationcorresponds to the request).

We decided to evaluate the performances of our methods with re-spect to two relevant aspects: allocation uniformity and allocation speed.

In order to do so, we devised three scenarios: Uniform, Biased andCorridor. Each one of them is characterized by the disposition of theclusters in space and the positioning of the area where the robots areinitially deployed.

The Uniform one has a central deployment area which is equallydistant from all the clusters in the environment, while the Biased onepresents some clusters closer to the deployment zone than the others.The Corridor one consists of a narrow arena, with the clusters linearlyarranged in the middle of it.

Concerning allocation uniformity, the Informed method performs bet-ter than the Naive and Probabilistic one on all the designed scenar-ios. However, no method is able to reach the optimal uniform allo-cation on any of the scenarios. Regarding allocation speed, the Naivemethod achieves a faster robot allocation than the Probabilistic and theInformed one on scenario Uniform and Biased, but it is outperformed byboth of them on the scenario Corridor. Furthermore, we observed thatthere exists a trade-off between the two aspects we are interested in,(i.e. a fast allocation comes at the expense of evenness in the distribu-tion of robots and viceversa). Our analysis showed that the Informedmethod achieves the best balance between uniformity and velocity ofthe allocation.

We also devised a metric to assess the difficulty of a scenario withrespect to the problem of performing a uniform allocation. In fact,

5.2 future work 89

by looking at the number of times that every cluster is seen by therobots, we are able to determine whether the distribution of theseviews is even across clusters or it is biased towards certain ones. Inthe latter case, we can safely assume that the scenario is more difficultsince the applied method has to compensate for the differences in therepartition of the views.

To summarize, our contribution consists of three methods to achievethe allocation of robots to tasks distributed in space and aggregatedto form collective activities. In terms of relative performances amongthe methods, the Naive method is the one achieving the fastest allo-cation while the Informed one allows to better distribute the robotsacross the clusters. Nevertheless, there is a strong influence of thetype of scenario on the performance of the system.

5.2 future work

Since the development and test of the methods has occurred onlyby means of simulation, our plan is to run experiments using the reale-puck robots and the real TAMs, in order to assess the transferabilityof our approach to an existing robotic platform, and to all the roboticsystems that share the same, minimal set of features required by themethods.

In addition, even though the presence of error in the odometryhas been realistically modeled and no communication is employed,further studies to verify the impact of the real sensor noise on theswarm performance should be performed.

Another interesting possibility would be to evaluate different prob-abilistic rules for the allocation of the robots, in order to have a morefine-grained control on the distribution of the robots across clusters.

A further study that could be made concerns the use of an inter-cluster recruiting behavior, as occurring in some species of ants, todirect robots from a cluster to another.

Last but not least, tests on the flexibility (i.e. the capacity of theswarm to dynamically adapt to changes in the requests), scalabilitywith respect to an increasing number of tasks and robots and robust-ness of the methods with respect to faults should necessarily be per-formed.

AA D D I T I O N A L P L O T S

This appendix includes the figures that have been discarded fromthe thesis due to the low readability on the paper version. The rel-evant data from these plots have been represented in the thesis in amore suitable way. In Section A.1 the plots of the quartiles and meansof the maximum error across cluster are displayed side by side for abetter visual comparison, arranged by scenario. In Section A.2 theplots of the quartiles and means of the number of allocated robotsare displayed side by side for a better visual comparison, arrangedby scenario.

90

A.1 maximum error 91

a.1

ma

xim

um

er

ro

r

020

0040

0060

0080

0010

000

02468

Tim

e st

eps

Tasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(a)

Nai

vem

etho

d

020

0040

0060

0080

0010

000

02468

Tim

e st

eps

Tasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(b)

Prob

abili

stic

met

hod

020

0040

0060

0080

0010

000

02468

Tim

e st

eps

Tasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(c)

Info

rmed

met

hod

Figu

reA

.1:C

ompa

riso

nof

the

max

imum

allo

cati

oner

rore

max

ofth

eth

ree

met

hods

onth

esc

enar

ioU

nifo

rmon

50

tria

lsof

10

00s

(10

00

0si

mul

atio

nst

eps)

each

.Eac

htr

iali

spe

rfor

med

wit

h20

robo

tsan

d25

avai

labl

eta

sks.

The

grey

area

corr

espo

nds

toth

ein

terq

uant

ilera

nge.

92 additional plots

020

0040

0060

0080

0010

000

02468

Tim

e st

eps

Tasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(a)

Nai

vem

etho

d

020

0040

0060

0080

0010

000

02468Ti

me

step

sTasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(b)

Prob

abili

stic

met

hod

020

0040

0060

0080

0010

000

02468

Tim

e st

eps

Tasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(c)

Info

rmed

met

hod

Figu

reA

.2:C

ompa

riso

nof

the

max

imum

allo

cati

oner

rore

max

ofth

eth

ree

met

hods

onth

esc

enar

ioBi

ased

on5

0tr

ials

of1

00

0s

(10

00

0si

mul

atio

nst

eps)

each

.Eac

htr

iali

spe

rfor

med

wit

h20

robo

tsan

d25

avai

labl

eta

sks.

The

grey

area

corr

espo

nds

toth

ein

terq

uant

ilera

nge.

A.1 maximum error 93

020

0040

0060

0080

0010

000

02468

Tim

e st

eps

Tasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(a)

Nai

vem

etho

d

020

0040

0060

0080

0010

000

02468

Tim

e st

eps

Tasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(b)

Prob

abili

stic

met

hod

020

0040

0060

0080

0010

000

02468

Tim

e st

eps

Tasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(c)

Info

rmed

met

hod

Figu

reA

.3:C

ompa

riso

nof

the

max

imum

allo

cati

oner

rore

max

ofth

eth

ree

met

hods

onth

esc

enar

ioC

orri

dor

on5

0tr

ials

of1

00

0s

(10

00

0si

mul

atio

nst

eps)

each

.Eac

htr

iali

spe

rfor

med

wit

h20

robo

tsan

d25

avai

labl

eta

sks.

The

grey

area

corr

espo

nds

toth

ein

terq

uant

ilera

nge.

94 additional plots

a.2

al

lo

ca

te

dr

ob

ot

s

020

0040

0060

0080

0010

000

05101520

Tim

e st

eps

Robots

Min

−Max

1Q−3

QM

edia

nM

ean

(a)

Nai

vem

etho

d

020

0040

0060

0080

0010

000

05101520

Tim

e st

eps

Robots

Min

−Max

1Q−3

QM

edia

nM

ean

(b)

Prob

abili

stic

met

hod

020

0040

0060

0080

0010

000

05101520

Tim

e st

eps

Robots

Min

−Max

1Q−3

QM

edia

nM

ean

(c)

Info

rmed

met

hod

Figu

reA

.4:C

ompa

riso

nof

num

ber

ofal

loca

ted

robo

tsR(t)

ofth

eth

ree

met

hods

onth

esc

enar

ioU

nifo

rmon

50

tria

lsof

10

00s

(10

00

0si

mul

atio

nst

eps)

each

.Eac

htr

iali

spe

rfor

med

wit

h20

robo

tsan

d25

avai

labl

eta

sks.

The

grey

area

corr

espo

nds

toth

ein

terq

uant

ilera

nge.

A.2 allocated robots 95

020

0040

0060

0080

0010

000

05101520

Tim

e st

eps

Robots

Min

−Max

1Q−3

QM

edia

nM

ean

(a)

Nai

vem

etho

d

020

0040

0060

0080

0010

000

05101520

Tim

e st

eps

Robots

Min

−Max

1Q−3

QM

edia

nM

ean

(b)

Prob

abili

stic

met

hod

020

0040

0060

0080

0010

000

02468

Tim

e st

eps

Tasks

Min

−Max

1Q−3

QM

edia

nM

ean

Opt

imal

(c)

Info

rmed

met

hod

Figu

reA

.5:C

ompa

riso

nof

num

ber

ofal

loca

ted

robo

tsR(t)

ofth

eth

ree

met

hods

onth

esc

enar

ioBi

ased

on5

0tr

ials

of1

00

0s

(10

00

0si

mul

atio

nst

eps)

each

.Eac

htr

iali

spe

rfor

med

wit

h20

robo

tsan

d25

avai

labl

eta

sks.

The

grey

area

corr

espo

nds

toth

ein

terq

uant

ilera

nge.

96 additional plots

020

0040

0060

0080

0010

000

05101520

Tim

e st

eps

Robots

Min

−Max

1Q−3

QM

edia

nM

ean

(a)

Nai

vem

etho

d

020

0040

0060

0080

0010

000

05101520Ti

me

step

sRobots

Min

−Max

1Q−3

QM

edia

nM

ean

(b)

Prob

abili

stic

met

hod

020

0040

0060

0080

0010

000

05101520

Tim

e st

eps

Robots

Min

−Max

1Q−3

QM

edia

nM

ean

(c)

Info

rmed

met

hod

Figu

reA

.6:C

ompa

riso

nof

num

ber

ofal

loca

ted

robo

tsR(t)

ofth

eth

ree

met

hods

onth

esc

enar

ioC

orri

dor

on5

0tr

ials

of1

00

0s

(10

00

0si

mul

atio

nst

eps)

each

.Eac

htr

iali

spe

rfor

med

wit

h20

robo

tsan

d25

avai

labl

eta

sks.

The

grey

area

corr

espo

nds

toth

ein

terq

uant

ilera

nge.

B I B L I O G R A P H Y

[Agassounon and Martinoli 2002] Agassounon, William ; Marti-noli, Alcherio: Efficiency and robustness of threshold-based dis-tributed allocation algorithms in multi-agent systems. In: Proceed-ings of the first international joint conference on Autonomous agents andmultiagent systems: part 3 ACM (Veranst.), 2002, p. 1090–1097

p. 22.

[Amé et al. 2006] Amé, Jean-Marc ; Halloy, José ; Rivault, Colette ;Detrain, Claire ; Deneubourg, Jean L.: Collegial decision makingbased on social amplification leads to optimal group formation. In:Proceedings of the National Academy of Sciences 103 (2006), Nr. 15,p. 5835–5840

p. 30.

[Anderson et al. 2002] Anderson, Carl ; Theraulaz, Guy ;Deneubourg, J-L: Self-assemblages in insect societies. In: Insectessociaux 49 (2002), Nr. 2, p. 99–110

p. 15.

[Arcaute et al. 2009] Arcaute, Elsa ; Christensen, Kim ; Sendova-Franks, Ana ; Dahl, Torbjørn ; Espinosa, Angela ; Jensen, Hen-rik J.: Division of labour in ant colonies in terms of attractive fields.In: Ecological Complexity 6 (2009), Nr. 4, p. 396–402

p. 29.

[Ashby 1962] Ashby, W R.: Principles of the self-organizing system.In: Principles of Self-organization (1962), p. 255–278

p. 6.

[Balas and Padberg 1976] Balas, Egon ; Padberg, Manfred W.: Setpartitioning: A survey. In: SIAM review 18 (1976), Nr. 4, p. 710–760

pp. 24 and 26.

[Balch and Hybinette 2000] Balch, Tucker ; Hybinette, Maria: So-cial potentials for scalable multi-robot formations. In: Robotics andAutomation, 2000. Proceedings. ICRA’00. IEEE International Conferenceon Bd. 1 IEEE (Veranst.), 2000, p. 73–80

pp. 16 and 21.

[Barca and Sekercioglu 2013] Barca, Jan C. ; Sekercioglu, Y A.:Swarm robotics reviewed. In: Robotica 31 (2013), Nr. 03, p. 345–359

pp. 17 and 18.

97

98 bibliography

[Benenson 1998] Benenson, Itzhak: Multi-agent simulations ofresidential dynamics in the city. In: Computers, Environment andUrban Systems 22 (1998), Nr. 1, p. 25–42

p. 10.

[Beni 2005] Beni, Gerardo: From Swarm Intelligence to SwarmRobotics. In: Swarm Robotics. Springer, 2005, p. 1–9

pp. 2, 6, 7, and 10.

[Berman et al. 2011] Berman, Spring ; Lindsey, Quentin ; Sakar,Mahmut S. ; Kumar, Vijay ; Pratt, Stephen C.: Experimental studyand modeling of group retrieval in ants as an approach to collectivetransport in swarm robotic systems. In: Proceedings of the IEEE 99

(2011), Nr. 9, p. 1470–1481

p. 16.

[Brambilla 2014] Brambilla, Manuele: Formal methods for the designand analysis of robot swarms, Universite Libre de Bruxelles, Disserta-tion, 2014

pp. 11, 12, and 14.

[Brambilla et al. 2013] Brambilla, Manuele ; Ferrante, Eliseo ;Birattari, Mauro ; Dorigo, Marco: Swarm robotics: a reviewfrom the swarm engineering perspective. In: Swarm Intelligence(2013), p. 1–41

pp. 11, 12, 13, and 15.

[Brambilla et al. 2009] Brambilla, Manuele ; Pinciroli, Carlo ;Birattari, Mauro ; Dorigo, Marco: A reliable distributed algo-rithm for group size estimation with minimal communication re-quirements. In: Advanced Robotics, 2009. ICAR 2009. InternationalConference on IEEE (Veranst.), 2009, p. 1–6

p. 16.

[Brutschy 2014] Brutschy, Arne: Enabling research on complex tasksin swarm robotics: Novel conceptual and practical tools. Brussels, Bel-gium, IRIDIA, Université Libre de Bruxelles, Brussels, Belgium,Dissertation, 2014

pp. 37, 38, 39, 46, and 47.

[Brutschy et al. 2014] Brutschy, Arne ; Garattoni, Lorenzo ;Brambilla, Manuele ; Francesca, Gianpiero ; Pini, Giovanni ;Dorigo, Marco ; Birattari, Mauro: The TAM: abstracting com-plex tasks in swarm robotics research. In: Swarm Intelligence (2014),p. submitted

p. 22.

bibliography 99

[Brutschy et al. 2010] Brutschy, Arne ; Pini, Giovanni ; Baiboun,Nadir ; Decugnière, Antal ; Birattari, Mauro: The IRIDIA TAM:A device for task abstraction for the e-puck robot / IRIDIA, Univer-sité Libre de Bruxelles. Brussels, Belgium, 2010 (TR/IRIDIA/2010-015). – Technical report

p. 38.

[Camazine 1991] Camazine, Scott: Self-organizing pattern forma-tion on the combs of honey bee colonies. In: Behavioral ecology andsociobiology 28 (1991), Nr. 1, p. 61–76

pp. 7 and 9.

[Camazine et al. 2001] Camazine, Scott ; Deneubourg, Jean-Louis ;Franks, N ; Sneyd, J ; Bonabeau, Eric ; Theraulaz, Guy: Self-Organization in Biological Systems. Princeton University Press, 2001

p. 4.

[Christensen et al. 2009] Christensen, Anders L. ; O’Grady, Re-han ; Dorigo, Marco: From fireflies to fault-tolerant swarms ofrobots. In: Evolutionary Computation, IEEE Transactions on 13 (2009),Nr. 4, p. 754–766

p. 16.

[Codling et al. 2008] Codling, Edward A. ; Plank, Michael J. ;Benhamou, Simon: Random walk models in biology. In: Journal ofthe Royal Society Interface 5 (2008), Nr. 25, p. 813–834

p. 21.

[Colorni et al. 1991] Colorni, Alberto ; Dorigo, Marco ; Maniezzo,Vittorio et al.: Distributed optimization by ant colonies. In: Pro-ceedings of the first European conference on artificial life Bd. 142 Paris,France (Veranst.), 1991, p. 134–142

p. 10.

[Colorni et al. 1994] Colorni, Alberto ; Dorigo, Marco ; Maniezzo,Vittorio ; Trubian, Marco: Ant system for job-shop scheduling. In:Belgian Journal of Operations Research, Statistics and Computer Science34 (1994), Nr. 1, p. 39–53

p. 26.

[Cooper 2008] Cooper, Steven J.: From Claude Bernard to WalterCannon. Emergence of the concept of homeostasis. In: Appetite 51

(2008), Nr. 3, p. 419–427

p. 5.

[Deneubourg et al. 1990] Deneubourg, J-L ; Aron, Serge ; Goss,Simon ; Pasteels, Jacques M.: The self-organizing exploratory pat-tern of the argentine ant. In: Journal of insect behavior 3 (1990), Nr. 2,p. 159–168

100 bibliography

p. 16.

[Dias et al. 2006] Dias, M B. ; Zlot, Robert ; Kalra, Nidhi ; Stentz,Anthony: Market-based multirobot coordination: A survey andanalysis. In: Proceedings of the IEEE 94 (2006), Nr. 7, p. 1257–1270

p. 28.

[Donald et al. 1997] Donald, Bruce R. ; Jennings, James ; Rus,Daniela: Information invariants for distributed manipulation. In:The International Journal of Robotics Research 16 (1997), Nr. 5, p. 673–702

p. 16.

[Dorigo and Birattari 2007] Dorigo, M. ; Birattari, M.: Swarmintelligence. In: Scholarpedia 2 (2007), Nr. 9, p. 1462

pp. 14 and 45.

[Dorigo et al. 2014] Dorigo, M. ; Birattari, M. ; Brambilla, M.:Swarm robotics. 9 (2014), Nr. 1, p. 1463. – revision #138643

p. 11.

[Dorigo et al. 1996] Dorigo, Marco ; Maniezzo, Vittorio ; Col-orni, Alberto: Ant system: optimization by a colony of cooper-ating agents. In: Systems, Man, and Cybernetics, Part B: Cybernetics,IEEE Transactions on 26 (1996), Nr. 1, p. 29–41

p. 6.

[Ducatelle et al. 2011] Ducatelle, Frederick ; Di Caro, Gianni A. ;Pinciroli, Carlo ; Gambardella, Luca M.: Self-organized cooper-ation between robotic swarms. In: Swarm Intelligence 5 (2011), Nr. 2,p. 73–96

p. 16.

[Eberhart and Kennedy 1995] Eberhart, Russ C. ; Kennedy, James:A new optimizer using particle swarm theory. In: Proceedings ofthe sixth international symposium on micro machine and human scienceBd. 1 New York, NY (Veranst.), 1995, p. 39–43

p. 6.

[Flocchini et al. 2008] Flocchini, Paola ; Prencipe, Giuseppe ; San-toro, Nicola ; Widmayer, Peter: Arbitrary pattern formation byasynchronous, anonymous, oblivious robots. In: Theoretical Com-puter Science 407 (2008), Nr. 1, p. 412–447

p. 15.

[Garnier et al. 2005] Garnier, Simon ; Jost, Christian ; Jeanson,Raphaël ; Gautrais, Jacques ; Asadpour, Masoud ; Caprari,Gilles ; Theraulaz, Guy: Aggregation behaviour as a source of

bibliography 101

collective decision in a group of cockroach-like-robots. In: Advancesin artificial life. Springer, 2005, p. 169–178

pp. 15 and 16.

[Gerkey and Mataric 2004] Gerkey, Brian P. ; Mataric, Maja J.:A formal analysis and taxonomy of task allocation in multi-robotsystems. In: The International Journal of Robotics Research 23 (2004),Nr. 9, p. 939–954

p. 22.

[Giusti et al. 2012] Giusti, Alessandro ; Nagi, Jawad ; Gam-bardella, Luca M. ; Di Caro, Gianni A.: Distributed consen-sus for interaction between humans and mobile robot swarms. In:Proceedings of the 11th International Conference on Autonomous Agentsand Multiagent Systems-Volume 3 International Foundation for Au-tonomous Agents and Multiagent Systems (Veranst.), 2012, p. 1503–1504

p. 17.

[Goldberg and Holland 1988] Goldberg, David E. ; Holland,John H.: Genetic algorithms and machine learning. In: Machinelearning 3 (1988), Nr. 2, p. 95–99

p. 13.

[Guerrero and Oliver 2003] Guerrero, José ; Oliver, Gabriel:Multi-robot task allocation strategies using auction-like mecha-nisms. In: Artificial Research and Development in Frontiers in ArtificialIntelligence and Applications 100 (2003), p. 111–122

p. 28.

[Gunn and Anderson 2013] Gunn, Tyler ; Anderson, John: Dy-namic heterogeneous team formation for robotic urban search andrescue. In: Procedia Computer Science 19 (2013), p. 22–31

p. 28.

[Gutiérrez et al. 2008] Gutiérrez, Alvaro ; Campo, Alexandre ;Dorigo, Marco ; Amor, Daniel ; Magdalena, Luis ; Monasterio-Huelin, Félix: An open localization and local communication em-bodied sensor. In: Sensors 8 (2008), Nr. 11, p. 7545–7563

pp. 34 and 50.

[Gutiérrez et al. 2010] Gutiérrez, Álvaro ; Campo, Alexandre ;Monasterio-Huelin, Félix ; Magdalena, Luis ; Dorigo, Marco:Collective decision-making based on social odometry. In: NeuralComputing and Applications 19 (2010), Nr. 6, p. 807–823

p. 16.

102 bibliography

[Howard et al. 2002] Howard, Andrew ; Mataric, Maja J. ;Sukhatme, Gaurav S.: Mobile sensor network deployment usingpotential fields: A distributed, scalable solution to the area cover-age problem. In: Distributed Autonomous Robotic Systems 5. Springer,2002, p. 299–308

pp. 16 and 21.

[Hsieh et al. 2008] Hsieh, M A. ; Halász, Ádám ; Berman, Spring ;Kumar, Vijay: Biologically inspired redistribution of a swarm ofrobots among multiple sites. In: Swarm Intelligence 2 (2008), Nr. 2-4,p. 121–141

pp. 30, 31, 38, and 46.

[Jakobi et al. 1995] Jakobi, Nick ; Husbands, Phil ; Harvey, Inman:Noise and the reality gap: The use of simulation in evolutionaryrobotics. In: Advances in artificial life. Springer, 1995, p. 704–720

p. 14.

[Jevtic et al. 2012] Jevtic, Aleksandar ; Gutiérrez, Alvaro ; And-ina, Diego ; Jamshidi, Mo: Distributed bees algorithm for taskallocation in swarm of robots. In: Systems Journal, IEEE 6 (2012),Nr. 2, p. 296–304

pp. 31, 38, and 46.

[Kalra and Martinoli 2006] Kalra, Nidhi ; Martinoli, Alcherio:Comparative study of market-based and threshold-based task allo-cation. In: Distributed Autonomous Robotic Systems 7. Springer, 2006,p. 91–101

p. 29.

[Knuth 1974] Knuth, Donald E.: Computer Programming as anArt. In: Communications of the ACM 17 (1974), December, Nr. 12,p. 667–673

p. v.

[Korsah et al. 2013] Korsah, G A. ; Stentz, Anthony ; Dias, M B.:A comprehensive taxonomy for multi-robot task allocation. In: TheInternational Journal of Robotics Research 32 (2013), Nr. 12, p. 1495–1512

pp. 22, 23, 24, 26, 43, 48, and 49.

[Krause et al. 2010] Krause, Jens ; Ruxton, Graeme D. ; Krause,Stefan: Swarm intelligence in animals and humans. In: Trends inecology & evolution 25 (2010), Nr. 1, p. 28–34

p. 8.

bibliography 103

[Krieger and Billeter 2000] Krieger, Michael J. ; Billeter, Jean-Bernard: The call of duty: Self-organised task allocation in a pop-ulation of up to twelve mobile robots. In: Robotics and AutonomousSystems 30 (2000), Nr. 1, p. 65–84

pp. 17 and 22.

[Kuhn 1955] Kuhn, Harold W.: The Hungarian method for theassignment problem. In: Naval research logistics quarterly 2 (1955),Nr. 1-2, p. 83–97

p. 23.

[Landau and Binder 2009] Landau, David P. ; Binder, Kurt: Aguide to Monte Carlo simulations in statistical physics. Cambridge Uni-versity Press, 2009

p. 30.

[Lin and Zheng 2005] Lin, Liu ; Zheng, Zhiqiang: Combinatorialbids based multi-robot task allocation method. In: Robotics and Au-tomation, 2005. ICRA 2005. Proceedings of the 2005 IEEE InternationalConference on IEEE (Veranst.), 2005, p. 1145–1150

p. 28.

[Liu and Nejat 2013] Liu, Yugang ; Nejat, Goldie: Robotic urbansearch and rescue: A survey from the control perspective. In: Jour-nal of Intelligent & Robotic Systems 72 (2013), Nr. 2, p. 147–165

p. 18.

[de Lope et al. 2013] Lope, Javier de ; Maravall, Darío ; Quiñonez,Yadira: Response threshold models and stochastic learning au-tomata for self-coordination of heterogeneous multi-task distribu-tion in multi-robot systems. In: Robotics and Autonomous Systems 61

(2013), Nr. 7, p. 714–720

p. 29.

[Low et al. 2004] Low, Kian H. ; Leow, Wee K. ; Ang Jr, Marcelo H.:Task allocation via self-organizing swarm coalitions in distributedmobile sensor network. In: AAAI Bd. 4, 2004, p. 28–33

p. 29.

[Lucas 2001] Lucas, GW: A tutorial and elementary trajectorymodel for the differential steering system of robot wheel actuators.In: The Rossum Project (2001)

p. 60.

[Mondada et al. 2005] Mondada, Francesco ; Bonani, Michael ;Guignard, André ; Magnenat, Stéphane ; Studer, Christian ; Flo-reano, Dario: Superlinear physical performances in a SWARM-BOT. In: Advances in Artificial Life. Springer, 2005, p. 282–291

104 bibliography

p. 15.

[Mondada et al. 2009] Mondada, Francesco ; Bonani, Michael ;Raemy, Xavier ; Pugh, James ; Cianci, Christopher ; Klaptocz,Adam ; Magnenat, Stephane ; Zufferey, Jean-Christophe ; Flore-ano, Dario ; Martinoli, Alcherio: The e-puck, a robot designedfor education in engineering. In: Proceedings of the 9th conference onautonomous robot systems and competitions Bd. 1, 2009, p. 59–65

pp. 2 and 33.

[Nouyan et al. 2008] Nouyan, Shervin ; Campo, Alexandre ;Dorigo, Marco: Path formation in a robot swarm. In: SwarmIntelligence 2 (2008), Nr. 1, p. 1–23

pp. 15 and 21.

[Oyekan and Hu 2013] Oyekan, John ; Hu, Huosheng: Ant RoboticSwarm for Visualizing Invisible Hazardous Substances. In: Robotics2 (2013), Nr. 1, p. 1–18

p. 54.

[Payton et al. 2001] Payton, David W. ; Daily, Michael J. ; Hoff,Bruce ; Howard, Michael D. ; Lee, Craig L.: Pheromone robotics.In: Intelligent Systems and Smart Manufacturing International Societyfor Optics and Photonics (Veranst.), 2001, p. 67–75

p. 15.

[Pfeifer et al. 2007] Pfeifer, Rolf ; Lungarella, Max ; Iida, Fumiya:Self-organization, embodiment, and biologically inspired robotics.In: Science 318 (2007), Nr. 5853, p. 1088–1093

p. 10.

[Pinciroli et al. 2013] Pinciroli, Carlo ; O’Grady, Rehan ; Chris-tensen, Anders ; Birattari, Mauro ; Dorigo, Marco: ParallelFormation of Differently Sized Groups in a Robotic Swarm. In:Journal of the Society of Instrument and Control Engineers 52 (2013),Nr. 3, p. 213–226

pp. 30 and 38.

[Pinciroli et al. 2012] Pinciroli, Carlo ; Trianni, Vito ; O’Grady,Rehan ; Pini, Giovanni ; Brutschy, Arne ; Brambilla, Manuele ;Mathews, Nithin ; Ferrante, Eliseo ; Di Caro, Gianni ;Ducatelle, Frederick et al.: ARGoS: a modular, parallel, multi-engine simulator for multi-robot systems. In: Swarm intelligence 6

(2012), Nr. 4, p. 271–295

pp. 39 and 40.

bibliography 105

[Pini et al. 2009] Pini, Giovanni ; Brutschy, Arne ; Birattari,Mauro ; Dorigo, Marco: Interference reduction through task par-titioning in a robotic swarm. In: Sixth International Conference onInformatics in Control, Automation and Robotics–ICINCO, 2009, p. 52–59

p. 22.

[Pini et al. 2011] Pini, Giovanni ; Brutschy, Arne ; Frison, Marco ;Roli, Andrea ; Dorigo, Marco ; Birattari, Mauro: Task partition-ing in swarms of robots: An adaptive method for strategy selection.In: Swarm Intelligence 5 (2011), Nr. 3-4, p. 283–304

p. 38.

[Qian and Cheng 2014] Qian, Binsen ; Cheng, Harry H.: A bio-inspired mobile agent-based coalition formation system for multi-ple modular-robot systems. In: Mechatronic and Embedded Systemsand Applications (MESA), 2014 IEEE/ASME 10th International Confer-ence on IEEE (Veranst.), 2014, p. 1–6

p. 26.

[Reynolds 1987] Reynolds, Craig W.: Flocks, herds and schools:A distributed behavioral model. In: ACM SIGGRAPH ComputerGraphics Bd. 21 ACM (Veranst.), 1987, p. 25–34

pp. 7 and 15.

[Rumbaugh et al. 2004] Rumbaugh, James ; Jacobson, Ivar ; Booch,Grady: Unified Modeling Language Reference Manual, The (2nd Edi-tion). Pearson Higher Education, 2004. – ISBN 0321245628

p. 47.

[Sahin 2005] Sahin, Erol: Swarm robotics: From sources of inspira-tion to domains of application. In: Swarm robotics. Springer, 2005,p. 10–20

p. 10.

[Sandhlom and Lesser 1997] Sandhlom, Tuomas W. ; Lesser, Vic-tor R.: Coalitions among computationally bounded agents. In:Artificial intelligence 94 (1997), Nr. 1, p. 99–137

p. 44.

[Sarker et al. 2014] Sarker, Md Omar F. ; Dahl, Torbjørn S ; Ar-caute, Elsa ; Christensen, Kim: Local interactions over globalbroadcasts for improved task allocation in self-organized multi-robot systems. In: Robotics and Autonomous Systems 62 (2014), Nr. 10,p. 1453–1462

p. 29.

106 bibliography

[Schweitzer 2003] Schweitzer, Frank: Brownian agents and activeparticles. On the emergence of complex behavior in the natural andsocial sciences. In: Springer Series in Synergetics. Berlin, Germany,Springer (2003)

p. 11.

[Shehory and Kraus 1998] Shehory, Onn ; Kraus, Sarit: Meth-ods for task allocation via agent coalition formation. In: ArtificialIntelligence 101 (1998), Nr. 1, p. 165–200

p. 44.

[Shen and Salemi 2002] Shen, Wei-Min ; Salemi, Behnam: To-wards distributed and dynamic task reallocation. In: IntelligentAutonomous Systems: IAS-7 (2002), p. 289

p. 27.

[Singh 1991] Singh, Munindar P.: Towards a Formal Theory ofCommunication for Multi-agent Systems. In: IJCAI Bd. 91 Citeseer(Veranst.), 1991, p. 69–74

p. 10.

[Spears et al. 2004] Spears, William M. ; Spears, Diana F. ;Hamann, Jerry C. ; Heil, Rodney: Distributed, physics-based con-trol of swarms of vehicles. In: Autonomous Robots 17 (2004), Nr. 2-3,p. 137–162

p. 13.

[Stirling and Floreano 2010] Stirling, Timothy ; Floreano, Dario:Energy efficient swarm deployment for search in unknown envi-ronments. In: Swarm Intelligence. Springer, 2010, p. 562–563

p. 21.

[Sutton and Barto 1998] Sutton, Richard S. ; Barto, Andrew G.:Reinforcement learning: An introduction. Cambridge Univ Press, 1998

p. 13.

[Theraulaz et al. 1998] Theraulaz, Guy ; Bonabeau, Eric ;Deneubourg, JN: Response threshold reinforcements and divisionof labour in insect societies. In: Proceedings of the Royal Society ofLondon. Series B: Biological Sciences 265 (1998), Nr. 1393, p. 327–332

pp. 7, 28, and 29.

[Turgut et al. 2008] Turgut, Ali E. ; Çelikkanat, Hande ; Gökçe,Fatih ; Sahin, Erol: Self-organized flocking in mobile robot swarms.In: Swarm Intelligence 2 (2008), Nr. 2-4, p. 97–120

p. 15.

bibliography 107

[Turing 1952] Turing, AM: The Chemical Basis of Morphogenesis.In: Philosophical Transactions of the Royal Society of London. Series B,Biological Sciences 237 (1952), Nr. 641, p. 37–72

p. 5.

[Varghese and McKee 2009] Varghese, Blesson ; McKee, Gerard: Areview and implementation of swarm pattern formation and trans-formation models. In: International Journal of Intelligent Computingand Cybernetics 2 (2009), Nr. 4, p. 786–817

p. 15.

[Vig 2006] Vig, Lovekesh: Multi-robot coalition formation, VanderbiltUniversity, Dissertation, 2006

pp. 19, 26, and 44.

[Vig and Adams 2005] Vig, Lovekesh ; Adams, Julie A.: Issuesin multi-robot coalition formation. In: Multi-Robot Systems. FromSwarms to Intelligent Automata Volume III. Springer, 2005, p. 15–26

p. 26.

[Votaw 1952] Votaw, DF: The personnel assignment problem. 1952

p. 23.

[Werfel et al. 2011] Werfel, Justin K. ; Petersen, Kirsten ; Nagpal,Radhika: Distributed multi-robot algorithms for the TERMES 3Dcollective construction system Institute of Electrical and ElectronicsEngineers (Veranst.), 2011

p. 16.

[Wilson 1984] Wilson, Edward O.: The relation between caste ra-tios and division of labor in the ant genus Pheidole (Hymenoptera:Formicidae). In: Behavioral Ecology and Sociobiology 16 (1984), Nr. 1,p. 89–98

p. 7.

[Winfield 2009] Winfield, Alan F.: Towards an engineering sci-ence of robot foraging. In: Distributed Autonomous Robotic Systems8. Springer, 2009, p. 185–192

p. 20.

spatial allocation in swarm robotics

Documents