intelligence through interactions- underwater robot ... studies/intelligence through... ·...

Helsinki University of Technology Laboratory of Automation Technology Series A; Research Reports No. 21. October 1999

INTELLIGENCE THROUGH INTERACTIONS -Underwater RobotSociety for Distributed Operations in Closed Aquatic Environment

Mika Vainio

Helsinki University of Technology Laboratory of Automation TechnologySeries A; Research Reports No. 21. October 1999

INTELLIGENCE THROUGH INTERACTIONS -Underwater RobotSociety for Distributed Operations in Closed Aquatic Environment

Mika Vainio

Dissertation for the degree of Doctor of Technology to be presented with due permission forpublic examination and debate in Auditorium T2 at Helsinki University of Technology (Espoo,Finland) on the 29th of October, at 12 o’clock noon.

Helsinki University of TechnologyDepartment of Automation and Systems TechnologyLaboratory of Automation Technology

Teknillinen korkeakouluAutomaatio- ja systeemitekniikan osastoAutomaatiotekniikan laboratorio

Distribution:Helsinki University of TechnologyLaboratory of Automation TechnologyP.O. Box 5400Tel. +358-9-4513304Fax. +358-9-4513308E-mail: [email protected]

Mika Vainio

ISBN 951-22-4734-8ISSN 0783-5477

Pica-set OyHelsinki 1999

To the gentle arts offly fishing

anddesigning

distributed autonomous robotic systems.

Both will definitely drive you crazy now and then...

iii

AbstractNowadays, robotics is moving towards adaptive, learning and cooperative robots.These robots constantly interact not only with the dynamic environment, but alsowith each other and with the persons using them. This large variety ofinteractions will produce behaviors superior to those performed by currentindustrial robots. The robots will be equipped to survive as a part of a complexsystem, where cooperation is essential for their survival. The collectiveintelligence emerging from these interactions justifies calling these systems, attheir highest level, robot societies, based on the analogy to structures found innatural environment.

This thesis describes a generic framework for multi-robot systems. Theframework is no longer just a research interest or technical challenge but verymuch a practical demand. As technical readiness increases rapidly, especially insensor technology and locomotion structures, the application domain for roboticsystems in general widens accordingly. Many of these future tasks can beperformed more efficiently with multiple robots and some of them can only beaccomplished with cooperative multi-robot systems.

In this thesis conceptual definitions for terms like societal robotic agent and robotsociety are given. Based on these definitions, a generic three-layer hybrid(reactive and deliberative components) control architecture for distributedautonomous robotic systems is introduced. This architecture is then tested on anovel distributed underwater robotic system. The robots operate in a closedaquatic environment performing a combined task of exploration and exploitation.The task in question consists of creating, maintaining and using an adaptivetopological map and searching and collectively destroying distributed dynamictargets (microbial growth spots). The nature of the system, along with the minimalperception (only a pressure sensor and a target detection sensor) and poormaneuverability (no thrusters, only specific weight alteration) in a complexenvironment, forced us to develop new algorithms for example for navigation.

Keywords. Multi-robot systems, robot society, control architecture, underwaterrobots

iv

PrefaceThis study has been conducted at the Automation Technology Laboratory of theHelsinki University of Technology during the years of 1992-1999, most of the timewithin the “Robot Society “ and “Intelligent Underwater Sensor/Actuator Societyfor Monitoring the Internal Status of Processes” projects. These projects havebeen supported by the Technology Development Centre of Finland (TEKES) andAcademy of Finland, both of which are gratefully acknowledged. I would also liketo thank the Emil Aaltonen Foundation and Tekniikan Edistämisäätiö for theirfinancial support. I would especially like to thank GETA (The Graduate School inElectronics, Telecommunications, and Automation) for providing continuity to myresearch in the form of long term funding from 1995 to 1999.

I wish to express my sincere gratitude to Professor Aarne Halme, head ofthe Automation Lab, for his encouragement, inspiration and expert guidanceduring this work, and, in particular, for allowing me a certain freedom in myresearch. It was absolutely vital for the success of this work.

I am grateful to my co-worker Mr. Pekka Appelqvist for joint work on theunderwater robotic system. Without him the system would not be operational. Wespent some long days together watching robots running their course time aftertime. It was not always interesting and without problems, but his company andknowledge were always greatly appreciated.

I am in great debt to Dr. Peter Jakubik for his incredible patience andwillingness to share his vast knowledge on the matters of concern. I would like tothank Mr. Torsten Schönberg for his extensive work on the project. Mr. PekkaKähkönen deserves special thanks for his innovative work concerning the initialmapping algorithm. Many thanks are further due to Mr. Antti Hakala (SofimationLtd., CEO) His input to the SUBMAR project was of great importance.

I would like to thank some other people from the laboratory for being partof the project: Dr. Arto Visala for his patience when I tested my ideas on him(very few of which passed!); Mr. Kalle Rosenblad for fast, precise andprofessional work on various electrical design matters; Mr. Tapio Leppänen forhis assistance in mechanical problems; Mr. Yang Wang, Mr. Sami Ylönen, Mr.Markku Kokko, Mr. Tommi Tuovila and Mr. Antti Matikainen for their participationin various phases of the software development; and, Ms Johanna Nikkinen forher valuable help in analyzing the results. My thanks are further due to Mr. JormaSelkäinaho and our secretaries for helping me with various problems beyondimagination. You made my life so much easier. Last but not least I would like tothank the whole staff of the lab for creating such a friendly and stimulatingatmosphere. And some people say that technology and engineers are boring. Ifthey only knew...

I thank my preliminary examiners Dr. Tapio Heikkilä and Professor MiguelA. Salichs for their valuable suggestions for enhancements, and my opponent Dr.Hajime Asama for finding time to make the long journey to Finland to examineand debate my views. Furthermore, I want to thank Ms Kathleen Tipton for herwork in correcting my English. The remaining mistakes are my own.

Finally, I would like to thank my family and my friends (mostly fly fishingbuddies). Without your support, love and patience none of this would have beenpossible. I owe you all.

Espoo, June 1999 Mika Vainio

v

ContentsAbstract iiiPreface ivContents vList of symbols and abbreviations viii

1. Introduction 11.1 Interactions and Intelligent Behavior 31.2 Cooperative Multi-Robot Systems 51.3 Motivation of the Dissertation 61.4 Main Contribution of the Dissertation 71.5 Thesis Outline 81.6 Author’s Contribution within the Research Group 9

2. Related Work 102.1 Introduction 112.2 Biological Background 11

2.2.1 Societal Structures 122.2.1.1 Bacteria 122.2.1.2 Social Insects 132.2.1.3 Types of Animal Societies 14

2.3 Ancestors in Engineering 162.3.1 Automated Guided Vehicles (AGVs) 16

2.3.1.1 Basic Structures 162.3.1.2 Central vs. Distributed Control 182.3.1.3 Need for Flexibility 18

2.3.2 Autonomous Mobile Robots (AMRs) 182.3.2.1 How It All Started 182.3.2.2 Traditional Subsystems 20

2.4 Transition of Paradigms in Robotics 212.5 Distributed Autonomous Robotic Systems 292.6 Towards the Next Millennium 37

2.6.1 Learning by Imitation 372.6.2 Evolutionary Robotics 38

3. Societal Robotic Agent 413.1 Introduction 413.2 Definition of an Autonomous Robotic Agent 42

3.2.1 Embodied and Situated 433.2.2 Ecological Niche 443.2.3 Learning 443.2.4 Knowledge Representation 45

3.3 What Does It Mean to Be Societal? 453.4 Formal Description 46

vi

4. Definition of Robot Society 494.1 Introduction 494.2 Economic Justification from the System Supplier Point of View 504.3 Main Principles 524.4 Main Features 54

4.4.1 Volume 544.4.2 Mission 544.4.3 The Role of Diversity 544.4.4 Decentralized vs. Centralized 55

4.5 Types of Interactions 564.5.1 Two Forms of Communication: Indirect and Direct 564.5.2 Man-Machine Interface 564.5.3 Interference: Conflicts, Competition and Deadlocks 57

5. Society Model 595.1 Introduction 595.2 Communication Structure: Avoid, Minimize and Forget 615.3 Behavioral Layer: Perceive and Interact 625.4 Task Layer: Parallel State Machines 625.5 Cooperative Layer: 64

Dynamic Group Formation through Communication5.6 Pros and Cons 65

6. Experimental Test-bed 676.1 Introduction 676.2 Underwater Society 67

6.2.1 Physical Society 706.2.2 Simulated Society 726.2.3 Communication System 726.2.4 Operator Interface 746.2.5 Self-Sufficiency 74

7. Mapping and Navigation 777.1 Introduction 777.2 Mapping Algorithm 78

7.2.1 Preliminary Results Using the Simulator 807.3 Common Environment Representation 817.4 Using Common Basic Map 837.5 Navigation 85

7.5.1 Test Results 887.6 Summary 90

8. Distributed Operations in Closed Aquatic Environment 918.1 Task Domain 91

8.1.1 Emulated Biomass Growth 928.1.2 Sensor Problems and Solutions 95

8.2 Control Architecture 968.2.1 Behavioral Layer 978.2.2 Task Layer 99

vii

8.2.3 Cooperative Layer 1008.3 Tests and Results 101

8.3.1 Reference Case -a Single Robot 1028.3.2 Group of Three Robots 1038.3.3 Group of Five Robots 106

8.4 Conclusions 111

9. Summary and Conclusions 1139.1 Main Results 1139.2 Future Work 114

References 115

Appendixes 131

A: Pseudocode for behaviors presented in Chapter 8.B: An example of communication log file.C: Single robot.D: Three robots without communication.E: Three robots with C-type communication.F: Five robots without communication.G: Five robots with C-type communication.H: Five robots with CC-type communication.

viii

List of symbols and abbreviations

a actionA a discrete set of agent actionsA biomassB agent’s behaviorc confidence valuecbirth link’s initial valueD death rate of the organismE link usage emptyF states which indicate the completion of the taskF link usage fulli inputI input functionL link matrixM FSA acceptorM miscellaneous behaviorN node arrayNA parameter for procedure ANB parameter for procedure BNC parameter for procedure CNnodes the number of traveled nodes, before links’ confidence values are

decreasedNi number of robotspk pressure measurement (k)q0 the initial stateQ the set of possible statesR radio communication moduleRi Robot( i )s current stateS a discrete set of environment statesS1,S2,S3 operational strategies at the task levelSi strategy (j)r reinforcement signalTkill if c < Tkill, the link is removedTrefill the time used for energy or poison refillingTtotal mission timeTwork the actual working time, i.e., Twork= Ttotal-TrefillV matrix for path-planning purposesW poison scalarδ the transition function mappingλ learning rateµ growth rate of the organism# link usage don’t careACTRESS ACTor-based Robot and Equipments Synthetic SystemAFSM Augmented Finite State MachineAGV Automated Guided VehicleAI Artificial IntelligenceAMR Autonomous Mobile RobotAPN Adaptive Place NetworkARSKA Autonomous Robot for Surveillance Key ApplicationsAuRA Autonomous Robot ArchitectureBeRoSH Behavior-based Multiple Robot System with Host for Object

Manipulation

ix

CBM Common Basic MapCEBOT CEllular roBOTics SystemCIM Computer Integrated ManufacturingCPU Central Processing UnitCRL Chicago Robot LanguageCS Classifier SystemCSMA/CD Carrier Sense Multiple Access with Collision DetectionCRC checksum calculation functionDAI Distributed Artificial IntelligenceDAMN Distributed Architecture for Mobile NavigationDARS Distributed Autonomous Robotic SystemsEP Evolutionary ProgrammingES Evolutionary StrategyES Environmental SensorFI Test value FI=W*TworkFSA Finite State AutomatonFMS Flexible Manufacturing SystemFPGA Field Programmable Gate ArrayGA Genetic AlgorithmGP Genetic ProgrammingGPS Global Positioning SystemHUTMAN Helsinki University of Technology’s Mobile Autonomous NavigatorIR_NX Short range IR sensor (X)IS Internal SensorMECANT MEChanical ANTMS Motor SchemaNN Neural NetworkPS Perceptual SchemaR2D2 May the Force be with you...RAP Reactive Action PackageRL Reinforcement LearningRS Receptor SchemaRUR Russums Universal RobotsSAGA Species Adaptation Genetic AlgorithmsS_C Capacitive sensor on the bottom of the robotS_G Tactile sensor on the palm of an one-degree of freedom gripperS_IR IR sensor in front of the robotSPA Sense-Plan-ActSS Self-sufficiency behaviorSSS Servo, Subsumption, SymbolicSUBMAR Smart Underwater Ball for Measurement and Actuation RoutinesS_VL Visible light sensor in front of the robotTA Task achieving behaviorTS Transmitter SchemaWWW World Wide Web

Chapter 1. Introduction

1

Chapter 1 IntroductionMuch has happened since the Czech Carl Capek introduced the term robot ina play named as RUR (Russums Universal Robots) presented for the firsttime in 1921 (Capek 1973). Capek’s idea of human-like servants, or slaves,was later eagerly transferred to the world of engineering. First generationindustrial manipulator type robots fit this description conceptually. Thesemachines were, like slaves, performing tasks explicitly programmed for them.This was the main stream in robot design for several decades, until the end ofthe 1970s, when a shift towards a new kind of conceptual paradigm started toemerge. This shift contained two important changes. The first major changewas to remove robots from fixed locations and give them the capability tomove. Mobile robots made designers face a series of new problems relatedto topics like limited energy resources, the need for versatile perception andthe limitations in mobility. Nevertheless, the benefits gained from this shiftwere so considerable, that the emphasis in robotic research was clearlydirected towards mobile robots. These robots were to free humans fromdangerous and boring tasks and at the same time increase productivity.Based on this highly technically boosted change, another major transition hasbeen occurring over the past ten years or so, namely the use of multiplerobots together.

Earlier, the reliable operation of a single mobile robot was considered to bemore than a handful and thus the idea of multiple robots working togetherseemed to be rather unrealistic. Thanks to rapid progress mainly inprocessors and sensors, real world multi-robot systems are, nevertheless,starting to emerge. We are gradually moving towards systems whereindividual robots that select their actions are not based only on their individualperception but also on interactions with the other robots within the system.This has allowed the increase in the overall complexity of a given task, as isshown in Figure 1.1, and thus introduced a whole new set of applications formobile robots.


2

ENVI

RO

NM

ENT

CO

MPL

EXIT

Y

TASK COMPLEXITY

INDUSTRIALMANIPULATOR ROBOT

AUTONOMOUS MOBILE ROBOT

mobility

multiplicity DISTRIBUTED AUTONOMOUS ROBOTS

Figure 1.1 Two main paradigm shifts in robotics.

When considering the justification of a multi-robot system in solving real worldproblems, the single most important criterion is the economic factor. The useof a single robot in a well-defined problem in a stabile and structuredenvironment is usually much more economically sound, than theconfiguration of a multi-robot system. Unfortunately, the world where mostautonomous mobile robots are operating is neither stabile nor structured.That is the main reason the trend in robotics is currently heading towardsmulti-robot systems. Furthermore, in many cases, the decomposing of acomplex task into parallel sub-tasks is the only reasonable approach or it atleast speeds up the performance. With some limitations, such as severalrobots working with the same sub-task, it can also increase the redundancy ofthe system. Redundancy itself can be of utmost importance in certain specialapplications, as in long-term and highly expensive planetary missions. Othernatural domains for multi-robots are tasks where the environmentalconditions set some limitations, for example for the size of the robot. In suchcases the only possible solution is in a way to divide the individual robot intoseveral smaller robots. As an example, one can consider the environment,where the robot has to go through some very tight spots, like in a nuclearpower plant, where a cleaning task has to be performed inside a highlyradioactive area. In addition, there can be tasks where successful completionof the mission requires close cooperation between the robots. Such a case isfor example the collective carrying of a large object. The performance of atask like this is far from being trivial. It requires some sort of interactionbetween robots, whether it is a direct communication, for instance throughradio transmissions or an indirect communication, such as through sensingthe forces in the object to be transported.

Collective transportation, as well as many other tasks normally related tomulti-robot systems, has a clear analogy with biological systems. A group ofants solves the problem through sensing the forces and torque in the object.Based on this information they change the direction of the forces accordingly


3

or, if needed, some ants change the position of their hold (Hölldobler andWilson 1990). Numerous parallels can be drawn from the naturalenvironment. Tested by evolution over millions of years these structures haveproven to be feasible in dynamic and hostile environments and can thusprovide valuable information and inspiration for similar types of engineeringtasks. When referring to ants, (Deneubourg 1996) even talks about "factorieswithin fortresses." He compares ant society to a well-defended factory thatgathers raw material from the outside world, and then uses it to maintain andextend its infrastructure and output, such as producing new individuals.

1.1 Interactions and Intelligent BehaviorTo clarify the difficult definition of intelligent behavior and intelligence ingenerally in robotics a simple example is presented. It is based on thefunctions of a real ant society, see, for example, (Sudd and Franks 1987).The initial situation is that members are randomly looking for food. If an antfinds food, it moves towards the nest leaving a trail of a chemical called apheromone, which will evaporate within a certain time period. If the foodsource is large enough and there are lots of ants, the result is a clear pathfrom the food to the nest. This kind of collective behavior can be easilytransferred to a multi-robot system. Let’s say that the task definition for therobots is to collect stones from an unknown environment. These stones aresituated in the environment such that there are some clear areas where thedensity of the stones is very high, i.e., there are clusters of stones, asillustrated in Figure 1.2a. When a robot finds a place where the concentrationof the stones is high it picks one up, and starts its way back to the base wherethe stones are supposed to be collected, see Figure 1.2b. Ants know thelocation of their nest and thus do not get confused about direction when theydetect a trail. This feature is easily replicated by installing a light source to thebase and by giving the robots the ability to detect the light gradient, i.e., dothe phototaxis. While moving toward the nest the robot carrying a stoneactivates its “pheromone” laying behavior. It just means that it starts to draw aline from the place it found the stone all the way back to the base, shown inFig 1.2c. The black "ink" of this pen stays on the floor only for a certain limitedtime.

Food

Robot

The Base

Robot carrying a stone

Figure 1.2a Initial situation. b One robot finds a stone and c While traveling, it starts to carry it to the base. leaves a trail of ink.


4

The robots have a pair of photo-diodes and an infrared LED. Because blackabsorbs infrared rays, the robots can detect the changes (black / white) onthe floor, shown in Figure 1.2d. Flexible traffic on this path is possible thanksto a simple obstacle avoidance behavior. This behavior makes the robot turnright (certain angle) and head in that direction when it detects an obstacle infront of it. After a short period of time the robot turns back to the left andreturns to the path. These simple rules create right-handed traffic, illustratedin Figures 1.2e and 1.2f. The trail stays active until the clustered stones arecollected.

Figure 1.2d Two other robots e The trail is strengthened. f Simple avoid obstacle behaviors detect the trail and provide collision free traffic. start to follow it.

After that the robots start looking for new productive areas. This kind ofcooperation with two separate "chained" swarms, one toward the base andthe other toward the stone concentration, can thus be accomplished withoutany active communication. The robots' operation is based only on theperception of the environment.

So what is the value of this example? It confirms the well-known fact, that “theintelligence is in the eyes of the observer” (Brooks 1991b). This simply meansthat if this scenario was presented to somebody he/she might think that thereare multiple robots on a mission to locate the stone quarry and then empty it.He/she might also think that the robots can communicate and have anefficient communication system. Furthermore, it presents how, with simplerules, we can mimic the performance of an ant society whose global behaviorseems to be highly complex. It demonstrates that through primitive and localinteraction between the members of an ant society (or a multi-robot system)global collective intelligence can emerge. It also illustrates that an intelligentbehavior is coupled with the environment where it is performed. In the aboveforaging scenario, an environment with evenly distributed stones would be avery difficult place for robots with the features described. Each stone retrievalwould initiate a path that would be useless for the others. Thus a behaviorconsidered to be intelligent in one environment is not necessarily so inanother environment.


5

1.2 Cooperative Multi-Robot Systems

Collective intelligence does not necessarily mean the same as cooperativebehavior. Selfish agents operating in the same environment can be said toexpress some sort of intelligence, but when can this be called cooperative.When multiple robots collect objects and avoid obstacles (static and dynamic)in the same restricted area, the behavior of the system is always collective.Robots’ primitive behaviors provide one type of intelligent behavior when themain criteria include survival and task completion. This kind of behavior is,nevertheless, far from being inherently cooperative. It becomes cooperative,however, when some sort of communication is included in the system.Whether it is passive (i.e., through the environment) or active, does notmatter. The main point is that robots share their knowledge with each other.In the above foraging scenario, the cooperation is achieved through passivecommunication, i.e., through the ink trail. An active communication wouldhave included inter-robot message transmissions about detected objectquarries. The literature on the matter of defining cooperation in multi-robotsystems is diverse. One definition for a cooperative behavior in robotics canbe found from (Cao et al., 1997):

Given some task specified by a designer, a multiple-robot systemdisplays cooperative behavior if, due to some underlyingmechanism (i.e., the “mechanism of cooperation”), there is anincrease in the total utility of the system.

This definition has one conceptual problem. It requires that the utility of thesystem can somehow be evaluated. In “toy problems” this is usually easy. Inforaging scenario example, the basic utility is simply the number of collectedobjects from the environment. It can include some other factors such as thespeed of the mission completion and survivability of the robots. Unfortunately,in most real world cases the utility itself is difficult to define not to mentionhow to accurately measure it. Each mission in a dynamic and complexenvironment is a unique case and comparison can be very difficult. However,if the tests are repeated extensively, some sort of statistical analyses aboutthe utility can be done. Another definition can be found in (Martinoli andMondada 1995). They stated that the difference between non-cooperativeand cooperative collective behavior is related to whether the task could in factbe completed with a single robot, (e.g., foraging) or whether a participation ofseveral robots is needed, as in transporting a large object.

Whatever the exact definition, one can easily predict that as the technologyimproves and some generic theory emerges, the number of distributedcooperative autonomous robotic system applications increases. Within a fewyears, various multi-robot systems move out from research laboratories intoeveryday life. Normal applications will include tasks like cleaning, monitoringand delivering in diverse environments. As an example, see Figure 1.3.Furthermore, a number of more revolutionary applications will probably alsoemerge. These will take multi-robot systems to distant planets, deep seanodule collection missions, mining operations, and when the time comesmaybe even inside human veins in search of tumors, which they will attack atclose range. In these scenarios the multiplicity of robots is more important


6

than the intelligence they possess. Just as in natural systems the intelligenceof the system emerges from the multiple interactions between the robots andthe environment. Deneubourg et al. (1992) state: “It is moving from the pointof view of nature lovers and admires of technology, to imagine that the nextrobots could be the nephews of modest animals that have been on the earthfor millions of years.”

Figure 1.3 An underwater robot society inside a closed aquatic environment.

1.3 Motivation of the Dissertation

Cooperative robots are constantly interacting not only with the dynamicenvironment, but also naturally with each other and with the persons who areusing them. This large variety of interactions will produce behaviors that willbe superior to those performed by current industrial robots. The robots will beequipped to survive as a part of a complex system, where cooperation isessential for their survival. The collective intelligence emerging from theseinteractions justifies calling these systems, at their highest level, robotsocieties. When the goal is to create a robot society, one is instantly facingmany problems, both conceptual and technical. First of all, what is a robotsociety anyhow? How can it be defined? Is it even possible to accomplishsomething artificial that could be referred as a society? And so on and soforth. When designing multi-agent systems, whether they are software orhardware agent based, some fundamental problems need to be solved.Parker (1994) cites the work done by (Bond and Gasser 1998) and listsfollowing elementary problems:

• How do we formulate, describe, decompose, and allocate problemsamong a group of intelligent agents?

• How do we enable agents to communicate and interact? • How do we ensure that agents act coherently in their actions? • How do we allow agents to recognize and reconcile conflicts?

To answer these questions this thesis presents a generic control architecturefor distributed autonomous robotic systems.


7

1.4 Main Contribution of the Dissertation

First, some conceptual definitions for terms like societal robotic agent androbot society are given. Related to these issues important matters such asinteraction, self-organization, adaptation and learning are also addressed.Based on these a generic control architecture for distributed autonomousrobotic systems is developed. All functions of distributed autonomous roboticsystems are obviously realized through their members. Members’ behaviorsare results from their own needs and from the constraints (dynamic by theirnature) set by the system, environment or operator. The developed model,shown in a simplified version in Figure 1.4, is based on a hierarchical three-layer model.

BEHAVIORAL LAYER

TASK LAYER

in ter-robotcom m unication

operator-robotcommunication

PERCEPTION-ACTION

COOPERATIVE LAYER

BEHAVIOURAL LAYER

TASK LAYER

inter-robotcom m unication


PERCEPTION-ACTION

COOPERATIVE LAYER

BEHAVIOURAL LAYER

TASK LAYER



PERCEPTION-ACTION

COOPERATIVE LAYER

BEHAVIOURAL LAYER

TASK LAYER



PERCEPTION-ACTION

COOPERATIVE LAYER

Figure 1.4 Simplified illustration of the control architecture.

It contains behavioral, task, and cooperative layers. Each of the layers isimplemented to the member. The behavioral layer is the most vital to a robot’ssurvival. The core of the layer is an Finite State Automaton (FSA) whichdetermines how to respond to certain stimuli from the environment, operator,and internal sensors in order to keep the robot operational. Furthermore, italso has a state for the actual performing of the task. This state has a lowerpriority than those serving self-sufficiency and operator's direct commands.The actual task achieving state is described more precisely on the task layer.Functions in this layer define how the robot should proceed when performingits tasks. At this level the robot tries to optimize its work by choosing the mostplausible strategies for each active task. The cooperative layer, on the otherhand, ensures that the society is operating as planned toward mission goals.For example, if a particular member, which is not doing so well, receives


8

information from the other society members, performing better than it, adecision to join that group should be made at the cooperative layer. Thisdecision is then combined at the behavioral layer to the information receivedthrough the robot’s own internal and external sensors. As a result, anappropriate action is taken and the total utility of the system increases.

The second main part of the contribution is based on the testing of the abovearchitecture on a novel distributed underwater robotic system, first presentedin (Halme et al., 1993). In this system the robots are operating in a liquidenvironment performing a combined exploration and exploitation task. Thenature of the system, along with the minimal perception system and poormaneuverability, forced us to develop new methods for navigation and pathplanning.

1.5 Thesis Outline

This thesis describes research where the goal has been to create a genericcontrol architecture for multi-robot systems, i.e. robot societies, for variousreal world problems. Nowadays, this kind of architecture is no longer just aresearch interest and technical challenge but very much a practical demand.As the technical readiness increases, the application domain widens rapidly.We are facing new tasks, which should be justifiably done with robotic agents.Many of these tasks can be performed more efficiently and robustly withmultiple robots. And some of them can only be solved with multi-robotsystems. In this work these multi-robot systems are named as societies basedon the analogy to the structures found in the natural environment. In the realworld, the behavior of a society is greatly influenced by the interactionsbetween the members, and between those members and the dynamicenvironment. By testing the developed control architecture not only insimulations, but also in a real underwater robotic system, the aim was to findvalid structures for tomorrow’s revolutionary robotic applications.

The organization of the dissertation is as follows:

Chapter 2: Related Work. This chapter reviews the current state of the art inthe field of Distributed Autonomous Robotic Systems. It also presents a shortoverview of the biological field as a source of inspiration, including the societalstructures found in many animal species. Furthermore, some relatedprinciples from the fields of Automated Guided Vehicles and AutonomousMobile Robots are also briefly discussed and various control architectures arepresented.

Chapter 3: Societal Robotic Agent. This chapter presents the main featuresof the elementary unit (the member) of a robot society. In addition, a formaldescription is given in the form of a Finite State Automaton.

Chapter 4: Definition of Robot Society. This chapter describes some principlecomponents when defining a robot society. In particular, inter-robot and robot-operator interactions are investigated.


9

Chapter 5: Society Model. This chapter presents the developed hierarchicalhybrid model for the control of multi-robot systems in cooperative taskperforming.

Chapter 6: Experimental Test-bed. This chapter illustrates the twoexperimental test-beds used in the dissertation, a multi-robot simulator and areal underwater multi-robot system. It also discusses the noteworthy problemof compatibility between results obtained through simulations and with realrobots.

Chapter 7: Mapping and Navigation. This chapter presents developedadaptive and robust mapping and navigation algorithms for underwater robotsoperating in a closed aquatic environment.

Chapter 8: Distributed Operations in Closed Aquatic Environment. Thischapter describes the actual experimental work done to verify how well anunderwater robot society performs the searching and destroying of distributeddynamic targets based on the developed control architecture and newnavigation algorithms.

Chapter 9: Summary and Conclusions. This chapter summarizes the maincontributions of the thesis and describes the work yet to be done.

1.6 Author’s Contribution within the Research Group

The work documented here was conducted during the period 1992-1999 in adynamic research group (from three to seven members). The author’sprincipal contribution to the group was the development of the feature basednavigation algorithm and the control architecture. The actual hardware designand communication system development were done mainly by PekkaAppelqvist. That work is to be published in detail in his forthcoming doctoraldissertation.

Chapter 2. Related Work

10

Chapter 2 Related Work

2.1 Introduction

The number of traditional industrial robots as part of modern factories has notincreased as dramatically as was expected earlier. These normal manipulatortype robots are more or less just machines carrying out their fixed programswith very little flexibility. Even Automated Guided Vehicles (AGVs), whichwere earlier considered triumphs for engineering science, seem nowadaysrather inflexible with their fixed tracks and centralized control structures. Ifthese machines are still called robots, what do we then call the machines thathave broken out of the factories and soon will be part of our everyday lives.These “creatures” will be moving among us, helping us accomplish varioustasks and in general have an important place in our society. So far thesekinds of machines have been given various names like service or advancedrobots. Their sizes, structures and behaviors are already very diverse. From afew centimeters up to huge machines, weighing several tonnes, they aregoing to become a growing necessity in our culture. To be able to do this,their intelligence level must far exceed present industrial robots. A clear trendseems to be toward adaptive, learning and highly interactive robots. Theserobots are constantly interacting not only with dynamic environments, but alsowith other robots and with the persons using them. This wide variety ofinteractions will produce behaviors superior to ones performed by currentindustrial robots.

Multi-robot research is clearly an inter-disciplinary field. It combines ideas andmethods from several disciplines including biology, computer science, andeven philosophy, as shown in Figure 2.1. Conventional robotics technologiesare linked to new and emerging ideas often inspired by the naturalenvironment. Nevertheless, only the latest inventions in the field ofelectronics, i.e., sensors and processors, have made it possible to transfersome of the fascinating and highly effective structures prevailing in natural


11

environment into functional engineering systems. When this development iscombined with the novel products from micro-machinery and bio-technology,some astonishing systems should be within our reach.

Background Discipline Technologies in Various Fields Conventional Robotics Technologies

Subjects of DARS

Biology

Computer Science

Physiology

Philosophy

Logic

Psychology

Autonomous Decentralized System

Self-Organization Phenomena

Artificial Life

Genetic Algorithm

Communication Network

Parallel Processing

Artificial Intelligence

Ergonomics

Distributed AI

Neural Network

Human Interface

Artificial Reality(Virtual Reality)

Multi-Robot Behavior

Self-Organization

Cooperative Operation

Distributed Planning

Distributed Sensing

Distributed Control

Coordinated Control

Shared Autonomy

Planning (Task Planning) (Path Planning)

Sensing

Sensor Fusion

Control

Tele-Presence(Tele-Existence)

Remote Control

Figure 2.1 Relationship between technologies. Adapted from (Asama 1994).

This chapter offers an overview of this wide area, nevertheless concentratingon two major conceptual cornerstones, namely biological and roboticresearch. First, some robust societal principles in natural environment andtested by time are illustrated. Various aspects of bacteria and social insectsare discussed and examples of complex behaviors are given. Next, as atechnical opponent for the solutions of nature, the main principles ofAutomated Guided Vehicles and Autonomous Mobile Robots are presented.These widely used machines are ancestors for elementary units in moderndistributed autonomous robotic systems. As the community of mobile robotresearchers grew considerably at the beginning of the 1980s, the problemsrelated to the use of traditional symbolic Artificial Intelligence methods forconstructing mobile robot architectures became evident. As a result, a newbranch emerged in the mid-1980s. Ever since, this approach, whetherreferred to as New AI or behavior-based, gained increasing popularity.Unfortunately, there seems to be no universal solution for the dilemma ofcreating a control architecture for mobile robots. Each approach has itsbenefits and drawbacks. As a result, common solutions nowadays aresystems where the benefits from both fields are combined into a singlesystem, called a hybrid. These controversial matters are also reviewed brieflyand some examples are given. After that, the concept of DistributedAutonomous Robotic Systems is presented through some well-knownoperational systems. Finally, some relevant new trends in the field arediscussed.

2.2 Biological Background

In this research we are trying to benefit the structures and functionsdeveloped during millions of years of evolution. In the natural environment thesurvival of the fittest has always been the guide line in the evolution process;organisms either adjust to changing environmental conditions or become


12

extinct. As a result, robust systems have developed, able to cope withsometimes very drastic and sudden changes. From an engineering point ofview the idea of a self-adapting robotic system is very appealing and there isdefinitely a growing need for these kinds of systems.

2.2.1 Societal StructuresThe term society for most people means our own society with its structures,laws and behaviors so complex, that most of the time we feel that we havedifficulty coping. Based on the knowledge about how we think and act it isquite safe to assume that there is no chance to create anything that comeseven close to human society. Luckily, humans are not the only creatures onEarth that have formed societies. Besides mammals, with a high level ofintelligence, there are other truly complex living organisms, whose evolutionhas taken a route just as brilliant. Instead of having one large unit, withsuperior intelligence, the natural environment has created systems where thisintelligence has been distributed over many small units. Even the lowest levelmicrobes, bacteria, have a tendency to form structures that have societalfeatures. Similar and more refined structures can be found in societiesformed by certain insect species, like ants, bees and termites. In thesesystems, as is usually the case in nature, the goal is never to find theoptimum solutions, but rather feasible solutions. These solutions areachieved through various levels of cooperation. In order to have a workingcooperation, these societal systems need some kind of basic rules tominimize or at least reduce the unnecessary interference and competitionbetween system members.

2.2.1.1 BacteriaUntil recently bacteria were more or less considered to be really simpleunicellular microbes, even though there have been studies proving theopposite even at the beginning of this century. It took decades before theprejudices were overcome, and the idea of bacteria as a multicellularorganism with abilities to form communities, hunt in groups and evencommunicate with each other was accepted (Shapiro 1988). All thesefeatures were previously considered to exist only in higher level organisms.Some of these features are actually quite astonishing. Myxobacteria havebeen commonly used as an example illustrating the aggregation and motionbehaviors among these small creatures. Time-lapse motion pictures wherethousands of bacteria are moving through extracellular slime extracted bythem in a very harmonic way, will convince even the most skeptical viewerthat social relationships actually exist among the colony members. Budreneand Berg (1995) show that Escheria coli bacteria are able to aggregate intostable patterns with remarkable regularity. This self-organization is mainlycaused by a one type of chemotaxis, where bacteria are move along thegradients of a chemical attractant they excrete themselves. Anotherfascinating feature can be found in some predatory species living in anaquatic environment. These bacteria have an elegant way to hunt and eat incomplex environment. The colonies secrete enzymes that dissolve theprotective outer layer of the prey microbes. The problem to be solved in orderto guarantee successful eating is to make sure that the enzymes haveenough time to have an effect on the prey colonies. The predator colony uses


13

the following strategy: it surrounds the prey (it takes actually the prey inside akind of pocket), and then emits the enzymes. This provides a maximal effectand the predator can use the nutrition of the prey colony without losingvaluable energy to the surrounding environment. Shapiro (1995) states thatbacteria derive clear advantages from multicellularity. These include strengthin numbers and the potential for specialization and the cellular division oflabor. Pennisi (1995) presents some new ideas concerning how bacteria usechemical messages to network with each other. When sometime in the futurerobot systems will have thousands of members, the simple but still effectiverules among these simplest forms of living things will hopefully provide uswith the means to control fully distributed artificial systems.

2.2.1.2 Social InsectsThe main biological inspiration in the multi-robot research has beenobtained from social insects, see, for example, (Wilson 1974). “Go tothe ant, you lazybones, consider its ways and be wise” (Bible,Proverbs 6:6-8) indicates clearly, that the incredible work done bythese little creatures has been known for thousands of years. In spiteof their low level of intelligence they have survived evolution’scompetition. These animals form seemingly chaotic structures,societies, that when studied more closely, show a high level ofdistributed intelligence. Their ability to survive comes from the highredundancy and structure of the society. Ant societies have severalfascinating features, such as chemical and tactical communication,asyncrohonicity, self-organization, coordinated behaviors, self-activation, stability, and so on. See (Hölldobler and Wilson 1990) forextensive presentation. These small animals, equipped with “simplerules,” can create rather complex global behaviors through localinteractions. If these rules could be properly isolated and formulatedthen maybe someday a real life-like society of robots with the samekind of astonishing robustness and adaptivity could be constructed.The reason why individuals have formed societies is the fact thatthey provide mutual advantages to the members of the society. Inthe natural environment these advantages include things like a moreefficient search for food, collective defense, coherent transferring,etc.

Kube and Zhang (1992) suggest that evolution in social insects hasreplaced human higher level reasoning by increasing the number ofsensors, which are directly coupled to behaviors. Behavior in socialinsects is usually considered a “prewired” program, which isexecuted when appropriate sensory stimuli are detected. Theanalogy to robotics is thus quite obvious. In (Kube and Zhang 1993)work done by Wilson, Durlach and Roth in 1958 is cited as anexample of a collective behavior invoked by a certain sensorystimulus, in this case by chemical odor. In an ant society, theworkers dispose of dead ants from the nest by carrying them out to a refusepile. By taking small amount of acetone extracts from dead corpses and thentouching live ants with it, Wilson was able to make the workers carry thoseunfortunate ants out of the nest to the refuse pile. For researchers one of the


14

biggest problems in social insect studies is how to conclude collectivebehavior from individual behaviors. In (Pasteels et al., 1987) it was pointedout that “collective behavior is not simply the sum of each participant’sbehavior, as others emerge at the society level.” The main question isundoubtedly how an ant society can build such complex structures eventhough individual ants seem to work in a rather inefficient way. In (Kube andZhang 1992) several possible approaches are listed to find the solution to thisquestion. First of all it may be that individual ants’ behaviors are much lessrandom than they appear. Ants are not considered as random particles butrather animals which can communicate and have various ways to dividelabor. Another view is to admit randomness at the individual ant level, butalso recognize that their collective reliability compensates well the individualinefficiency caused by so-called behavioral variance. It is argued that thisvariance will eventually ensure the performing of the social activity (such asnest building).

The mechanism involved in task-achieving collective behavior is a kind ofpositive feedback, which can be also described as doing what the neighbor isdoing (Kube and Zhang 1993). When this is coupled with simple rules andinvoked by a certain stimulus this kind of system can generate a performancethat exceeds the sum of its parts. Beckers et al. (1995) use the termstigmergy to describe a similar collective behavior. They defined stigmergy(term invented by French biologist P. Grasse in 1959) as “the production of acertain behavior in agents as a consequence of the effects produced in thelocal environment by previous behavior.” The use of stigmergy can also berelated to other features besides building structures. These naturally includetrail recruitment in ants, where the use of pheromones among workers isbased on the local conditions (i.e., the better the quarry, the higher thequantity of the pheromone). This trail recruitment can be rather complexincluding the selection and exploitation of the richest food source in theneighborhood or the selection of the shortest path between the nest and apotential food source (Beckers et al., 1995). The trail following in social andcellular systems is studied in detail in (Keshet 1993), where various modelsare presented to describe this behavior. Camazine (1993) suggests, that thesociety level complex behaviors (in this case self-organizing processes likepattern formation on the combs of honey bee colonies, collective nectarsource selection by honey bees and brood sorting in ants) develop as anautomatic consequence of large sub-units interacting concurrently. In additionto these papers there are numerous studies (see, e.g., (Corbara et al., 1993),(Deneubourg et al., 1990), (Goss et al., 1993), and (Gutowitz 1993)), wherethe behaviors of real animals are modeled and then transferred intosimulators. All together, these examples show that social insects can solveglobal problems based on interactions between single workers, and betweenworkers and the environment. Terms like “the social machine” or “thesuperorganism” are thus well deserved.

2.2.1.3 Types of Animal SocietiesAnimals proof positive that working solutions can be obtained without anyneed for complex control and communication structures. There are numerousdifferent animal societies. Parker (1992) presents a categorization made by


15

Tinbergen (1953), where societies have been divided into two broadcategories: those that differentiate and those that integrate. The social insectcolonies are good examples of differentiated societies. The members of thesesocieties are blood relatives that have a division of labor. Members aredivided into casts and the individual exists for the good of the society, and istotally dependent upon the society for its existence. These groups canaccomplish tasks that are impossible for an individual member to achieve.Another interesting example of these kinds of societies is the one formed bynaked mole rats. These rodents form societies that have all the features ofeusociality or “true sociality”, including at least the following characteristics:two generations live together, reproduction is restricted to a few individualsand non-breeders cooperate in caring for the offspring of breeders. See(Sherman et al., 1992) for details. The other type of societies is formed basedon the attraction of individual animals to each other. The members are notblood relatives. These individuals are driven by selfish motivation, whichmakes them form groups in order to benefit from them. Good examples ofthese kinds of societies are wolf packs and schools (synchronized behavior)or shoals (loosely coordinated, see Figure 2.2) of fish. In these cases thesociety exists for the good of the individual and not the other way around. Thereasons that these two types of societies have evolved in the naturalenvironment are similar for both types of societies: protection againstpredators (or the benefit of hunting together), and the coordination of tasks tobe accomplished (collecting food, forming and moving in swarms, etc.).

Figure 2.2 A schoal of carps. See (Helfman et al. 1997) for details of fishes as social animals.

Another approach is to study how cooperation and altruism can emergeduring evolution. “Weak” altruism is defined as a behavior that benefitsanother individual more than the one producing the behavior. The otherextreme is the case, where an individual does something for the good of theothers even though it causes some harm to itself. This is called strongaltruism. Both of these behaviors are common in societies called “ultrasocial.”These kinds of systems have individuals who are ready to sacrificethemselves for the defense of the other society members. There are only afew of these kinds of systems. These include certain insect species andhumans.


16

2.3 Ancestors in Engineering

2.3.1 Automated Guided Vehicles (AGVs)Over the last 15 years the development of the factory automation has beenextremely rapid. The continual battle over markets and market sensitivity toeven small changes have caused large modifications to corporate structures.In order to survive, companies have been forced to seek new operationmodels. Efficiency, flexibility and controllability have been the keywords in thisstruggle for existence. The long through-put times, immense storage of rawmaterials, unfinished and finished products of earlier are now long gone, atleast in the majority of companies. A traditional functional layout has revertedto a combination of cells, product families, and so on. One of the mainchallenges was to increase actual working time at the factory. There weretraditional means of accomplishing this: reducing set-up times, increasing themodularity of the products, and so on. Nevertheless, the increase inunmanned production had the greatest effect on increased work time. In thispoint acronyms such as FMS (Flexible Manufacturing System) and CIM(Computer Integrated Manufacturing) became a regular part of ourvocabulary. And as an essential part of these concepts the material flows hadto be re-organized. Automatic storage, portal robots and Automated GuidedVehicles (AGVs) started to be widely used. In (Muller 1983) an AGV isdefined to be a vehicle whose task is to transport materials usually along thehorizontal level. These vehicles are computer controlled, battery-driven anddriverless electric tractors. There are four places in a modern factory whereAGVs are most frequently used: part of an assembly system, transportationof the products, material handling, and operating as a working bench or as apart of a flexible conveyor belt. In a wider context AGVs can be used inalmost every kind of production from the car industry to paper manufacturing.

2.3.1.1 Basic StructuresEven though there are several companies manufacturing AGVs, the followingfive major components are always onboard: the material handling equipment,chassis, energy source, moving and steering systems and computer basedcontrolling system. The material handling equipment is the actual interface tothe manufacturing system. Usually it is some kind of tool especially designedfor the job to be done, like a roller conveyor on top of an AGV. A newerapproach is to place a portal robot onboard an AGV and use it as a flexiblepart of the working machines loading system. The chassis of the vehicle isalso totally dependent on the task. For example, the weight of the material tobe handled naturally dictates constraints for the design. Nowadays, thesecurity aspects have been taken into consideration much more than earlier:the safety bumpers and ultrasonic obstacle detection sensors are common intoday's AGVs. The energy source usually takes the largest part of thevehicle's space. The energy fill-up can be different in different systems: arobot can go to a station were the used batteries are replaced or it can go to astation where it connects itself to a "force line" for loading. The movingsystem in indoor environment is always electric but outside some kind ofcombustion engine can be possible. There are numerous different steeringsystems, the placement of the wheels and the number of steering wheels,which produce the required movements. The cheapening of the processors


17

has made it possible to use more efficient components for the onboardcontrolling task and thus have better and faster control systems.

An AGV -system can be divided both in the physical and in the informationsense into two separate levels. In the physical sense, the division is into thetrucks and traffic system. In the information sense this division is into truckcontrol and into traffic control systems.

The AGV guidance system usually contains the following parts (Muller 1983):

• The actual guiding, which means to hold the vehicle in a desired route.This guidance is normally based on a cable which is laid into the factoryfloor. This cable contains wires which provide an inductive field for theAGV to follow. The other way is to follow optically a line painted on thefloor. This kind of guidance is easy to implement, but there are severalpotential interferences, for example dirt. In the future the tendency will beto eliminate any fixed physical routes and thus increase the flexibility.

• The selection of the route, which means to get the vehicle where it issupposed to go. These methods are based on the knowledge of the actualplace and direction of the vehicle. These values are then compared to theinitial correct values and the corrections are made based on thedifferences.

• The control of the moving / steering motor, which means to control theactual movement of an AGV (its acceleration, velocity, etc.)

• The control of the material handling equipment, which is totally dependenton the case.

Besides these tasks the AGV has to be able to transform information to theother machines in the network and also to receive commands. In smallsystems there is no actual need for computer-based central control, and theaddresses are given to AGVs at stopping stations, such as in loading stations.The other possibility is to program the vehicle to travel a fixed route, and ifneeded, the station to be passed gives a stop signal. During loading, thestation provides the address for the delivery of the load and so on. If there isno central control system, AGV should be capable of choosing its own route.Nowadays, it is quite common to combine the centralized and the vehicle'sown control system in one operative system.

Traffic control has basically two tasks: to take care of the communicationbetween the network and the vehicles and to prevent vehicles from collidingwith each other. In a traditional AGV system this is done by using the cablenetwork's features. In newer vehicles obstacle avoidance is implemented, forexample, with ultrasonic sensors. Traffic control system in normal systems isa microcomputer system, which can be found from the lowest level of thecontrol hierarchy. Its main tasks are:

• closing and opening some part of the network• passing on the real position information to route guidance• giving the desired goal coordinates to the vehicle's computer


18

Besides these tasks, the traffic control controls the security equipment, opensthe doors and transfers commands to the material handling stations.

2.3.1.2 Central vs. Distributed ControlAs stated earlier, large AGV systems usually need some kind of centralizedcontrol in order to avoid collisions, to control the driving at crossings and tocommunicate with the other systems. In distributed control, the central or maincomputer decides which vehicle gets which job and where the rest of thevehicles are waiting for their next mission. The information on the state of thenetwork is delivered by the block (part of the whole system) level's computer.When a certain AGV has been selected for a certain task, the information ofthe mission is transferred to the block's computer where the AGV is at thatmoment. This computer takes care of the routing in its block. It communicateswith the AGVs, with other blocks' computers, and with the main computer. Incentralized control the main computer takes care of everything. It has a certaintasks handling software which communicates directly with all AGVs. This unitcontinually monitors the position of each and every AGV position. This centralmachine gives operation instructions to the AGVs only up to the next address.

2.3.1.3 The Need for FlexibilityAnother definition for a modern AGV could be: an AGV is a vehicle whichoperates driverless under distributed/centralized control as a firm part of amodern flexible factory. The AGV should be free of any fixed routes, centralcontrol and in general it could (or should) be called an Autonomous MobileRobot. These robots react fast to demands from the production line. Taskallocation should be done on-line at the robotic level.

2.3.2 Autonomous Mobile Robots (AMRs)When one tries to define what term autonomous mobile robot actually standsfor, one understands very fast that there is no exact definition for it.Sometimes, a machine is said to be an autonomous mobile robot “when it canmove as a whole in some controlled way with some degree of autonomy andperform a certain task” (Todd 1986). This sentence is very general and it doesnot reflect how mobile, or how autonomous the robot has to be. The robot canmove in two dimensional or in three dimensional space, it can operate fullywithout any guidance from the operator, or it can receive instructions now andthen from the operator. Meystel (1991) gives another definition: “Anautonomous mobile robot doesn't require any communication line in order tounderstand its "master" and properly perform the assignment.”

2.3.2.1 How It All StartedOne of the earliest projects combining a robot with Artificial Intelligence wasthe robot Shakey at the Stanford Research Institute in 1967, see (Nilson1984). The robot was linked to a computer via radio and was programmedwith FORTRAN. Even though Shakey was "the robot" for several years,based on the integration of the hierarchical levels of the software, it was stillfar from being a robot that could be called completely autonomous. After thatseveral projects were started to study wheel based mobile robots, see, forexample, (Moravec 1983). It seemed that suddenly everybody was anxious tobuild mobile robots.


19

After the ground was laid with the wheeled robots, one of the next goals wasto build a robot that could walk. Ohio State University's "Hexapod" (1976-1977) is usually considered to be the first operative multi-legged system in theworld. It had electric motor drives for all motions and worked in a "master-slave" regime with an umbilical cord. The main concern for this robot was tokeep the motion of the body uniform while the system of legs performed somecomplex motion (Meystel 1991). One widely referenced robot of the time wasbuilt by H.R. Everett at the Naval Postgraduate School (1980-1983), see(Everett 1995). It was one of the earliest robots having multiple sensors andbeing fully controlled by its on-board computer. The primary target for thisrobot, named Robart I (the second generation Robart II), was to serve as amobile test-bench for research and experimentation in the areas of ArtificialIntelligence (AI), computer interface techniques, and other similar problemfields. Robart II had many features now common to mobile robots.

During the 80s and early 90s, there have been numerous autonomous mobilerobot programs in progress and today an active mobile robot researchprogram at a good technical university is almost a fact. Earlier, the robot wasthe actual object of the study, but now it is usually only an instrument in thestudy. The mobile robot, the wonder of the 70s and 80s, has became a basictool for many applications outside the safety of research institutes, as shownin Figure 2.3. For comprehensive presentation of robot evolution, the readershould see (Rosheim 1994). It reviews the fascinating history of thesemachines.

Figure 2.3. Autonomous mobile robots are not necessarily turtle type test-beds suitable onlyfor office floors. Helsinki University of Technology’s Automation Technology Laboratory hasdeveloped over the past 15 years various types of AMRs. These include, for example, ARSKA(four-wheel drive) all-terrain vehicle, MECANT (giant six-legged) walking machine, RoboBall(spherical robot), a rowboat capable of operating autonomously or according to voicecommands and many more. See www.automation.hut.fi for more details.


20

2.3.2.2 Traditional SubsystemsWhen a traditional mobile robot is in concern, it is relatively easy to list themain subsystems that it should have:

• The energy system can be purely electric, meaning that the energy comesfrom an accumulator, or from batteries, or as well from solar cells or from"burning cells." The power source can also be a combustion engine, whichchanges the energy of the fuel to electricity. Both systems have theirbenefits and drawbacks. A battery-based system can be small, but thelifetime is normally rather short. On the other hand, if a combustion engineis used, a longer operation time can be achieved, but noise and gases cancause serious problems, especially if working indoors.

• The actuation system takes care of the actual mechanical motion of therobot. The system can use electric, electric-hydraulic or electric-pneumaticmeans to accomplish the movement. An electric actuation system is verycommon. The electric-hydraulic system combines electric control signalsand actuators, which use liquid in some form, such as in hydrauliccylinders. The electric-pneumatic system uses air as a means of activationas in pneumatic grippers.

• The moving system gives the robot its mobility. The most common way isto use wheels or tracks. These systems can also be mixed together. Overthe past 15 years walking (and climbing) machines have also become veryimportant targets for several studies, see (Virk et al., 1999). Controllingthese robots is far more difficult than the control of normal ones. Inaddition to these two main systems, there are a few others though lessfrequently used, like propel, propulsion and hovercraft based just to namefew.

• The sensor system has two main tasks. It follows the robot's inner stateand monitors the environment where the robot is operating. This systemusually consists of several types of sensors, the ability to control thesesensors and to process the information from them. Sensors must becarefully selected to get the best possible coverage of all variables, whichare to be measured. The most common sensor system used in mobilerobots includes various combinations of camera, ultrasonic, infrared andtactile components. For comprehensive presentation, see (Everett 1995).

• The piloting system's main task is to control the moving and energysystems. This subsystem operates as a link between separatesubsystems and actuators. It contains all the necessary algorithms tooperate the robot according to what is needed.

• The navigation system has only one function to accomplish: it has to beable to define the robot's place and orientation in some defined coordinatesystem. It can be either absolute or relative. The relative coordinatesystem is usually used when the robot operates in a certain restricted areawhich boundaries are known. The absolute coordinate system is usedwhen the robot's place in the global coordinate system is essential for thetask. This system is usually used in robots which are working outdoors inan unknown environment. The robot's navigation system can be one of


21

several systems available today or it can be a combination of them. Themost widely used systems include dead reckoning, inertial, beacons,visual + landmarks and GPS (Global Positioning System). The normalpractice is to use a hybrid system, where two or more of the abovesystems are combined. This increases the fault tolerance of the systemand provides a way to nullify navigation errors. Various sensor fusionmethods can be used for the actual combination.

• The planning and guiding system's main task is to make sure that therobot gets to its goal location and if possible via the best route. Thecriteria, which dictate the route to be used depend greatly on the case.Sometimes the time used is the key factor and sometimes the energy isthe primary resource to be optimized. Planning is no minor task in anunknown environment.

• The man/machine interface ensures that the operator can communicatewith the robot as well as monitor it. Normal communication systemsinclude radio and light-based solutions.

• The work system takes care of the actual task performance. The form ofthis system is completely based on the mission. It's very common thatthese systems include manipulators, some work tools (e.g., a drill or agripper) or inspection equipment (e.g., camera). This system can beseparately controlled, and in many cases, even though the robot itself isautonomous in moving, the actual work task can be tele-operated.

2.4 Transition of Paradigms in Robotics

The term Artificial Intelligence covers a wide field of methods, all aiming toproduce computational forms of intelligent systems. Earlier normalautonomous mobile robot architecture was deliberative and hierarchical, suchas NASREM (Albus et al., 1987) or Hierarchical Nested Controller (Meystel1987) just to name two of the widely referenced ones. These systemstypically have a clear subdivision of functionality into modules thatcommunicate with each other in a predetermined manner. These plannersrely on world models and the subdivision usually depends on spatial andtemporal constraints, as illustrated in Figure 2.4.

Strategic GlobalPlanning

TacticalIntermediate Planning

Short-Term Local Planning

Actuator Control

SPATIAL SCOPE

HIERARCHICAL PLANNER WORLD MODEL TIME

HORIZON

GlobalKnowledge

LocalWorldModel

Immediate SensorInterpretations

Long-Term

Real-Time

Global

Immediate Vicinity

ACTIONS SENSING

Figure 2.4 Deliberative/hierarchical planning. Adapted from (Arkin 1998).


22

Since 1985 the robotics related field of AI has been separating into two maincategories: the first is the main-stream approach and known as knowledge-based, classical AI or top-down AI. The new approach, on the other hand, canbe called behavior-based, new AI or bottom-up AI. In the Introduction part of(Allen et al., 1990), Nils J. Nilsson postulates the following:

”Because there are animals that seem too simple to be capable ofplanning but are nevertheless capable of quite complex behavior(consider the bee, for example), some artificial intelligence researchers(Brooks 1986) have decided to concentrate first on systems that actbut that do not plan. It is as if those researchers believed that theontogeny of artificial intelligence must recapitulate the phylogeny ofnatural intelligence. The present volume contains most of the importantwork of researchers (myself included) who predict that we can bypassthose eons of evolutionary history that produced only dull animalswhich made no plans. We want to build machines straightaway thatwould rank high on the evolutionary scale, perhaps machines as goodas or better than humans are in terms of thinking about what we aregoing to do before doing it. It’s worth a try! And judging from the resultsreported here, we are making good progress.”

This is quite strong comment! It describes machines that should be betterthan humans in terms of thinking before doing. It is clear that this approach ismore suitable for non-situated and non-embodied intelligence research thanfor studying autonomous mobile robots. A chess playing machine Deep Bluecan defeat even the Grand Masters, but is it actually intelligent? Is it enoughto go through a huge data base containing statistics of almost everycompetition game ever played, or should there be some kind of ability tomake decisions based on incomplete data in a dynamic and non-structuredworld?

In (Maes 1992) the new approach is presented along with a comparison withthe traditional approach. Maes lists some typical characteristics normallyrelated to both of these approaches. Knowledge-Based AI features includethe following:

• It models isolated, often advanced competencies, like chess playing,where the expertise has more depth than width.

• Systems are closed, meaning that their only connection to theenvironment is usually through the user, which describes the environmentwith a symbolic language.

• These systems deal with one problem at a time.• Their internal structures are static (excluding an interpreter).• They are not adaptive to changing situations.• In the case of an autonomous system a central system with all of the

above characteristics is augmented with a perception module and anexecution module which take over part of the role of the human interface.


23

In contrast, Maes states, that Behavior-Based AI typically studies thefollowing type of systems:

• The system usually has multiple integrated competencies.• The system is open or situated in its environment.• The emphasis is on autonomy.• Rather than on knowledge, the emphasis is on the resulting behavior of the

system. The internal structures are active “behavior producing” modules asopposed to static “knowledge structures.”

Arkin (1998) presents a similar division for these two main fields, namingthem as reactive and deliberative, as shown in Figure 2.5.

DELIBERATIVE REACTIVE

Purely Symbolic Reflexive

SPEED OF RESPONSE

PREDICTIVE CAPABILITIES

DEPENDENCE ON ACCURATE, COMPLETE WORLD MODELS

Representation-dependentSlower responseHigh-level intelligence (cognitive)Variable latency

Representation-freeReal-time responseLow-level intelligenceSimple computation

Figure 2.5 Robot control system spectrum. Adapted from (Arkin 1998).

Even though this kind of division may seem to be quite controversial and hasbeen the reason behind many lively debates in the literature, the fundamentaldifference is nevertheless quite clear. In his paper, which actually representsthree-layer hybrid architectures, Gat (1998) states: “Subsumption (i.e., certainbehavior-based control architecture) achieved dramatic early success in thearea of collision-free robot navigation. While SPA (Sense-Plan-Act) -basedrobots were pondering their plans, subsumption-based robots were zippingaround the lab like R2D2. By the common metric that speed equalsintelligence, subsumption appeared to be a major breakthrough.” The needfor symbolic representation and accurate planning was clearly challenged withopen and highly biologically inspired systems, where behavioral modulesprovided direct connections between sensors and actuators. See (Maes1995) for overview.

This new bottom-up approach has been implemented to numerous physicalsystems, i.e. to mobile robots that are situated in the world and are more orless autonomously performing some tasks, see, for example, (Brooks and Fly1989), (Brooks 1990a), (Brooks 1990b), (Brooks 1991a), and (Brooks 1992).


24

The control structure of these “Robot Beings” or “Artificial Creatures” wasusually done with computational components running in parallel with directcouplings to sensors and actuators. This kind of realization differs greatlywhen compared to an older framework, where sensing, modeling, planningand action components were connected more or less in a sequential way.Brooks (1991b) lists four key aspects that describe this approach:

• Situatedness: “The robots are situated in the world: they do not deal withabstract descriptions, but with the here and now of the environment thatdirectly influences the behavior of the system.” “The world is its own bestmodel“

• Embodiment: “The robots have bodies and experience the world directly -their actions are part of a dynamic with the world, and the actions haveimmediate feedback on the robots’ own sensation.” “The worldgrounds regress”

• Intelligence: The robots are observed to be intelligent (more than just CPUbased). “Intelligence is determined by the dynamics of interaction with theworld”

• Emergence: The intelligence of the system emerges from the system’sinteraction with the world. “Intelligence is in the eye of theobserver”

The first point states that map building has always been one of the hardestproblems to solve in robots operating in dynamic environments. There arevery few traditional systems that are capable of operating in real worldconditions, and they have one common feature: they require a lot ofcomputation and a highly sophisticated perception system. Brooks underlinesthe fact that in many cases, as is also case in the natural environment, theagent does not require very precise map of its environment. In many cases nomap is needed at all. This is definitely the case when the mission and theenvironment are suitable for more or less random exploring. The otherimportant point is the fact that only by building real robots will we be able toenvision what is happening in the environment. The number of variables istoo large, so by the time we could construct a “perfect” simulator, particularlywhen it comes to the environment, we would already have done the testingwith a real robot. This problem between simulated and real robots has beenunder extensive research, see (Jakobi et al., 1995). The last two points dealmore or less with the main subject of this thesis, i.e., that the intelligence indistributed autonomous robotic systems is obtained through diverseinteractions (Brooks 1991c).

The two best known examples of behavior-based systems are probablysubsumption architecture (Brooks 1986) and motor schemas (Arkin 1987).Both of these approaches have been tested in numerous real world roboticsystems. In subsumption architecture the traditional vertical Sense-Model-


25

Plan-Act model was replaced with a new hierarchical decomposition. In thismodel each task-achieving behavior is represented as a separate layer.These layers operate towards individual goals concurrently andasynchronously. A behavior can be implemented with an augmented finitestate machine (AFSM) or with behavioral abstractions (rules). Stimulus orresponse signals can be suppressed or inhibited by some other behaviors, asis shown in Figure 2.6.

Behavioral ModuleInput Wires

Inhibitor

Suppressor

Output Wires

Reset

I

SR

Figure 2.6 Basic module in subsumption architecture.

There are no central world models or global sensor representations. Sensorinputs are connected directly to the appropriate behaviors. Higher-levelcompetencies can subsume simpler lower level behaviors through a fixedpriority hierarchy, see Figure 2.7.

Collision-FreeWandering

Convex-BoundaryTracing

General BoundaryTracing

Stroll

Avoid

Align

Correct

Figure 2.7 Incremental interaction of the basic navigation behaviors resulting in boundarytracing. Adapted from (Mataric 1990).

The other major approach, i.e., motor schemas, is best known for the workdone by Ronald Arkin, see, for example, (Arkin 1987), (Arkin 1989), and(Arkin 1990). The motor schema differs from other behavioral approachesaccording to (Arkin 1998) in several ways:

• Behavioral responses are represented with vectors (potential field method)providing a continuous response encoding.

• Coordination is achieved by vector addition.


26

• No predefined hierarchy exists for coordination. Behaviors are configuredat real-time based on robot’s intentions, capabilities and environmentalconstraints.

• Each behavior can contribute in varying degrees (i.e., relative strengths) tothe robot’s overall response.

• Perceptual uncertainty can be considered directly (an input) to behavior’sresponse.

Motor schema behavior contains one or more perceptual schemas(sometimes divided into perceptual subschemas), which provide theenvironmental information for that behavior. Each motor schema produces anaction vector (both orientation and magnitude components), that defines howthat behavior would like to move the robot. The outputs of several motorschemas are then combined through vector summation. As a result the robothas a single global vector (usually somehow normalized) that defines thedirection and speed of the next movement. This procedure is repeated againas fast as possible. As a result we have a robot capable of movingcontinuously and smoothly in a dynamic environment. In Figure 2.8 receptorand transmitter schemas are expressed as separate units to provide a kind ofhomeostatic control, see (Arkin 1998) for details.

ES1

ES2

ES3

INTERNALSENSORS

MOTORS

TRANSMITTER SCHEMAS

VECTOR

ROBOTMOTOR SCHEMASENVIRONMENTAL SENSORS

ENVI

RO

NM

ENT

BROADCAST MEDIUM

RS1 IS1

IS2

TS1

TS2

RS2 RS3

PS1

PS3

PS2

Key:RS - Receptor SchemaTS - Transmitter SchemaPS - Perceptual SchemaMS - Motor SchemaIS - Internal SensorES - Environmental Sensor

Figure 2.8 Process diagram for reactive component of Autonomous Robot Architecture(AuRA). Adapted from (Arkin and Balch 1997)

Inspired by the good results achieved by the early works, e.g., abovesubsumption and motor schemas architectures, as well as challenged by theemerged limitations and problems, a wide range of behavior-basedarchitectures surfaced. These include, for example, Animate AgentArchitecture (Firby et al., 1998), see Figure 2.9, Circuit Architecture (Kaelblingand Rosenschein 1991), Action-Selection (Maes 1990), DAMN (Rosenblatt1995) and Samba (Riekki 1998).


27

Task

Task

TaskTask

Task

Task Agenda RAPSystem

World ModelRAP Interpreter

RAP Libary

RAP

RAP RAP

RAP

Task Subtasks

State RAP Methods

Enable/Disable Skills

Signals

Success / FailureSketchy Plan

CRL System

Skill Manager

Disabled Skills

Skill 1

Skill 8

Skill 5

Skill 3

Skill 7Skill 6Skill 4Skill 2

Enabled Skill Schedule

Sensor Values Channels

Read Read Write Constrain

Sensors Actuators

SignalsSkillsSkills

Figure 2.9 Animate Agent Architecture. Adapted from (Firby et al., 1998). This two-layerarchitecture can be considered to exist between behavior-based and hybrid architectures. Atthe lowest level CRL system controls concurrent perceptual-motor skills in real-time. The RAP(Reactive Action Package) system is designed to carry out multi-step plans in dynamicenvironments.

Arkin (1998) presents four assumptions normally listed as reasons why to usepurely reactive systems: the environment lacks temporal consistency andstability, the robot’s immediate sensing is adequate for the task at hand, it isdifficult to localize a robot relative to a world model and symbolicrepresentational world knowledge is of little or no value. Arkin continuesstating that in some environments these assumptions are not simply true, forexample in some restricted and highly structured area, with no extra dynamicobjects other than the robot, deliberative planning can and should beconducted. Noreils and Chatila (1995) commented on the necessity to haveplanning included to a control architecture: “We are not interested in insectsthat wander aimlessly in the environment, nor in machines that performrepeatedly the same task, but in robots capable of achieving complex andvariable tasks in various environments and situations. To fulfill their task,these robots must plan for their actions, i.e., be able to predict theirconsequences, and to order them to achieve some goal. A robot should beprogrammable.” However, when the robot is to work in a highly dynamic andun-structured environment, behavior-based reactivity is definitely needed.This is the main reason why so many of the modern autonomous robots areusing some sort of hybrid controllers. Arkin (1998) cites work done by Lyons(1992) and states, that there are three different ways to combine planning andreaction: hierarchical integration (different activities, time scales and spatial


28

scope), planning to guide reaction (advice, configuration) and coupledplanning-reacting (each guiding the other concurrently).

Autonomous Robot Architecture (AuRA) was one of the first architectures(Arkin 1986) to combine deliberative planning and reactive (schema based)control systems into a single hybrid one, see Figure 2.10. The hierarchicalplanning system consists of a mission planner, spatial reasoner and plansequencer. This system is coupled to the reactive schema based controller.The highest level of AuRA is a mission planner that sets the high-level goals,for example given by the operator. The spatial reasoner uses a prioriknowledge to come up with a sequence of sub-paths that the robot musttravel in order to complete the its mission. The plan sequencer then translateseach of these sub-paths into a set of motor behaviors. After that these setsare sent to the robot and the reactive control part takes over. The operation ofthis part was explained earlier in this chapter, see Figure 2.10. Thedeliberative part “sleeps” until a failure has been detected. In case of a failurethe deliberative part tries to correct the situation. This happens throughreinvoking one stage at a time starting from the plan sequencer. If it can notsolve the problem the spatial reasoner is activated and finally if necessary themission planner contacts the operator in order to get help.

Plan Recognition User Profile

Spatial Learning

Opportunism

On-lineAdaptation

Learning User Input

User Intentions

Spatial Goals

Mission Alterations

Teleautonomy

Mission Planner

Spatial Reasoner

Plan Sequencer

Schema Controller

Motor Perceptual

Actuation Sensing

REPR

ESENTATIO

N

HierarchicalComponent

Reactive Component

Figure 2.10 High level AuRA Schematic. Adapted from (Arkin and Balch 1991).

Another well documented hybrid architecture is Atlantis, see, for example,(Gat 1991) and (Gat 1998). Three-layer Atlantis, shown in Figure 2.11,combines a deliberator (planning and world modeling) and controller(managing collection of primitive activities) by using a sequencer. Thesequencing layer operates through conditional sequencing based onsuccessful subtask completion or the detection of a failure. When thesequencer notices a problem, it can ask help from the deliberator. The outputof the deliberator is considered as advice and is not thus always followed.

Activation

Results

Status

Query

CONTROLLER

SEQUENCER

DELIBERATOR

SENSORS ACTUATORS

Figure 2.11 The Atlantis architecture (Gat 1998).


29

(Lyons and Hendriks 1992) presented yet another way of combining reactivityand planning. Their Planner-Reactor architecture, shown in Figure 2.12, usesa planner continuously to modify (to adapt) an executing reactive controlsystem based on changes in the environment and agent’s underlying goals. Ina way, planning is used to remove or correct errors in performance as soonas they happen.

GOALS

ADAPTATION

PERCEPTION

ACTION

SENSING

PLANNER

REACTOR

WORLDREACTIONS

PERCEPTIONS

Figure 2.12 Planner-Reactor Architecture. Adapted from (Arkin 1998).

Other hybrid architectures include SSS (Connell 1991), Agent Architecture(Hayes-Roth et al., 1993), Generic Robot Architecture (Norelis and Chatila1995), and (Schneider-Fontan 1999). For a comprehensive presentation onthe matter of behavior-based robots, the reader should see (Arkin 1998). Ithas over 500 references concerning related issues.

2.5 Distributed Autonomous Robotic Systems

Due to the extensive work with single autonomous mobile robots in the early1980’s, and the development of suitable algorithms and hardware, a new fieldworking with multiple autonomous robots was born. Fascinated by the hugepossibilities lying ahead and challenged by endless problems emerging due tomultiplicity, a large number of researchers started to work with distributedautonomous robots, see (Taipale and Hirai 1992). Cao et al. (1997) state, thatduring 1987-1995 well over 200 papers have been published in the field ofcooperative mobile robotics alone. Over the past few years, the number ofrelated papers has been even bigger and this seems to be the trend of thefuture as well (see, DARS92, DARS94, DARS96 and DARS98).Asama (1994) defines Distributed Autonomous Robotic Systems (DARS) asfollows: “Distributed Autonomous Robotic Systems are the systems whichconsist of multiple autonomous robotic agents into which required function isdistributed. In order to achieve given missions, the agents work cooperativelyto operate and /or process tasks.”

The main research subjects in DARS are, according to Asama (1994), asfollows:

1. Multi-robot behavior deals with the group formation and consistentbehavior among that group.

2. Distributed sensing solves problems related cooperative sensing amongautonomous agents.


30

3. Distributed planning handles problems linked to planning among multipleagents. The key issue here is to avoid deadlocks and conflicts in taskassignment, task planning, path planning, etc.

4. Distributed control deals with the control of a single robot with distributedmultiple processors with parallel processing.

5. Coordinated control handles the control of multiple robots in somecoordinated manner. The control can be carried out (at least partly) by thesupervisor but can be incorporated into individual agents as well.

6. Cooperative operation deals with various problems related to multiple robotcooperation. These include conflict resolution, collaborative teamorganization, role assignment and resource sharing. In general some sortof communication, whether it is direct or indirect, is very essential to thissubject.

7. Self-organization studies how the required function is distributed intomodules and how the system reconfigures autonomously and dynamicallyaccording to the situation. In a physical system this means that we havesystems where modules connect to each other with some kind of couplingmechanism. In information processes this subject handles concepts likecollective and swarm intelligence. One of the best examples of this field isdefinitely the concept of the Cellular Robotic System. This system consistsof many components that are called “cells.” Each cell has simple functions,a database and knowledge. Through self-organization these simple cellscan combine and form a structure that can perform many morecomplicated tasks. The literature on the matter is rather large, but thereader should see at least (Fukuda et al. 1989) and (Fukuda and Ueyama1994).

8. Shared autonomy considers the matter of a cooperative system betweenmachines and humans. This interface is very important and includesconcepts like tele-presence and virtual reality.

Fukuda and Ueyama (1994) list several advantages of distributed (ordecentralized) autonomous robotic systems over centralized robotic systems.These include simplicity, modularity, load variance, cooperative andcoordinate abilities, exchangeability, variety, response ability, mutualdiagnosis and miniaturization. Simplicity comes from the decomposition of thesystem into a number of elementary units. These units are robotic units withsimple functions. Modularity provides opportunities to easily vary thecomposition of the system easily. Load variance provides fault tolerance andreduction of load. Cooperative and coordinate abilities improve the ability tocarry out the required tasks. Exchangeability is based on the modularity of thesystem and provides flexibility, fast response and fault tolerance to thesystem. Variety is related to the configuration of the system. It is based on thebehavioral repertoire of the elementary units, i.e., the simpler the units andthe more difficult the task, the more units that can be used for the completionof the task. The Response ability is naturally increased based on the multiplefunctional elements in the system. Mutual diagnosis benefits from the self-diagnosis of the elementary units. Miniaturization is based on the simplicity ofthe elementary units and modularity of the system.


31

When we are some day able to produce hundreds or thousands of miniaturerobots, we will face totally new problems, such as the question of controllingthese huge distributed systems. This kind of approach deals closely withterms like swarm and swarm intelligence. (Deneubourg et al., 1992) definesswarm as follows: “A swarm is defined as a set of (mobile) agents which areliable to communicate directly or indirectly (by acting on their localenvironment) with each other, and which collectively carry out a distributedproblem solving. In this sense we refer to functional self-organization, sincethis emerges from the swarm’s internal dynamics and its interaction with theenvironment. The swarm functioning induces both the genesis of functionalcollective patterns which characterize the differentiation and spatial-temporalorganization of the agents of the swarm and also the parallel organization ofthe material elements in the environment upon which each agent acts.” In(Hackwood and Beni 1992) and in (Beni and Hackwood 1992a) swarmintelligence was furthermore defined to be a property of "systems of non-intelligent robots exhibiting collectively intelligent behavior." This intelligenceis studied through the perception of the external environment. See also (Beniand Hackwood 1992b). In (Sugawara and Sano 1996) the collective behaviorof a multi-agent system was studied through real robots experiments andsimulations using foraging as the task. It is one of the few papers where aquantitative analysis has been performed. The main idea of the paper was toverify Beni’s claim that swarm intelligence emerges from interaction among Nunits only when N exceeds some critical limit. The robotic experimentsillustrated clearly, that when the items to be collected were distributed evenlythe interaction only weakened the performance. On the other hand when theitems where localized the interactions between members improved theperformance. This is a rather obvious result and several other papers havereached the same conclusion. These papers also include work done by ourgroup, see (Schönberg 1993) for details.

Unfortunately, we are not yet capable of producing hundreds or thousands ofreal robots. Instead, we have to prove our ideas with a handful of robots.Next, some of the existing multi-robot systems are shortly presented. Thispresentation is by no means complete due to the high number of activeprojects, but it should nevertheless give the reader some idea of the currentstatus of the field. For more comprehensive presentation see, (Cao et al.1997).

CEBOT (CEllular roBOTics System) (Fukuda et al. 1989) is one of the firstand most well-known systems in the field. This decentralized, hierarchicalarchitecture was inspired by the cellular organization of biological entities. Thesystem is dynamically reconfigurable. The cells (robots) can be physicallyconnected to others and can reconfigure based on needs and constraints setby the environment, task or operator. The literature is very large including, forexample, (Fukuda et al., 1991), (Fukuda et al., 1993), (Kawauchi et al., 1993),(Kawauchi et al., 1994), and (Fukuda and Iritani 1994). Fukuda and Ueyama(1994) summarize the majority of this literature along with a part concerning arelated field of micro robotics.


32

ACTRESS (Asama et al. 1989) is also one of the oldest and most citedsystem in distributed autonomous robots. ACTRESS (ACTor-based Robotand Equipments Synthetic System) consists of robotors (robots andworkstations) forming a heterogeneous system suitable for various taskperformings. Topics like communication protocols, negotiations methods andcollision avoidance are handled in numerous papers including, for example,(Asama et al., 1991a) and (Asama et al., 1991b).

In (Mataric 1992a), (Mataric 1992b), (Mataric 1993a), and (Mataric 1993b) agroup behavior is viewed as a collection of basic, primitive behaviors, builtfrom simple local interactions, and combined into more complex actions.Instead of trying to analyze some arbitrary complex behaviors with statisticalmethods, Mataric describes actual behaviors emerging from local interactions.These interactions between individual agents do not have to be complex toproduce complex global consequences. Five basic interaction primitives havebeen listed: safe-wandering, following, dispersion, aggregation, and homing.These serve as building blocks for more complex behaviors. For example,flocking, i.e, "the ability of a group of agents to move as a coherentaggregate without pre-specified leaders and followers” can be constructedwith a combination of homing, dispersion, aggregation and safe-wandering,as is shown in Figure 2.13. For more comprehensive representation, see(Mataric 1994a).

homing

dispersion

aggregation

safe-wandering

flocking

sensory inputs

effectoroutputs

basicbehaviors

compositebehaviors

Figure 2.13 The implementation of flocking behavior. Adapted from (Mataric 1994a). Directcombinations are marked with a plus sign inside a circle. Temporal combinations are markedwith a multiplier sign inside a circle.

In (Mataric 1994b) and (Mataric 1997) the problem of cooperation isapproached through social learning. Also called observational learning, itincludes aspects like how to perform a behavior, through imitation, and whento perform it through social facilitation. In (Mataric 1994a) imitation is definedas “the ability to observe and repeat the behavior of another animal or agent.”Social facilitation on the other hand means that the agent can selectivelyexpress a behavior which is already a part of the agent’s repertoire. Althoughsomebody might consider, that these social reinforcement learning tasks aresimple, they are nevertheless implemented with a rather large group of realrobots (so called Nerd Herd) working in a dynamic world and thus deserve tobe named state-of-the-art. Real world is nothing like a 2D grid world usuallyused in learning tests. See also (Mataric 1995a) and (Mataric 1995b).


33

Heterogeneous teams of robots are main targets of a research presented in(Parker 1994). Novel architecture, ALLIANCE, uses the basic structures ofsubsumption, but includes some extra features, namely behavior sets and akind of motivational system. Motivational behaviors, shown in Figure 2.14,accept, besides sensor values and inhibition from other behaviors, also inter-robot communication. This way, a robot has some kind of idea what theothers are doing. Motivational behaviors enable certain behavior set anddisable the others. This active behavior set will then perform certain task.Parker introduces two functions (impatience and acquiescence) to guide therobot’s operation through overall motivation level. L-ALLIANCE is anextension to ALLIANCE. It uses reinforcement learning to adjust certainparameters controlling behavior set activation. See also (Parker 1992) and(Parker 1996).

Motivational Behavior



Behavior Set 0

Behavior Set 2

Behavior Set 1

Layer 2

Layer 0

Layer 1

Sensors

Inter-RobotCommunication

Actuators

cross-inhibition

Figure 2.14 ALLIANCE architecture. Adapted from (Parker 1994).

Kube (1997) presents the results from his extensive study (see, e.g., (Kube1992), (Kube and Zhang 1992) and (Kube and Zhang 1993)) of obtainingmeaningful cooperation without explicit communication. Inspired by thebehaviors of social insects, Kube studies collective box pushing task. In hissystem robots (both simulated and real ones) search for a box, alignthemselves correctly, escape stagnation (Kube and Zhang 1994) andtransport it to the goal location. What is especially noteworthy in thisresearch, is the fact, that Kube uses as many as 11 real robots in his tests.

Cooperation (i.e., collision avoidance) between Yamabico robots whilemoving in corridors and through intersections is studied in (Premvuti and Yuta1990) and (Yuta and Premvuti 1992). Furthermore, important issues relatedto the organization of multi-robot systems, such as active or non-activecooperation, level of independence, and types of communication, were alsoaddressed.

GOFER architecture (Caloud et al., 1990) deals with distributed problemsolving with multiple mobile robots using traditional AI techniques. A central


34

task planning and scheduling system communicates with robots and hasknowledge of the status of the tasks and the robots.

Stilwell and Bay (1993) introduce a new material transport system conceptusing swarms of army ant-like robots. This system is composed of smallautonomous robots (mostly simulated), without any central controller, and withminimal inter-robot communication. In such system, teams of ant-like robotswould swarm around a palletized load, go under the pallet and lift it up. Afterthat they would transport it to a new location and then lowered it down. Thiskind of idea has several advantages: small units can get to places wherelarger units can not, and then would organize themselves according to what isneeded, i.e., how large the load is. The actual transportation was done with asingle dynamic leader. This leader is chosen for example just by checkingwhich of the robots is closest to the target. After that the others will mimic thebehavior of a caster and "follow" the leader. If something happens to theleader it is replaced with a new robot, and the transportation can continueagain. Johnson and Bay (1995) present furthermore how a behavioralcontroller with four behaviors (orientation, force, pallet contact, height) can beused with vector summation as a coordination mechanism for this demandingcollective transportation task.

(Ohkawa et al., 1998) present a method for controlling distributedautonomous robots in their task sharing through simple rewards given by theoperator. The architecture consists of sensor, behavior, selector, evaluationand frustration modules, as shown in Figure 2.15. The sensor module followswhat the other robots are doing. The selector module selects one of thebehaviors that fits to the situation. The behavior module has all the elementalbehaviors. The evaluation module estimates the results of behavior selectionand learns simultaneously through reinforcement learning and good behaviorselection rules. The frustration module’s task is to change the behavior if therobot chooses the same behavior for a while. The problem of correctevaluation is approached by designing an algorithm that changes theevaluation based on the rewards received from the operator during themission. The system was tested in simulations. The results indicated that theoperator could control a group of robots just by giving simple rewards.

ENVIRONMENT

CurrentBehavior

Other Robots'sBehaviors

Evaluation

BehaviorSelector

Frustration

Behavior 1

Behavior N

Behavior 2...

NewBehaviorLearning

S

.

.

.

Sensor Module

Evaluation Module

Selection Module

Frustration Module

Behavior Module

Switch

Figure 2.15 Algorithm for spontaneous cooperative behavior. Adapted from (Ohkawa etal., 1998).


35

MacKenzie et al. (1997) use schema-based architecture for multi-robotmissions. In the Societal Agent Theory (MacKenzie 1996), a singlerepresentational syntax is used to express teams of physical agents as wellas sensorimotor behaviors. (Arkin 1992) and (Balch and Arkin 1994) proofedthat team cooperation could arise without active inter-robot communication.

BeRoSH (Behavior-based Multiple Robot System with Host for ObjectManipulation) concept presents a multiple robot system using a behavior-based dynamic cooperation strategy, see (Wang et al., 1994) and Wang etal., (1996). BeRoSH maintains a host as its leader, and has homogeneousbehavior-based robots with limited manipulation skills. The host generatesgoals for the robots and offers some extra assistance, but does not calculatetarget object dynamics or force distribution for dynamic cooperation.Experimental results demonstrate, that by using the behavioral controller,shown in Figure 2.16, the robots (with simple one degree of freedom gripper)are able to transport a target object cooperatively to the target, even thoughthere is an unknown obstacle on the route.

Check Task Performing

Push and Pull Object

Keep Touching

Touch Object

I

S

S

Output

Keep Touching

Manipulating ObjectLevel 4

Level 3

Com

munication

Sensors

Figure 2.16 An advanced behavioral controller for manipulation. Adapted from (Wang et al., 1996).

Halme et al. (1993) and Halme et al. (1996a) document early results obtainedwith a developed simulator, shown in Figure 2.17. The test scenario includesrobots collecting objects (foraging) from an initially unknown 2D environmentand simultaneously collectively creating a map of the environment. The basicconcept was verified and some extensive simulation runs were performed insearch of general principles for a model robot society, in this case a foragingsociety. For example, an optimum communication range was sought andsome behavior adapted from an ant society (e.g., the distribution of energybetween society members) were also tested. A more profound simulationstudy can be found in (Schönberg 1993). Additionally it presented anautonomous mobile robot, HUTMAN, as the prototype for the member of themodel society, see Figure 2.18. The design of this autonomous robot ispresented in (Vainio 1993).


36

Figure 2.17 Society simulator. See Figure 2.18 HUTMAN, the model society member. (Schönberg 1993) for details.

Vainio et al.(1995a) present an ALife based approach toward robot societies.During the testing with the society simulator it became evident, that a dynamicsystem (robot society) operating in a dynamic world is rarely in a static state.Everything seemed to have an effect on everything else and thus the controlof so-called operational parameters through normal ways was almost animpossible task. These operational parameters are those variables thatcontrol the performance of the society members (e.g., communication range,obstacle definition, curvature radius, energy refilling parameters, and so on).To cope with this problem Genetic Algorithms were applied to the control ofthe society. Out of several parameters the two most important were chosen.These two include the range of communication and the range of obstacledetection. These parameters were coded to the genome of the members ofthe society. After that a random population was created and a simulation run(generation one) was started. When the simulation was terminated someevolutionary operators (one-point crossover, mutation) were used accordingto rank based selection and elitism. Elitism means that the best individualalways transfers its genome to the next generation (this way the best possibleindividual will always stay with the process). The rank based selection, as thename reveals ranks the individual according to their fitness. After that acertain percentage of the individuals are thrown out of the game and theGaussian operations are applied to the rest. The fitness mentioned abovefollows how well the members are working as individuals as well as part of thesociety. The simulations revealed that even though the algorithm used is notthe best alternative due to the small population size, it still works fairly well.With a different generation definition, a clear short term adaptation could beverified and the overall performance of the whole society improved with thenumber of generations. As an extra benefit the system produced a kind ofautomatic search procedure for correct operational parameters: by followingthe evolution of the communication range value through the generations, forexample, a good overall value could easily be found. As an interesting resultthis value was very close to the value found earlier through extensivesimulations to be the number one solution for that particular environment.


37

(Vainio et al., 1995b) illustrate how a group of autonomous mobile robots (therobot society) can perform stone foraging (exploit) and mapping (explore)tasks in an initially unknown environment. Even though the testing was donein a simulated environment, special emphasis was attached to real worlddemands. This means that the mechanisms defining how the society couldoperate under various degrees of guidance from the operator were studied.The tool to control the behavior of the society was built into the members.Each member had a task-specific fitness function, which followed how wellthe member was performing. This simple function provides an easy way forthe operator to control the society. Simply by changing the ratio betweenexploring and foraging values the operator can guide the society, for example,to perform a rapid exploration. The concept was tested in an environmentwhere the working area was divided into two separate areas by a barricademade out of stones to be collected. The structure of the environment was notknown by the members, so they couldn’t predict the presence of a stonebarricade. The software of the members included a “voting” behavior. Thisbehavior allows the members of the society to make a common decisionabout the existence of the barricade. The behavior is based on thedevelopment of the fitness function. When a member detects that itsperformance is decreasing (i.e., the fitness function is reduced) it askswhether there are other members in the same kind of situation. If there areother members, with a similar experience to the questioner, a commondecision will be made and the group will start to collaborate. This kind ofgroup behavior initiated by a sort of auto-stimulation will lead to a fairly fastbreakthrough of the barricade. Thus, it will also allow the fitness function toincrease, because of the exploration of the new areas. The most importantindividual result or conclusion came in the form of the knowledge that byusing a simple fitness function-based strategy, the society was able tooperate in a complex environment without any highly sophisticated sensorysystem in any members. This kind of collective decision making was sufficientto solve the existence of a wall. There was no need to include any specificsensor or perception strategy for the robots.

2.6 Towards the Next Millennium

As the field of multi-robot systems gradually started to maturate after tenyears of intensive research, some new ideas and new techniques were boundto come along. These new ideas interested initially only a small portion of theresearchers in the field, but as usual, success creates success. “Hot” topicstoday include, among others, learning by imitation and evolutionary robotics.These concepts are briefly described.

2.6.1 Learning by ImitationThere are actually two main approaches to gain intelligence throughinteractions. The usual way to do it in robotics is the one inspired by socialinsects, i.e., a lot of simple units with simple interactions. There is anotherway to increase an individual’s (or system’s) intelligence. This approach isinspired by higher level animals, such as primates. The number of individuals(robots) in these systems is smaller, but their level of intelligence is higher.


38

They are learning by imitation, i.e., by monitoring what the others are doingand then by imitating the same behavior, these robots can improve theirperformance. This kind of approach has its roots in learning experimentsperformed with higher level animals, like primates. The problem with thesesystems is the need to have a good perception system, because the learningrobot has to be able to understand what the teaching robot is doing in certainenvironmental conditions. This approach was much inspired by “the socialintelligence hypothesis,” suggesting that primate intelligence originally evolvedto solve social problems and was only later on extended to problems outsidethe social domain (Dautenhahn 1995). As an example, Dautenhahn presentsa real robot scenario, where the development of social relationships betweenthese robots occurs through imitation based on collection and use of so-calledbody images. These body images mean that the robot gradually learns theproperties of other interacting robots, by keeping bodily contact and at thesame time observing what they are doing. Other related works include (Hayesand Demiris 1994), (Kuniyoshi 1994) and (Bakker and Kuniyoshi 1996).

2.6.2 Evolutionary RoboticsThe normal way to build a robot controller is to decompose this complexproblem into separate sub-problems and then solve these problems one byone. But when this problem becomes more and more complex (i.e. the robothas to survive in a very dynamic environment), this kind of approach is almostimpossible to perform: it isn’t always possible to even know how todecompose the problem, as interactions are very much coupled with theenvironment and along with the growing complexity the number of theseinteractions increases rapidly.

One solution to this difficult problem is to use the Darwinian evolutionapproach where the human designer’s part is greatly diminished and theproblem solving is done through artificial evolution. (Harvey 1993) defines itas follows: ”The artificial evolution approach will maintain a population ofviable genotypes (chromosomes), coding for cognitive architectures, whichwill be inter-bred and mutated according to a selection pressure. Thispressure will be controlled by a task-oriented evaluation function: the betterthe robot performs its task the more evolutionary favored is its cognitivearchitecture.”

The wide field of Evolutionary Algorithms has been divided into severalnevertheless very closely related sub-fields. These include GeneticAlgorithm(GA), Evolutionary Programming (EP), Evolutionary Strategy (ES),Classifier System(CS) and Genetic Programming (GP). Out of those, GA, CSand GP, in particular have been under extensive study in autonomous agentresearch. For example (Almassy and Verschure 1992) and (Ram et al., 1994)apply GAs to improve the performance of an autonomous agent. (Vainio etal., 1995a) presents a way to improve the performance of a robot society in aforaging task. Besides these traditional GA implementations, there areseveral papers concerning the use of Genetic Programming in autonomousagents. The concept of evolving real computer programs directly from scratchhas led to some very inspiring studies like (Reynolds 1992). The actual


39

textbooks on Evolutionary Algorithms include, for example, (Holland 1975),(Goldberg 1989), (Koza 1992a), (Koza 1994) and (Mitchell 1996).

Harvey et al. (1992) present three ways to implement the control system of anautonomous robot through artificial evolution. These are:

• An explicit control program, in a high level language, for example, by usinggenetic programming techniques, see (Koza 1992b), (Reynolds 1994) and(Sims 1994).

• A mathematical expression mapping inputs to outputs, for example apolynomial transfer function.

• A blueprint for a processing structure, a network of simple processingelements.

Cliff states in (Harvey et al., 1995) that neural networks are best suited toevolving robot controllers because:

• NNs are well suited to noisy environments.• In principle any function can be approximately computed by an appropriate

NN (i.e., Universal Function Approximator Theorem).• Any dynamic system can be approximated by an appropriate continuous-

time dynamic neural network.• Recurrent networks can have an internal state over time.

General readings of the field include, for example, (Rumelhart and McClelland1986), (Haykin 1994) and (Anderson 1995). (Pfeifer 1996) lists some well-known positive features usually related to these systems: these include faultand noise tolerance, the ability to generalize and the fact that the learning isintrinsic (or “emergent”). Generally speaking NNs are divided into twocategories based on the method of learning, i.e., supervised or unsupervisedlearning. Usually when autonomous mobile robots are under considerationthe supervised learning takes place before the actual operation starts. Thismeans that the weights are fixed and thus these systems will become unableto react to changes in the environment. Nevertheless there are some ratherinteresting studies where this approach has been used. The classical book onthe matter is (Braitenberg 1984). It presents a series of very simple vehicles.Their sensors and actuators are connected with simple neural controllers butthey can nevertheless perform rather interesting behaviors.

Usually, the main goal in autonomous agent design is to provide the agentswith some means of adapting themselves to dynamic environments. Thisadaptation is gained by providing the networks with the ability to buildthemselves based on the incoming information combined with the historydata. The actual learning must be based on the internal “motivations” and byvarious error signals received from the agent’s decision. The neural networkcan have various forms, but the most commonly used structures includeWinner-Takes-All (see, e.g., (Gaussier and Zrehen 1994)), various ofKohonen’s topological maps variations, see, e.g., (Lambrinos et al., 1995))and general recurrent neural nets, (see, e.g., (Yamauchi and Beer 1994) and


40

(Cliff et al., 1993)). The actual evolution can be done in software or in somecases also directly in hardware. These two approaches are presented below.

Evolution in softwareIn this approach basic building blocks are adaptive noise-tolerant dynamicalneural networks. These networks can be recurrent and should operate in realtime. In (Harvey 1992) a modified version of the genetic algorithm (SpeciesAdaptation Genetic Algorithms, SAGA) is used. Traditional GAs work with afixed and well-defined search-space. This kind of approach is not well suitedto problems where the task domain is not defined very well and can becomeeven more complex over a long time. SAGA was developed to cope withthese problems. In this algorithm the genotypes can increase in lengthindefinitely and encode more complex phenotypes. Because the artificialevolution of a robot controller usually requires hundreds or thousands ofindividual robot controller evaluations, some sort of use of a simulator is oftenneeded to cope with time constraints. But, these simulators should beadequate, especially when it comes to describing the environment and therobot. Noise should also be included at all levels. Most of the work has beendone completely with simulated robots/animats, for example, (Beer andGallagher 1992) and (Cliff et al. 1993), but some have transferred thesesimulation results onto real robots, see, for example, (Jakobi et al., 1995),(Nolfi et al., 1994), and (Miglino et al., 1995). In addition to these paperswhere the evolution was conducted in simulation and then tested in a realrobot there is actually only one group that has completed work solely with areal robot. This is the work done by Floreano and Mondada with theKhephera robot. This very small robot has proven to be excellent for thesekinds of evolutionary experiments. It is easy to connect to yourworkstation/PC and let it run for days if needed, through a flexible tether,which is used for external computation and power. Floreano and Mondadahave reported their work for example in (Floreano and Mondada 1994a),(Floreano and Mondada 1994b), (Floreano and Mondada 1996a,) and(Floreano and Mondada 1996b). In these papers the robot is connected to aworkstation with a tether. Their next step is to use a special energy systemwhich will enable a continuous operation with multiple robots at the same timewithout any direct coupling to outside. The artificial evolution will occur onlybetween the robots working at the site. Mataric and Cliff (1996) give anexcellent overview on these matters along with the state-of-the-art in the field.

Evolution in hardwareMost of the work done with artificial evolution in robotics is done bydeveloping robot control software. Thompson (1995) presents experimentswhere the actual robot hardware, in the form of semiconductor circuits, couldevolve directly. He uses a reconfigurable hardware (Field Programmable GateArray (FPGA)), which could be used as the substrate for evolving recurrentasynchronous networks of high-speed logic gates. The actual controller taskwas to produce a wall-avoidance behavior in a differential-drive wheeledrobot, just by using two sonars pointing left and right in the robot’s direction oftravel. For more details, see (Thompson et al. 1996). The direct evolving ofphysical hardware will obviously increase the speed of artificial evolution andmake it possible to exploit the physical properties of the implementation.

Chapter 3. Societal Robotic Agent

41

Chapter 3 Societal Robotic Agent3.1 Introduction

Humans, like many other species, are social creatures by nature. We live insocieties with certain values and rules. These societal variables and thevalues for them (i.e. correct responses to certain stimuli) are transferred to thefollowing generations through culture. When is an artificial agent consideredto have a societal nature? Is it enough that it operates in a multi-agentenvironment, or is it necessary to have active communication betweenagents? One field especially closely related to these matters is DistributedArtificial Intelligence (DAI), see, for example, (Bond and Gasser 1988). Itsmain topic of interest is to study how multiple artificial agents (softwareagents) interact and behave. The field is normally divided into two sub-fields:Distributed Problem Solving and Multi-Agent Systems. Distributed ProblemSolving deals with agents that work together to solve common problems.Cooperation can be achieved by dividing the main goal or the main task intosub-tasks. These sub-tasks are then allocated to different agents in thesystem. Another way to cooperate is to gather information in a distributed wayand then fuse the gained data into a single representation. Multi-AgentSystems consist of autonomous agents that may have a common goal, butthey can also have personal goals to proceed. Furthermore, there is nocommon high level control mechanism, which forces the agents to coordinatetheir actions and plans. This sub-field deals with problems more directly linkedto robotics, such as incomplete knowledge of the environment, overlappinggoals and most importantly the need to interact with each other. Terms likebelief, intentions, motives and so on are actively used in this research: anagent has its own motives to perform certain tasks. It has intentions to carryout an action in future and belief in what the others are doing or will do undercertain circumstances. But is this enough? Do societal structures emerge inthese kinds of conditions or is something else needed? In (Epstein and Axtell1996) artificial societies have been developed with simple rules. The resultsare quite good and life-like features emerge when a system’s complexity isincreased. Terms like immigration, regression, etc. are verified and studied.The problem with this study and with all similar studies is the simplicity of the


42

environment where the agents live. At its simplest it is nothing but a grid-typeworld, where movements are restricted to four principal directions. Sensorinformation is always ideal or white noise is added to increase the uncertaintyof the sensor. Furthermore, hundreds or thousands of generations of agentslive and die in these experiments. The main question is thus: how useful arethe structures and algorithms obtained from these “virtual” systems for realphysical multi-robot systems? We postulate, among others, that in order to beuseful in the real world, the systems used should not only be capable ofintelligent behavior, but be very much situated and embodied.

This chapter gives a short definition for the societal robotic agent. The agentshould be capable of surviving in a complex environment with incompletesensory information and non-ideal actuators. It should have the opportunity tointeract with other agents (robotic or humans) sharing its world. The maindesign principles are listed and the selected formal notation is presented.

3.2 Definition of an Autonomous Robotic Agent

Contrary to sci-fi literature, present and future robots do not resemblehumans: a lawnmowing robot looks like a traditional lawnmower, a floorcleaning robot looks like a normal human-operated cleaning machine, and soon. Only in the distant future when we try to build a multi-task robot with highmaneuverability and versatile manipulation, could the robot have a somewhathuman-like outlook. Those days are far off. Nevertheless, like humans, asocietal robotic agent has to have certain basic features very similar to ours. Ithas to be able to sense what is going on around it (perception), it has to beable to move around in the world it is living in (actuation), and it has to havesome kind of intelligent way to combine sensing with action (“cognition”).Additionally, when the agent in concern is supposed to be societal, then somesort of interaction option with another societal agent (communication) must beavailable. At the moment there seems to be no solid theory to back up thisapproach. The dynamic interactions in the system makes it very difficult toanalyze by traditional means (i.e., dynamical systems, control theory, etc.).The work done at the autonomous agents field provides nevertheless someinsights into this problem domain. (Pfeifer 1996) states: “It seems that it wouldbe premature to ask for a theory of autonomous agents. This is why westarted with a set of principles that can help us formulate our beliefs about thenature of intelligence in a compact way.” (Pfeifer 1996) lists seven mainprinciples:

• The “Fungus Eaters” principle states that the agents should beautonomous, self-sufficient, embodied and situated. The term FungusEaters comes from a book by (Toda 1982), where it stands for anautonomous agent working in some far away planet collecting ore andeating fungus for energy.

• The principle of the ecological niche means that the agents are alwaysdesigned for some particular environment, i.e. to operate in a particularniche. Thus agents should only be evaluated and compared with respect


43

to that particular environment. That is to say, that the intelligence of arobotic system cleaning the windows of a skyscraper can not be comparedto the intelligence of a group of lawnmowing robots working on a golfcourse. Both are optimized for their own task and environment andcomparison would just be a waste of time.

• The principle of parallel, loosely coupled processes is directly biologicallymotivated. The intelligence of an agent should be based on a largenumber of decentralized processes. Even though, in many cases, theprogram in a robot runs on a single processor, this principle can beapplied through parallel programming or though some real time kernelopposite raditional sequential programming.

• The principle of sensory-motor coordination states that the interaction withthe environment is to be conceived as a sensor-motor coordination. Itinvolves agent’s sensors, control architecture, effectors and the agent as awhole. It also means that classification, perception and memory should beviewed as sensory-motor coordination rather than as individual modules.

• The “value” principle implies that there has to be some sort of valuesystem inside an agent. This system should be based on self-supervisedlearning mechanisms (i.e. there is no universal teacher) employingprinciples of self-organization (both temporal and spatial).

• The principle of “ecological balance” underlines the fact that there has tobe a match between the “complexity” of the sensors, the actuators and theneural substrate, i.e. there is no sense in building an autonomous agentwith a high-speed processor and only very few simple sensors.

• The principle of “cheap design” combines three separate ideas: the use ofsystem-environment interactions which actually make them more robust;the use of Occam’s razor (i.e. choose the simplest construction possible)and the exploitation of the constraints of the ecological niche.

3.2.1 Embodied and SituatedTo fit under the definition of robot, the system has to have a physical body.Through this body, including sensors and actuators, the robot interacts withthe surroundings. To be of any use, the robot must always operate in somespecific environment. The problem with simulators is the fact that “simulatorsare doomed to succeed.” In other words the program code always has somedeviation from the real case, mostly concerning the description of theenvironment and the sensors. Gaussian noise is rarely the correct way tomodel the problems in perception. However, there are some preliminarysimulation results indicating that a highly realistic description of the sensorsand proper adding of noise can produce results that can be compared to theperformance of real robots, see, for example, (Jakobi et al. 1995). But when ahuge amount of work has been invested in creating an identical world withaccurately modeled sensors and actuators, the question arises as to whetherthe same amount of time and effort should have been used for testing the realembodied and situated robots.


44

3.2.2 Ecological NicheIn the natural environment an organism is bound to a certain environment.The size of this “niche” can vary depending on the obvious aspects like thesize and mobility of the animal, the supply of food, the predator-preyrelationships, and so on and so forth. Due to the incomplete perception anddeficient actuation capabilities, this same principle must be applied toautonomous agents as well. In most cases the agent is designed for particularenvironment. Its sensors are chosen to meet the challenges of theenvironment and the actuators are built to cope with a given task in the workenvironment. Move it to another environment and you will most likely faceserious problems. As Brooks (1990a) elegantly states: “Elephants don’t playchess.”

3.2.3 LearningTo be able to improve its own performance, an agent has know how it isperforming. Mataric (1994a) lists two main purposes why learning should beincluded in autonomous agent design, adapting to external and internalchanges and simplifying built-in knowledge. She continues and states that amodel for situated learning should minimize the learner’s state space andmaximize learning at each trial. Both of these goals are rather obvious. Thefirst deals with the fact that in situated agents the state space can easily bevery large. Some sort of problem decomposition can be the only reasonablesolution. The second point is related to the fact that energy and time arevaluable in situated agent design. Mataric (1994a) furthermore states:“Reinforcement learning in situated domains can be formulated as learningthe conditions necessary and sufficient for activating each of the behaviors inthe repertoire such that the agent’s behavior over time maximizes receivedreward.”

As a reference of reinforcement learning the reader should see (Kaelbling etal., 1995). It gives a clear view to historical works and presents a broadselection of current research works. In (Kaelbling et al., 1995) the standardreinforcement-learning model is explained as follows: “an agent is connectedto its environment via perception and action. On each step of interaction theagent receives as input, i, some indication of the current state, s, of theenvironment; the agent then chooses an action, a, to generate as output. Theaction changes the state of the environment, and the value of this statetransition is communicated to the agent through a scalar reinforcement signal,r. The agent’s behavior, B, should choose actions that tend to increase thelong-run sum of values of the reinforcement signal. It can do this over time bysystematic trial and error, guided by a wide variety of algorithms. Formally,the model consists of

• a discrete set of environment states, S;• a discrete set of agent actions, A; and• a set of scalar reinforcement signals; typically {0, 1}, or the real numbers.

An input function I, determines how the agent views the environment state;we will assume that it is the identity function (that is, the agent perceives theexact state of the environment).” See Figure 3.1 for illustration.


45

B

I

R

T

i

r

s

a

ENVIRONMENT

ROBOT

Figure 3.1 Model of reinforcement learning. Adapted from (Kaelbling 1993).

See (Kaelbling 1993) for some practical examples of reinforcement learning inembedded systems. In (Balkenius 1995) the various aspects of learning areconsidered mainly through various animal learning theories. It deals withseveral types of learning methods in the creation of complete artificial nervoussystems for simulated artificial creatures.

However complex the learning of an autonomous agent (i.e. robot) can be inthe real world, it is always many times more difficult when we are consideringmulti-robot systems. Work on this challenging matter include, for example,(Parker 1994), (Mataric 1994a), (Balch 1998), (Michaud and Mataric 1998)and (Michaud and Mataric 1999).

3.2.4 Knowledge RepresentationArkin (1998) argues that in order to be useful, representation must have somerelationship with the external world where the robot is operating and it mustprovide the robots some “tools” to predict what is going on just outside itssensor range. This knowledge can be represented in a short-term behavioralmemory or in some sort of long-term memory map, whether given a priori orcollected during the “lifetime” of the robot. Pettersson (1997) lists differentrepresentations used to store the knowledge. These includecentral/distributed, homogeneous/ heterogeneous, uniform/non-uniform,explicit/implicit and symbolic/descriptive. Distribution as normal increases thefault tolerance, but some sort of extensive communication between modulesis usually required. When uniform representation has been chosen, thecontrol (i.e. adding, deleting, changing) of the knowledge is simple. Theproblem is of course related to the different needs of different knowledge.Homogeneous knowledge representation guarantees that all the modules canaccess and use the same information and thus possibly produce moreintelligent behavior. Symbolic representation has always been closely relatedto the classical AI, where the whole intelligence is usually based on symbolicdescription.

3.3 What Does It Mean to Be Societal ?

In order to work as a part of a societal structure, the member of the systemmust have some kind of way to understand what the other members aredoing. In a way it has to be equipped with social skills. It has to know what toexpect from the others while on the other hand, know what it must itselfprovide to them. This can happen basically two different ways: direct


46

communication between members or indirect communication through theenvironment. The concept of direct communication is easy to understand.Two or more members interact with each other (or with the operator in somecases). This form of communication is dealt with more precisely later on. It is,however, appropriate to state, that direct communication is never free. Itrequires extra components, energy and software. Thus, indirectcommunication is in many cases more appealing. Indirect communicationmeans that the agent can somehow understand how other members’behaviors affect the environment. This is not always easy to provide.

3.4 Formal Description

The proposed formal description of the societal robotic agent is based on theFSA (Finite State Automaton) structure and behavioral action model. Thetheory of FSA is wellknown and several studies using this method to describerelations in robotics have been done previously, such as (Brooks 1986) and(Arkin and MacKenzie 1994). An FSA acceptor M can be specified by thequadruple (Q, δ, q0, F), where Q is the set of possible states where a robotcan be, δ is the transition function mapping the current state qi to the nextstate qi+1 using various types of inputs (i.e. from sensors, communication,calculations, etc.), q0 ∈ Q is the initial state, and F ⊆ Q consists of stateswhich indicate the completion of the task. In the model developed in thisthesis each of these states includes a certain behavioral pattern (or sub-states), which the robot performs when it enters the state. The statetransitions are determined by the “desires” or “needs” of the robot. These inturn are conducted by single logical variables (digital or analog) or by morecomplicated performance evaluation functions or by a combination of both.Table I illustrates this notation.

Table I: General state definitionName STATEType: self-sufficiency | task achieving |

miscellaneousFrom: state1 |...| staten

Trigger_in: logical variables(digital or analog)performance evaluation functions

combinationSub-task(s): none

ST1,...,STn

Trigger_out: logical variables(digital or analog)performance evaluation functions

combinationTo: state1 |...| staten

Algorithm: pseudocode

There are three different types of states in our system: task achieving (TA),self-sufficiency (SS) and miscellaneous (M). These types have a fixed priorityrelation. The most important states with the highest priorities are naturallystates dealing with self-sufficiency. This means that whenever a state


47

belonging to this class is activated, it has direct control over a robot’s action.Normally, these states include at least energy refilling and stagnationrecovery behaviors. Task achieving states are always related to the status ofthe society. If there is active communication between the members of thesociety, the relevant information is always included in the state performance.If there is no direct communication, the work done by the other members ofthe society affects the performance of an individual indirectly, because thesensors of the robot constantly give information on what the others are doing.These states are usually rather complex, and can not be represented withoutdividing the state into sub-states. These sub-states operate inside a taskachieving state, and their interface surfaces are the same as their “mother”state. Miscellaneous states can be both, i.e. individual or society related. Theyinclude the rest of the states. Usually the end state (i.e. termination of theoperation) is included in this group.

To illustrate this formal description an example is given (Figure 3.2). Therobot concerned is a two wheeled turtle-type robot. It is equipped with thefollowing sensors: radio communication module (R), visible light sensor infront of the robot (S_VL), IR sensor in front of the robot (S_IR), tactile sensoron the palm of a one-degree of freedom gripper (S_G), eight short range IRsensors around the robot’s body (IR_N1 - IR_N8) or generally (IR_NX), andcapacitive sensor on the bottom of the robot (S_C).

The task is to look for a ball in the environment, pick it up, and then locate theBase_station and then get to the Base_station, and drop the ball there. Whiledoing this the robot should avoid obstacles (i.e. blocks and walls). The ball isilluminated with visible light and should thus by detected with an S_VL sensor.The Base_station is illuminated with an IR -light and should be detected withan S_IR sensor. The arrival of the ball to the gripper is detected with an S_Gsensor. The obstacles are detected with eight short range IR sensors. Thecapacitive sensor S_C is used to detect that the robot has arrived to theBase_station (i.e. the floor of the Base_station is covered with a metal plate).Radio link (R) is used for mission initiation.

RobotBase

_stati

on

Obstacle

Ball

VISIBLE LIGHT

IR-lamp

Metal plate

IR LIGHT

Gripper

S_G

IR_N1IR_N8

IR_N5

S_C

S_IR

S_VL

R

Figure 3.2 The task scenario.

All of the sensors are considered to be binary (i.e. ON / OFF), and there is noneed to consider problems such as the interference between IR-light on theBase_station and the IR-light from short range sensors. Furthermore, there isno need to consider IR_NX sensors separately but rather to use them as awhole package (i.e. IR_NX is either ON or OFF). Moreover, the problems


48

related to the grasping and holding of the ball during the transportation can beomitted (i.e. when S_G changes from OFF to ON, the ball is in the gripperuntil released when the robot arrives at the Base_station). Based on thisinformation the following definition can be given: M = {{start, Look_for_ball,Pick_up_ball, Obstacle_avoidance, Look_for_Base, Get_to_Base, Drop_ball,End},δ, start,{End}},where δ contains the transition information given in TableII. The developed FSA (one possible) is illustrated in Figure 3.3.

START

1

END

2

Look_for_ball

3

Obstacle_avoidance

4

Pick_up_ball

5

Look_for_Base

6

Drop_ball

7

S_VL_ON

S_VL_OFF

IR_NX_ON

IR_NX_OFF

IR_NX_ON

S_G_ON

S_G_OFF

IR_NX_OFF

IR_NX_ON

S_IR_OFF

S_G_ON

S_G_OFF

Get_to_Base

8

S_IR_ON

S_IR_ON &&S_C_OFF

S_IR_OFF

S_IR_ON &&S_C_ON

IR_NX_OFF

IR_NX_ONR_ON

R_OFF

Figure 3.3 FSA for ball retrieval task

Table II: Transition function mappingsq input δ(q , input)

start R_OFF startstart R_ON Look_for_ball

Look_for_ball S_VL_OFF Look_for_ballLook_for_ball S_VL_ON Pick_up_ballLook_for_ball IR_NX_ON Obstacle_avoidancePick_up_ball S_G_OFF Pick_up_ballPick_up_ball S_G_ON Look_for_Base

Look_for_Base S_IR_OFF Look_for_BaseLook_for_Base S_IR_ON Get_to_BaseLook_for_Base IR_NX_ON Obstacle_avoidanceGet_to_Base S_IR_OFF Look_for_BaseGet_to_Base S_IR_ON & S_C_OFF Get_to_BaseGet_to_Base S_IR_ON & S_C_ON Drop_ballGet_to_Base IR_NX_ON Obstacle_avoidance

Drop_ball S_G_ON Drop_ballDrop_ball S_G_OFF End

Obstacle_avoidance S_IR_ON Obstacle_avoidanceObstacle_avoidance S_IR_OFF RETURN to previous state

End ALL End

This rather trivial example is here to illustrate the chosen formal presentationlater used in Chapters 5 and 8.

Chapter 4. Definition of Robot Society

49

Chapter 4 Definition of Robot Society4.1 Introduction

Living systems have two main behaviors that have priority over everythingelse. The first one is self-preserving behavior. The second one is the species-preserving behavior. In some species in some special conditions the secondbehavior will rule out the first. This is called strong altruism. Klopf (1982)suggests that another primary behavior would be what he calls stimulation-preserving behavior, which means that knowledge acquisition and play arealso vital to living systems.

We have tried to include all these behaviors in our definition of robot society.In short: the members should be able to monitor their energy level andacquire more before their “death” (self-preserving), the members should beable to work as a society fulfilling the task that was given to the society(species -preserving) and finally we are also trying to achieve knowledgeacquisition in order to provide the members with a way to improve their overallutility.

The reason why individuals have formed societies is the fact that they providemutual advantages to the members of the society. In the natural environmentthese advantages include things like a more efficient search for food,collective defense, coherent transferring, etc. Similar ideas can easily beconnected to robot societies as well. In (Jakubik et al., 1992) we suggestedthe following:

“The difference between the natural and the artificial societies is thatin the natural societies all tasks are performed for the sake of survival,though in doing so one might find suprising achievements, too. Whilein natural societies one may regard these accomplishments as sideeffects, in machine societies they are the objectives.”


50

As an example of such “side effect,” one can consider the situation, whereants form a living bridge over a small gap in the ground. In (Halme 1992) theconcept of robot society was in general defined as a set of unit robots ormachines which are mobile and can communicate with each other. Theseunits (members of the society) can be of a different type (i.e., class or cast) ifa proper structure requires that. Society is always structured via itsinformation and control mechanisms, which define the information and controlflows inside the society as well as the interface between the society and theoperator. The operator monitors and/or controls the society functions. Thesociety can be autonomous in the same sense as a conventional robot whenit performs tasks according to a given program. The operator should alwaysbe able to access this program when needed.

4.2 Economic Justification from the System Supplier Point ofView

When designing an autonomous mobile robot for operation in a dynamic andat least partly unstructured environment, the designer is facing the ultimatequestion: how to ensure the robot gets the job done? This question raisesrespectively many new questions, such as: how to prevent deadlocksituations, how to consider sensor or actuator malfunction, or how to be surethat the robot generally possesses all the relevant information about the taskand the environment in particular. This kind of approach leads us easily to theworld of probabilities. We start to talk about the probabilities of malfunction inmotors or in sensors. Through some extensive testing, we could probablycome up with a reasonable distribution model for these problems. If thesedistributions deviate even slightly from 100% performance in the reasonabletime range (as they always do) for example in the case of some sensor, weare in trouble. If the task is to design a system that performs the given taskwith 100% certainty, something has to be done. The normal engineeringsolution for this dilemma is to increase the redundancy of the whole system.In the case of a robot this usually includes the duplication (or even triplication)of almost everything possible starting from the energy source all the way tothe processor: time consuming and very expensive, indeed.

Usually, it is “relatively” easy to make a robot perform most of the tasksnormally related to modern autonomous mobile robots, like searching andcollecting items. For example in the item collecting task, the robot justwanders around, detects an object, checks whether it is of the right type, andif so picks it up and transports it to a defined location. And then back it goesagain. No problem, until the robot faces a situation which is not included in itsknowledge base or in its behaviors. As a result the robot gets stuck in somedead end created by some truck driver, or drops into an elevator pit due tosome problems with the elevator’s doors. These examples may sound far-fetched and unreasonable, but weird things happen in the real world. Whatwill the designer do after rescuing the robot from the deadlock or afterbuilding a new one? He or she makes some adjustments to the code andmaybe adds some new sensors or increases the mobility to cope with these


51

surfaced problems. This cycle continues until finally the designer has a robotcapable of performing the given mission in all possible cases imaginable inthat particular environment. But, at what cost? The timetable has beenadjusted several times and the cost for the robot has gone sky high and canno longer be justified economically. Putting it bluntly, a few special perception-action couplings (or a small portion of state space) have become extremelyexpensive for the system deliverer.

Because 100% completion of the mission is in many cases essential we haveto go through this long and expensive cycle. Luckily, there is another way toreach the same outcome. The main idea is to substitute the minordeficiencies in individual robot operation not by doing additional software orhardware iteration but rather by multiplying the individual robot. Thus we canmake sure, that the mission can be accomplished. The problem here is, ofcourse, to define the required number of robots for the task. Interferencebetween the robots due to some resource sharing problems will actuallydictate the optimum number of robots in a particular task domain. Some tasksare nevertheless so difficult to perform that instead of copying a couple ofhomogeneous highly advanced (i.e. in perception-action couplings) robots, itis wiser to create a system consisting of several individually not so intelligentrobots. These robots can even be divided into groups. Each group is formedwith homogeneous robots capable of performing a certain task or tasks.Individual groups are thus heterogeneous. This gives us a chance to buildcheaper, smaller and simpler robots, which makes it possible to increase theredundancy of the system by increasing the number of robots in eachparticular group. If the mission to be performed consists of tasks which can beperformed in any order, this approach is very easy to implement. Usually,however, these tasks are somehow either temporally or spatially related andmust thus be performed in a particular order. This naturally increases theneed to have some kind of communication system between the groups for thecoordination of the work. If an inter-robot communication system must beimplemented, then naturally the time and material used for that purpose willincrease the total cost.

To justify the use of multiple robots instead of a single refined and optimizedrobot, we have to consider all the related costs. The actual comparison startsfrom the point where we have a single robot capable of handling almost everypossible situation on its route to mission completion. If we select the route,which includes the optimization we are going to spend a considerable amountof time and money for additional software and hardware designing andtesting. On the other hand, we can start immediately after reaching a certainlevel of competence to multiply the individual robot and save thus money andtime. Both of the these cases are illustrated in Figure 4.1.

The benefits of a society are quite clear. The redundancy of a society issuperior compared to a single robot solution, even though this robot wouldhave much more intelligence than the members of the society. A malfunctionin a single mobile robot on Mars costs billions, whereas the same happeningin a robot society would only decrease the performance by a few withpercentage points. Furthermore, a multi-robot system is usually much more


52

suitable for tasks that support parallelism, such as for exploring. The othermain benefit is the total modularity found in the control and thecommunication systems. In addition to the tolerance towards the malfunctionand thus removal of some members from the system’s active operation, thesesystems will accept the adding of new members. Introduce them to theworking area and let them loose, they will take their place in the societyimmediately.

INITIALDESIGN

TEST

PROBLEMS 95%performance

ADDITIONAL DESIGN:

SOFTWAREHARDWARE

TEST

PROBLEMS

ADDITIONAL DESIGN:

SOFTWAREHARDWARE

TEST

100% MISSIONCOMPLETION

99%performance

INITIALDESIGN

TEST

PROBLEMS 95%performance

MULTIPLICATION

TEST

100% MISSIONCOMPLETION

COST_1 = hardware + additional design(software+hardware) + TIME_1

time

cost

&

COST_N = N*hardware+ TIME_NTI

ME_

1

TIM

E_N

TIME_1>TIME_N

Figure 4.1 Two optional routes (individual robot on the left, multiple robots on the right) for~100 % mission accomplishment from the system supplier’s viewpoint. The diagram is highlysimplified.

4.3 Main Principles

There are clearly some general principles that should be connected to thedefinition of a robot society. It is of course possible to create a society ofrobots that doesn’t have all these features, but in general the society shouldhave the following properties, at least to some extent.

• Constrained local interactions: This feature combines two reallyimportant features, namely communication (active or passive) between themembers and sensing of the surrounding environment.

• Autonomy (i.e. self-sufficiency): This means that the society should beable to survive in the environment where it is operating. Usually it meansthat the members of the society can monitor their energy level and refillwhen needed. As well, the members should have some means ofrecovering from stagnation without any help from outside (i.e. from the


53

operator). These features are not always easy to achieve and in mostcases help from the operator is needed from time to time.

• Asynchronicity: This means that the members are individuals and theiractions are in many cases independent and self-activated. Direct couplingbetween sensing and actuating ensures this.

• Coordinated behavior: In some cases the society (or at least a limitedgroup of the society) has to be able to operate in a coordinated way. Sometasks literally require the “hands” of several robots at the same time.Various types of spatial self-organization, such as clustering, evolution ofrequired spatial structures as well as the dissipation of these structures arethen needed. To do this some sort of stability is required. It means that therobots have to have some kind of way to change their behaviorautomatically. As well as this automatic behavior switching, the termemergent behavior has become a quite frequently used term. Basically itmeans that as a result of rich interactions between behavioral modules andthe environment, the performance of the robot (the member) can be moreor less unexpected and in some cases very hard to predict and analyze.This kind of behavior is not necessarily always positive for the operation ofthe member and thus this concept should be studied closely. In manycases, it seems that these behaviors have been the designer’s goal fromthe beginning. One just has to wonder if they can then be called emergentbehaviors. On the other hand something totally unexpected in anengineering case is rarely a desirable feature. Nevertheless, it has to besaid that in multi-robot systems especially, the overall behavior of thewhole system is in many cases unexpected due to the rich interactionsbetween the members and the environment and could thus be describedas emergent behavior.

• Goal oriented: Because our concept of robot society is stronglyengineering solution, there are clearly some features that the system musthave. These include controllability and of course task performing capability.Controllability means that the operator has to have some way tocommunicate with the system and to be of any use the system obviouslyhas to have means to perform whatever task it was assigned to do.

• Knowledge representation: To be able to work efficiently as a part of thesociety the member needs a way of representing knowledge. This isrequired not only for member level operations, like mapping theenvironment, but especially for operating as a functional unit of a largersystem. Some sort of knowledge of the societal structure is needed forsuccessful mission accomplishment.

• Decision making: While operating in a dynamic environment with otheragents, the member needs to have some kind of problem solvingcapability. The member is facing almost continuously various types ofproblems which need to solved. In many cases these problems go down tomaking proper decisions, i.e. choosing the right ways to operate. Earlierthis kind of problem was solved by using an extensive heuristic, where


54

numerous rules decided which operation to take in each particularsituation. Unfortunately a dynamic system like a robot society has so manyuncertainties, that this kind of system is very hard, if not totally impossibleto construct. What is needed instead is some kind of adaptive flexiblesystem that gives the member the possibility to survive in changingconditions based on its perception and interactions from the othermembers.

• Capability to learn: This point is closely related to the decision-makingcapability. In many cases decision making is also under the learningprocess. By learning, the member improves its decision-makingcapabilities. But the need to learn is not restricted to this case only. Thereare several other areas which need to have some kind of possibility tolearn. These include the learning of better communication and controlsolutions. One of the most frequently used method is ReinforcementLearning (RL), where the member is learning the proper way to operateunder certain conditions. This kind of sensor-action policy learning is verycommon, but there are several problems attached to the use of thismethod. Other methods often include Neural Nets (NNs) and GeneticAlgorithms (GAs), where the perception and action modules are coded toan NN and a GA is used to improve the performance of this net. When weare considering a multi robot system one very important feature to learn isthe way an individual member should operate as a part of the society. Inthis case the member should have a way of receiving some informationabout how the society is doing, and how it is doing in it. If some othermember is doing far better than it is, then some sort of learning (orcopying) of the other’s behavior is needed.

4.4 Main Features

4.4.1 VolumeThe outlook of a society is highly dependent on the number of agents in it.The simplest society is naturally the case of two agents and at the other endof this scale are the cases where the "social insects" approach has beenchosen (i.e. there are "almost" an infinite number of agents in a society). Theoptimum number of units in a society depends on the application. It wouldseem trivial that an increase in the number of society members wouldautomatically increase the total result of the society in a linear way, at least incases where a parallel approach to the accomplishment of the task ispossible. This is usually true to a certain extent, after which the increase willhave no positive effect on the society's operation. This is due to interferencebetween the members in the system.

4.4.2 MissionEach robot society is designed separately for each individual mission. Thereare no free lunches in the field of multi-robot design, i.e. there are no generalrobot societies for various tasks. The mission must be either preprogrammedto members of the society, for example in the form of behaviors, or then it willgradually become evident for them for example through their onboard value


55

system. Robots performing some particular tasks at some phase of themission will receive positive feedback from the environment and/or from theother society members and will then act accordingly.

4.4.3 The Role of DiversityA society may be homogeneous or heterogeneous. Robots can be physicallyidentical (i.e. similar sensors, actuators, etc.) but are still consideredheterogeneous due to the different behaviors. The composition of a societyhas direct linkage to the control solutions; some kind of distributed control hasbeen used normally with a homogeneous case, while a heterogeneoussociety is usually controlled with some degree of centralized control. This kindof division is actually quite natural. In a homogeneous case all the membersare alike and the system can easily be understood to work in a parallel way.In a heterogeneous society more complex control solutions are needed. It is,however, possible to cope with this dilemma in a behavior-basedheterogeneous system as presented, for example, in (Parker 1996). See(Parker 1994) for a more precise description. Balch (1998) introduced theterm social entropy to provide a way of measuring the level of behavioraldiversity in a multi-robot system.

Ota and Arai (1993) and Ota et al. (1993) classify groups of robots into twocategories: static and dynamic. In a static group members don't change,whereas in a dynamic group members can change. A static group is a groupof members that know they are part of a group. The motion planning is basedon the group. This static feature is needed, for example, when the group iscarrying something so large or heavy, that a single drop-out from the groupwould led to a failure in accomplishing the given task. A static group is alwaysless flexible than a dynamic group. In certain tasks a static group is almostequal to a single larger unit; no failures are allowed. A dynamic group on theother hand can change it's form during the mission and will thus survive fromsome degree of malfunctions. Another way to measure the reconfigurability ofa group is to study the possible division between leaders and followers. In thestatic case the leader(s) and follower(s) are decided before the actualmission. This kind of strategy requires considerable motion control at thebeginning of a task. For example, when a group of robots is carrying a largeload towards a certain target, the leader has to place itself in the correctlocation. But if no classification has been made, each member can be either afollower or a leader according to the need and the situation.

Homogeneous societies are in principle more flexible. They continue tooperate even though some of the members in the society "die". In aheterogeneous case the "dead" of an agent can be fatal for the function of thewhole society. Another benefit which a homogenous society has is that thebehavior of an agent is predictable to the other members of the society, whichdecreases the need for an active communication. The use of aheterogeneous society, on the other hand, makes it possible to give thesystem more complicated tasks, because different groups can be designed toperform certain sub-tasks. It reduces the overall complexity of individualmembers.


56

4.4.4 Decentralized vs. CentralizedThe control structure of a society is one of the basic problems to be solvedbefore attempting to implement a robot society. This structure always has twoparts: the control of an individual member and the overall control of thesociety. In the global control the main question concerns superior units, whichhave some level of centralized control over the other members or, is thiscontrol transferred out of the society to the operator. The drawback of acentralized architecture is its vulnerability. If the "controller robot" shouldbreak down, the whole system is out of order and won't work. In contrast, ifwe have used distributed control, meaning that there are equal members inthe society, we have a much more redundant system on our hands. Theproblem of a distributed control is of course the increase in thecommunication between members and the fact that the group of totallyautonomous robots is quite difficult to implement without any centralguidance. The local control is of course totally dependent on the globalcontrol. The more distributed the global control is, the more complex the localcontrol must be.

4.5. Types of Interactions

4.5.1 Two Forms of Communication: Indirect and DirectIndirect communication takes place, when agents are communicating only bychanging and detecting the changes in their environment. In the animal worldants communicate through chemical messages (as well as through tactilemessages), and bacteria emit bacteriocin to provide information to the othermembers in their colony. Beckers et al. (1995) pointed out that a stigmergy,“the production of a certain behavior in agents as a consequence of theeffects produced in the local environment by previous behavior”, is not strictlyrestricted to building structures, but can also be found for example in ant trailrequirements. Thus in many cases stigmergy and indirect communication canbe considered to represent the same phenomena.

The active communication of a multi-robot system consists of three separatefeatures according to (Dudek et al. 1993): range, topology and bandwidth.The range, within a society, that a member can communicate with others canvary from zero to covering the whole area in which the society's members arelocated. In a practical application with some real robots the best solution isusually somewhere between these two extremes. A certain type of fixedradius around the member is probably the most convenient way to implementthe communication. The topology defines with what kind of rules thecommunication occurs. For example, is it possible to address a certainmember of the society, or is there a hierarchy which could dictate the protocol(e.g., a tree or graph). One common way is to communicate with everymember inside the defined communication range. The bandwidth indicateshow the price for communication has been evaluated. In simulations thecommunication is usually free, but in real applications it consumes valuableprocessing time and adds extra components to the design. At the other end ofthe scale is the situation where there is no active communication due to thevery high cost. In this case the required communication is done passively, bychanging and by detecting changes in the environment.


57

Fukuda and Sekiyama (1994) identify the most important aspects ofcommunication in multi-robot systems: the synchronization of action,information exchange and negotiations. In some tasks the synchronization isvital for a successful task completion. Information exchange is naturallybeneficial, that is, if information is valid (i.e., correct and new). Negotiationprovides a way, for example, to optimize and organize the work load. See(Numaoka and Takeuchi 1992) and (Ozaki et al., 1994).

4.5.2 Man - Machine InterfaceFrom an engineering point of view, the man-machine interaction is vital. Theinterface should provide, if possible, a real-time connection to the system.This can be done through each and every member, through some particularlocation (e.g., a base station) or through some special society members (inthe case of a heterogeneous society). The improvements in data transmissionand in network software have provided some very interesting possibilities inthis matter, such as the use of the WWW in (Suzuki et al., 1996).

4.5.3 Interference: Conflicts, Competition and DeadlocksIn most tasks, parallel by their nature, the use of multiple homogeneousrobots improves the performance of the society. The improvement can belinear or even sub-linear. But in most cases at some point the performanceimprovement stops even though the number of members is increased. Insome cases the performance of the whole system even decreases. Thereason for these results is obviously the increase in interference between therobots. The interference can be caused mainly by the conflicts in space, incommunications media or in a manipulable object (Cao et al., 1997). Theproblem with space is a natural consequence when a large number of robotsis operating in a finite space. Due to the physical constraints set by theenvironment, the robots are frequently moving in the same sub-areas, thusincreasing the possibility the conflict in space. To cope with this problem therobots spend a lot of time avoiding each other or recovering from temporaldeadlocks. Naturally, this reduces the active and productive working time. Thesecond main area of conflict is the communications media. Whether therobots communicate with infrared light or with radio frequency, theinterference increases as the number of robots increases or the number ofmessages is increased. Usually, a proper communication protocol takes careof the problem, but generally some part of the information is lost and must betaken into consideration when designing these systems. The third mainproblem is apparent when the members are collectively trying to move anobject. The robots have to have some kind of contact with the manipulableobject. The grasping or pushing locations on the object are numbered. Thus,a large number of robots have difficulty finding a plausible place for eachwhen participating in the operation.

When the operator is trying to find the optimum number of robots for aparticular task, three important parameters must be considered: the physicalproperties of the robot, the operational environment, and the specific task.Nowadays, there is no use in talking about general all-purpose robots.Unfortunately, we have a long way to go to reach that point. Rather, we


58

should talk about specific robot systems for a specific task. Each of thesethree main parameters are tightly related to others and in order to create anoperational system these parameters should be optimized together. To furtherillustrate this point a simulation test was performed (Vainio et al., 1998). Thesimulator used in the test is represented in detail in Chapter 6. The task wasto find distributed algae spots from a closed aquatic environment and thencollectively remove them (see Chapter 8 for details). Throughout testing theenvironmental conditions (i.e. initial algae sizes and growth rates, distancerelated communication probabilities, flow, etc.) were kept constant. Theinterference caused mainly by the competition for space was studied byvarying the size of the society (3,5,7,10,15 robots). Three robots were clearlytoo small a group to finish the job, while, 15 always caused a deadlock. This“traffic jam” was caused by a large group of robots simultaneously trying toenter an up-going narrow pipe. When comparing societies with 5, 7 and 10members, it became obvious that there were no significant differences in taskperformance nor in elapsed time, as can be seen in Figure 4.2. Nevertheless,the increase in the number of robots in operation had a correlation to theuntimely “death” of some robots. Consistently, out of 10 robots, 2-3 ran out ofenergy, and out of 7 robots 1-2 “died.” When the size of the society was 5,they all survived and performed the required task. Thus it seems that theoptimum number of robots for the task would be ≈ 5. This number is, ofcourse, highly dependent on the number and growth rate of the algae spots,as well as some other operational parameters, such as the threshold for whento actively start looking for energy. These matters are more closely discussedwhen the actual tests are presented in Chapter 8.

= Change from explore to exploit state

Figure 4.2 The task performance of various sized societies. It illustrates how societies toolarge (15 robots) or small (3 robots) fail in the exploiting task, whereas medium sized societies(5, 7, 10) are able to finish the task approximately at the same time (Vainio et al., 1998).

Chapter 5. Society Model

59

Chapter 5 Society Model5.1 Introduction

This chapter presents the developed generic control architecture, shown inFigure 5.1, for cooperative robotic systems (i.e. societies). Earlier versions ofthis architecture can be found in (Vainio et al., 1998a), (Vainio et al., 1998b)and (Vainio et al., 1999). This kind of architecture must be able to directlycontrol an individual as well indirectly affect (improve) the performance of thewhole society. The idea is to provide a framework for different types ofsocieties in various tasks. Each task domain sets its own requirements for themodel and must be adjusted accordingly.

All functions of the society are obviously realized through its members. Themembers’ behaviors result from their own needs and from the constraints(dynamic by their nature) set by the society or by the operator. Each memberhas a way to represent the status of the society by certain variables. Thesevariables summarize all the information that an individual agent has about itsown performance and the performance of the other society members. Thesevariables are usually expressed explicitly, but they can as well be presentedimplicitly with a sort of self-organizing process, for example, in a neuralnetwork.

The society is defined with a three level hierarchy: cooperative, task, andbehavioral. Each layer deals with some essential part of the robot’s survivaland mission completion in a dynamic environment with incomplete sensoryinformation. The model can be called a hybrid, because it contains bothreactive and deliberative components. Reactivity is used, as usually, at thelowest level, where a fast response to stimuli is needed. Planning, on theother hand, is related to performing the task. The robot must be able to planits actions if it has some sort of symbolic knowledge, i.e. a topological map, ofthe environment.


60

Each of these levels is implemented to the members. The cooperative layerwill ensure that the society is operating as planned toward mission goals. Forinstance, if four society members are performing well in a foraging task bycollecting objects from nearly the same location, then at the cooperative layer(which is still happening in each individual member involved) a decision on anobject quarry will be made. After that it is reasonable to recruit someadditional members to work with the same quarry and thus improve theperformance of the society. The task layer control is highly domain specificand includes problems like how to initiate a certain strategy in a particulartask. The behavioral layer is the core. Its main task is to provide tightperception action coupling and keep the robot operational. These differentlayers are explained in detail later on.

STRATEGY1

1

STRATEGY2

2

STRATEGY3

3

STRATEGY4

4

PERCEPTION ACTION

ENVIRONMENT

INTERNALRESOURCES

BEHAVIORAL LAYER

TASK LAYER

STRATEGY1

1

STRATEGY2

2

STRATEGY3

3

STRATEGY4

4

STRATEGY1

1

STRATEGY2

2

STRATEGY3

3

STRATEGY4

4

COOPERATIVE LAYER

task_1

task_N

task_2

R2

R3

R4 Rk

R1 Rk-1. . .

. . .

ta s k _ N

ARBITRATION

R 1R 2

R N

ST1 ST2 STNtask_1

R 1R 2

R N

ST1 ST2 STNtask_2

R 1R 2

R N

ST1 ST2 STNtask_N

ta s k _ 2

ta s k _ 1

BUFF

ER

communication out

group_1

group_2

group_M

com

mun

icat

ion

in

from operator

from other robots

to o

pera

tor &

oth

er ro

bots

self-suff._1

1

self-suff._2

1

self-suff._N

1

tasks

2

misc._1

3

misc._2

3

misc._N

3

Figure 5.1 Simplified diagram of the three-layer control architecture.


61

Being situated and embodied as stated earlier, an individual robot is tightlycoupled to the surrounding environment. It has to be able to survive withincomplete sensory information and with non ideal actuators. It can eitherhave a map or some kind of a priori knowledge about the area where it isoperating, or it can work only by the sensor information, which it receiveswhile working. In both cases, the robot needs to get a lot of information aboutits inner state and about the environment. The sensor reading frequencyvaries naturally based on the sensor and the task in concern, but in generalthe more dynamic the environment is the higher the frequency should be.

5.2 Communication Structure: Avoid, Minimize and Forget

As stated earlier communication is never free. Our approach is to avoid it asmuch as possible, minimize the frequency and contents of messages whennecessary and to provide the system with a way to forget obsoleteinformation. The two forms of communication in a society, i.e. operator-robotsand inter-robot, are presented next.

Robot’s function is to perform tasks requested by an operator. In many cases,the whole mission is performed without any help from the operator. However,in some cases, a human is a solid part of the system and controls some or allfunctions of an active society. The operator continuously receives informationfrom the society members. This information gives the operator some meansto understand and evaluate what is happening during a mission. Based onthis the operator can guide the society out of some unwanted states. Forexample some kinds of deadlocks can occur and in those cases small helpfrom the operator can solve the problem. In this control architecture thesociety has three separate operational modes: autonomous, semi-autonomous and non-autonomous. Autonomous means here that after themission has been started, the society is on its own. Semi-autonomous standsfor the case where the operator wished to modify the society’s operationduring the mission, for example by giving a certain task a temporal priorityover others. The non-autonomous mode is active when the society iscompletely under the operator’s control. This mode has a lot of similaritieswith normal robot controlling, where the robot is told exactly where to moveand what to do. The status of the society is dynamic, so changes from ahigher level of autonomy to a lower level during a mission is allowed andsometimes can even be vital for the successful performance of the society.

It is of crucial importance that the amount of inter-robot communication is keptto a minimum, especially when the size of the society grows. This isaccomplished by restricting the contents of messages. Only data about one’sown personal performance and status is transferred. The individual robot thenmakes its own decision about the status of the society. In general messagesare only sent a couple of times and no handshaking is required. Part of theinformation is lost, but it is not fatal to the operation of the society. Inter-robotcommunication is dealt later on in more detail.


62

5.3 Behavioral Layer: Perceive and Interact

This layer forms an interface between the robot and the environment. Thelayer receives information through various sensors (internal and external) andreceives communication messages from the operator. The reading cycles ofthese sensors can basically have two forms: they can either be timing basedor event based. Some sensors are read with a certain frequency, whereascommunications system is normally an event-based system, i.e. a message isread when it arrives.

The information is then fed to the actual control structure represented in theform of a Finite State Automata (FSA), see Figure 5.2. This state machinehas two main functions to perform: keeping the robot operational (i.e. self-sufficient) and reaching the given goals (i.e. task performing). Besides thesetwo main groups, there is one further group of states. It is usually related tospecial conditions and is normally linked to the termination of the mission (i.e.miscellaneous). Only one state of the Automata can be active at a time. Inmost scenarios the robot has to perform several tasks simultaneously. Thisprevents the use of several task states at the behavioral layer’s Automata.The output of the FSA is the control of actuators and some communicationmessages. Normally this means giving different motors speed and durationcommands (i.e. normally direction and velocity).

self-suff._1

1

self-suff ._2

1

self-suff._N

1

tasks

2

misc._1

3

misc._2

3

misc._N

3

Figure 5.2 The main FSA at the behavioral layer, containing three types of tasks: those thatkeep the robot operational (self-sufficient), one that gets the actual job done (tasks) and a fewothers usually related to the termination of the mission (miscellaneous). Only one state at atime can be active.

5.4 Task Layer: Parallel State Machines

The solution for the parallel task performance is to represent all tasks in asingle state at the behavioral layer FSA. This state is then represented in


63

detail on the higher layer, called task layer. This layer contains severalindependent FSAs each representing a single task, as shown in Figure 5.3.Every state machine has access to all the information available on the lowerlayer. The states in these machines are labelled as strategies because theycontain a behavioral pattern, i.e. how to proceed in reaching an individualtask’s goal. Strategies and state transitions are based on the performanceevaluation. If a certain strategy provides good results the robot should keep it,otherwise it should move on to another strategy for improvement. This exploit-explore dilemma is always present when reinforcement learning is applied. Inorder to find an optimum, the robot has to take risks and choose strategies(policies) that it has never tried before. Only by doing this can it avoid thelocal optimum pitfall and improve its performance in the long run. To providethe robot with at least a vague idea of what is going on, the control systemshould incite the robot to test the initial strategies at the beginning of themission. Unfortunately, because the environment changes all the time whilethe mission proceeds, the strategy superior at the beginning can loose itscharm later on. This is why the change of states should be quite frequent, andif possible some sort of evolution of new or combined strategies should beincluded in the design.

Unlike the lower behavioral layer, where only one state has access to theactuators, there has to be an arbitration mechanism on the task layer to solvethis problem. There are many possible ways to do this. These include, forexample, fixed priority, voting, summing and several others. See (Arkin 1998)for a comprehensive presentation. Common to all is that they will ensure thatthere is only one output command at a time for the actuators.

If a higher priority state at the behavioral layer’s FSA becomes active (e.g.,the robot is running out of energy), the necessary information about each taskis stored in buffers. When the robot program starts to perform the actual taskagain it retrieves the needed information from these buffers.

strategy_1

1

strategy_N

N

strategy_2

2

Figure 5.3 The FSA for an individual task. States represent the strategies selected.


64

5.5 Cooperative Layer: Dynamic Group Formation throughCommunication

An inherent property of the society concept is incomplete communicationbetween members. Only part of the transmitted information is available for theother members, while some messages are processed after a long delay. Dueto these delays, the value of these messages should be decreased. Thecooperative layer is active only if there is an active communication system. Itcontains all the information the robot has gained through inter-robotcommunication. When a mission starts, the individual robot has no idea of thesize of the society. Through communication (messages include senders ID) itgains knowledge of the other robots. A dynamic table is used to store thesemessages and a time label is attached to the messages upon arrival. Thislabel is decreased as time goes by until it reaches some threshold and isremoved from the table. Usual ways to provide this fading effect is through alinear or exponential decreasing. Linear decreasing provides a constantdecrease and goes to zero. The exponential approach, on the other hand,provides a large decrease at the beginning, but stores the information for alonger period. There are several potential types of messages that could beused, but a performance result for the chosen strategy is naturally valuable tothe other robots. Other valuable messages include, for example, informationabout selected topological targets. Common to all is that every task has itsown dynamic result table. See Figure 5.4 for an example.

Robot1 (R1) Robot2 (R2) . . . Robotk (Rk)Strategy1

(S1)result[S1][R1]time[S1][R1]

result[S1][R2]time[S1][R2]

result[S1][Rk]time[S1][Rk]

Strategy2(S2)



result[S2][Rk]time[S2][Rk]

. . .Strategyn

(Sn)result[Sn][R1]time[Sn][R1]

result[Sn][R2]time[Sn][R2]

result[Sn][Rk]time[Sn][Rk]

Figure 5.4 A dynamic result table for an individual task. It contains information about thesuccess of society members. A time label is set to maximal value, when a message isreceived and then gradually decreased.

Based on these stored messages and forgetting factors, the robot can forman internal picture of what is happening inside the society. Each robot,represented in the table, belongs to certain groups (or castes). A group isformed by robots that have selected the same strategy in a particular task.This means that an individual robot can belong maximally to as many groupsas there are different tasks and that the maximal size of a group is the totalnumber of robots in the society. A group can be formed also through someother criteria, such as the spatial distribution (i.e. robots operating at the samelocation) or the volume of the group (e.g. join the largest or smallest group).


65

These groups are, if not originally fixed, dynamic by nature and change as themission proceeds.

Robot (Ri) compares its own performance to the relative performance of thegroups. First it calculates its own maximal relative average of success asfollows:

successrobot= max{Performancerobot[Sj]} (5.1) j ε [1,n]

where

[ ] [ ][ ] [ ][ ][ ][ ]

Performance Sresult S R time S R

time S Rrobot j

j i j i

m im

n=

=�

*

1

(5.2)

It then compares this value to the maximal success values of active groups(i.e. robots operating in the same strategy),

successgroup= max{Performancegroup[Si]} (5.3) i ε [1,n]

where

[ ]( [ ][ ] [ ][ ] )

[ ][ ]Performance S

result S R time S R

time S Rgroup i

il

k

l i l

i ll

k= =

=

�

�

1

1

* (5.4)

If successgroup is greater than successrobot it means that the best group isperforming relatively better than the individual. Thus the robot should changeits operational strategy in that particular task to the strategy indicated bysuccessgroup. On the other hand if successrobot is greater than successgroup thenthe robot should ignore the group information. Robot performs this procedureeach time a certain task’s strategy ends. This ensures that the individual robotbenefits from the information it has received from the other society members.Normally there are only a few actual working tasks and in those tasks thereare usually just a couple of possible strategies. So, unless the size of thesociety is very large, these calculations can be done without any problemsrelated to time or computational power.

5.6 Pros and Cons

The developed model belongs to the wide class of three layered controlarchitectures. It is a hybrid model. The main goal was to keep it simple, use apriori information when available and relevant but at the same time provide afast response when needed. The model relies heavily on the establishedtheory of Finite State Automata, which has been used extensively in mobile


66

robotics for decades already. Information extracted from a non idealcommunication exchange between robots is stored in dynamic task tables.Based on these tables an individual member of the society has an idea ofwhat the other members are doing and how well they are doing it. Due to thestructure of these tables and the message forgetting factor, the status of thesociety, as an individual it sees, is not fixed, but dynamic, and so triesconstantly to find ideal group structures for each particular spatial andtemporal instances, naturally within reasonable time limits.

Out of all found control architectures suitable for multi-robot systems,ALLIANCE (Parker 1994) seems to have the most similarities with the controlarchitecture presented here. In ALLIANCE individual layers of competencethat are always active, can be considered to represent states other than tasksin our behavioral layer’s FSA. Behavior sets, on the other hand, represent inour model the individual tasks represented with tasks state in the behaviorallayer’s FSA and operated on the task layer. The cooperation layer in ourmodel has its counterpart in ALLIANCE as motivational behaviors. Thesebehaviors take sensor readings, inhibitory feedback from other activebehaviors, internal motivations and inter-robot communication as inputs. Theoutputs of these behaviors decide which behavior set becomes active. Oncea behavior set is activated, other behavior sets are deactivated. In our modelthe cooperative layer uses the inter-robot communication to decide whichstrategy should be activated in each individual task.

The model scales up pretty well, because additional members will onlyincrease the amount of communication and thus the sizes of the task tables.The calculations based on these tables are simple and performed ratherseldom, so it is not a real problem. Ironically, this same feature, i.e. anincomplete communication system, is the main cause for the problemsencountered when using this model: if some critical message is lost we cannot guarantee an optimal behavior at the society level. Some individuals cantemporally even work against the society, although this is case in the naturalenvironment where large societies exist. On some occasions, this kind ofbehavior can even be beneficial to the society in the long run, because these“renegade” individuals can direct the performance of the whole societytowards the next sub-optimal state, when conditions change rapidly in theworking environment and their “poor” strategy is suddenly the best alternative.

The operation of the control architecture presented is validated in Chapter 8,where it is put into use with a real autonomous distributed underwater roboticsystem.

Chapter 6. Experimental Test-bed

67

Chapter 6 Experimental Test-bed6.1 Introduction

Problems related to the use of simulators in mobile multi-robot systems arewell-known. No matter how well programmed the simulator might be, it stillcan’t replace real situated and embodied robots working in dynamicenvironments. Diverse interactions between individual robots and betweenrobots and the environment are in most cases the actual force that createscomplexity at the system level. Unfortunately a concept where tens orhundreds of robots are supposed to work at the same time is very hard toimplement without the use of a simulator. The main idea in our research wasto apply the simulator when testing some simple algorithms inspired by socialinsects and thus providing insights into how these simple creatures with verylimited rules can produce such complex global behaviors. After this initialphase the focus of research was to be transferred to physical realizations ofthe concept.

6.2 Underwater Society

While studying the basic structures of robot society with the foraging society,presented briefly in Chapter 2.5, it became evident that there actually was apractical demand for another type of robot society. The scenario of anunderwater robot society, sometimes called the Bacterium robot society, wasfirst presented in (Halme et al., 1993). More recent papers include (Halme etal., 1996b), (Vainio et al., 1996) and (Vainio et al., 1997). The main goal ofthis study, along with theoretical considerations, was to design andmanufacture an underwater robot society to work inside 3D processes as anextra feature linked to a traditional automation system, as illustrated in Figure6.1.


68

BA

T

T

T

P

P

G

G

A

B

PGT

BUS

Figure 6.1 An underwater robot society provides information outside fixed sensory locationsand transports chemical agents (A, B) to wherever they are needed. As an optional feature ofa traditional automation system, the society enables faster responses to problems andreduces the volumes of needed chemical agents.

In the processing industry, the question of how to monitor the internal state ofthe process in real-time and how to make local adjustments in mixing, flow orreaction conditions, if needed, is one of the major problems. Normally thesensors used in monitoring are fixed and thus will only provide informationfrom certain selected parts of the process. Local adjustments have been hardto implement, if not totally impossible, and thus the control has been basedmore or less on the overall control of the system. This may lead e.g. to anextensive use of chemicals, which is both expensive and often causesunwanted residuals and pollution. One solution for these problems could be amobile sensor/actuator robot society, which is capable of operating inside theprocess (i.e. measuring, transporting reagents and collecting samples).

Another natural application area can be found in the field of clean and wastewater processes. As a case example, we presented in (Halme et al., 1997)the possibility to use our underwater robot society for leak detection in cleanwater lines. We proposed a system where several robots work as a societyand improve the overall estimate of the leaks through multi-Bayesian teamdecision making. Figure 6.2 illustrates the main idea in the simple case wherea leak occurs in a pipe segment.


69

FLOW

SUBMAR

Input Output

Leak.2

Leak.1

Radio/ultrasonicpassive transmitter nro. 2

AMPL

ITUDE

AMPLI

TUD

E

Radio/ultrasonicpassive transmitter nro. 1

=NODES / =POSSIBLE LEAKS

Mesh

Figure 6.2 Water leak detection inside a long main water line.

Several identical robots (Figure 6.3) are put inside main water lines through aparticular two valve input station. These robots will travel mainly passivelywith the pressurized water and simultaneously measure several parameters inorder to detect problems, usually various sizes of leaks. At the other end ofthe mission, the robots are “captured” with the aid of a special metal meshinto another two valve output station. The localization of the robot can beachieved using several methods, but once again a fusion of several systemswill probably be the answer to most problems. A straightforward approachsuch as the time of the travel estimation should yield good results, especiallyif some sort of extra sensory information is combined. These include forexample the use of radio or ultrasonic beacons installed in some places alongthe route to be inspected. For a detailed presentation see (Halme et al.,1997).

Figure 6.3 Robot society member inside a pipe.


70

6.2.1 Physical SocietyThe practical testing of the society is done in laboratory’s demo processenvironment, shown in Figure 6.4. It consists of tanks, pipes and a jet-flowpump. The volume of the transparent demo process is 700 liters.

Figure 6.4 Society in test environment.

The society consists of small sized (diameter 11 cm) spherical robots calledSUBMARs (Smart Underwater Ball for Measurement and ActuationRoutines). These robots move passively along liquid flow or actively bycontrolling their specific weights. SUBMAR, shown in Figures 6.5a-6.5c, isequipped with a micro-controller, several sensors, tank actuators and a shortrange radio for communication. The figures represent the latest generationSUBMAR. See (Appelqvist et al., 1997) and (Appelqvist et al., 1998) fordetails. The sensors implemented depend on the application at hand.Typically sensors for temperature, pressure, pH, and conductivity are useful inmany applications. Inertia sensors could be used for measuring accelerationsand for navigation purposes. The tanks are used for controlling the specificweight, taking samples and carrying a cleaning agent. Energy is carried in abattery back. Due to the fact that motion energy is taken mainly from flow


71

currents, the consumption of energy is relatively small and the operation timelong, in practice several hours.

Figure 6.5a SUBMAR.

Figure 6.5b Cross-section (Appelqvist et al., 1997).

A / D converter

expansion bus

I / Oserial

I / Oparallel

utilities CPU80C166

128 KSRAM

128 KFlashEPROM

memorydecoder driver

circuit

driver circuit

chemical tank motor

divingtankmotor

radio communication

infraredLEDs

status &indicatorLEDs

signalamplifier

signalamplifier

signalamplifier

temperaturesensor

pressuresensor

infraredphototransistorsstep-up power

converter,5V supply

battery pack3.6V, NiMH

oscillator32 MHz

currentmonitoring

programloading

voltagemonitoring& safetyshut-down

Figure 6.5c Electronics block diagram (Appelqvist et al., 1998).


72

The following generations of robots, intended to be used in certain industrialenvironments, will be tailored to each case separately. Harsh environmentalconditions will be demanding of the casing and components. Theminiaturization technics and bio-sensor technology will provide sensors andactuators enabling some very small robots in the near future.

6.2.2 Simulated SocietyIn order to validate the behavior of a robot society as a solution for a complexreal world problem, before we could produce more than a couple of realrobots, a simulator was required. The simulator was coded with Open GL inSilicon Graphics Indigo2 to represent the 3D world with complex dynamics(Figure 6.6). The extensive calculation makes on-line simulation quite slow.However when the graphics updating is off, the speed is much faster and thesimulator can provide the statistical data needed. Basic flow dynamics arecombined with the features extracted from the real process. The otherfeatures of the simulation, such as the robots’ control structures, collisionhandling and diffusion of the poison are based on the testing done with a realrobot. As well, the flow vectors have their own impact on the diffusion rateand its direction. The operator can adjust various parameters such as thecommunication range, algae growth, number of members, etc. On the lowerwindow the contents of a selected parameters are shown. It illustrates, forexample, the energy level of the robot. In addition, it presents how the robotlearns its environment in the form of a topological map.

Figure 6.6 The appearance of the 3D society simulator.

6.2.3 Communication SystemThe behavior of the society must be controllable outside and thus the societyhas to have an information connection to the controller. However, it isimportant from a practical point of view that this connection is not built into


73

every member of the society, but rather to the information system. This isbecause a society may include a large number of members that are located inplaces where a communication system is difficult to build. Basically, thecommunication in a society is carried out on a member-to-member basis.

Communication in the society is divided into two categories: between operatorand robots, as well as inter-robot communication. Status messages and themeasurement data are sent from robot to operator, while robots can receivehigher-level control commands for further operation. Members carry and passon short messages, distributing information throughout the whole society.Since all data exchange is carried out under a common frequency with half-duplex radio modules, some sort of protocol is required to prevent overloadingof the communication network. It is evident that some part of the informationwill nevertheless be lost. However, if the communication structure is welldesigned, some loss will not harm the functioning of the society, and in theworst case only some actions will be delayed.

As a solution, a CSMA/CD type of protocol has been implemented toSUBMARs to enable messaging with minimal loss of information. Theprotocol frame is presented in Figure 6.7. See (Premvuti and Wang 1994) forsimilar approach.

Item Bytes ExplanationMessage title :STX 1 Start byteVersion 1 Version numberReceiverID

1 Receiver / Broadcast

Sender ID 1 SenderMessagetype

1 CMD, INFO, ACK,...

MessageID

1 Used for ACKs

Datalength

2 Length of payload

Title CRC 2 Title checksumPayloadCRC

2 Payload checksum

Payload :Datamax.

65520 Actual message

Figure 6.7 Message frame for communication (Appelqvist et al., 1998).

There are some built-in features in the protocol to support effectivecommunication: messages can be sent only to certain members, or broadcastto all in the range. In addition, the most important messages can be sent withan acknowledgement request, to ensure that the message was receivedcorrectly. The number of automatic retries can be set, as well as time-outs


74

before them. CRC checksums are calculated separately for the message titleand for the actual message, the payload. To minimize the amount oftransferred data, only a collection of fixed messages is used. The protocoldoes not determine the format of messages.

6.2.4 Operator InterfaceUser interface allows the operator to control the society and get on-lineinformation from the robots. It features protocol settings, different types ofsending, and logging of received data onto files. Each robot has its own logfile, which allows detailed performance analysis for the task executionevaluation. Figure 6.8 shows a display from this software, which runs at anNT workstation. The radio module is operated via the RS-232 port.

Automatic mission control is a software client for the previously mentioneduser interface. It allows preprogrammed mission controls for task execution,i.e. the messages will be automatically sent at a given time. The TCP/IPconnection to the user interface also enables running of the programs fromdistant location through the Internet.

Figure 6.8 User interface running in NT environment.

6.2.5 Self-SufficiencyContrary to simulated worlds, in real world robotic application there is no suchthing as an endless energy resource. One of the most important (if not the


75

most important) value that the robot must monitor is its energy status.Compared to a normal 2D environment, a liquid 3D world makes this taskchallenging. In the demo process there are basically two different ways toimplement the loading of energy. It is possible to provide a galvanic contactby using a space where a section of the robot is out of the water. This can bedone if the volume of the diving tanks is large enough. The other possibility isto implement it with inductive energy transferring. However, because the reallife time of a battery pack is much longer than the mission length in ourmission scenarios, we decided to avoid the problem of constructing a physicalrecharging station. Instead, we included an IR -lamp in the system. The lightof this lamp represents the incoming energy, which the robot can “absorb”locally through its phototransistors.

In our system the incoming flow of energy is considered to be unlimited as isusually the case in systems that can be easily reached, for example, a normalfactory environment. Here it means that there is an endless supply of energyflowing into the recharging location, and every time a robot arrives at thatlocation its batteries are loaded with maximum current. On the other hand, ifthe working environment is a deep sea floor or space, the situation changes.In those cases energy is a very valuable resource and its use should alwaysbe optimized. For example if the system consists of several homogeneousrobots and one robot acquires a malfunction which increases the use of thecommon energy supply, the system should be able to shut it down for thesake of the whole system.

Much like most niches in the natural environment, the amount of givenincoming resource is more or less constant (but not indefinite) over a certainperiod of time. The more animals there are, same species or not, sharing thisresource, the fiercer the competition. As a result some will do better thanothers, some will be forced to leave their niches, and some will eventually failin their task (to reproduce) and die. Steels (1995a; 1995b; 1996) presentedan ecologically inspired system, where the incoming flow of energy was keptconstant and the robots had to compete for their recharging energy not onlywith each other but also against some lamps using the same energy, thusacting as parasites. The robots could increase their share of the resource bytrying to “kill” the lamps by colliding with them. Each collision reduced thecurrent the lamp used and if enough collisions took place the lamp wasswitched off for a while, before it woke up and started to compete again. Thiskind of system creates an interesting scenario where one can study forexample strategies emerging among robots. Some level of cooperation canbe seen when several robots were hitting a single lamp until it switched of.Furthermore, some sort of selfish behavior could also be detected in the formof a robot waiting around the recharging station while the others were busykilling the lamps. To create a similar system, we could easily limit the energyloading voltage (i.e. the current) by relating it to the strength of algae growths.The smaller this value is the longer a robot has to stay in a recharginglocation in order to get its energy storage refilled. Naturally, the longer it is inrecharging the longer it is out of productive work and the smaller the loadingenergy gets. This kind of approach removes the need to create some sort ofartificial fitness function to follow the success of the society in the algae


76

removal task. Both of these approaches (i.e. constant and variable energy),referred to “Ecology” and “Engineering”, are illustrated in Figure 6.9.

+--

Constant energy flow 9.4 V

Ualgae_1

Ualgae_2

[0 - 4.7 V]

[0 - 4.7 V]

Erecharging [ 0 - 9.4 V ]

loading ON

loading OFF

IR lamp

algae_1

algae_2

EngineeringEcology

Figure 6.9 The energy recharging system. Two different approaches were designed. One hasan endless supply of energy flowing into the robots through an IR lamp with a voltage of 9.4Volts. This case, named Engineering, represents a situation where it is easy to transfer energyto the system. It was used in our tests. The other case, named Ecology, represents a casewhere other units (i.e. algae growths) compete with robots and the voltage level fluctuates. Itremains to be tested.

Chapter 7. Mapping and Navigation

77

Chapter 7 Mapping and Navigation7.1 Introduction

In order to perform distributed operations, the members of the society, with avery limited ability to move actively, require some kind of spatialrepresentation model of their environment. This representation can be inCartesian coordinates or in a relative coordinate system. Theserepresentations are usually not at the sensor level, but on a higher level. Thefused information in these representations is usually added to a world model.The form and meaning of these representations and the world modeldepends on the case. In some cases the robot doesn't need any kind of worldmodel at all. It can operate purely on a situation-action base. On the otherhand there are applications where the robot has to have an exact map of itssurroundings in order to be able to work. Luo and Kay (1991) list five differenthigh-level representations: Spherical Octree, Occupancy Grids and NeuralNets, Graphs, Labelled Regions and Production Rules. These are presentedbriefly below.

1. Spherical Octree:A spherical octree is an octonary tree structure that at its first level separatesthe spherical perspective view of the environment into eight octantscorresponding to the children of the root node (the entire sphericalenvironment perceived by the robot). Objects in the environment can berepresented in the octree by recursively subdividing octants containing part ofthe object into eight more octants at the next lowest level and merging octantsthat are completely contained by the object into one octant at the next highestlevel, repeating this process to represent the object at increasingly finerresolutions.

2. Occupancy Grids and Neural Nets:This representation allows integration of information from many different typesof sensors. Bayesian estimation is used to fuse together each sensor's


78

probabilistic estimate as to whether a cell in the grid is occupied by an object.The resulting grid is useful, for example, when determining a free path for therobot to travel. This same idea is also modified for neural nets, where themagnitude of each neuron's activation corresponds to the probability that thecell it represents is occupied.

3. Graphs:Graph structures have been used to represent the local and topologicalfeatures of the environment to avoid having to define a global metric relationbetween non adjacent nodes in the graph.

4. Labelled Regions:Sensory information can be used to segment the environment into regionswith properties that are useful for spatial reasoning. The known characteristicsof different types of sensory information can be used to label some usefulproperty of each region so that symbolic reasoning can be performed at ahigher level in the control structure.

5. Production Rules:The use of production rules in a control structure allows for a wide range ofwell-known artificial intelligence methods to be used for path planning andlearning purposes.

In a complex underwater environment any absolute localization method isdifficult to construct and requires some sort of active beacon system or ahighly sophisticated inertia system. The concept of the robot society providesa simple way to solve this problem. An individual member’s map need not behighly accurate. It is enough that different process parts can be clearlyrecognized. The cooperation between society members will then make theirmaps more detailed. In (Mataric 1991) it was pointed out that even though thefixing and finding of landmarks among animals is usually based on vision, itisn’t obligatory. Animals are known to use various of types of landmarks,including tactile and auditory. Mataric presented a neurobiologially-feasiblecognitive mapping, implemented into an autonomous mobile robot, wherelandmarks were defined as combinations of the robot’s motion and sensoryinputs. The map produced by the robot contained nodes (i.e. landmarks) andtopological links between different nodes, which indicate their spatialadjacency. We chose to represent the environment with a related method byusing a strongly connected directed graph. It is represented in the form of anadjacency matrix indicating the topology of the graph.

7.2 Mapping Algorithm

Mapping is based on processing a single variable, namely pressure. Thisvalue is used to detect events (i.e. nodes) where the motion characterchanges. A change indicates that motion changes from downwards to acertain level, from a level downwards, from upwards to a level and from alevel upwards, as shown in Figure 7.1. The data obtained this way is naturallylimited and open to error. To be able to reconcile this feature some kind ofadaptive method has to be used. In (Yamauchi 1995) a concept called APN


79

(Adaptive Place Network) was introduced. This provided a spatialrepresentation and learning capability for a mobile autonomous robot. Amodified version of this method was implemented into our system and isdescribed briefly in the following:

UP to LEVEL (UL)

DOWN to LEVEL (DL)

LEVEL DOWN (LD)

LEVEL UP (LU)

Figure 7.1 Node types

When a new link is created it is also assigned a confidence value c∈[0,1].This value basically estimates how real the link is, i.e. does it really exists or isit just some kind of sensor error or due to a collision between societymembers. At the beginning of the lifespan of a link the value c is set to cbirth. Ifthe robot travels through a link then the value of that link is increased with thefollowing equation:

ct+1 = λ +(1-λ)*ct (Eq.7.1)

where λ is the learning rate. After a certain number of nodes (Nnodes) havebeen detected the values of all links are reduced according to the followingformula,

ct+1 = (1-λ)*ct (Eq.7.2)

When this value goes below a threshold (Tkill), the link disappears from theadjacency matrix. If all links connecting the node to the graph are deleted,then the node is also removed from the graph. Collisions with other memberscreate nodes which are, however, eventually removed from the graph. Thepseudocode of the make.map -algorithm is shown in Table III and exampledata from a run are illustrated in Figure 7.2.


80

Table III: make.map -algorithm1. Take one new pressure measurement (pi).2. Create input vector, i.e. three consecutive pressure measurements Ij

=[pi,pi-1,pi-2].3. From Ii calculate: Si = (pi - pi-2)4. IF two consecutive S values are on the opposite side of threshold (T or -T) THEN a new node (a new event) has been found.5. The new node is compared to the linked nodes of the current node.

IF matching node is found (i.e. the type of event is the same, and thedifference between pressure values is smaller than T)

THEN it is considered the current node. ELSE the new node is compared to all known nodes.

IF matching node is found THEN it is considered the current node.

6. IF on at least two previous algorithm cycles a link from the current node tothe new node has been found

THEN update current node data by averaging the pressure value andreturn

the current node. ELSE add a new node to the graph and return it.7. IF a link exists from the previous to the returned node

THEN increase the link confidence (Eq. 7.1) ELSE add a new link with confidence value cbirth.8. Decrease all link confidences after every Nnodes found nodes (Eq. 7.2)9. IF a link confidence drops below Tkill THEN remove the link.10.IF a node is left without links THEN remove the node.11. Go to step 1

Figure 7.2 An example run with the following parameters: λ = 0.25, Nnodes = 20, Tkill = 0.1, cbirth= 0.5. From the pressure values (depth) various parts of the demo process can be recognized.The length of this particular run is about 1 hour 30 minutes. A new node is always markedwith a cross, and a circle represents a known node.

7.2.1 Preliminary Results Using the SimulatorAlong with these initial tests implemented with a real robot, the societysimulator was used to study the behavior of the society. The preliminarysimulations verified the obvious fact that the parameters (learning rate, etc.) in


81

the make.map- algorithm had a very strong effect on the form and size of themap. In the following tests these parameters were fixed and the only variablewas the number of members in the society. First we studied how a singleagent behaves in a simulator. When the simulation terminated, the topologyof the process was quite well-known to the agent. The size of the adjacencymatrix was still the same and the confidence values for the nodes were quitehigh, thus indicating clearly the existence of the learning capability (Figure7.3a - 7.3b).

194 193 7 182 41 41 59 59 7 185 191 156 156 107 107 192 1070 194 0 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 193 0 0 0 60 81 0 0 0 0 0 0 35 0 0 0 0 622 7 83 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 03 182 0 0 0 0 0 0 0 0 0 59 0 0 0 0 0 0 04 41 0 0 0 0 0 81 0 0 0 0 0 0 0 0 0 0 05 41 0 0 0 0 0 0 81 0 0 0 0 0 0 0 0 0 06 59 0 0 0 0 0 0 0 81 0 0 0 0 0 0 0 0 07 59 0 0 0 0 0 0 0 0 81 0 0 0 0 0 0 0 08 7 0 0 81 0 0 0 0 0 0 0 0 0 0 0 0 0 09 185 0 0 0 0 0 0 0 0 0 0 0 0 0 59 0 0 010 191 0 37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 011 156 0 0 0 0 0 0 0 0 0 0 0 0 35 0 0 0 012 156 0 0 0 0 0 0 0 0 0 0 35 0 0 0 0 0 013 107 0 0 0 0 0 0 0 0 0 0 0 0 0 0 59 0 014 107 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 78 015 192 0 78 0 0 0 0 0 0 0 0 0 0 0 0 0 0 016 107 0 0 0 0 0 0 0 0 0 0 0 0 0 0 37 0 0

Figure 7.3a An adjacency matrix for a single agent after 7.3b Topological map drawn onfull simulation time. op of the process picture.

Next the number of members was increased to 10. The results are shownbelow. The size of the adjacency matrix is obviously larger due to the multiplecollisions caused by the presence of 10 society members. The confidencevalues are thus predominantly quite low and the topological map is also ratherinadequate. When the simulation proceeds, the main nodes will become moreevident and the topological map will acquire a familiar form (see Figures 7.4a- 7.4b). Even though the members have numerous collisions, these eventsare forgotten quite quickly.

193 42 42 59 58 14 109 192 194 111 190 20 194 155 155 59 59 76 76 43 43 1940 193 0 69 0 0 0 0 0 0 0 57 0 0 0 42 0 30 0 0 0 0 0 01 42 0 0 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 02 42 0 0 0 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 03 59 0 0 0 0 68 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 04 58 0 0 0 0 0 0 0 0 0 0 0 67 0 0 0 0 0 0 0 0 0 05 14 0 0 0 0 0 0 0 0 66 0 0 0 0 0 0 0 0 0 0 50 0 06 109 0 0 0 0 0 0 0 57 0 0 0 0 0 0 0 0 0 0 0 0 0 07 192 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 08 194 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 09 111 0 0 0 0 0 0 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 010 190 45 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 011 20 0 0 0 0 0 65 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 012 194 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 013 155 0 0 0 0 0 0 0 0 0 0 0 0 0 0 42 0 0 0 0 0 0 014 155 0 0 0 0 0 0 0 0 0 0 42 0 0 0 0 0 0 0 0 0 0 015 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 016 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 017 76 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 018 76 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 019 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 50 020 43 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5021 194 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Figure 7.4a An adjacency matrix for an agent after full 7.4b Topological map drawnsimulation time with 10 member society on top of the process picture.

7.3 Common Environment Representation

At a certain phase, the map built by the make.map -algorithm in each robotreaches a stabile form, where only small temporary perturbations emerge.The size of the map stays within reasonable limits due to the reinforcementalgorithm presented above. The maturation of the map can be easily detected


82

by following the number of new nodes vs. old nodes. On a matured map theold nodes dominate clearly and only occasionally is a new node detected. Atthis point a robot will fix its map and notify the operator. In order to be of anyuse, the members of the society should all have the same map. Thecombination of maps can be produced in various ways ranging from a fullyautomatic method to a map created by the operator, as shown in Figure 7.5.

Robot_1make_map

1

Robot_2make_map

2

Robot_Nmake_map

3

Robot_1basic_map

4Robot_2

basic_map

5Robot_N

basic_map

6

COMBINATIONinter-robot

7

COMBINATIONoperator

8

COMMON BASIC MAP (CBM)

9

checkCBM

10

OK

PROBLEMS

Figure 7.5 The flowchart for the creation of the Common Basic Map (CBM). First, individualrobots use the make.map- algorithm to acquire their basic maps. These maps are combinedeither through inter-robot communication or by the operator. As a result every robot has thesame CBM.

The automatic combination is performed through inter-robot communication.Several methods are available for this combination. When all the members(or at least most of them) have reached the point where the map hasmaturated, each robot transmits the map to the other robots. After receivingthe maps a robot uses an algorithm that combines maps according to certainrules. The rules contain factors like matching particular nodes, checking theconfidence values for particular links, etc. As a result each robot has the


83

same map that fuses the distributed perception information the societypossesses. One way to carry out the map fusion is to allow the individualwhose basic map is ready first to give its map to the others. This approach isfast and easy to implement, but does not guarantee that the society has thebest possible map.

The map fusion task can also be given to the operator. After receiving themaps through the radio, the operator can either use a special algorithm to dothe combination or do it manually. The operator can also include someadditional features in the map even though the robots haven’t noticed them.

Whichever of the previous methods is used, the society members have acommon representation, the Common Basic Map (CBM), of the environmentafter this phase. This map provides them with a change for cooperation. Thequality of the map is verified from time to time through a checking procedure.During the sequence each robot tracks how well CBM represents the currentsituation of the environment. If problems are detected (i.e. many new nodesappear) then the make.map -algorithm is run again. As a result the old CBM isreplaced with an updated CBM. With the demo process, the CBM has theform as illustrated in Figure 7.6.

23

7 6

12

1118

17 16

1514

13

9

type: UP to LEVEL(UL)depth: -33 cm

N 4

L[13][14]type: EMPTY(E)lenght: 87 cmtravel time: ~20 sec

depth

0 cm

-192 cm

510 8

14

Figure 7.6 The basic map drawn on top of the demo process layout. Nodes are representedas circles and arrow heads indicate the directions of links. The two gray boxes describes thedata structures in node array N and link matrix L.

7.4 Using Common Basic Map

The map is given to the robots in the form of a matrix L (links) and an array N(nodes). Starting from a known location the robots are immediately on themap. After that they just follow the possible routes on the map. There areusually no more than three possible links from a particular node. When a


84

node is detected the robot compares node type and pressure value to thepossible nodes in the map. Usually there is a match and the robot stays onthe map (Old node in Figure 7.7). Sometimes, however, no match can bedetected. This means that the robot has either collided with another robot orhas moved in an unusual way, for example interrupted a dive before reachingthe node at the bottom of the tank. With a mismatch, the robot neglects thefollowing node, because non-matching nodes come always in pairs, as canbe seen in Figure 7.7. After that the robot tries to match the next node to anynode on the map. In most cases the node is unambiguous (New match inFigure 7.7). There are some nodes, however, which are ambiguous. In thesecases the robot has more than one potential match, and it can not know forsure where it is (e.g., R2 initial location (troubles) in Figure 7.7). Based on theCommon Basic Map, shown in Figure 7.6, when a robot arrives at node 2,there is no way to tell at which node it will end up next, if it has emptied thediving tanks. The possible nodes include 3, 11 and 15. This is due to thewater flow which may either suck it into the vertical pipe or allow it to rise insome of the tanks. The problem is solved, however, by checking the nextnode. The main principles of this algorithm, called follow.map, are shown inFigures 7.7 and 7.8.

Old node 7Old node 6

Old node 5

New Match 2

troubles!!

New Match 2

Robot 1 (R1)

Robot 2 (R2)

COLLISION -no matching nodes -no transmissions -skip the next node

R1 initiallocation

R2 initiallocation

Figure 7.7 The main features of the follow.map -algorithm. The operation of the algorithm isbased on the Common Basic Map (CBM). See text for details.


85

current node = work_node

1

confusionpossible in

CBM?

2

Possible next nodesin CBM are (A,B,C)ambiguos=TRUE

3

node found4

ambiguos=TRUE(possible nodes

are A, B ,C )

5

matchA or B or C

6

YES

NO

YES

ambiguos=FALSEwork_node=(A or B or C)

7

YES

linked nodeexists inCBM?

8

NO

skip_next_node=TRUE9

NO

match anynode inCBM ?

11

NO

skip_next_node=TRUE

12

NO

skip_next_node=FALSE

13

YES

skip_next_node=FALSE

14

YES

NONO

work_node=Y

15

YES

work_node=X

16

YES

Figure 7.8 The flowchart of the follow.map -algorithm.

7.5 Navigation

The path planning of an individual robot is based on the fact that theCommon Basic Map is a strongly connected directed graph. From every nodethere is access to all other nodes. The structure of the map makes it possibleto use Floyd’s algorithm (Sedgewick 1988) to calculate the shortest pathbetween each node. This calculations can be done in advance based on theCommon Basic Map’s link matrix L. The algorithm takes L as an input andprovides an output in the form of the same sized matrix V. Matrix V is used inanother algorithm, which calculates the shortest node trail from a start nodeto a goal node. This algorithm uses depth-first search. It is also run in


86

advance at the beginning of the mission after the Common Basic Map hasbeen obtained. This information is stored in a struct called ROUTE.

The navigation method must take into account robot limitations inmaneuverability and must thus be very straightforward. It is based on thestatus of the diving tanks while traveling from one node to another on themap. There are three different modes for link usage: full(F), empty(E) anddon’t care(#). Full means that the link represents diving. Empty indicates thatthe robot is moving upwards going up based on its buoyancy. Don’t caremeans that the status of the tanks is irrelevant for the use of the link. Such alink represents, for example, vertical pipes, where a strong flow moves therobot in any case. All this information is stored in matrix L indicating how therobot can move from one node to another.

The actual navigation is based on the combined use of the struct ROUTE,matrix L and array N. As an example consider the case where the robot hasto go from its current node(6) to a goal node(12)(Figure 7.9). It fetches theroute trail (6 � 12) from the struct ROUTE. The trail contains nodes6,7,2,3,12. After that the robot uses matrix L to check the motor status fromone node on the trail to the next. As a result it receives the followinginformation (6�7: F), (7�2: F), (2�3: #) and (3 �12: E). Next, the robottracks backwards from the goal node(12) to find out where the status of theactuators should change. In this particular case it happens when the robotreaches node 2. The link between nodes 2 and 3 is marked as don’t care(#)and can be considered to represent E. The navigation strategy for this casemust start with a dive in order to move from node 6 to node 2. In node 2 therobot should empty the tanks. This should take the robot to the goal node 12.

Navigation task (current node = 6 | goal node = 12)Get ROUTE [6][12] = 6 - 7 - 2 - 3 - 12

Traillink 3-12: motors Elink 2-3: motors #link 7-2: motors Flink 6-7: motors FIn node 6: Dive(F)In node 2: Surface(E)

Node 6:Dive

Node 2: SurfaceNode 3

Node 4: failureReplan, Get ROUTE[4][12]

Node 12: success Stop

Node 7

Figure 7.9 The navigation is based on the use of tank actuators in particular nodes.


87

If the robot detects that it has lost the trail, i.e. that it finds itself in some othernode than those listed on the node trail, it simply plans again starting fromthe current node. Loosing the trail for a while is something that has to beaccepted due to the limited maneuverability. The complete algorithm isillustrated in Figure 7.10.

Common Basic Map1

Calculate shortest paths for CBM by using Floyd's algorithm

output V

2

Calculate path sequences for each start-goal node pairs by using a

depth-first search and V --->

struct ROUTE ={{0},{1,2},{1,2,3},...,{1,2,3,5,...,X}}

{{2,3,4,1},{0},{2,3},...,{2,3,4,7,...X}}...

{{X,1},{X,1,2},...,{0}}

3

goal node is given4

Use follow.map to get current node5

Get route trail from current node to goal node from ROUTE(current, X, Y,.., Z, goal)

6

Backtrace motor status starting from goal node to current node in order to find out the

nodes where to run motors

7Get motor status (F / E / #)

for the links on the route trail from current node to goal node

(from matrix L) (current-> X: F) | (X->Y: F) ...|

(Z->goal: E)

8

Use follow.map to get current node

9

current node=

goal node

10

current nodeon

route trail?

11

NO

NO

NO

END

12

YES

current node=

run motor node

12

YES

Run motors (F / E / #)

13

YES

Figure 7.10 The path planning and navigation algorithm is based on the Common Basic Mapand route information available from it. It also uses the follow.map -algorithm to keep robot onthe map and to recognize when the target has been reached.

7.5.1 Test ResultsThe navigation task (from node 6 to node 12) shown in Figure 7.9 was testedwith a single robot. Fifteen separate runs were performed and the duration


88

and the routes of the runs were recorded. The results are shown in Figure7.11.

Navigation from node 6 to node 12 (15 runs): C1(directly), C2(one round), C3(up once), C4(up three t imes), C5(up tw ice &one round)

time

used

for n

avig

atio

n [s

econ

ds]

0

50

100

150

200

250

300

350

CASE1(9/15) CASE2(1/15) CASE3(3/15) CASE4(1/15) CASE5(1/15)

0

50

100

150

200

250

300

350

Figure 7.11 Results from a navigation task, node 6 to node 12. Five different cases wererecorded during 15 trials. Each bar represents the average time (if more than one sample) forthat particular case.

Out of 15 test runs in 9 cases, the robot was able to navigate directly to thegoal location (Figure 7.12). The results from separate runs are shown inFigure 7.13.

Start

Goal

Direct navigation from node 6 to node 12 (9 runs):average=81.8, min.=77, max.=87, std.=3.7

tests

727476788082848688

1 2 3 4 5 6 7 8 9

Figure 7.12 Direct navigation Figure 7.13 Individual tests for direct navigation. Standard deviation (std) is very small.

Once the robot was unable to rise directly to the target location due to astrong current, and it was sucked into the vertical tube. After that, the robotre-planned and succeeded in its navigation, as is shown in Figure 7.14.


89

Start

Goal

Figure 7.14 The robot was forced to perform one extra round.

In three times out of 15 trials, the robot became stuck in the zero current zoneof the large round tank. After it ejected the water from its tanks, it rose to thesurface and noticed it had failed in its navigation and had to re-plan (Figure7.15). After re-planning, the robot was able to reach its target destination.

Start

Goal

Figure 7.15 Zero current conditions in a certain area at the bottom of the largest tank causedthe robot to miss the target on the first attempt. The second attempt was successful.

Besides these three main cases, there were also two additional cases. Once,the robot was stuck in the zero current zone three times in a row beforesucceeding in its mission. Another time it was stuck twice in a row, after,which it had to do one extra round before mission completion.

In order to verify the problematic flow conditions in the demo process, thenavigation task from node 12 to node 6 was tested (Figure 7.16). It wasknown that there was strong turbulence on that route. The results from 15separate test runs, shown in Figure 7.17, indicate clearly that the turbulencestrongly affects any navigation task including bypassing it.


90

Start

Goal

Navigation results from node 12 to node 6 (15 trials): Average=79.8, min.=59, max.=137, std.=19.38

tests

0

20

40

60

80

100

120

140

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Figure 7.16 Navigation task from node 12 Figure 7.17 Results from 15 separate test runs.to node 6. Standard deviation is large due to turbulence.

In some cases a robot can get stuck for several minutes or even moveagainst the current in the pipe. In full a scale operation, presented in Chapter8, a special behavior was introduced to recover from these kinds of situations.An inherent feature of a robot society provides a solution for this problem. Ifthere is a robot in turbulence, another robot will collide with it and push itonwards. Sometimes this helping robot will get stuck in the turbulence but thefollowing robot will assist it, and so on.

7.6 Summary

In this chapter an adaptive mapping algorithm for a closed aquaticenvironment is presented. Individual robot’s maps are combined in a singlecommon representation. This Common Basic Map is then used as thefoundation for a highly robust navigation algorithm. Finally, the operation ofboth of these algorithms is verified through navigation tests with a robot. Areliable operation is needed when the actual task, i.e. the neutralizing ofdistributed targets, is performed in Chapter 8.

Chapter 8. Distributed Operations in Closed Aquatic Environment

91

Chapter 8 Distributed Operations in Closed Aquatic Environment8.1 Task Domain

The task studied in the demo process is an exploration and exploitation task:searching and destroying distributed targets in the closed processenvironment. The distributed targets are supposed to be microbial algaegrowth spots inside the water system. Each target has dynamic behavior; if analgae growth is not completely destroyed in a certain location, it continues togrow. This leads to an interesting problem: what is the optimal strategy tocontrol the society in this kind of task? Should the society first try to locate allthe algae growths before starting to remove them, or should it operateimmediately as soon as the first growth has been detected? Both theenvironmental (only active vertical movement and strong currents withturbulence) and the hardware constraints (energy and chemical resources)have an effect on this decision.

The cleaning task considered deserves some comment from the practicalpoint of view. Algae growth occurs usually in a location where water standsstill. A standard solution to the problem is to insert as much cleaning agent asis needed to guarantee a minimum concentration over the whole volume ofliquid. Needless to say, this approach is ineffective. The SUBMAR societyprovides an alternative solution where multiple small robot cooperate andtransport the minimum amount of chemical needed straight to the areaswhere algae growths have been detected (Figure 8.1). This kind of policy canconserve a large amount of chemical and can also be very effective providedthat the algae is not too widely distributed.


92

Figure 8.1 A single society member performing the algae removal behavior. The upper leftpicture shows how the robot has detected an algal growth and lands on the bottom. In the nextfigure it starts to output a chemical in order to recruit other members. Below left, the memberoutputs the chemical at maximum speed, and in the last picture the robot has already left thetank and the “poison” slowly disperses.

8.1.1 Emulated Biomass GrowthBecause working with real algae in laboratory conditions is impractical, aspecial LED panel system was developed for the demo process. The systemcan be used to describe the growth of the biomass, for example bacteria, oralgae, which tends to occur in closed water systems. In this system thegrowth spots are presented by infrared LED and phototransistor matrixpanels, illustrated in Figure 8.2.

EMULATEDALGAE AGGLOMERATION

IR LED & IR phototransistormatrixUSER

INTERFACE

PROCESSCONTROLSTATION

I/O -CARD

IR LED actuators

IR photo-transistorsensors

ALGAE MODEL

GROWTHCOMPUTATIONIN REMOTE PC

7

Figure 8.2 Emulated algae system.


93

Light emitted from LEDs represent oxygen or some other gas produced by themicrobial growth, which in reality could be detected by robots with anappropriate sensor. The emulated organism reacts to the presence of anycleaning agent through its phototransistors. Robots have infraredphototransistors as dissolved gas sensors; while lighting their LEDs, robotsemulate the spreading of the cleaning agent for algae removal. There arethree (only two were used in these tests) independent growth agglomerationsinstalled at different locations in the demo process. This enables the analysisand evaluation of parallel multi-tasking in spatially distributed task execution.Each algae growth spot consists of four phototransistor panels. Theelectronics driving and sampling the actual emulated spots are controlled viaan I/O-card. Transmission of infrared light can be analog or frequencymodulated. Analog is more realistic, since the signal level is dependent of themeasuring distance, which is a familiar real-life situation.

Calculation of the current level of biomass and its growth rate at each spot iscarried out by a remote PC. The behavior of the biomass is modeled with ageneralized growth curve typical to most biological growth processes, see, forexample, (Stanier et al. 1990). The status of an algae growth, A, (i.e. thebiomass) is based on a formula which indicates how the derivative of thealgae is related to the growth and natural death of the cells, as follows:

dAdt

D A= −( )*µ (Eq. 8.1)

where µ is the growth rate and D is the death rate of the organism. The valueof µ depends on the limiting substrate (e.g. nitrogen). The death ratebecomes meaningful when the age of the cells increases or when poisonoussubstrates (i.e. a cleaning agent) are released into the environment. Theactual equation used in our model is discretized from the Equation 8.1:

A t A t e D t( ) ( )* ( )+ = −1 µ ∆ (Eq. 8.2)

If there is a cleaning agent release the value of D is related to theconcentration of the cleaning agent (Poison). This value can be detectedthrough the four inputs from the phototransistor panels. Poison.max is amaximum concentration of the cleaning agent in the liquid (i.e. the maximumvalue of the phototransistor panel ≈ 4.7 Volts). If Poison reaches Poison.maxvalue, it indicates that the release of the agent had the maximum effect. Thevalue of D is thus a function of Poison and Poison.max.

With a single robot attack, the duration of this action is rather short. It equalsthe time that it takes for a robot to release the contents of its cleaning agenttank and for the agent to dissolve into the aquatic environment. If severalrobots release their chemical agents approximately at the same time, thevalue of Poison is naturally high but the duration of the attack is short. On theother hand, if robots release their cleaning agents one after another, the valuePoison is smaller, but the duration of the attack is respectively longer. The keyquestion is to find a balance between the duration and the amplitude of thecleaning action.


94

The generalized growth curve of a bacterial culture consists of four separatephases. These include the lag phase, exponential phase, stationary phaseand death phase. These phases are shown in Figure 8.3, where the biomass(A) value, produced by the model, is plotted. During the exponential phase,there is an attack made by a single robot. As a result the value of biomassdrops for a while, but it starts to grow immediately after the poison isdissolved.

Sensor sensitivity

Environment restrictions

Lag

phas

e

Expo

nent

ial p

hase

Stat

iona

ry p

hase

Dea

th p

hase

Effe

ct o

fpo

ison

ing

Biom

ass

Time [sec]

Figure 8.3 Growth curve of the emulated biomass. It illustrates the four characteristic phases.An attack is visible around 3800 seconds.

The behavior of the emulated biomass is illustrated in Figure 8.4. Each ofthese small diagrams represents how the biomass reacts to various lengths ofcleaning agent (i.e. poison) releases. It also considers how well the cleaningagent reaches the biomass (i.e. how well the phototransistor panels detectthe incoming infrared light).

0 500 1000 1500 20001

1.5

2

2.5

3

3.5

4

4.5

5

0 500 1000 1500 20002.5

3

3.5

4

4.5

5

One attack2/4 panels detect max. poison

biom

ass

time [seconds / 2] time [seconds / 2]

biom

ass

One attack1/4 panels detect max. poison

0 500 1000 1500 20000

1

2

3

4

5

0 500 1000 1500 20001

1.5

2

2.5

3

3.5

4

4.5

5

Two consecutive attacks2/4 panels detect max. poison

biom

ass

time [seconds / 2]

biom

ass

time [seconds / 2]

Two consecutive attacks1/4 panels detects max. poison

0 500 1000 1500 20000

1

2

3

4

5

0 500 1000 1500 20000

1

2

3

4

5

Three consecutive attacks2/4 panels detect max. poison

Three consecutive attacks1/4 panels detects max. poison

biom

ass

biom

ass

time [seconds / 2]time [seconds / 2] 0 500 1000 1500 20000

1

2

3

4

5

0 500 1000 1500 20000

1

2

3

4

5

time [seconds / 2] time [seconds / 2]

biom

ass

biom

ass

Four consecutive attacks2/4 panels detect max. poison

Four consecutive attacks1/4 panels detects max. poison

Figure 8.4 The behavior of the emulated biomass for 1, 2, 3 and 4 consecutive attacks.Panels are adjusted so that an individual robot can, in ideal conditions (i.e. very close and inright orientation) accomplish the effects presented. Thus, by performing three consecutiveattacks with two panels seeing the maximal light, a single robot can neutralize a maximal sizebiomass growth.


95

8.1.2 Sensor Problems and SolutionsAs is usually the case with a real robot system, the sensory system is far frombeing ideal and the mobility of the robot is hardly ever as good as it shouldbe. The conditions of the working environment change constantly and onlythe most sophisticated calibration procedures can keep the robot optimizedfor the current conditions. These built-in sensory deficiencies must becompensated by developing extra features into algorithms. Our applicationcase is a good example of this kind of case.

Phototransistors are used to detect the infra-red light emitted by the algaegrowths. To simplify the design of the emulated algae, both parts of thesystem, i.e. the robot and the algae, are made with the same components.Unfortunately this issued some constraints that had to be taken into account.For example, we couldn’t keep the algae panels on all the time, becauseduring that time the panel’s photo-transistors are useless, because theydetect the light of the panels. This forced us to introduce a ten-second cycle,where the first five seconds are reserved for the IR LEDs (i.e. algae)activation and the other five seconds for detection of IR-light (i.e. poisonrelease) emitted by the robots. When a robot arrives at a possible algaelocation, it will measure the level of IR-light for a duration of ten seconds at afrequency of 1 Hz. During that time it should be able to detect the algaegrowth (i.e. five measurements at a high level and five at a low level). If therobot detects a constant high level, then it has to decide whether it is causedby an another robot performing the poisoning task (i.e. emitting the IR-light) orby the energy recharging station. There is only one recharging location and itis well known to all robots after a short period of operation. Thus, when arobot detects nothing but high values in a known algae location, it knows thatthere is at least one other robot actively performing a poisoning. When thishappens, the robot will keep on measuring algae until the other robot(s)stop(s) poisoning. Judging from this period, the robot can deduce without anyactive communication that others are working on the same algae location,and use this information for its own decision making.

In our demo process there are three types of algae locations, as shown inFigure 8.5. The differentiation of the type is based on the movement of therobot when it detects the algae growth. For each type the robot has a specificoperational procedure (A, B, C). The most difficult case is the one where analgal growth is situated where there is a strong current (e.g. in pipes). Thecurrent makes the robot pass the algae within a few seconds. If the passingoccurs when the light (algae) is off, the robot usually can not detect thegrowth. Nevertheless, if a robot detects this type of algae growth it attempts toverify it by trying to detect it again on NA consecutive rounds (procedure A).The value of the growth is then the largest value detected.

Another type of growth occurs where there is some sort of circular of turbulentcurrent. This is the case for example on the surface on an open tank. Therethe robot circulates on the surface close to the process walls and NB valuesare taken (procedure B). The closer to the wall the robot gets, the strongerthe sensor value. The orientation of the robot has of course some effect on


96

the sensory reading. The robot is able to make a decent estimate of theamount of algae. The largest value is again selected.

The third case is due to poor mobility in X-Y directions (Z equals depth).When a robot comes to a location where there is a zero current (e.g. on theceiling of a tank) its sensor value is more or less constant and depends totallyon the location to which it arrived. With no X-Y maneuverability, the only waythe robot can get a better idea of what is happening is by making additionaldives (procedure C). These dives provide some movement in the X and Ydirections. We used two extra dives and received thus three separatemeasurement series (NC measurements each). Once again the largest valuewas selected.

Algae location type: no current

Algae location type: strong current

Algae location type:circular current

procedure C

procedure B

procedure A

Figure 8.5 Three different types of algae growth locations. For each type there is a particularmeasurement procedure. See text for detail.

8.2 Control Architecture

In Chapter 5 a generic three layer control architecture for distributedautonomous robotic systems was presented. In this chapter the model isapplied to the task domain described above. The model is illustrated in Figure8.6. The main parts of this model are explained below in detail.


97

R2

R3

R5R1

BUFF

ER

PERCEPTION ACTION

ENVIRONMENT

INTERNAL RESOURCES

BEHAVIORAL LAYER

TASK LAYER

COOPERATIVE LAYER

exploit

explore

ARBITRATION

e x p lo r e

e x p lo it

BUFF

ER

communication out

group_1

group_2

com

mun

icat

ion

in

from operator

from other robots

to o

pera

tor &

oth

er ro

bots

R 1R 2

R 5

ST1 ST2explore

ST3

R 4

R 3

ST4R 1R 2

R 5

ST1 ST2exploit

ST3

R 4

R 3

R4R4

group_4

group_3

attackclosestalgae

1

attacklargestalgae

2attack

smallestalgae

3

makemap

1

use map

3

fix map

2

check map

4

target detection sensor

energy

"poison" poison release

diving tanks

pressure sensor

recover

1

load

1

tasks

2

notepad3

end

3

Figure 8.6 The developed control architecture for distributed operations in a closed aquaticenvironment.

8.2.1 Behavioral LayerThe robot reads the pressure and target detection (i.e. phototransistors)sensors once per second. With the same frequency it also checks the statusof its internal resources (i.e. energy and cleaning agent) and controls the


98

actuators (i.e. diving tanks and poison release). Receiving and sending amessage is naturally an event based function performed only when needed.

In the case of the search and destroy mission considered here, the number ofstates included in the design of the behavioral layer FSA is five, i.e. Q ={recover, tasks, load, notepad, end}, as is shown in Figure 8.7. The statesload and recover represent low level behaviors with highest priority (i.e. self-sufficiency behaviors). The actual task achieving behavior is the tasks state.The recover state is the initial state, q0, that the robot enters, when the poweris turned on. It is also a kind of emergency state, active when the robotdetects that something is wrong with its mobility (e.g. its location has notchanged even though it should have). In this state the robot starts to use itsactuators extensively. The robot changes its status to notepad (miscellaneousbehavior), if recover has failed to make it mobile again. In this state the robotno longer moves actively, and its only useful feature is to operate as a kind ofbeacon or message mediator until its energy resources are used (i.e.abnormal termination). The robot enters load, when it detects that either itsenergy or poison level has reached some threshold value. In the end state(miscellaneous behavior), the robot has completed the mission, the missiontime is finished or the operator has given the command to abort the mission(i.e. normal termination). Then the robot navigates to a defined location(home) waiting to be removed from the demo process. For pseudo codepresentation of the behaviors see Appendix A. A general description of thesoftware structure is illustrated in Figure 8.8.

recover

1

load

1

tasks

2

notepad3

end

3

Figure 8.7 The behavioral layer FSA. The states recover and load belong to the self-sufficiency states. The tasks state is the actual task achieving state, and the notepad and endstates form the miscellaneous state group.


99

Going with the flow

Semi-active movement

Landmark detection

Map learning

Goal-Directed Navigation

Landmarks

Map fixed

recover

load

exploit

notepad

end

DIVING TANKS

POISON TANKS

S S

S

S S SSEN

SOR

S AN

D C

OM

MU

NIC

ATIO

N

Figure 8.8 General description of the software structure. Higher priority behaviors cansuppress lower level behaviors. For pseudocode presentation, see Appendix A.

8.2.2 Task LayerIn this task domain there are two tasks to be represented on this layer,namely exploit and explore. In explore the robot moves around the processtrying to create the basic map. This map is then combined with the otherrobots’ basic maps. The result is the Common Basic Map (CBM). The robotthen uses this map and checks its quality from time to time. The FSA for theexplore task is shown in its simplified version in Figure 8.9. In these tests,however, the explore task is fixed to the state use map. This can be seenfrom Figure 8.6, where at the highest cooperative layer every robot is fixed tothat particular strategy (i.e. ST3). In this state the robot uses CBM and thefollow.map algorithm (Chapter 7.4), in order to be able to navigate. The fixingwas done in order to achieve as identical conditions as possible for the maintarget of these tests, i.e. exploit.

makemap

1

use map

3

fix map

2

check map

4

attackclosestalgae

1

attacklargestalgae

2attack

smallestalgae

3

Figure 8.9 FSA for the explore task. Figure 8.10 FSA for the exploit task.

In the exploit task the robot performs the poison release at a specific targetlocation. While operating in the explore task (especially when creating aCommon Basic Map) a robot can detect an arbitrary number of growth spots.Each spot is registered with an information field containing the location (in the


100

Common Basic Map) and the volume. Based on this information the robotthen selects its next strategy. Naturally, there exist several possibleoperational strategies, but here only three plausible strategies wereconsidered: attacking the closest algal growth(S1), attacking the largestdetected algal growth(S2), attacking the smallest detected algal growth(S3).The FSA for exploit task is shown above in Figure 8.10. In the exploit task therobot monitors the chosen algae’s status by storing the largest sensory valueof the target algal growth (procedure A or B or C, see Chapter 8.1.2), i.e.exploit = algaenode(i). The location of the growth is always connected (directlyto some node or temporally related to two consecutive nodes) to the CommonBasic Map. The exploit performance evaluation functions is as follows:

∆exploitrobot(t) = exploitrobot(t-1) - exploitrobot(t). (Eq. 8.3)

In other words, the robot follows how a selected algal growth behaves duringthe performance evaluation cycle. At the beginning of the mission, the robottests all available strategies in a row. It receives a ∆exploit value for each ofthe strategies. After this the robot chooses the most profitable one as its nextstrategy. Each strategy’s ∆exploit value is updated after testing. Furthermore,if the chosen algae growth is already deceased, the robot heads towardssome other known algae location instead, as is illustrated in Figure 8.11.

test all strategies select best strategy monitor mission ends

When the robot detects that one of the algal spots is eliminated(i.e. below sensor detection), the robot changes its target to another known algal growth. If there are two growth spots and one dies first, therobot navigates to the other. If itis also dead, it will navigate back to the first, and so on. This feature makes sure that if a growth isn't completely dead, the robot will detect it as soon as it has reached a certain volume.

At the beginning, every robot tests each strategy in a row.

After initial testing, the robot selects the best performed strategy asthe next strategy. If inter-robot communication isactive, the robot compares its own results to the results of the groups, each representing a set ofrobots performing the samestrategy (for a particular task).

When the operator has ordered the robot to abortthe mission, or the mission time is full, the robot navigates back to the home base.

Figure 8.11 The “watchdog” behavior ensures that the algae growths eventually die.

8.2.3 Cooperative LayerEven though there is no direct communication between the robots in thesociety, the members can still receive information about the others throughthe environment. For example, if a robot detects that the volume of an algaegrowth is decreasing without its own active operation upon it, it concludes thatthere must be other robots performing the removal task on the same growth(or alternatively there is some problems with its sensor readings).Nevertheless, when an active communication, no matter how simple, isallowed an individual robot can use task related information received from theothers. The communication used in our system is critical to the distancebetween sender and receiver, and the locations of the robots in the process.The robot either receives the whole message or otherwise it is omitted due tothe protocol. An inherent property of the society concept is incompletecommunication between members. The robot stores the received messagesin a dynamic task table with a time label and then calculates relative successvalues for each group, formed by robots performing an individual task’sparticular strategy, as was shown in Chapter 5.


101

In these tests, the message between members contains only two types ofinformation: strategy information and corresponding ∆exploit value. Thusthere can be as many groups as there are available strategies, and thenumber of robots (Ni) in a group can vary from one to the total number ofrobots in operation. For each of these groups the robot is aware, an averagevalue of performance is calculated with Equation 5.4. And again, the strategywith the highest ∆exploitgroup(i) value is considered to represent society’sstrategy. Next the individual robot compares this best group’s (i.e. strategy’s)time related average performance to its own time related largest value. If thegroup’s value is bigger then the next strategy chosen is the group’s.Otherwise, the robot chooses the next strategy based on its own beststrategy. The whole procedure was explained in detail in Chapter 5.

8.3 Tests and Results

There were actually two main research goals. The first was to study how thesize of the society affects the performance, i.e. to find out what kind ofbalance there would be between the increase in labor vs. increased resourceconflicts caused mainly by the lack of space. The second main research goalwas to find out how much the cooperation obtained through the describedcontrol architecture could help the performance of an individual and thus thewhole society. Both of these results were obtained from two types of tests,with or without inter-robot communication. A society of 3 and 5 robots weretested and then compared to the reference case, where only a single robotwas operating. In each test the development of both the two algae growthspots and successful cleaning agent (poison) releases were recorded, as isshown in Figure 8.12.

3 rob o ts (C ), 1 /010 7 , g rowth 1

02468

10121416

1

106

211

316

421

526

631

736

841

946

1051

1156

1261

1366

time / 2 [s]

volts poison

biomass

3 ro b o ts (C ), 1 /01 0 7 , g row th 2

02468

1012

1

106

211

316

421

526

631

736

841

946

1051

1156

1261

1366

time / 2 [s]

volts poison

biomass

Figure 8.12 Biomass volume (solid line) in one particular run. Poison releases, i.e., attacks,are shown with dashed line. Left figure indicates, how the circular currents make poisonattacks very non-uniform in growth location 1. Figure on the right, on the other hand, presentsmore stable poison attacks in growth location 2. This is due to almost zero current conditions.

The test value for the success is the total volume of living biomass, i.e. thesum of biomass volumes in growth spots 1 and 2. The performance forseparate cases, i.e. communication and no communication, was calculated bytaking an average of the total biomass of the test runs.


102

Due to the fact that there is no actual energy recharging station in the realsystem, the energy consumption of the robot was based on an energyconsumption function. When the robot just goes with the flow measuring butnot actively driving motors or making the poison attack, it consumes 1 unit /second. If it drives motors or performs an attack the consumption doubles.The initial energy resource for the robot was set at 3600 units. This meansthat it would be sufficient for the whole mission, if the robot did nothing otherthan measuring. This is not the case, so in practice the robot has to refill itsenergy resources, as is presented in Chapter 6.3.5. The speed of refilling is700 units per second when the robot is at the energy station node and detectsthe incoming energy (i.e. IR-light). It starts to navigate towards the energystation when the energy level drops below 540, immediately after the attackand measurement cycle ends. If the level for some reason drops below thelower limit of 360, the robot stops everything and starts to navigate towardsthe energy node. The use of a cleaning agent (i.e. poison) is based on aneven simpler function. The initial volume is 240, which equals four full (60second) attacks. When these attacks have been completed, the robot headstowards the recharging station. The speed of refilling for a cleaning agent is50 units / seconds.

The mission starts by giving each robot an ID number. After that a startcommand is issued to the whole system, as shown in Figure 8.13. Appendix Bshows a log file containing out-going messages of a single robot during amission. At the beginning of a test both of the algae growths start from thevalue 4.5 V. After a while, they reach the maximum volume, represented byvoltage 4.7 V. Thus, when the robots start their active performance, thevolume of the algae in the process is as large as it gets. The maximummission time was set at 60 minutes after which the robots navigate to theupper part of the largest tank from where they could be easily removed fromthe process.

Give me ID !

Give me ID !

ID: 1

ID: 1

Give me ID !

Give me ID !

ID: N

ID: N

START

START

START

time

Mission started

Mission started

Mission started

OPE

RATO

R

RO

BOT

1RO

BO

T N

Mission started

Figure 8.13 Interaction diagram for the mission startup.

8.3.1 Reference Case -a Single RobotIn order to find out how much effect an individual robot can have, single robottests were performed(see Figure 8.14).


103

1 robot, 5 runs, average (bold)

0

2

4

6

8

10

1 96 191

286

381

476

571

666

761

856

951

1046

1141

1236

1331

1426

1521

1616

time / 2 [s]

tota

l bio

mas

s [V

]

Figure 8.14 The results from five separate runs demonstrate clearly that a single robot ishelpless against full grown algae growths. More precise results from this single robot case canbe found in Appendix C.

These tests also served as testing for a developed communication protocol,because there were no interference caused by multiple robots. In Figure 8.15an interaction (one-way) diagram for navigation is shown. Robot sentmessages indicate how the navigation proceeds. The operator can thus followthe journey of the robot.

time

OPE

RATO

RR

OBO

T X

GoalNode 000Y

Ntrl=000Y000Z

Ntyp=00080003

Motors on

Old 000W

On trail

Old 000Z

Motors on

Old 000YOld 000C

GoalNode 000YNtrl=000Y000Z

Ntyp=00080003Motors onOld 000W

On trailOld 000Z

Motors onOld 000COld 000Y

Figure. 8.15 Interaction diagram for route planning and navigation.

8.3.2 Group of Three RobotsNext the test was performed with three robots not having communicationabilities (Figure 8.16). Three robots were able to reduce the volume of thealgae, but not eliminate it completely during the mission time. The curves,


104

shown in Figure 8.17, illustrate clearly how, at the beginning, the robotsperform poison attacks approximately at the same time (i.e., the curves havea very similar form). Later on the randomness of the process reduces thiseffect and the curves disperse.

Figure 8.16 Group of three robots performing algae attack on growth location 1, which issituated on node 6. See Figure 7.6 for picture of the Common Basic Map.

3 robots (NC), 10 runs, average (bold)

0

2

4

6

8

10

1 95 189

283

377

471

565

659

753

847

941

1035

1129

1223

1317

1411

1505

1599

time / 2 [s]

tota

l bio

mas

s [V

]

Figure 8.17 The average total biomass curves (10 runs) for a group of three robots with nocommunication. Three times out of 10 the group was able to eliminate the algae.

Communication was then included in the system. In this basic type ofcommunication, labeled C-type, robots transfer their success value for algaeremoval. This value is obtained by subtracting the measurement taken after


105

the attack from the measurement taken before attack. These results areillustrated in Figure 8.18.

3 robots (C), 10 runs, average (bold)

0

2

4

6

8

101 96 191

286

381

476

571

666

761

856

951

1046

1141

1236

1331

1426

1521

1616

time / 2 [s]

tota

l bio

mas

s [V

]

Figure 8.18 The total biomass curves (10 runs) for group of three robots with C-typecommunication. Five times out of 10 the group was able to perform the given task.

To demonstrate the difference between communication and nocommunication, the average curves were plotted in Figure 8.19.

3 robots, 10 runs

time / 2 [s]

tota

l bio

mas

s [V

]

0123456789

10

1

123

245

367

489

611

733

855

977

1099

1221

1343

1465

1587

3nc

3rc

Figure 8.19 The average curves for a group of three robots with and without C-typecommunication.

Figure 8.19 illustrates that there is no significant improvement in theperformance of the group when C-type communication is allowed. The reasonfor this is obvious. There are only three robots with a very limitedcommunication structure, i.e. local communication and no repeated sending.This effectively eliminates any real improvements. Moreover, the bad


106

orientation of algae detection sensors (phototransistors) causes someadditional problems especially in C-type communication, where only an algaeattack result is transmitted. Thus there are two possibilities for erroneousmeasurements, i.e. the algae value before and after attack. More detailedresults can be found in Appendixes D and E.

8.3.3 Group of Five RobotsIn order to find out when the benefit from the mutual communication wouldkick in, two extra robots were added to the system. The idea was not just toincrease the shear power of the poisoning task, but also to provide morecommunication traffic and thus also some improvement for the use ofavailable distributed information. First, the reference case (i.e. nocommunication) was performed. These ten runs along with the average areshown in Figure 8.20, details for which are available in Appendix F.

5 robots (NC), 10 runs, average (bold)

0

2

4

6

8

10

1 78 155

232

309

386

463

540

617

694

771

848

925

1002

1079

1156

1233

1310

time / 2 [s]

tota

l bio

mas

s [V

]

Figure 8.20 No communication. Five robots were able to kill the algae in every instance.

Next, the group was given the ability of C-type communication. These resultsare shown in Figure 8.21 and detailed information is available in Appendix G.

5 robots (C), 10 runs, average (bold)

0

2

4

6

8

10

1 70 139

208

277

346

415

484

553

622

691

760

829

898

967

1036

1105

1174

time / 2 [s]

tota

l bio

mas

s [V

]

Figure 8.21 Communication active (C-type). Algae was again killed in every case.


107

Next, new type of communication was introduced. This CC -typecommunication contains a slightly greater number of messages (Figure 8.22).Every time a robot receives a measurement value for an algae growth, ittransmits it to the others.

time

OPE

RA

TOR

RO

BO

T X

RO

BO

T Y

Old 000XProcedure 1 0937

Neutralize targetCease fireAttack OKProcedure 1 0912AA000X091200020025

GoalNode 000Z

AA000X091200020025

IRC message AA received

IRC message AA received

Procedure 1 0937

Procedure 1 0912

GoalNode 000Z

Old 000XProcedure 1 0937Neutralize target

Cease fireAttack OK

Procedure 1 0912AA000X091200020025

IRC message AA receivedGoalNode 000ZGoalNode 000Z

Figure 8.22 Interaction diagram for CC-type communication. Robot X starts to attack an algaegrowth located at node 000X. It informs the other robots, in this diagram robot Y, of thevolume of the algae before and after the poison attack. It also sends the normal successmessage, as in this case AA000X091200020025. After receiving that message robot Y sendsan acknowledgement message back to robot X.

These received values are stored in the tables similar to the ones presentedin Chapter 5.5 (labeled with a forgetting factor). Robot uses these values inorder to get an estimate for a particular algae growth. This kind of behaviorreduces the change of robot having a false value for the growth even thoughits measurement procedure provides a totally erroneous result. The resultsare shown in Figure 8.23 and details are available in Appendix H.

Figure 8.23 Communication active (CC-type). Each test was successful.

5 robots (CC), 10 runs, average (bold)

0

2

4

6

8

10

1 61 121

181

241

301

361

421

481

541

601

661

721

781

841

901

961

1021

time / 2 [s]

tota

l bio

mas

s [V

]


108

The averages in all three cases are plotted in Figure 8.24. Figure 8.25illustrates the mission completion times sorted in an increasing order.

time / 2 [s]

tota

l bio

mas

s [V

]

0123456789

10

1

116

231

346

461

576

691

806

921

1036

1151

1266

5 robots (NC)

5 robots (C)

5 robots (CC)

Figure 8.24 Averages of the three separate cases.

Mission completion

tests

time

/ 2 [s

]

0

200

400

600

800

1000

1200

1400

1 2 3 4 5 6 7 8 9 10

5 robots (NC)

5 robots (C)

5 robots (CC)

Figure 8.25 Duration of missions for the three separate cases.

One of the main reasons for the relatively small improvement (Figure 8.24),are the sensors of the robot. Besides the problems with algae detectionalready mentioned, the robots had a problem in releasing the cleaning agent.Even though they were attacking the right place, the poor orientation of therobots allowed them to waste their cleaning agent. Due to the structure of thedemo process (i.e. two algae growths in optimal locations) the recharging


109

station was forced to be set at a non-ideal location at the bottom of the largesttank (node 2). This location is problematic due to the strong current from thetank to the horizontal pipe. The robots often fail to refill their resources. Thiscauses long delays during which the robots are out of productive work, i.e.,the exploit task suffers. This can have a very dramatic effect on the results. Inorder to avoid this distortion in the results, the time robots used to obtainenergy or poison (Trefill), was removed from the final results. This way theactual working times (Twork=Ttotal-Trefill) could be compared.

As stated earlier, there are three important factors that should be consideredwhen results are analyzed: mission completion time, number of dead robotsand the amount of used poison. With the current energy usage, the robotsusually survive the whole mission without the need to refill the energy. Thus ina normal case, i.e. no malfunction in robots, there are no dead robots. Thepoison on the other hand lasts for only four attacks and must usually berefilled. Depending on the individual run, in a five robots case, robot uses onaverage 4.8 poison attacks (without communication) or 5.4 poison attacks(with communication). It means that each robot has to refill once during amission. The actual parameters when judging the performance of the societyare the time used and the poison. The goal is naturally to eliminate the algaeas fast as possible and with as little poison as possible. To make theindividual runs comparable, each run is scaled according to the largestamount of poison used, i.e. W = poisonused / poisonmax and the final value forindividual test runs is FI = W*Twork. The results are shown below in Figures8.26, 8.27 and 8.28.

5 robots (NC)

tests

time

/ 2 [s

]

0

200

400

600

800

1000

1200

1400

1 2 3 4 5 6 7 8 9 10

Mission time (MT)

Working time (MT -refill)

Scaled (poison)working time

Figure 8.26 Five robots (no communication) with scaled mission times.


110

5 robots (C)

tests

time

/ 2 [s

]

0

200

400

600

800

1000

1200

1 2 3 4 5 6 7 8 9 10

Mission time (MT)


Scaled (poison)working time

Figure 8.27 Five robots (C-type communication) with scaled working times.

5 robots (CC)

tests

time

/ 2 [s

]

0100200300400500600700800900

1000

1 2 3 4 5 6 7 8 9 10

Mission time(MT)


Scaled(poison)working time

Figure 8.28 Five robots (CC-type communication) with scaled working times.

Above, scaled results are combined in Figure 8.29, where three cases areshown, sorted according to increasing order. The differences between thecases start to emerge. When no communication and CC-type communicationcases are tested with the T-test (equal variances based on the F-test), it canbe stated that the mean of mission completion times with CC-typecommunication is statistically smaller than the mean of the non-


111

communication case (α=0.05). In other words, the CC-type communicationconsiderably improves the performance of the society.

Scaled working time, 5 robots

tests

time

/ 2 [s

]

0100200300400500600700800900

1000

1 2 3 4 5 6 7 8 9 10

NC

C

CC

Figure 8.29 Combined scaled working times.

When comparing the results from tests with or without communication theactual time that the communication has an effect on the performance occursafter the initial strategies have been tested. The average time for testingthese strategies with 5 robots case 18 minutes with no communication, 17.4minutes with C-type communication and 18.6 minutes for tests with CC-typecommunication. The average mission completion time for 5 robots with C-typecommunication is 28.1 minutes and with CC-type communication 27.4minutes. Only 10.7 minutes (C-type) and 8.8 minutes (CC-type) on averageare left for communication to have an effect on the results.

8.4 Conclusions

Once again, it became obvious that the real world is the only place where newideas should be tested in distributed autonomous robotics. The importance ofthe volume was clearly demonstrated, as shown in Figure 8.30. There is noquestion that if we had added more members to the society, the results havebeen even better. To what limit, is the key question. At some point theinterference between the robots starts to play such a large role that it actuallydecreases the performance of the whole society. This remains to be testedwhen we can increase the number of robots in the society.


112

Averages: 1 robot(5 tests), 3 robots(10 tests), 5 robots(10 tests)

time / 2 [s]

tota

l bio

mas

s [V

]

0

1

2

3

4

5

6

7

8

9

101 78 155

232

309

386

463

540

617

694

771

848

925

1002

1079

1156

1233

1310

1387

1464

1541

1618

1695

1 robot

3 robots (NC)

3 robots (C)

5 robots (NC)

5 robots (C)

5 robots (CC)

Figure 8.30 The combined results clearly illustrate that size does matter. The differences inperformance between 1 and 3 robots and 3 and 5 robots are very clear.

The other main research question about the importance of communicationwas not as clearly answered. Unexpected sensor properties along with localcommunication and short operational times after initial strategies diminishedthe positive effect of cooperation caused by inter-robot communication. Innear the future the amount of communication will be increased throughresendings and by having robots repeat messages as links. This research willbe documented in another upcoming doctoral dissertation from our researchgroup.

Chapter 9. Summary and Conclusions

113

Chapter 9 Summary and Conclusions9.1 Main Results

This thesis describes research where the goal has been to create a generalframework for the designing of multi-robot systems for various real worldproblems. Nowadays, this kind of framework is no longer just a researchinterest and technical challenge but very much a practical demand. Astechnical readiness increases, the application domain widens rapidly. We arefacing new tasks, which should be justifiably, performed with robotic agents.Many of these tasks can be performed more efficiently with multiple robots.Some of them can only be solved with multi-robot systems. In this work theserobotic systems are referred to as societies based on the analogy to thestructures found in the natural environment. In the real world, the behavior ofa society is greatly influenced by the interactions between the members, andbetween the members and a dynamic and often even hostile environment.

Based on the above framework a generic control architecture for distributedautonomous robotic systems was developed. All functions of distributedautonomous robotic systems are obviously realized through their members.The members’ behaviors are results from their own needs and from theconstraints (dynamic by nature) set by the system, environment or operator.The model developed is a hierarchical three-layer model containingbehavioral, task and cooperative layers. The behavioral layer keeps the robotoperational, the task layer makes the robot work towards defined goals andthe cooperative layer acts as an interface between the individual and thesociety, giving it the possibility to gain from the society’s collectiveintelligence.

Chapter 9. Summary and Conclusions

114

The second main contribution of the thesis is based on the fact that thetesting of the above architecture was conducted on a novel distributedunderwater robotic system. It required much innovative algorithmicdevelopment. The robots were operating in a closed liquid environmentperforming a combined exploration and exploitation task. The nature of thesystem, along with the minimal perception system and poor maneuverability,forced us to develop novel mapping, navigation and path planning algorithms.

9.2 Future Work

As is usually the case with empirical research, the knowledge gained raisesmore questions than it answers. This was the case here as well. Even thoughmany parts of the system were tested initially in a powerful and realisticsimulator it was the real tests with physical robots that revealed the problemsrelated to multi-robot systems operating in complex environments. Thegreatest problems were once again caused by the robots’ sensory systems.Erroneous transmitted information caused some unexpected results andforced us to find ways to overcome this problem. This goal was partlyreached, but there are still many ways to improve the performance of thesystem through more effective inter-robot communication. This will be ourmain research topic in the future. Furthermore, new strategies for the exploittask will be tested. These include fixed “always attack the largest growth” and“always attack the smallest growth”. The former should guide the robots tochange their targets according to their estimation of its volume, whereas thelatter should direct the robots to kill the smallest growth first.

Concerning the actual physical system, we are trying to increase themaximum size of the society. Then we could really see when the mutualinterference would actually start to decrease the society’s performance. Weare also planning to increase the number of emulated algae growths from twoto three. This should increase the benefit of the inter-robot communication byreducing the probability of choosing the right target based on incorrectinformation.

To summarize, by further testing the developed generic control architecture ina real underwater robotic system, we are hoping to find valid structures to beused in tomorrow’s revolutionary robotic applications, wherever they might be.

References

115

ReferencesAlbus J, McCain H., Lumia R.: NASA/NBS Standard Reference Model forTelerobot Control System Architecture (NASREM), NBS Technical Note 1235,Robot Systems Division, National Bureau of Standards, 1987.

Allen J., Hendler J., Tate A.(Eds.): Readings in Planning, Morgan KaufmannPublishers, Inc., San Mateo, USA, 1990.

Anderson J. A.: An Introduction to Neural Networks, The MIT Press,Cambridge, Mass., 1995.

Appelqvist P., Halme A., Schönberg T, Vainio M., Wang Y.: Designing SimpleCooperative Sensor/Actuator Robots for Liquid Process Environments,IEEE/ASME International Conference on Advanced Intelligent Mechatronics,CD-ROM, Tokyo, Japan, 1997.

Appelqvist P, Vainio M., Halme A.: Mechatronical Design of UnderwaterSensor/Actuator Robots for Cooperative Task Execution, Mechatronics '98,The 6th UK Mechatronics Forum International Conference, Skövde, Sweden,pp. 249 - 254, 1998.

Arkin R. C.: Path Planning for a Vision-Based Autonomous Robot,Proceedings of the SPIE Conference on Mobile Robots, Cambridge, MA, pp.135-170, 1986.

Arkin R. C.: Motor Schema-Based Navigation for a Mobile Robot: AnApproach to Programming by Behavior, Proceedings of the IEEE InternationalConference on Robotics and Automation, Raleigh, NC, pp. 264-271, 1987.

Arkin R. C.: Motor Schema-Based Mobile Robot Navigation, InternationalJournal on Robotics Research, Vol. 8, No. 4, August, pp. 92-112, 1989.

Arkin R. C.: Integrating Behavioral, Perceptual, and World Knowledge inReactive Navigation, Designing Autonomous Agents, Pattie Maes (Ed.), TheMIT Press, Cambridge, Mass., pp. 105-122, 1990.

Arkin R. C.: Cooperation without Communication: Multi-agent Schema BasedRobot Navigation, Journal of Robotic Systems, Vol. 9, No. 2, pp. 351-364,1992.

Arkin R. C., MacKenzie D. C.: Temporal Coordination of PerceptualAlgorithms for Mobile Robot Navigation, IEEE Transactions on Robotics andAutomation, Vol. 10, No. 3, pp. 276 - 286, 1994.

Arkin R. C., Balch T. R.: AuRA: Principles and Practice in Review, Journal ofExperimental and Theoretical Artificial Intelligence, Vol. 9, No. 2, 1997.

References

116

Arkin R. C.: Behavior-Based Robotics, The MIT Press, Cambridge, Mass.,1998.

Asama H., Matsumoto A., Ishida Y.: Design of an Autonomous andDistributed Robot System: “ACTRESS", Proceedings of IEEE/RSJ Int.Workshop on Intelligent Robots and Systems, Tsukuba, Japan, pp. 283-290,1989.

Asama H., Habib M. K., Endo I., Ozaki K., Matsumoto A., Ishida Y.:Functional Distribution among Multiple Mobile Robots in An Autonomous andDecentralized Robot Systems, Proceedings 1991 IEEE InternationalConference on Robotics and Automation, Sacramento, CA, April, pp. 1921-1926, 1991a.

Asama H., Ozaki K., Itakura H., Matsumoto A., Ishida Y., Endo I.: CollisionAvoidance Among Multiple Robots Based on Rules and Communication,Proceedings IEEE/RSJ International Workshop on Intelligent Robots andSystems, Osaka, Japan, pp. 1215-1220, 1991b.

Asama H.: Trends of Autonomous Robotic Systems, Distributed AutonomousRobotic Systems, Asama H., Fukuda T., Arai T., Endo I., Asama, (Eds.),.Springer-Verlag, Tokyo, pp. 3-8, 1994.

Bakker P., Kuniyoshi Y.: Robot See, Robot Do: An Overview of RobotImitation, AISB’96 Workshop on Learning in Robots and Animals, pp. 3-11,1996.

Balch T., Arkin R. C.: Communication in Reactive Multiagent RoboticSystems, Autonomous Robots, Vol.1, No.1, pp. 27-52, 1994.

Balch T.: Behavioral Diversity in Learning Robot Teams, Ph.D. Thesis,College of Computing, Georgia Institute of Technology, 1998.

Balkennius C.: Natural Intelligence in Artificial Creatures, Ph.D. Thesis,University of Lund, Cognitive Studies 37, Sweden, 1995.

Beer R., Gallagher J.: Evolving Dynamic Neural Networks for AdaptiveBehavior, Adaptive Behavior, Vol. 1, No. 1, pp. 91 - 122, 1992.

Beckers R., Holland O. E., Deneubourg J. L.: From Local Actions to GlobalTasks: Stigmergy and Collective Robotics, Artificial Life IV, Brooks R. A. andMaes P. (Eds.), The MIT Press, Cambridge, Mass., pp. 181-189, 1995.

Beni G., Hackwood S.: The Maximum Entropy Principle and Sensing inSwarm Intelligence, Toward a Practice of Autonomous Systems (Proceedingsof the First European Conference on Artificial Life), Varela F. J. and BourgineP.(Eds.), The MIT Press, Cambridge, Mass., pp. 153 - 160, 1992a.

References

117

Beni G., Hackwood S.: Coherent Swarm Motion under Distributed Control,Proceedings of International Symposium on Distributed Autonomous RoboticSystems, Wako, Saitama, Japan, pp. 39 - 52, 1992b.

Bond A., Gasser L.: Readings in Distributed Artificial Intelligence, MorganKaufmann, 1988.

Braitenberg V.: Vehicles: Experiments in Synthetic Psychology, The MITPress, Cambridge, Mass., 1984.

Brooks R. A.: A Robust Layered Control System For a Mobile Robot, IEEEJournal of Robotics and Automation, Vol. RA-2, No.1, pp. 14-23, 1986.

Brooks R. A., Flynn A.: Fast, Cheap and Out of Control, Memo 1182,Massachusetts Institute of Technology Artificial Intelligence Lab. Cambridge,Massachusetts, 1989.

Brooks R. A.: Elephants Don’t Play Chess, Robotics and AutonomousSystems, 6, pp. 3-15, 1990a.

Brooks R. A., Maes P., Mataric M. J., More G.: Lunar Base ConstructionRobots, IEEE International Workshop on Intelligent Robots and Systems, pp.389 - 391, 1990b.

Brooks R. A.: Challenges for Complete Creature Architectures, From animalsto animats (Proceedings of the First International Conference on Simulation ofAdaptive Behavior), Meyer J.-A. and Wilson S. W. (Eds.), The MIT Press,Cambridge, Mass., pp. 434 - 443, 1991a.

Brooks R. A.: Intelligence without Representation, Artificial Intelligence, 47,pp. 139 - 160, 1991b.

Brooks R.A.: Intelligence without Reason, Memo 1293, MassachusettsInstitute of Technology Artificial Intelligence Lab, Cambridge, Massachusetts,1991c.

Brooks R. A.: Artificial Life and Real Robots, Toward a Practice ofAutonomous Systems, Varela F. J. and Bourgine P. (Eds.), The MIT Press,Cambridge, Mass., pp. 3 - 10, 1992.

Budrene E. O., Berg H. C.: Dynamics of Formation of Symmetrical Patternsby Chemotactic Bacteria, Nature, Vol. 376, Number 6, July, 1995.

Caloud P, Choi W, Latombe J-C, Le Pape C., Yim M.: Indoor AutomationWith Many Mobile Robots, Proceedings IEEE/RSJ International Workshop onIntelligent Robots and Systems, pp. 67-72, 1990.

Camazine S.: Collective Intelligence in Insect Colonies by Means of Self-Organization, Proceedings of European Conference on Artificial Life,Brussels, Belgium, pp. 158 - 173, 1993.

References

118

Capek K.: RUR, translated by P. Selyer, Washington Square Press, NewYork, 1973.

Cao Y. U., Fukunaga A. S., Kahng A. B.: Cooperative Mobile Robotics:Antecedents and Directions, Autonomous Robots, Vol. 4, No. 1, pp. 7-28,1997.

Cliff D., Harvey I., Husbands P.; Explorations in Evolutionary Robotics,Adaptive Behavior, Vol. 2, No. 1, pp. 71 -104, 1993.

Connell J. H.: SSS: A Hybrid Architecture Applied to Robot Navigation, IEEEInternational Conference on Robotics and Automation, Nice, France, pp.2719-2724, 1991.

Corbara B., Drogoul A., Fresneau, Lalande S.: Simulating the SociogenesisProcess in Ant Colonies with MANTA, Proceedings of European Conferenceon Artificial Life, Brussels, Belgium, pp. 224 - 235, 1993.

DARS92: Proceedings of International Symposium on DistributedAutonomous Robotic Systems, Saitama, Japan, Sep. 21-22, 1992.

DARS94: Distributed Autonomous Robotic Systems, Asama H., Fukuda T.,Arai T., Endo I., (Eds.), Springer-Verlag, Tokyo, 1994.

DARS96: Distributed Autonomous Robotic Systems 2, Asama H., Fukuda T.,Arai T., Endo I.,(Eds.), Springer-Verlag, Tokyo, 1996.

DARS98: Distributed Autonomous Robotic Systems 3, Lueth T., Dillmann,Dario P., Wörn H., (Eds.), Springer-Verlag, Berlin, 1998.

Dautenhahn K.: Getting to Know Each Other- Artificial Social Intelligence forAutonomous Robots, Robotics and Autonomous Systems, Vol. 16, Nos. 2-4,December, 1995.

Deneubourg J. -L., Goss S., Franks N., Sedova-Franks A., Detrain C.,Chretien L.: The Dynamics of Collective Sorting Robot-like Ants and Ant-likeRobots, From Animals to Animats (Proceedings of the First InternationalConference on Simulation of Adaptive Behavior), Paris, France, pp. 356-363,1990.

Deneubourg J.-L., Theraulaz G., Beckers R.: Swarm-Made Architectures,Toward a Practice of Autonomous Systems (Proceedings of the FirstEuropean Conference on Artificial Life), Varela F. J. and Bourgine P.(Eds.),The MIT Press, Cambridge, Mass., pp. 123-133, 1992.

Deneubourg J.-L.: Collective Intelligence in Natural and Artificial Systems,Distributed Autonomous Robotic Systems 2, Asama H., Fukuda T., Arai T.,Endo I.(Eds.), Tokyo, Springer-Verlag, pp. 12-13, 1996.

References

119

Dudek G., Jenkin M., Milios E., Wilkes D.: A Taxonomy for Swarm Robots,Proceedings of the IEEE/RSJ International Conference on Intelligent Robotsand Systems, Yokohama, Japan, pp. 441-447, 1993

Epstein J. M., Axtell R.L.: Growing Artificial Societies: Social Science from theBottom Up, The MIT Press, Cambridge, Massachusetts, 1996.

Everett H.R.: Sensors for Mobile Robots, A.K. Peters, Wellesley, MA, 1995.

Firby R. J., Prokopowicz P. N., Swain M. J.: The Animate Agent Architecture,Artificial Intelligence and Mobile Robots, Kortenkamp D., Bonasso R. P.,Murphy R.(Eds.),AAAI Press / The MIT Press, Menlo Park, CA, pp. 243-275,1998.

Floreano D., Mondada F.: Automatic Creation of an Autonomous Agent:Genetic Evolution of a Neural-Network Driven Robot, From Animals toAnimats 3 (Proceedings of the Third Intl. Conf. on Simulation of AdaptiveBehavior), Cliff D., Husbands P., Meyer J.-A., Wilson S. W. (Eds.), The MITPress, Cambridge, Massachusetts, pp. 421 - 430, 1994a.

Floreano D., Mondada F.: Active Perception, Navigation, Homing andGrasping: An Autonomous Perspective, Proceedings of the From Perceptionto Action Conference, Lausanne, Switzerland, pp. 122 - 133, 1994b.

Floreano D., Mondada F.: Evolution of Homing Navigation in a Real MobileRobot, IEEE Transactions on Systems, Man and Cybernetics-Part B, 26,1996a.

Floreano D. and Mondada F.: Evolution of Plastic Neurocontrollers forSituated Agents, From Animals to Animats 4,(Proceedings of the 4thInternational Conference on Simulation of Adaptive Behavior), Maes P.,Mataric M. J., Meyer J.-A., Pollack J. and Wilson S. W. (Eds.), The MITPress, Cambridge, Massachusetts, pp. 402 - 410, 1996b.

Fukuda T., Nakagawa S., Kawauchi Y, Buss M.: Structure Decision Methodfor Self Organizing Robots Based on Cell Structures-CEBOT, Proceedings ofthe 1989 IEEE International Conference on Robotics and Automation, pp.695-700, 1989.

Fukuda T., Buss M., Hosokai H., Kawauchi Y.: Cell Structured RoboticSystem CEBOT: Control, Planning and Communication Methods, Roboticsand Autonomous Systems 7, pp. 239-248, 1991.

Fukuda T., Sekiyama K., Ueyama T., Arai F.: Efficient CommunicationMethod in the Cellular Robotics Systems. IEEE/TSJ International Conferenceon Intelligent Robots and Systems, Yokohama, Japan, pp. 1091-1096, 1993.

Fukuda T., Ueyama T.: Cellular Robotics and Micro Systems, World Scientificin Robotics and Automated Systems -Vol.10, World Scientific, 1994.

References

120

Fukuda T., Sekiyama K., Communication Reduction with Risk Estimate forMultiple Robotic System, Proceedings of the IEEE International Conferenceon Robotics and Automation, pp. 2864-2869, 1994.

Fukuda T., Iritani G.: Intention Model and Coordination for CollectiveBehavior in Robotic System, Distributed Autonomous Robotic Systems,Asama H., Fukuda T., Arai T, Endo I. (Eds.), Springer-Verlag, pp. 255-266,1994.

Gat E.: Reliable Goal-directed Reactive Control for Real-world AutonomousMobile Robots, Ph.D. Thesis, Department of Computer Science, VirginiaPolytechnic Institute and State University, 1991.

Gat E.: Three-Layer Architectures, Artificial Intelligence and Mobile Robots,Kortenkamp D., Bonasso R. P., Murphy R.(Eds.), AAAI Press / The MITPress, Menlo Park, CA, pp. 195-210, 1998.

Gaussier P., Zrehen S.: A Constructivist Approach for Autonomous Agents,Artificial Life and Virtual Reality, Thalmann N. M. and Thalmann D. (Eds.),John Wiley and Sons Ltd, pp. 98 - 114, 1994.

Goldberg D. E.: Genetic Algorithms in Search, Optimization, and MachineLearning, Addison-Wesley, Massachusetts, 1989.

Goss S., Deneubourg J.L, Beckers R., Henrotte J.-L: Recipes for CollectiveMovement, Preprints from the European Conference on Artificial Life(ECAL'93), Brussels, Belgium, pp. 400 - 410, 1993.

Gutowitz H.: Complexity-Seeking Ants, Proceedings of European Conferenceon Artificial Life (ECAL’93), Brussels, Belgium, pp. 429 - 439, 1993.

Hackwood S., Beni G.: Self-organization of Sensors for Swarm Intelligence,Proceedings of the IEEE International Conference on Robotics andAutomation, Nice, France, pp. 819-829, 1992.

Halme A.: Robot Societies - Some Basic Considerations, MEL, Tsukuba,Japan, (unpublished memo), 1992.

Halme A., Jakubik P., Schönberg T., Vainio M.: The Concept of RobotSociety and Its Utilization, Proceedings of the IEEE International Workshopon Advanced Robotics, Tsukuba, Japan, pp. 29-35, 1993.

Halme A, Jakubik P, Schönberg T, Vainio M: The Concept of Robot Societyand Its Utilization in Future Robotics, Advanced Robotics & IntelligentMachines, Gray J. O. and Caldwell D. G. (Eds.), IEE, London, UK, pp. 255 -270, 1996a.

Halme A, Appelqvist P, Jakubik P., Kähkönen P., Schönberg T., Vainio M.:Bacterium Robot Society-A Biologically Inspired Multi-Agent Concept forInternal Monitoring and Controlling of Processes, IEEE/RSJ International

References

121

Conference on Intelligent Robots and Systems, Osaka, Japan, pp. 1707-1714, 1996b.

Halme A, Vainio M., Appelqvist P., Jakubik P., Schönberg T., Visala A.:Underwater Robot Society Doing Internal Inspection and Leak Monitoring ofWater Systems, Proceedings of SPIE Sensor Fusion and DecentralizedControl in Autonomous Robotic Systems (vol. 3209), Pittsburgh, USA, pp.190-199, 1997.

Harvey I.:Species Adaptation Genetic Algorithms: The Basis for a ContinuingSAGA, Toward a Practice of Autonomous Systems, Varela F. J., Bourgine P.(Eds.), The MIT Pres, Cambridge, Mass., pp. 346-354, 1992.

Harvey I, Husbands P., Cliff D.: Issues in Evolutionary Robotics, TechnicalReport CSRP 219, Univeristy of Sussex School of Cognitive and ComputingSciences, 1992.

Harvey I.: The Artificial Evolution of Adaptive Behavior, Ph.D. Thesis, TheUniversity of Sussex, 1993.

Harvey I., Cliff D., Husbands P.: Evolutionary Approaches to AutonomousAgents, Prints from Advanced Study Institute on Practice and Future ofAutonomous Agents, Monte Verita, Switzerland, 1995.

Hayes G., Demiris J.: A Robot Controller Using Learning by Imitation,Borkowski A., Crowley J. L.(Eds.), Proceedings of the 2nd InternationalSymposium on Intelligent Robotic Systems, Grenoble, France, July, pp. 198 -204, 1994.

Hayes-Roth B., Lalanda P., Morignot P., Pfleger K., Balabanovic P.: Plansand Behavior in Intelligent Agents, Technical Report KSL-93-43, KnowledgeSystems Laboratory, Stanford University, Stanford, CA, 1993.

Haykin S.: Neural Networks- A Comprehensive Foundation, MacmillanCollege Publishing Company Inc., New York, NY, 1994.

Helfman G. S., Collette B. B., Facey D. E.: The Diversity of Fishes, BlackwellScience Inc., Malden, Massachusetts, 1997.

Hölldobler B., Wilson E. O.: The Ants, Belknap Press, 1990.

Holland J. H.: Adaptation in Natural and Artificial Systems, The University ofMichigan Press, Ann Arbor, MI, 1975.

Jakobi, N., Husbands, P., Harvey, I.: Noise and the Reality Gap: The Use ofSimultation in Evolutionary Robotics, Advances in Artificial Life (Proceedingsof the 3rd European Conference on Artificial Life), Moran, F. Moreno, A.,Merelo, J. Chacon, P. (Eds.) Springer-Verlag Lectures notes in ArtificialIntelligence 929, pp. 704-720, 1995.

References

122

Jakubik P., Schönberg T., Vainio M., Halme A.: Robot Societies- A BasicStudy (Advancing report 1), Helsinki University of Technology, Finland,(unpublished memo), 1992.

Johnson P., Bay J.: Distributed Control of Simulated Autonomous MobileRobot Collectives in Payload Transportation, Autonomous Robots, Vol. 2, No.1, pp. 43-64, 1995.

Kaelbling L., Rosenschein S.: Action and Planning in Embedded Agents,Designing Autonomous Agents, Maes P. (Ed.), MIT Press, Cambridge, MA,pp. 35-48, 1991.

Kaelbling L.P.: Learning in Embedded Systems, The MIT Press, Cambridge,Mass, USA, 1993.

Kaelbling L. P., Littman M. L., Moore A. W.: Reinforcement Learning: ASurvey, Prints from Advanced Study Institute on Practice and Future ofAutonomous Agents, Monte Verita, Switzerland, 1995.

Kawauchi Y., Inaba M., Fukuda T.: A Principle of Distributed Decision Makingof Cellular Robotic System(CEBOT), Proceedings of 1993 IEEE InternationalConference on Robotics and Automation, pp. 833 - 838, 1993.

Kawauchi Y., Inaba M., Fukuda T.: Evolutional Self-organization ofDistributed Autonomous Systems, Distributed Autonomous Robotic Systems,Asama H., Fukuda T., Arai T, Endo I. (Eds.), Springer-Verlag, Tokyo, pp.243-254, 1994.

Keshet L. E.: Trail Following as an Adaptable Mechanism for PopulationBehaviour, Proceedings of European Conference on Artificial Life, Brussels,Belgium, pp. 326-344, 1993.

Klopf A. H.: The Hedonistic Neuron, A Theory of Memory, Learning, andIntelligence, Hemisphere, Washington, USA, 1982.

Koza J. R.: Genetic Programming: On the Programming of Computers byMeans of Natural Selection, The MIT Press, Cambridge, Mass., 1992a.

Koza J. R.: Evolution of Subsumption Using Genetic Programming, Toward aPractice of Autonomous Systems, Varela F. J. and Bourgine P. (eds.), TheMIT Pres, Cambridge, Mass., pp. 110-119, 1992b.

Koza J. R.: Genetic Programming II: Automatic Discovery of ReusablePrograms, MIT Press, The Cambridge, Mass., 1994.

Kube C. R.: Collective Robotic Intelligence: A Control Theory For RobotPopulations, Master's Thesis, University of Alberta, 1992.

Kube C. R.: Collective Robotics: From Local Perception to Global Action,Ph.D. Thesis, Computing Science, University of Alberta, 1997.

References

123

Kube C. R., Zhang H.: Collective Robotic Intelligence, From Animals toAnimats 2 (Proceedings of the Second International Conference onSimulation of Adaptive Behavior), Meyer J-A, Roitblat H. L., Wilson S. W(Eds.), The MIT Pres., Cambridge, Mass., pp. 460 - 468, 1992.

Kube C. R., Zhang H.: Collective Robotics: From Social Insects to Robots,Adaptive Behavior, Vol. 2, No. 2, pp. 189-219, 1993.

Kube C. R., Zhang H.: Stagnation Recovery Behaviors for CollectiveRobotics, Proceedings of the International Conference on Intelligent Robotsand Systems, pp. 1883-1890, 1994.

Kuniyoshi Y.: The Science of Imitation -Towards Physically and SociallyGrounded Intelligence, Technical Report RWC TR-94001, Real WorldComputing Project, Tsukuba-shi, Ibaraki-ken, Japan, 1994.

Lambrinos D., Scheier C., Pfeiffer R.: Unsupervised Classification of Sensory-Motor States in a Real World Artifact Using a Temporal Kohonen Map,Proceedings of the International Conference in Artificial Neural Networks,Paris, October, pp. 467 - 472, 1995.

Lyons D.: Planning, Reactive, Encyclopedia of Artificial Intelligence, S.Shapiro (Ed.), 2nd ed., John Wiley and Sons, New York, pp. 1171-1182,1992.

Lyons D., Hendriks A.: Planning for Reactive Robot Behavior, Proceedings ofthe IEEE International Conference on Robotics and Automation, Nice,France, pp. 2675-2680, 1992.

Luo R. C., Kay M. G.: Multisensor Integration and Fusion in IntelligentSystems, Autonomous Mobile Robots: Perception, Mapping, and Navigation,S.S.Iyengar, A.Elfes, (Eds.), IEEE Computer Society Press, pp. 201-231,1991.

MacKenzie D. C., Arkin R. C., Cameron J. M.: Multiagent MissionSpecification and Execution, Autonomous Robots, Vol. 4, No. 1, pp. 29-52,1997.

MacKenzie D. C.: A Design Methodology for the Specification of Behavior-Based Robotic Systems, Ph.D. Thesis, College of Computing, GeorgiaInstitute of Technology, Atlanta, 1996.

Maes P.: Situated Agents Can Have Goals, Robotics and AutonomousSystems, Vol.6, pp. 49-70, 1990.

Maes P.: Behavior-Based Artificial Intelligence, From Animals to Animats 2(Proceedings of the Second International Conference on Simulation ofAdaptive Behavior, Meyer J.-A., Roitblat H. L., Wilson S. W, (Eds.), The MITPress, Cambridge, Mass., pp. 2-10, 1992.

References

124

Maes P.: Modelling Adaptive Autonomous Agents, Artificial Life- An Overview,ed. Langton C., The MIT Press, Cambridge, Mass, pp. 135-162, 1995.

Martinoli A., Mondada F.: Collective and Cooperative Group Behaviours:Biologically Inspired Experiments in Robotics, Proceedings of the FourthInternational Symposium on Experimental Robotics ISER-95, Khatib O,Salisbury J. K. (Eds.), Stanford, US, June-July, Lecture Notes in Control andInformation Sciences, Springer-Verlag, pp. 3-10, 1995.

Mataric M. J.: A Distributed Model for Mobile Robot Environment-Learningand Navigation, MIT EECS Master's Thesis, Jan 1990, MIT AI Lab TechReport AITR-1228, May 1990.

Mataric M. J.: Navigating With a Rat Brain: A Neurobiologically-InspiredModel for Robot Spatial Representation, From Animals to Animats (FirstInternational Conference on Simulation of Adaptive Behavior), Meyer J-A,Wilson S.(Eds.), The MIT Press, Cambridge, Mass., pp. 169-175, 1991.

Mataric M.J.: Minimizing Complexity in Controlling a Mobile Robot Population,Proceedings of the 1992 IEEE International Conference on Robotics andAutomation, Nice, France, pp. 830-835, 1992a.

Mataric M. J.: Designing Emergent Behaviors: From Local Interactions toCollective Intelligence, From Animals to Animats 2 (Proceedings of theSecond International Conference on Simulation of Adaptive Behavior), MeyerJ.-A., Roitblat H.L., Wilson S. W (Eds.), MIT Press, Cambridge, Mass, pp.432-441, 1992b.

Mataric M. J, Marjanovic M. J: Synthesizing Complex Behaviours byComposing Simple Primitives, Preprints from the European Conference onArtificial Life, Brussels, Belgium, pp. 698-707, 1993a.

Mataric M. J.: Designing Emergent Behaviors: From Local Interactions toCollective Intelligence, From Animals to Animats 2 (Proceedings of theSecond International. Conference on Simulation of Adaptive Behavior),Meyer J.-A., Roitblat H.L., Wilson S. W (Eds.), The MIT Press, Cambridge,Mass., pp. 432-441, 1993.

Mataric M. J.: Interaction and Intelligent Behavior, MIT EECS PhD Thesis,1994a.

Mataric M. J.: Learning to Behave Socially, From Animals to Animats 3(Proceedings of the Third International Conference on Simulation of AdaptiveBehavior), Cliff D., Husbands P., Meyer J.-A., Wilson S. W (Eds.), MIT Press,Cambridge, Mass., pp. 453-462, 1994b.

Mataric M. J.: Issues and Approaches in the Design of CollectiveAutonomous Agents, Robotics and Autonomous Systems, Vol. 16, Nos. 2-4,December, pp. 321-331, 1995a.

References

125

Mataric M. J.: Designing and Understanding Adaptive Group Behavior,Adaptive Behavior Vol. 4, No.1, December, 1995b.

Mataric M. J., Cliff D.: Challenges in Evolving Controllers for Physical Robots,“Evolutionary Robotics”, special issue of Robotics and Autonomous Systems,Vol. 19, No. 1, October, 1996.

Mataric M. J.: Learning Social Behavior, Robotics and Autonomous Systems,20, pp. 191-204, 1997.

Meystel A.: Nested Hierarchical Controller for Intelligent Mobile AutonomousSystem, Intelligent Autonomous Systems, Hertzberger L. O., Groen F. C. A.(Eds.), Elsevier Science Publisher B. V., Amsterdam, pp. 416-448, 1987

Meystel A.: Autonomous Mobile Robots Vehicles with Cognitive Control,World Scientific Publishing Co. Pte. Ltd., Singapore, 1991.

Michaud F, Mataric M. J.: Learning from History for Behavior-Based MobileRobots in Non-stationary Conditions", joint special issue on Learning inAutonomous Robots, Machine Learning, Vol. 31, Nos. 1-3, pp. 141-167, andAutonomous Robots, Vol. 5, Nos. 3-4, pp. 335-354, 1998.

Michaud F, Mataric M. J.: Representation of Behavioral History for Learning inNonstationary Conditions, Robotics and Autonomous Systems, Vol. 29, No.2, Nov 30, 1999 (forthcoming).

Miglino O., Lund H. H., Nolfi S.: Evolving Mobile Robots in Simulated andReal Environments, Artificial Life, Vol. 2, No. 4, 1995.

Mitchell M.: An Introduction to Genetic Algorithms, The MIT Press,Cambridge, Mass., 1996.

Moravec H. P.: The Stanford Cart and the CMU Rover, Proceedings of theIEEE, Vol. 71, No. 7, pp. 872-884, 1983.

Muller T.: Automated Guided Vehicles, IFS Publications Ltd., UK, 1983.

Nilsson N.: Shakey the Robot, Technical Note No. 323, Artificial IntelligenceCenter, SRI International, Menlo Park, CA, 1984.

Nolfi S., Floreano D., Miglino O., Mondada F.: How to Evolve AutonomousRobots: Different Approaches in Evolutionary Robotics, Proceedings ofArtificial Life IV, Brooks R. A. and Maes P. (Eds.), MIT Press, Cambridge,Mass., pp. 190-197, 1994.

Norelis F. R: Toward a Robot Architecture Integrating Cooperation betweenMobile Robots: Application to Indoor Environment, The International Journalof Robotics Research, Vol. 12, No 1, February, pp. 79-98, 1993.

References

126

Noreils F. R., Chatila R. G.: Plan Execution Monitoring and ControlArchitecture for Mobile Robots, IEEE Journal of Robotics and Automation,Vol. 11, Nro. 2, pp. 255-266, 1995.

Numaoka C., Takeuchi A: Collective Choice of Strategic Type, From Animalsto Animats 2 (Proceedings of the Second International Conference onSimulation of Adaptive Behavior), Meyer J-A, Roitblat H. L., Wilson S. W(Eds.), The MIT Press, Cambridge, pp. 469-477, 1992.

Ohkawa K., Shibata T., Tanie K.: Method for Controlling a Group of Robots byan Operator, Distributed Autonomous Robotic Systems 3, Lueth T., Dillmann,Dario P., Wörn H., (Eds.), Springer-Verlag, Berlin, pp. 309-318, 1998.

Ota J., Arai T.: Motion Planning of Multiple Mobile Robots Using DynamicGroups, Proceedings of the IEEE International Conference on Robotics andAutomation, Atlanta, Georgia, pp. 28-33, 1993.

Ota J., Arai T., Kurabayashi D.: Dynamic Grouping in Multiple Mobile RobotsSystem, Proceedings of the IEEE International Conference on IntelligentRobots and Systems, Yokohoma, Japan, pp. 1963-1970, 1993.

Ozaki K., Asama H., Ishida Y., Yokota K., Matsumoto A., Kaetsu H., Endo I.:Negotiation Method for Collaborating Team Organization among MultipleRobots, Distributed Autonomous Robotic Systems, Asama H., Fukuda T., AraiT., Endo I., (Eds.), Springer-Verlag, Tokyo, pp. 199- 210, 1994.

Parker L. E.: Adaptive Action Selection for Cooperative Agent Teams, Fromanimals to animats 2 (Proceedings of the Second International Conference onSimulation of Adaptive Behavior, Meyer J-A, Roitblat H. L., Wilson S. W(Eds.), The MIT Press, Cambridge, Mass., pp. 12 - 23, 1992.

Parker L.E.: Heterogeneous Multi-Robot Cooperation, PhD Thesis,Massachusetts Institute of Technology, Artificial Intelligence Laboratory, MA,February 1994, MIT-AI-TR 1465, 1994.

Parker L. E.: Multi-Robot Team Design for Real-World Applications,Distributed Autonomous Robotic Systems 2, Asama H., Fukuda T., Arai T.,Endo I., (Eds.), Springer-Verlag, Tokyo, pp. 91 - 103, 1996.

Pasteels J. M., Deneubourg J.-L., Goss S.: Self-Organization Mechanisms inAnt Societies: Trail Recruitment to Newly Discovered Food Sources, FromIndividual to Collective Behavior in Social Insects, Pasteels J. M. andDeneubourg J.-L. (Eds.), Birkhäuser Verlag, Basel, pp. 155 - 175, 1987.

Pennisi E.: The Secret Language of Bacteria, New Scientist, Vol. 16,September, pp. 30 - 33, 1995.

Pettersson L: Control System Architectures for Autonomous Agents - ASurvey Study, Technical Report, Department of Machine Design, RoyalInstitute of Technology (KTH), Stockholm, Sweden, 1997.

References

127

Pfeifer R.: Building 'Fungus Eaters': Design Principles of Autonomous Agents,From animals to animats 4,( Proceedings of the 4th International Conferenceon Simulation of Adaptive Behavior), Maes P., Mataric M. J., Meyer J.-A.,Pollack J., Wilson S. W. (Eds.), The MIT Press, Cambridge, Massachusetts,USA, 1996.

Premvuti S., Yuta S.: Consideration on the Cooperation of MultipleAutonomous Mobile Robots, IEEE International Workshop on IntelligentRobots and Systems, Tsuchiura, Japan, pp. 59-63, 1990.

Premvuti S., Wang J.: A Medium Access Protocol (CSMA/CD-W) SupportingWireless Inter-Robot Communication in Distributed Robotic Systems,Distributed Autonomous Robotic Systems, Asama H., Fukuda T., Arai T,Endo I. (Eds.), Springer-Verlag, Tokyo, pp. 165-175, 1994.

Ram A., Arkin R., Boone G., Pearce M.: Using Genetic Algorithms to LearnReactive Control Parameters for Autonomous Robotic Navigation, AdaptiveBehavior, Vol. 2, No. 3, pp. 277 - 304, 1994.

Reynolds C. W.: An Evolved, Vision-Based Behavioral Model of CoordinatedGroup Motion, From Animals to Animats 2, Proceedings of the SecondInternational Conference on Simulation of Adaptive Behavior, Meyer J.-A.,Roitblat H. L., Wilson S. W (Eds.), The MIT Press, Cambridge, Mass., pp.384- 392, 1992.

Reynolds C.W.: Evolution of Corridor Following Behavior in a Noisy World,From Animals to Animats 3 (Proceedings of the Third InternationalConference on Simulation of Adaptive Behavior), Cliff D., Husbands P.,Meyer J.-A. and Wilson S. (Eds.), The MIT Press, Cambridge, Mass., pp.402-410, 1994.

Riekki J.: Reactive Task Execution of a Mobile Robot, Ph.D. Thesis, InfotechOulu and Department of Electrical Engineering, University of Oulu, 1998.

Rosheim M. E.: Robot Evolution -The Development of Anthrobotics, JohnWiley & Sons Inc., New York, NY, 1994.

Rosenblatt J.: DAMN: A Distributed Architecture for Mobile Navigation,Working Notes, AAA1 1995 Spring Symposium on Lessons Learned forImplemented Software Architectures for Physical Agents, Palo Alto, CA, pp.167-178, 1995.

Rumelhart D., McClelland J.: Parallel Distributed Processing, The MIT Press,Cambridge, Mass., 1986.

Schneider-Fontan M.: Plan Execution Based on Active Perception: AddingHints to Plans, Autonomous Robots, Vol. 6, Nro. 1, pp. 53-68, 1999.

References

128

Schönberg T.: Simulator Studies of the Robot Society Concept, LicentiateThesis, Helsinki University of Technology, 1993, (not published).

Schöner G., Dose M., Engels C.: Dynamics of Behavior: Theory andApplications for Autonomous Robot Architectures, Prints from AdvancedStudy Institute on Practice and Future of Autonomous Agents, Monte Verita,Switzerland, 1995.

Sedgewick R.: Algorithms (2nd edition), Addison-Wesley, New York, USA,1988.

Shapiro J. A.: Bacteria as Multicellular Organisms, Scientific American, Vol.258, No. 6, pp. 62 - 69, 1988.

Shapiro J. A.: The Significances of Bacterial Colony Patterns, BioEssays, Vol.17, No. 7, pp. 597 - 607, 1995.

Sherman P. W., Jarvis J. U. M., Braude S. H.: Naked Mole Rats, ScientificAmerican, Vol. 264, August, pp. 42 - 48, 1992.

Sims K.: Evolving 3D Morphology and Behavior by Competition. Proceedingsof Artificial Life IV, Brooks R. A. and Maes P. (Eds.), The MIT Press,Cambridge, Mass., pp. 28 - 39, 1994.

Stanier R. Y., Ingraham J. L., Wheelis M. L. and Painter P. R.: GeneralMicrobiology (5th Ed.), London, Macmillan Education Ltd., 1990.

Steels L.: The Artificial Life Roots of Artificial Intelligence, Artificial Life- AnOverview, Langton C. (Ed.), The MIT Press, Cambridge, Mass, pp. 75- 110,1995a.

Steels L.: Intelligence -Dynamics and Representations, The Biology andTechnology of Intelligent Autonomous Agents, Steels, L. (Eds.), Berlin,Springer-Verlag, 1995b.

Steels L: Discovering the Competitors, Adaptive Behavior,Vol. 4, No. 2, pp.173 - 199, 1996.

Stilwell D. J., Bay J. S.: Toward the Development of a Material TransportSystem using Swarms of Ant-like Robots, Proceedings of the IEEEInternational Conference on Robotics and Automation, Atlanta, USA, pp 766-771, 1993.

Sudd J. H., Franks N. R: The Behavioural Ecology of Ants, Chapman andHall, New York, USA, 1987.

Sugawara K., Sano M.: Cooperative Acceleration of Task Performance:Foraging Behavior of Interacting Multi-Robots System, DistributedAutonomous Robotic Systems 2, Asama H., Fukuda T., Arai T., Endo I.(Eds.), Springer-Verlag, Tokyo, pp. 233-242, 1996.

References

129

Suzuki T., Fujii T., Asama H., Yokota K., Kaetsu H., Mitomo N., Endo I.:Cooperation between a Human Operator and Multiple Robots forMaintenance Tasks at a Distance, Distributed Autonomous Robotic Systems2, Asama H., Fukuda T., Arai T., Endo I. (Eds.), Springer-Verlag, Tokyo, pp.50 - 59, 1996.

Taipale T., Hirai S.: A Comparative Study of Multi-Robot Systems, Bulletin ofthe Electrotechnical Laboratory, Vol. 56, No. 8, pp. 30-60, 1992.

Thompson A.: Evolving Electronic Robot Controllers that Exploit HardwareResources, Advances in Artificial Life (Proceedings of the 3rd EuropeanConference on Artificial Life), Moran, F. Moreno, A., Merelo, J. Chacon, P.(Eds.) Springer-Verlag Lectures Notes in Artificial Intelligence 929, pp. 640 -656, 1995.

Thompson A., Harvey I., Husbands I.: Unconstrained Evolution and HardConsequences, Towards Evolvable Hardware, Sanchez E., Tomassini M.(Eds.), Springer-Verlag Lecture Notes in Computer Science, 1996.

Tinbergen N.: Social Behavior in Animals, Chapman and Hall Ltd, GreatBritain, 1953.

Toda M: Man, Robot and Society, The Hague, Nijhoff, 1982.

Todd D. J: Fundamentals of Robot Technology, Kogan Page Ltd, London,UK, 1986.

Vainio M.: Design of an Autonomous Small-Scale Mobile Robot, HUTMAN,for robot society studies. Master's Thesis, Helsinki University of Technology,1993, (not published).

Vainio M., Schönberg T., Halme A., Jakubik P.: Optimizing the Performanceof a Robot Society in Structured Environment through Genetic Algorithms.Advances in Artificial Life ( 3rd European Conf. on Artificial Life, Granada,Spain, June, 1995), Moran F., Moreno A., Merelo J. J., Chacon P. (Eds.),Lecture Notes in Artificial Intelligence 929, Subseries of Lecture Notes inComputer Science, Springer-Verlag, Berlin, Germany, pp. 733-746, 1995a.

Vainio M., Schönberg T., Halme A.: Evolving of a Fitness Based OperationStrategy for a Robot,Proceedings of the 2nd IFAC Conference on IntelligentAutonomous Vehicles, Espoo, Finland, pp. 372 - 377, 1995b.

Vainio M., Halme A., Appelqvist P., Kähkönen P., Jakubik P., Schönberg T.,Wang Y.: An Application Concept of an Underwater Robot Society,Distributed Autonomous Robotic Systems 2, Asama H., Fukuda T., Arai T.,Endo I. (Eds.), Springer-Verlag, Tokyo, pp. 103 - 114, 1996.

Vainio M., Halme A., Jakubik P., Schönberg T., Appelqvist P.: RobotSocieties- From Simulator Studies Towards a Real World Application, AAAI

References

130

Fall Symposium Technical Report FS-97-02 (Socially Intelligent Agents),Cambridge, USA, pp. 137 - 139, 1997.

Vainio M., Appelqvist P., Schönberg T., Halme A.: Group Behavior of aMobile Underwater Robot Society Destroying Distributed Targets in a ClosedProcess Environment, Proceedings of the 3rd IFAC Conference on IntelligentAutonomous Vehicles, Madrid, Spain, pp. 112-117, 1998a

Vainio M., Appelqvist P., Halme A.: Generic Control Architecture for aCooperative Robot System, 1998 IEEE/RSJ International Conference onIntelligent Robots and Systems, Victoria, Canada, pp. 1119-1125, 1998b

Vainio M., Appelqvist P., Halme A.: Mobile Robot Society for DistributedOperations in Closed Aquatic Environment, accepted to Robotica (1999).

Virk G. S., Randall M., Howard D.,(Eds.): Proceedings of the 2nd InternationalConference on Climbing and Walking Robots (CLAWAR 99), ProfessionalEngineering Publishing Limited, Bury St Edmunds and London, UK, 1999.

Yamauchi B., Beer R.: Integrating Reactive, Sequential, and LearningBehavior Using Dynamical Neural Networks, From Animals to Animats 3(Proceedings of the 3rd International Conference on Simulation of AdaptiveBehavior), Cliff D., Husbands P., Meyer J.-A., Wilson S. W (Eds.), The MITPress, Cambridge, Mass., pp. 382 - 391, 1994.

Yamauchi B. M.: Exploration and Spatial Learning in Dynamic Environments,Case Western Reserve University, PhD Thesis, 1995.

Wang Z.-D., Nakano E., Matsukawa T.:Cooperating Multiple Behavior-BasedRobots for Object Manipulation, Distributed Autonomous Robotic Systems,Asama H., Fukuda T., Arai T., Endo I. (Eds.), Springer-Verlag, Tokyo, pp.371-382, 1994.

Wang Z.-D., Nakano E., Matsukawa T.: A New Approach to Multiple Robot’sBehavior Design for Cooperative Object Manipulation, DistributedAutonomous Robotic Systems 2, Asama H., Fukuda T., Arai T., Endo I.(Eds.), Springer-Verlag, Tokyo, pp. 350-361, 1996.

Wilson E. O.: The Insect Societies, Harvard University Press, Cambridge,1974.

Yuta S., Premvuti S.: Coordinating Autonomous and Centralized DecisionMaking to Active Cooperative Behaviors Between Multiple Mobile Robots,Proceedings of the IEEE/RSJ Conference on Intelligent Robots and Systems,pp. 1566-1574, 1992.

131

Appendixes

Appendix A: Pseudocode for behaviors presented in Chapter 8.

Appendix B: An example log file (1 robot)

Appendix C: Tests (1 robot)

Appendix D: Tests (3 robots, no communication)

Appendix E: Tests (3 robots, C -type communication)

Appendix F: Tests (5 robots, no communication)

Appendix G: Tests (5 robots, C -type communication)

Appendix H: Tests (5 robots, CC -type communication)

APPENDIX A: Pseudocode for the behaviors presented in Chapter 8

Name recoverType self-sufficiency

Algorithm

IF (node_now==old_node) {counter1=counter1+1;}

IF(counter1>LIMIT1) // IN { old_state=state_now; state_now=recover; exploit=0; explore=0; counter1=0; IF (tanks == FULL) drive_tanks(EMPTY); ELSE IF (tanks == EMPTY) drive_tanks(FULL); counter2=counter2+1; }

IF ((state_now==recover)&&(current_node!=old_node)) // OUT { state_now=old_state; explore=1;exploit=1; } IF (state_now==recover)&&(counter2>LIMIT2) // TO notepad {state_now=notepad;}

old_node=node_now;

Name notepadType miscellaneous

Algorithm

IF (state_now==notepad) { eliminate(motors); // minimizes energy consumption mediate(incoming_messages); // acts as a message link station }

Name endType miscellaneous

Algorithm

IF((operator_command==1)||(mission_time==FULL)) { state_now=end; goal_node=home; navigate(current_node, goal_node); }

IF((state_now==end)&&(current_node==home)) {halt;} // OUT


Name loadType self_sufficiency

Algorithm

IF ((energy_level< LIMIT_HIGH) && (strategy_done==1)) // IN { old_state=state_now; old_goal=goal_node; state_now=load; goal_node=energy_node; navigate(current_node, goal_node); energy_out=1; explore=0;exploit=0; }ELSE IF (energy_level<LIMIT_LOW) // IN { old_state=state_now; old_goal=goal_node; state_now=load; goal_node=energy_node; navigate(current_node, goal_node); energy_out=1; explore=0; exploit=0 }ELSE IF (poison_level==0) // IN { old_state=state_now; old_goal=goal_node; state_now=load; goal_node=energy_node; navigate(current_node, goal_node); poison_out=1; explore=0;exploit=0; }IF((energy_out==1) || (poison_out==1)) { IF(current_node==energy_node) { IF ((energy_level<full_energy) || (poison_level<full_poison)) { refill; counter3=counter3+1; IF(counter3>LIMIT3) // Failure in refilling {state_now=recover;} // TO recover } ELSE // OUT { energy_out=0; poison_out=0; state_now=old_state; goal_node=old_goal; explore=1;exploit=1; } } ELSE {navigate(current_node, energy_node);} }


Name tasksType task achieving

Algorithmstate_now=tasks; //DEFAULTexploit=1;explore=1;

Name exploitType task achieving

Algorithm

IF((exploit==1)&&(map.ready==1)) // IN{ IF((strategy1=1)&&(strategy2==1)&&...&&(strategyN==1)) {all_tested=1;}

IF(all_tested==0) //NEW STRATEGIES EXIST { IF(strategy1==0) { do(strategy1); strategy1=1; } ELSE IF(strategy2==0) { do(strategy2); strategy2=1; }

...

ELSE IF (strategyN==0) { do(strategyN); strategyN=1; } }

IF (all_tested==1) // ALL STRATEGIES TESTED { IF(communication==0) { next_strategy==best(own_strategy); do(next_strategy); } ELSE IF (communication==1) { IF(best(group_strategy)>best(own_strategy)) {next_strategy=best(group_strategy);} ELSE {next_strategy=best(own_strategy);} do(next_strategy); } } }


Name exploreType task achieving

Algorithm

IF (explore==1) { IF(make==1) { make.map; //See Chapter 7.2 for description of algorithm fix=1; make=0; }

IF(fix==1) { fix.map; // See Chapter 7.3 for description of operator and fix=0; // inter- robot based methods map.ready=1; }

IF(map.ready==1) { use.map; // See Chapter 7.4 for description of follow.map // and route.planning algorithms counter5=counter5+1; IF(counter5 > LIMIT5) { test.map; // See Chapter 7.3 for detailed description IF(test==0) //Map obsolete. New map must be obtained { map.ready=0; make=1; counter5=0; } ELSE //Map OK {counter5=0;} } } }

APPENDIX B: Log file containing messages from a single robot during a test.

My ID is: 2 Robot receives ID number My tank mode is: 2 Robot starts a pressure calibration dive Mission start NewMatch0006 Calibration dive completed. Robot arrives to node 6. Ntrl=00060002 Robot plans its route to next goal node(12). Ntrl tells that in nodes 6

and 2 robot drives motors.These messages are removed from the rest of the log.

Ntyp=00080003 Ntyp tells that in node 6 robot takes water in (8) and in node 2 it ejects water outThese messages are removed from the rest of the log.

Motors on Informs that motors are on.These messages are removed from the rest of the log.

Old 0007 Robot has reached node 7. On trail Everything going as planned.

These messages are removed from the rest of the log. Old 0008 Old 0010 Old 0002 Old 0012 Robot arrives to the goal node. Procedure2 0937 Robot performs the measurement procedure and gets en estimate

for the algae growth. Neutralize target Robot starts poison release. Cease fire Poison attack stops after 60 seconds (one dose) Attack OK Procedure2 0938 Robot performs a new measurement procedure in order to see

the effect of its attack. AA0011093800010001 Inter-robot message. Tells to the other robots that it has been in

node 12 (11 in robot's program), measured value for thealgae is 938, performs strategy 1 and that the resultfor that strategy was 0001. If the result is smaller than1000 it means that it is negative (1000 is added to positive results)

GoalNode 0006 New goal node is 6 (fixed, used for testing strategies at the beginning) Old 0013 Robot starts to travel Old 0014 Old 0003 Old 0004 Old 0005 Old 0006 Procedure1 0941 Strategy 0002 Procedure1 0945 Target found from N0006 Neutralize target Cease fire Attack OK Procedure1 0940 AA0005094000021005 Strategy 0003 GoalNode 0012 Old 0007


Old 0002 Old 0012 IRC message AA received Robot receives a message from some other robot Procedure2 0434 Target found from N0012 Neutralize target Cease fire Attack OK Procedure2 0667 AA0011066700030233 GoalNode 0006 Old 0013 Old 0014 Old 0003 Old 0004 Old 0005 Old 0006 Procedure1 0683 Target found from N0006 Neutralize target Cease fire Attack OK Procedure1 0349 AA0005034900021334 GoalNode 0012 OUT OF POISON Robot runs out of poison after four full-time attack Old 0007 Starts to navigate towards node 2, which is the energy node. Old 0002 Arrives to the refilling location. POISON LOAD OK Refilling successful. Old 0012 Robot reaches goal node. Procedure2 0008 Measurement procedure gives a value 8, which indicates

that the algae is dead or that the measurement has failed totally. GoalNode 0006 Robot doesn't start an attack, but takes a new goal node. Old 0004 Old 0005 Old 0006 Robot reaches the goal node. Procedure1 0141 Measurement procedure tells robot, that this algae is alive. Navigation OK Target found from N0006 Attack starts. Neutralize target Cease fire Attack OK Procedure1 0931 New measurement procedure gives a value of 931. AA0005093100020790 Strategy 0003 Robot changes strategy to strategy 3 (attack the smallest algae) GoalNode 0012 and takes a new target. Old 0007 Old 0002 Old 0012 Robot arrives to the goal node Procedure2 0940 and gets a new measurement, which tells it that algae in node 12 is alive.


Target found from N0012 Neutralize target Cease fire Attack OK Procedure2 0920 AA0011092000031020 GoalNode 0006 Robot changes its goal node, because the other algae growth is smaller. Old 0013 Old 0003 Old 0004 Old 0005 Old 0006 Robot reaches goal node. Procedure1 0008 Measurement procedure indicates that the algae is already dead. GoalNode 0012 Robot changes its goal node. Old 0007 Old 0002 Old 0012 Robot reaches goal node. Procedure2 0009 Measurement procedure indicates that the algae is already dead. GoalNode 0006 Robot changes its goal node. Old 0013 Old 0014 Old 0003 Old 0004 Old 0005 Old 0006 Robot reaches goal node. Procedure1 0008 Measurement procedure indicates that the algae is already dead. GoalNode 0012 Robot changes its goal node.

131

Appendixes C-H

Appendix C: Tests (1 robot)

Appendix D: Tests (3 robots, no communication)

Appendix E: Tests (3 robots, C -type communication)

Appendix F: Tests (5 robots, no communication)

Appendix G: Tests (5 robots, C -type communication)

Appendix H: Tests (5 robots, CC -type communication)

NOT AVAILABLE IN THIS PDF DOCUMENT

HELSINKI UNIVERSITY OF TECHNOLOGY AUTOMATION TECHNOLOGY LABORATORY RESEARCH REPORTS

No. 8 Salminen, R.,Robust pole placement control of a trolley crane system, May 1992.

No. 9 Sievänen, R.,Construction and identification of models for tree and stand growth, November 1992.

No. 10 Zhang, X., Halme, A.,A summary of the study of bioelectrochemical fuel cell by using Saccharomyces cerevisiae, January 1994.

No. 11 Yang, H.,Using landmarks for the vehicle location measurement, February 1994.

No. 12 Halme, A., Zhang, X.,Experimental study of bioelectrochemical fuel cell using bacteria from Baltic sea, February 1995.

No. 13 Zhang, X.,Aspects of modelling and control of bioproceses: Application of conventional approach, and functional state concept, October 1995.

No. 14 Yang, H.,Vision methods for outdoor mobile robot navigation, November 1995.

No. 15 Wang, Y.,Spherical rolling robot, February 1996.

No. 16 Hartikainen, K.,Motion planning of a walking platforms designed to locomote on natural terrain, November 1996.

No. 17 Zhang, X., Halme, A.,Effect of size and structure of a bacteria fuel cell on the electricity production and energy conversion rate,March 1997.

No. 18 Visala, A.,Modeling of nonlinear processes using Wiener-NN representation and multiple models, November 1997.

No. 19 Xu, B.,An interactive method for robot control and its application to deburring, November 1998.

No. 20 Zhang, X. Halme A.,A biofilm reactor for a bacteria fuel cell system, August 1999.

ISBN 951-22-4734-8ISSN 0783-5477

intelligence through interactions- underwater robot ... studies/intelligence through... ·...

Documents