Adaptive Co-operative Mobile Robots

A thesis

submitted to the

Cardiff University

in candidature for the degree o f

Doctor of Philosophy

by

Medhat Hussein Ahmed Awadalla

iIntelligent Systems Laboratory

Manufacturing Engineering centre

Cardiff University

United Kingdom

2005

UMI Number: U584751

All rights reserved

INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed,

a note will indicate the deletion.

Dissertation Publishing

UMI U584751Published by ProQuest LLC 2013. Copyright in the Dissertation held by the Author.

Microform Edition © ProQuest LLC.All rights reserved. This work is protected against

unauthorized copying under Title 17, United States Code.

ProQuest LLC 789 East Eisenhower Parkway

P.O. Box 1346 Ann Arbor, Ml 48106-1346

Synopsis

This work proposes a biologically inspired collective behaviour for a team o f co­

operating robots. Collective behaviour is achieved by controlling the local interactions

among a set o f identical mobile robots, each robot performing a set o f sim ple behaviours

in order to realise group goals. A m odification o f the subsumption architecture is

proposed for implem enting control o f individual robots. This architecture is adopted

because it is computationally inexpensive and potentially suitable for low-level reactive

and reflexive behaviours.

In this scenario, the individual behaviours o f the robots have different aims, which may

cause conflict. To address this issue, a fuzzy logic-based approach for m ultiple behaviour

coordination within each robot is proposed.

The work also focuses on the developm ent o f intelligent multi-agent robot teams capable

o f acting autonomously and o f collaborating in a dynamic environm ent to achieve team

objectives. A knowledge-based software architecture is proposed that enables these

robots to select co-operative behaviours and to adapt their perform ance during the

specified time o f the mission. These abilities are important because o f uncertainties in the

environmental conditions and because o f possible functional failures in some team

members. Improvement in team perform ance is achieved by updating the control o f the

robots based on knowledge acquired on-line. This architecture is implemented in a

simulated team o f mobile robots performing a proof-of-concept collaborative task. The

results show a significant improvement in overall group performance and the robot team

is able to achieve adaptive cooperative control despite dynamic changes in the

environm ent and variation in the capabilities o f the team members. Finally, a task

involving real mobile robots is undertaken to demonstrate a practical, though simplified,

implementation o f the proposed collective behaviour.

Dedication

This dissertation is dedicated to my mother, my wife, my son M ohannad and my family

who gave me my aspirations during this work.

iv

Acknowledgements

I would like to thank my supervisor Prof. D. T. Pham for his excellent supervision,

continuous encouragem ent and support. He was a brilliant supervisor.

I also want to thank all my colleagues at the Intelligent System Laboratory, especially

Shankir, Jerom e, M ark, Slava, Jan, Dayal, Fahmy, Sameh, Afshin, who were very good

to me and very helpful whenever I needed them.

Declaration

This work has not previously been accepted in substance for any degree and is not

being concurrently submitted in candidature for any degree.

Signed... h r A z k r r . ...(Medhat Awadalla - Candidate)

Date....

Statement 1

This thesis is the result o f the candidate’s own investigations, except where

otherwise stated. Other resources are acknowledged by footnotes giving

references.

Signed........................... T ........ ( Medhat Awadalla - Candidate)

Date.....

Statement 2

I hereby give consent for my thesis, if accepted, to be available for photocopying

and for inter-library loan and for the title and the summary to be made available to

outside organisations.

Signed .T7...\ . . . . . . . . . \. . . . 7T7. . . . . . . . . . . . . ( Medhat Awadalla - Candidate)

Date S.

v i

Contents

Synopsis ii

Dedication iv

Acknowledgements v

Declaration vi

Contents vii

List o f Figures x

Abbreviations xiv

Nomenclature xv

Chapter 1 - Introduction 1

1.1 Preliminaries 1

1.2 Research Objectives 3

1.3 Organisation o f the Thesis 4

Chapter 2 - Background 6

2.1 Preliminaries 6

2.2 Co-operative M obile Robotics Classification 7

2.3 Collective Behaviour o f Social Insects 10

2.4 Behaviour Coordination 14

2.4.1 Arbitration ASM s 16

2.4.2 Command Fusion ASM s 16

2.4.3 Priority-Based Arbitration (Subsumption Architecture) 17

2.4.4 State-Based Arbitration 17

vii

2.4.4.1 Discrete Event Systems (DES) 17

2.4.4.2 Tem poral Sequencing 18

2.4.4.3 Bayesian Decision Analysis 18

2.4.4.4 Reinforcem ent Learning Approaches to Action Selection 21

2.4.5 W inner-takes-AU: Activation Networks 21

2.4.6 Voting Based Com m and Fusion 22

2.4.7 M ultiple Objective Behaviour Coordination 22

2.4.8 Superposition Based Command Fusion (Potential Fields) 24

2.5 Awareness Effect on M obile Robot Co-operation 28

2.6 Fuzzy Logic Systems (FLSs) Basic Structure and Design Elements 31

2.6.1 Fuzzification Interface 31

2.6.2 Knowledge Base 32

2.6.2.1. Data base 32

2.6.2.2. Rule base 35

2.6.3 Decision M aking Logic 40

2.6.4 Defuzzification strategies 41

2.7 Summary 43

Chapter 3 - Bio-logical Inspired Collective Behaviour and Co-operatingMobile Robots 44

3.1 Preliminaries 44

3.2 Collective Dynamic Target Tracking 45

3.3 M odified Subsumption Architecture 47

3.4 Simulation 54

vi i i

3.5 Experim ents and Discussion 55

3.6 Summary 69

Chapter 4 - Fuzzy-Logic-Based Behaviour Coordination in Multi-Robot Systems 70

4.1 Preliminaries 70

4.2 Fuzzy Based Behaviour Coordination 71

4.2.1 Command Fusion 71

4.2.2 Fuzzy-Logic-Based Dynamic Target Tracking Behavioural Architecture 74

4.2.2.1 Target Following Behaviour 77

4.2.2.2 Obstacle Avoidance Behaviour 80

4.2.2.3 Fuzzy Com m and Fusion 83

4.2.2.4 Defuzzification 83

4.3 Experim ents and Discussion 85

4.4 Summary 96

Chapter 5 - Knowledge-Based Architecture for AdaptiveCo-operative Mobile Robots 97

5.1 Preliminaries 97

5.2 Adaptive Co-operative Action Selection Architecture 98

5.2.1 Assum ptions 99

5.2.2 Architecture M echanism 101

5.3 M onitoring and Performance Evaluation 103

5.4 Architecture Control Strategy 104

5.4.1 Off-line Learning Phase 105

ix

5.4.2 On-line Learning Phase 106

5.5 Feed-Forward Neuro-fuzzy Technique 106

5.5.1 Param eter Learning Algorithm 113

5.5.2 Pattern and Batch M odes o f Training 119

5.6 Experim ents and Discussion 120

5.7 Summary 130

Chapter 6 - Mobile Robots Hardware and Experiments 131

6.1 Preliminaries 131

6.2 Construction o f a team o f mobile robots 132

6.3 Experiments and Discussion 139

6.6 Summary 150

Chapter 7 - Conclusions and Further Work 151

7.1 Contributions 151

7.2 Conclusions 152

7.3 Further Work 153

Publications 154

References 155

X

List of Figures

Chapter 2:

Figure 2.1: An example o f FSA encoding a door traversal behaviour(Pirjanian, (1999) 19

Figure 2.2: The architecture used in the sensor selection approach(Kristensen, 1996). 20

Figure 2.3: The distributed architecture for mobile robot navigation(Rosenblatt, 1995). 23

Figure 2.4: Two approaches to command fusion. Top: combiningindividual decisions. Bottom: combining individual preferences. 27

Chapter 3:

Figure 3.1(a): State diagram o f the perspective o f an individual robot. 48

Figure 3.1(b): State diagram o f the perspective o f the group. 49

Figure 3.2: The Subsumption architecture. 51

Figure 3.3: The behaviour architecture o f a target - tracking robot. 53

Figure 3.4(a): Initial environm ent (one target, different obstacles,and fifteen robots). 58

Figure 3.4(b): Intermediate stage with the robots tracking the target. 59

Figure 3.4 (c): Robots having captured the target even though someo f them failed to carry out their tasks 60

Figure 3.4(d): Final stage with all robots having captured the target. 61

Figure 3.5(a): M obile robots tracking a dynamic target in cluttered 62and uncluttered environm ents

Figure 3.5(b): The effect o f behaviours conflict on the performanceo f dynamic target tracking. 63

Figure 3.5(c): M obile robots tracking a dynam ic target in cluttered anduncluttered environm ents with varying numbers o f robots 64

Figure 3.6: The effect o f changing the obstacle density ondynamic target-tracking performance 65

Figure 3.7: Tracking dynamic m ultiple targets in a cluttered environm ent. 66

Figure 3.8: M ulti-targets (two targets) tracking with varying the num bero f robots in different environm ents 67

Figure 3.9: Robots could not track all the targets. 68

Chapter 4:

Figure 4.1: Fuzzy coordination o f behaviours. 73

Figure 4.2: Fuzzy logic-based behavioural architecture. 75

Figure 4.3: An example o f a situation where a robot wants tom ove towards a target in the proximity o f obstacles. 76

Figure 4.4: Fuzzy rules for target following. 78

Figure 4.5: An example o f com puting desired direction. 79

Figure 4.6: Fuzzy rules used for obstacle avoidance. 81

Figure 4.7: Robot turning directions 82

Figure 4.8: Steps to determ ine the proper turning direction 84

Figure 4.9(a): Initial environm ent (one target, two obstacles and two robots). 86

Figure 4.9(b): Intermediate stage without coordination. 87

Figure 4.9(c): Intermediate stage with coordination. 88

Figure 4.10(a): Initial environm ent (one target, one obstacle and two robots). 89

Figure 4 .10(b): Intermediate stage without coordination. 90

xii

Figure 4.10(c): Intermediate stage with coordination. 91

Figure 4 .1 1(a): Initial environm ent (one target, one obstacle and four robots). 92

Figure 4.11 (b): Intermediate stage without coordination. 93

Figure 4 .1 1(c): Intermediate stage with coordination. 94

Figure 4.12: Tracking a dynamic target with varying number o f robots. 95

Chapter 5:

Figure 5.1: Adaptive co-operative action selection architecture. 100

Figure 5.2(a): Structure o f the proposed neuro-fiizzy system. 108

Figure 5.2(b): Basic structure o f a node in the network. 108

Figure 5.3(a): Initial environm ent (two robots randomly situated in anenvironm ent with a long box). 122

Figure 5.3(b): The robots reach the box. 123

Figure 5.3(c): The left robot starts to push the box. 124

Figure 5.3(d): Both o f the robots successfully completed their mission. 125

Figure 5.3(e): Only one robot reached the box because o f the obstacle. 126

Figure 5.3(f): The robot starts to push from the left side. 127

Figure 5.3(g): The robot m oves to the other end to push. 128

Figure 5.3(h): The robot continues back and forth pushing. 129

Chapter 6:

Figure 6.1 (a): The mobile robots 133

Figure 6.1(b): The target 134

Figure 6.2: Robot sensors board 135

xi i i

Figure 6.3: The schematic diagram for the sensors board 136

Figure 6.4: The emitted signal (a) for the target (b) for the obstacles 138

Figure 6.5(a) Initial environm ent (one target and two robots). 141

Figure 6.5(b): Intermediate stage with the robots tracking the target. 142

Figure 6.5(c): Final stage with the robots having captured the target. 143

Figure 6 .6 (a): Initial environm ent (one target and three robots). 144

Figure 6 .6 (b): Intermediate stage with the robots tracking the target. 145

Figure 6 .6 (c): Final stage with the robots having captured the target. 146

Figure 6.7(a): Initial environm ent (one target, different obstaclesand two robots). 147

Figure 6.7(b): Intermediate stage with the robots tracking the target. 148

Figure 6.7(c): Final stage with the robots having captured the target. 149

x iv

ABBREVIATIONS

AID Analogue to digital converter

ASMs Action selection m echanism s

ASP Action selection problem

BP Back propagation learning algorithm

COA Centre o f area

COG Centre o f gravity

DAMN Distributed architecture for mobile navigation

DES Discrete event systems

FLS Fuzzy logic system

FSA Finite state autom ata

I/O Input/output address

IR Infrared sensor

MAC M ark and cover motion rule

M ODM M ultiple objective decision making

M OM M ean o f the maximal value

R2D2 Relative robot direction detector

VLSI Very large scale integration

SO Self-organising behaviour

XV

NOMENCLATURE

Chapter 2:

U Total potential field

Uatt Goal attractive potential field

Urep Obstacle repletion potential field

BP backpropagation

DA-FNN fuzzy neural network with differentiable activation

functions

FLC fuzzy logic controller

FLS fuzzy logic system

GM P generalised modus ponens

GM T generalised modus tolens

NB negative big

NS negative small

x v i

Chapter 4:

(Xj

a 1 and a 2

B1 and B2

u *

0

R1 and R2

P

AND

Min

Chapter 5:

PB

Mi,my and ay

Of

Relative applicability o f the behaviour

Relative applicability for behaviourl and behaviour 2

Robot behaviours

Defuzzified crisp output

Target angle

Fuzzy rules

M embership function degree

Fuzzy conjunction connector

Fuzzy implication

positive big

The mem bership function o f the j th term at the term set describing the il fuzzy variable

Centre (or mean) and width (or variance) o f bell­shaped function

Activation function o f the ith node at the kth layer o f the netw ork

t h thInput o f the n layer node corresponding to the i fuzzy variable ( the superscript n corresponds to the layer num ber)

thLink weight at the n layer

Learning rate

N j N um ber o f term s at the ith term set

T (x) = {t U t ; ,. -,T*j} Term set o fx

5 Link weight that connects the ith output numeratorwn,J node to the output term nodes

Link weight that connects th< node to the output term nodes

th5 Link weight that connects the i output denom inatorw dij

5t h

5 Propagated error from layer five to the j node at the1J ith term set at layer four

th^4 Propagated error from a node at the i output term setn at layer four to the rth rule node at layer three

t hg4 Total propagated error from layer four to the r rule

node at layer three

g3 Propagated error from the mth rule node o f the ruleljm nodes that share the same j* term node at the i^ input

term set at layer two

g3 Total propagated error from layer three to the j th termIJ node at the ith term set at layer two

2 Propagated error from the j th term node at the ith termij set at layer two to the ith input node at layer one

5

^2 Total propagated error from layer two to the ith input1 node at layer one

31 Learning rate

M (x) = {m 'x,M x v .,M x} Set o f m em bership functions that is associated with thei linguistic term s at the term set

M x

/ k ( ) and a k () Aggregation function and activation function o f a node

at the kth layer

x v i i i

mij and crij Centre (or m ean) and width (or variance) o f the bell­shaped function o f the j* term o f the ith input linguistic variable

w ni and w di

8ni and §di

Aw

xi> f \ , u \ » a| and w ’

/ ? , a ? a n d w jj

f ] , u? and a?

/■-, U" and a- J u ’ uu au

/ 5ni, u5ni and a 5ni

/di> u^i and al

f t > yj and a f

HaM> ^bW’HA (x) and n B (x)

E

Link weights associated with each output variable

node at layer six

Propagated error from layer six to numerator and denom inator nodes at layer five

The change o f the error

The Ith input, aggregation function, net input, activation function and weight link ofthe ith node at layer one o f the network

Aggregation function, activation function and weight Link o f the j th node o f the ith group of nodes at layer two

Aggregation function, net input and activation function o f the rth rule node at layer three

Aggregation function, net input and activation function o f the j th node o f the ith group o f nodes at layer four

Aggregation function, net input and activation function o f the ith num erator node at layer five

Aggregation function, net input and activation function o f the ith denom inator node at layer five

Aggregation function, crisp output and activation function o f the ith output node at layer six

Two fuzzy membership functions and their com plem ents

Error function

x ix

q r^ and T1 Robot r\ is aware that another robot r^ is working with

a certain task T 1

x (t) and y (t) *nPut Pattem and the desired output pattern

The desired output and current network outputy(t) and y n e t( t )

w Adjustable free parameter

a ' and a"- The arg um ent o f softmin and softmax functions andits complement.

x, y and z Linguistic variables

XX

Chapter 1

Introduction

1.1. Prelim inaries

The increasing current interest in m obile robots and in particular, m obile robot team s,

is due to their applicability to a w ide range o f tasks. Exam ple tasks suitable for

m obile robots include nuclear and hazardous waste cleanup, m ining - including

m aterial rem oval, search and rescue operations, mine sweeping for both m ilitary and

hum anitarian purposes, space m issions, lifting and carrying o f m aterials, surveillance

and sentry, as well as underw ater excavation.

A m ulti-robot system or team consists o f a group o f robots that can take specific roles

w ith in that organisation. The team m ay be com posed o f individual robots that either

differ or are sim ilar in structure and capability, i.e., either heterogeneous or

hom ogeneous. Furtherm ore, it m ight have co-operative individuals w orking together

tow ards a m utual goal, or it could be com posed o f rivals com peting for som e lim ited

resources.

B iological agents, for exam ple social insects, have been m anifestly successful in

exploiting the natural environm ent in order to survive and reproduce. Scientists are

interested in understanding the strategies and tactics adopted by such natural agents to

im prove the design and functionality o f com puter-based artificial agents (robots).

They observe how these social insects locally interact and co-operate to achieve

1

com m on goals. It seem s that these creatures are program m ed in such a way that the

required global behaviour is likely to em erge even though some individuals m ay fail

to carry out their tasks.

In th is work, a biologically inspired collective behaviour for a team o f co-operating

m obile robots is proposed. This behaviour em erges by controlling the local

interactions betw een a num ber o f identical m obile robots perform ing a set o f sim ple

behaviours. A m odification o f the subsum ption architecture is proposed for

im plem enting the control o f the individual robots. The context o f tracking a dynam ic

target is used to illustrate the proposed approach.

Since the control in behaviour-based system s is distributed am ong a set o f specialised

behaviours, the behaviours o f the robots have different aims, w hich m ay cause

conflict. Therefore, it is necessary to obtain an appropriate trade-off betw een the

objectives o f the robots that can po ten tia lly conflict.

To address this issue, a fuzzy logic-based approach for behaviour coordination is

proposed. Fuzzy logic is robust in the presence o f system and external perturbations.

It is straightforw ard to design and im plem ent and efficient at representing know ledge

for system s that deal w ith continuous variables. The fuzzy rule form at m akes it easy

to write sim ple and effective behaviours for a variety o f tasks w ithout having to use

com plex m athem atical m odels. It is possib le to adopt a w eighted com bination o f

behaviours, which gives sm oother control. The local nature o f fuzzy rules allow s one

to identify the fired rules im m ediately and m akes it possible to m odify specific

behaviours quickly.

2

The focus o f developm ent in co-operative robotic system s is to construct team s o f

robots able to accom plish m issions that cannot easily be achieved, i f at all, using

single robots. The potential advantages o f co-operative system s over single robot

solutions include increased fault to lerance, sim pler robot design, w idened application

dom ains, and greater solution efficiency. However, the use o f m ultiple robots

introduces additional issues o f robot control that are no t present in single robot

solutions. Forem ost am ong these is the question o f how to achieve globally coherent

and efficient behaviours from the interaction o f robots lacking com plete global

inform ation.

The robots need to be responsive to continual changes in the capabilities o f robot team

m em bers and to changes in the state o f the environm ent and m ission. They should be

aware o f the actions o f their team m ates and have the ability to adapt to these dynam ic

changes. To address this issue, a know ledge-based software architecture is proposed

that enables these robots to perform co-operative behaviours and adapt their

perform ance during the specified tim e o f the m ission. This im provem ent in team

perform ance is achieved by updating the control o f the robots based on know ledge

acquired on-line.

1.2. Research Objectives

The overall aim o f this research w as to develop intelligent m ulti-agent robot team s

capable o f acting autonom ously and collaborating in a dynam ic environm ent to

achieve team goals.

3

To reach the aim o f the research, the fo llow ing objectives were set:

1- To use the collective behaviour o f sim ple creatures to enable a team o f robots to

accom plish a com plex task, such as a dynam ic target tracking task, and to m odify

the subsum ption architecture to be suitable for controlling the robots.

2- To find a new technique for solving conflicts betw een the contradictory

behaviours o f each robot.

3- To generate a know ledge-based softw are architecture for a team o f robots to

enable them to select appropriate actions and adapt the m ission perform ance to

deal w ith uncertainties in the environm ent or changes in the capabilities o f team

m em bers.

4- To construct a team o f m obile robots for real experim ents to investigate the

proposed techniques.

1.3. Organisation of the Thesis

The rem ainder o f the thesis is organised as follows. C hapter 2 review s the

background literature relevant to the w ork presented in this thesis. This includes

literature on co-operating m obile robots, the collective behaviour o f sim ple creatures

such as social insect colonies, the action selection problem (ASP), behaviour

coordination, robot aw areness and the basic com ponents o f fuzzy logic system s.

C hapter 3 describes the m odification o f the subsum ption architecture and exam ines

the collective behaviour o f social insects as applied to co-operating m obile robots in

the context o f dynam ic target tracking.

C hapter 4 proposes a fuzzy logic approach for behaviour coordination in m ulti-robot

system s.

Chapter 5 discuses the developm ent o f a know ledge-based system for m ulti-agent

robot team s and proposes a softw are architecture to enable m ultiple robots to perform

co-operative behaviours and adapt their perform ance during the specified tim e o f the

m ission.

C hapter 6 focuses on the design o f a team o f m obile robots and real experim ents to

illustrate an im plem entation o f som e o f the proposed ideas.

C hapter 7 concludes the thesis and suggests areas for further investigation.

5

Chapter 2

Background

2.1. Preliminaries

D uring the last few decades, m ajor research efforts have been directed tow ards

im proving the perform ance o f individual m obile robots through the use o f advanced

sensors and actuators and the application o f intelligent control algorithm s. This was

m ainly driven by the need to perform increasingly com plex real tim e tasks. As a

result, individual m obile robots have becom e very sophisticated. M ore recently, an

alternative approach to achieving com plex tasks using m ultiple co-operative

autonom ous m obile robots has been investigated (Hu and Gan, 2005; M elhuish et al.,

1998; Alam i et al., 1998; Hu et al., 1998; A rkin, 1990; M ataric, 1998, 1996). G roups

o f m obile robots have been constructed, w ith the aim o f studying such issues as group

architecture, resource conflict, m obile robots co-operation and learning.

Collaboration increases the perform ance o f a robot team w ithout requiring significant

m odifications to individual robot capacities. Collaboration m ay be obtained using

com m unication schem es, im plicit com m unication via the environm ent or sim ple

explicit com m unication schem es. By these m eans, the task accom plished by the team

can be m ore com plex and its perform ance enhanced w ithout losing the autonom y or

increasing the com plexity o f individual robots. In some cases (G hanea-H ercock and

B am es, 1996; Boehringer et al., 1995; M ataric et al., 1995; M artinoli, 1999a, 1999b),

6

the task m ay require collaboration for it to be successfully perform ed at all, w here a

single robot is not able to carry out the task alone. Such tasks can be defined to be

“strictly collaborative” .

2.2. Co-operative M obile Robotics Classification

Research in the field o f co-operative m obile robotics has increased substantially in

recent years. M ost o f this research has concentrated on how to obtain the desired

interaction dynam ics betw een agents (robots) to increase the overall team

perform ance. This field can be broadly categorised into two groups: “co llective”

(sw arm type) co-operation and “in ten tional” co-operation.

Collective robotics is usually behaviour-based and characterised by distributed control

o f hom ogeneous robot team s. The desired collective behaviour is obtained as an

em ergent property o f the interaction m echanism designed into each robot. The

approaches developed and the problem s addressed are for hom ogeneous robot team s

only, in which each robot has the sam e capabilities and control algorithm .

A dditionally, issues o f efficiency are largely ignored. The types o f tasks im plem ented

take inspiration from social insect societies, such as ants and bees.

A num ber o f researchers have investigated ‘sw arm ’ robotics. Steels (1990) presented

sim ulation studies o f several dynam ic system s to achieve em ergent functionality w ith

application to the collection o f rock sam ples on a distant planet. D rogoul and Ferber

(1992) undertook sim ulations o f foraging and chain-form ing robots. A rkin et al.

(1993) im plem ented research concerned w ith sensing and com m unication for tasks

such as foraging. M ataric (1992) described the im plem entation o f group behaviours

for physical robots such as dispersion, aggregation and flocking. K ube and Zhang

(1992) detailed an em ergent control strategy applied to a group o f physical robots

perform ing the task o f locating and push ing a brightly-lit box.

“ Intentional” robotics achieves co-operation am ong a lim ited num ber o f typically

heterogeneous robots perform ing several distinct tasks. Such system s norm ally

em ploy either central control or a m ix o f central and distributed control.

In an intentional co-operative system , the robots often have to deal w ith som e kind o f

efficiency constraint that requires a m ore directed type o f co-operation than is found

in collective co-operative system s. Furtherm ore, the robots are usually required to

perform several d istinct tasks. These m issions thus usually require a sm aller num ber

o f robots involved in m ore purposeful co-operation, although the individual robots

involved are typically able to perform usefu l tasks on their own. Such system s require

a robust allocation o f subtasks to robots, to m axim ise the efficiency o f the team , and

p roper coordination am ong team m em bers, to allow them to com plete their m ission

successfully. M ost existing w ork on heterogeneous physical robots uses a traditional

artificial intelligence approach, w hereby the robot controller is divided into m odules

for sensing, w orld m odelling, planning, and acting. This is the so-called sense-m odel-

p lan-act paradigm , in contrast to the functional decom position m ethod used in

behaviour-based approaches.

M any researchers have investigated these intentional co-operative system s. N oreils

(1993) addressed one such sense-m odel-plan-act control architecture, w hich includes

three layers o f control. The planner level m anages coordinated protocols, decom poses

tasks into sm aller sub-units, and assigns the sub-tasks to a netw ork o f robots. The

control level organises and executes the tasks o f the robots. The functional level

provides controlled reactivity. This architecture was applied to two m obile robots

perform ing box pushing.

Caloud et al. (1990) presented another sense-m odel-plan-act architecture, which

includes a task planner, a task allocator, a m otion planner and an execution m onitor.

Each robot had goals to achieve, e ither based on its own current situation or v ia a

request by another team m em ber.

A sam a et al. (1992) described a robot system called ACTRESS, w hich addressed the

issues o f com m unication, task assignm ent and path planning am ong heterogeneous

robotic agents. Their approach revolves prim arily around a negotiation fram ew ork,

w hich allow s robots to recruit help w hen needed. They dem onstrated their

architecture on m obile robots perform ing a box-pushing task.

In general, co-operative (both sw arm and intentional) approaches to robotics should

include m echanism s within the control software o f each robot that allow s team

m em bers to recover from dynam ic changes in their environm ent or in the robot team .

Researchers have recognised that a m ore prom ising approach for the developm ent o f

co-operative control m echanism s is by the inclusion o f learning algorithm s (H u and

9

Gu, (2005, 2004); Cragg and Hu, 2005; A costa and Hu, 2003a; 2003b). M uch w ork in

particular has been carried out in the field o f m ulti-agent learning (M inguez and

M ontano, 2005; Elfw ing, 2004; W eiss et al., 1996; M ataric et al, 1995). A pplications

include predator/prey scenarios (Korf, 1992; Tan, 1993; G asser et al., 1989; Levy and

Rosenschein, 1992; Stephens and M erx, 1990), m ulti-robot soccer team s (D uhaut et

al., 1998), and box-pushing tasks (S tilw ell and Bay, 1993; K ube and Zhang, 1992;

Sen et al., 1994).

2.3. Collective Behaviour o f Social Insects

Collective behaviour is dem onstrated in any type o f system where patterns are

determ ined not by som e centralised body, bu t instead by the interactions o f a group o f

decentralised bodies (Fong et al., 2003; K ristina and Aram, 2002). There is no need

for centralised authority at all, no r for explicit com m unication betw een interacting

bodies.

Collective behaviour dem onstrates also a fundam entally im portant principle that has

been beneficial to nature and hum ans alike, nam ely that some objectives are easier to

accom plish in a group rather than by an individual. This interaction does not

necessarily require a high level o f intelligence, or even com m unication betw een the

participating bodies, yet objectives m ay be accom plished that are outside the scope o f

an individual. M any exam ples o f collective behaviour can be found in nature, e.g.

flocking o f birds, term ites building enorm ous mounds, and ants collectively carrying a

large grasshopper back to the nest to be used as food. A flock o f birds m anoeuvring

10

through the air is quite im pressive. There is no a leader bird telling the other birds

which way to m ove. Each bird sim ply has an instinctive behaviour to react to the

other birds around it, and when they all fly together the result is a collective behaviour

called flocking. A nother exam ple is ants, which have m inim al form s o f

com m unication and are considered to have very low intelligence, yet arm y ants are

able to m ove large objects thousands o f tim es heavier than them selves back to their

nest (Franks, 1989). One ant could no t direct the all other surrounding ants to return

that object, and could not m ove the object itself. It is also the case that an ant could

not determ ine the weight o f the entire object by simply tugging on it. H ow ever, the

collective behaviour that results in successful com pletion o f the an ts’ objective is due

to a genetic trait possessed by the ants. As shown by these exam ples, collective

behaviour provides a m eans for very sim ple creatures to accom plish com plicated

objectives.

Social insects can process m any sensor inputs, m odulate their behaviour according to

m any stim uli, including interactions w ith nest-m ates, and take decisions on the basis

o f a large am ount o f inform ation. The success o f social insects lies m ainly in their

self-organising behaviour (SO), w here com plex behaviour em erges from the

interactions o f individuals that exhibit sim ple behaviour by them selves (Parker et al.,

2005; Tarasew ich and M cM ullen, 2002). They can also solve problem s in a changing

environm ent (flexibility) and give the h ighest level o f perform ance even though some

individuals fail to perform their tasks (robustness). M ore and m ore researchers are

interested in this exciting way o f achieving a form o f artificial intelligence - swarm

intelligence - in which it is attem pted to link the functioning principles o f insect

colonies to the design principles o f artificial system s. For exam ple, Bay and U nsal

11

(1994) described the design and developm ent o f a class o f small m obile robots

intended to be sim ple, inexpensive and physically identical, thus constituting a

hom ogeneous team o f robots. They derive their usefulness from their group actions,

perform ing physical tasks and m aking co-operative decisions as a coordinated team.

Because o f their behavioural resem blance to their insect counterparts, they have been

nam ed “arm y-anf ’ robots.

B onabeau et al. (1999) and K ube and B onabeau (2000) stated that social insects such

as bees, ants and term ites all function collectively as groups, and efficiently

accom plish a range o f tasks in order to m aintain their societies.

Kube and Zhang (1992, 1994) exam ined the problem o f controlling m ultiple

autonom ous robots based on observations m ade from the study o f social insects. They

proposed m echanism s that allow ed populations o f behaviour-based robots to perform

tasks w ithout centralised control or use o f explicit com m unication.

Chantem argne and H irsbrunner (1999) presented a collective robotics application

w hereby a pool o f autonom ous robots regroup objects that are distributed in their

environm ent. There is no supervisor in the system, the global task is not encoded

explicitly within the robots, the environm ent is not represented w ithin the robots, and

there is no explicit co-operation protocol betw een the robots. Instead, the global task

is achieved by virtue o f em ergence and self-organisation.

M artinoli and M ondada, (1995; 1998), M artinoli et al., (1997a; 1997b; 1999a; 1999b)

and M artinoli (1999) focused on the hardw are tools needed to m onitor team

12

perform ances as well as those needed to achieve collective adaptive behaviours. They

presented a sim ple bio-inspired collective experim ent, nam ely the gathering and

clustering o f random ly distributed passive seeds. V aughan et al. (2001a; 2001b)

show ed a team o f real m obile robots that co-operated based on the ant-trail-follow ing

behaviour and the dance behaviour o f bees to robustly transport resources betw een

two locations in an unknown environm ent.

A nt-inspired solutions to various search problem s have been dem onstrated (D origo et

al., 1996; Deneubourg et al., 1990, 1991; Beckers et al., 1994), as has chem ical trail

laying and following in robots (Sharpe and W ebb, 1998; Russell, 1999).

W agner et al. (1998) and W agner and B ruckstein (1995) described an ant-inspired

m ethod for exploring a continuous unknow n planar region. Such a m ethod m ight

em ploy robots w ith lim ited sensing capabilities but with the ability to leave m arks on

the ground to cover a closed region for the purposes o f cleaning a floor, painting a

wall, or dem ining a m ine field. A m ark and cover (MAC) rule o f m otion is proposed

using tem porary m arkers (“pherom ones”) as a m eans o f navigation and indirect

com m unication.

Ijspeert et al. (2001) investigated collaboration in a group o f sim ple reactive robots

through the exploitation o f local interactions. A test-bed experim ent is proposed in

w hich the task o f the robots is to pull sticks out o f the ground - an action that requires

the collaboration o f two robots to be successful. The experim ent is im plem ented in a

physical set-up com posed o f a group o f m obile robots, and in W ebots, a three

dim ensional sim ulator o f m obile robots, (M ichel, 1998).

13

As m entioned above, through the use o f collective behaviour inspired by social

insects, sim ple tasks that require a sm all num ber o f m obile robots working in

uncluttered environm ents can be accom plished. The question o f interest is whether

this collective behaviour approach can help accom plish com plex tasks, such as

dynam ic target tracking, which require m ore collaboration, interaction, coordination

and aw areness am ong a large num ber o f robots working together in a highly cluttered

and dynam ic environm ent.

2.4. Behaviour Coordination

In behaviour-based robotics, the control o f a robot is shared betw een a set o f

purposive perception-action units, called behaviours (M urrieta-Cid, 2003; Parker,

2002; Schultz and Parker, 2003; A rkin, 1999; Pirjanian and Christensen, 1997). Based

on selective sensor inform ation, each behaviour produces im m ediate reactions to

control the robot with respect to a particu lar objective, i.e., a narrow aspect o f the

overall task o f the robot such as obstacle avoidance or wall following. Behaviours

w ith different and possibly incom m ensurate objectives m ay produce conflicting

actions that are seem ingly irreconcilable. Thus, a m ajor issue in the design o f

behaviour-based control system s is the form ulation o f effective m echanism s for

coordination o f the behaviours in a robot. This is known as the action selection or

behaviour coordination problem (Pirjanian, 1998).

B ehaviour coordination is generally recognised as one o f the m ajor open issues in

behaviour-based approaches to robotics. It can be split into two conceptually different

problem s: (1) how to decide w hich behaviour(s) should be activated at each m om ent;

14

and (2) how to com bine the results from different behaviours into one com m and to be

sent to the effectors o f the robot. These are called the behaviour arbitration and the

com m and fusion problem s, respectively.

N um erous action selection m echanism s (ASM s) have been proposed over the last

decade and these can be classified into a num ber o f logical groups. M ackenzie et al.

(1997) classified them into state-based and continuous m echanism s. W ith a state-

based A SM , in a given state, only a relevant subset o f the behaviour repertoire o f the

robot needs to be activated. W ith a continuous ASM , there are no discrete states and

the w hole behaviour repertoire is available for activation.

Saffiotti (1997) divided A SM s into arbitration and comm and fusion m echanism s,

corresponding respectively to the state-based and continuous approaches o f

M ackenzie et al. A rbitration is concerned w ith “how to decide which behaviour to

activate at each m om ent” and com m and fusion is concerned with “how to com bine

the results o f different behaviours into one com m and to be sent to the effectors o f the

robot” .

Based on these classifications, it seem s that ASM s can be best classified according to

one m ain characteristic, nam ely w hether the ASM can handle only one or m ultiple

behaviours sim ultaneously.

15

2 .4 .1 . Arbitration ASMs

Arbitration m echanism s select one behaviour from a group o f com peting behaviours,

and give it ultim ate control o f the system (the robot) until the next selection cycle.

This approach is suitable for arbitrating betw een the set o f active behaviours in

accordance w ith the changing objectives and requirem ents o f the system under

varying environm ental conditions. A rbitration m echanism s for action selection can be

classified as priority-based, state-based and winner-takes-all. In priority-based

m echanism s, an action is selected based on priorities assigned in advance. Thus,

behaviours w ith higher priorities are allow ed to take control o f the robot. State-based

m echanism s select a set o f behaviours that is com petent to handle the situation

corresponding to the given state. Finally, in winner-takes-all m echanism s, action

selection results from the interaction o f a set o f distributed behaviours that com pete

until one behaviour w ins and takes control o f the robot.

2.4.2. Command Fusion ASM s

Com m and fusion com bines recom m endations from m ultiple behaviours to form a

control action that represents their consensus. This approach allows all the behaviours

to contribute sim ultaneously to the control o f the system in a co-operative rather than

a com petitive m anner. C om m and fusion m echanism s can be divided into voting

techniques, superposition techniques and m ultiple objective behaviour coordination

techniques.

16

V oting techniques interpret the output o f each behaviour as votes, and then select the

action that receives the largest num ber o f votes. Superposition techniques combine

behaviour recom m endations using linear com binations. Finally, m ultiple objective

behaviour coordination techniques provide a formal theoretic approach to m aking

decisions based on m ultiple objective decision theory.

2.4.3. Priority-Based Arbitration (Subsumption Architecture)

The subsum ption architecture (Brooks, 1986) represents a priority-based arbitration

m echanism , where behaviours w ith h igher priorities are allowed to subsum e the

output o f behaviours w ith low er priorities. This architecture is covered in m ore detail

in chapter three.

2.4.4. State-Based Arbitration

2.4.4.I. Discrete Event Systems (DES)

B ehaviour selection is accom plished using state-transition (Kosecka, 1993) w here,

upon detection o f a certain event, a shift is made to a new state and thus to a new

behaviour. U sing this form alism , system s are m odelled in term s o f finite-state

autom ata (FSA), w here states correspond to the execution o f actions o r behaviours

and where events, which correspond to observations, cause transitions betw een these

states.

17

2.4.4.2. Temporal Sequencing

The tem poral sequencing approach is also know n as perceptual sequencing (Arkin

and M ackenzie, 1994) and is very sim ilar to the discrete-event system s approach. A

finite-state autom aton is used to sequence betw een a series o f behaviours based on

perceptual triggers. At each state, a d istinct behaviour is activated and perceptual

triggers cause transitions from one state to another. See the exam ple in figure 2.1.

2.4.4.3. Bayesian Decision Analysis

The approach o f sensor p lanning w ith Bayesian Decision Analysis (K ristensen, 1996)

is used to address the problem o f sensor selection, i.e., which sensors to use for w hich

purpose. Sensor selection can be considered a special case o f action selection, where

the actions are certain sensor operations. It operates according to the purposive

paradigm , where the system consists o f a set o f purposive m odules sim ilar to

behaviours.

For exam ple, the problem in figure 2.2 is to decide which sensors to allocate to which

purposive m odules in order to accom plish a given task, declared by the m ission

planning-m odule.

18

StartStop

door traverseduser-command

TraverseDoor

door not traverseddoor found

FindDoor

obstacle_on_path

obstacle avoided

door not found

AvoidObstacle

obstacle_on_path

Figure 2.1: An exam ple FSA encoding a door traversal operation (Pirjanian, (1999)

19

(-►

actuator!

sensor r

actuator s

sensor!

purposive module n

purposivemodule2

purposivemodulel

user interface/higher level planning

mission planning

sensor planning

Figure 2.2: The architecture used in the sensor selection approach(K ristensen, 1996)

2 0

2.4.4.4. Reinforcement Learning Approaches to Action Selection

A fundam entally different approach to action selection is to learn the action selection

m echanism (H um phrys, 1997; Lin and Lu, 1996). O f the several learning approaches

proposed, the m ost prom ising is reinforcem ent learning. R einforcem ent learning in

this context operates to induce, based on trial and error, a perception-to-action

m apping that m axim ises some rew ard.

The robot leam s the perception-action m apping, know n as a policy, by exploring

actions that lead to some rew ard. The rew ard function is designed so as to encourage

desired behaviours and suppress unw anted ones. Thus, the robot will select actions

that m axim ise the expected rew ard.

2.4.5. Winner-takes-All: Activation Networks

W ith this approach, the system consists o f a set o f behaviours or competence m odules

w hich are connected to form a netw ork. In th is network, each behaviour is described

by the preconditions under w hich it is executable, the effects after successful

execution in the form o f add-lists and delete-lists and the activation level, which is a

m easure o f applicability o f the behaviour (M aes, 1989). W hen the activation level o f

an executable behaviour exceeds a specified threshold, it is selected to furnish its

action.

21

2.4.6. Voting-Based Command Fusion

To m anage the ongoing tasks o f an agent so that action conflict is m inim ised and

desired levels o f com pliance with overall goals are achieved, each behaviour votes for

one action, w hich is suitable from its point o f view. The votes received from all

behaviours are summ ed for each action and the action with the largest num ber o f

votes is then selected. For exam ple, D A M N is a distributed architecture for m obile

robot navigation (Rosenblatt, 1997; R osenblatt and Thorpe, 1995). It consists o f a set

o f behaviours (figure 2.3) that pursue the system goals, based on the current state o f

the environm ent. Each behaviour votes for o r against each action w ithin the current

possible set o f actions. The action w ith the m axim um weighted sum o f received votes

is then selected, where each behaviour is assigned a weight, which reflects the relative

im portance or priority o f the behaviour in a given context.

2.4.7. Multiple Objective Behaviour Coordination

W ith this approach, m ultiple behaviours are blended into a single m ore com plex

behaviour that seeks to select the action that sim ultaneously satisfies all behavioural

objectives as far as possible. In (P irjanian and Christensen, 1997; P irjanian and

M ataric, 2000) m obile robot navigation and co-operative target acquisition exam ples

are given, in which the principles o f m ultip le objective decision-m aking (M O D M ) are

dem onstrated. Sim ulated as well as real-w orld experiments show that a sm ooth

blending o f behaviours according to the principles o f M ODM enables coherent robot

behaviour.

2 2

weights

votes

DAMN arbiter

commands

behaviour n

behaviour 2

behaviour 1

mode manager

vehicle controller

Figure 2.3: A distributed architecture for m obile robot navigation(R osenblatt, 1995)

2 3

2.4.8. Superposition-Based Command Fusion (Potential Field)

The potential-field approach, introduced in ( Khatib, 1986), is an approach to m otion

planning where the robot, represented as a point in configuration space, m oves under

the influence o f an artificial potential field produced by an attractive force at the goal

configuration position and repulsive forces at the obstacles. Action selection in this

case corresponds to a m ove, at each configuration, in the direction indicated by the

negative gradient o f the total potential U. The potential function U is constructed as

the sum o f two potential functions:

U = U att+U rep (2.1)

w here Ua n is the attractive potential associated with the goal and Urep is the

repulsive potential associated w ith the obstacles.

M uch w ork has been carried out in behaviour coordination and action selection that

does not directly relate to the above.

Saffiotti et al. (2000) espoused desirab ility functions as an effective w ay to express

and im plem ent com plex behaviour coord ination strategies within a single robot. The

desirability function approach w as ex tended to deal with the behaviours o f team s o f

robots. The authors showed that desirability functions offer a convenient tool to

incorporate and blend individual objectives and team objectives.

2 4

Y am ada and Saito (1999) described an action selection m ethod for m ultiple m obile

robots perform ing box pushing in a dynam ic environm ent. The robots are designed to

need no explicit com m unication, and to be adaptive to dynam ic environm ents by

changing their active set o f behaviours. The researchers proposed a m echanism that

changed the active behaviour set depending on the situation.

Hu et al. (1998) presented a feasible solution for a team o f autonom ous m obile robots

to function in a co-operative m anner. To realise coordination, a m ulti-channel infrared

com m unication system was developed to exchange m essages am ong m obile robots.

Tw o exam ples o f flocking and shared experience learning were given to dem onstrate

the perform ance o f the system.

Due to their co-operative nature, com m and fusion m echanism s prom ise im proved

perform ance over arbitration-based m echanism s. H ow ever, there are draw backs that

should be highlighted. W here a linear com bination m echanism is em ployed, the

obtained solution m ight be far from the required one. C om m and fusion system s are

also costly both in com putation tim e and hardw are, and unnecessarily so i f system

accuracy is not critical. Furtherm ore, in m ulti-objective m echanism s, it is difficult to

control the robots even heuristically to m eet all objectives.

Fuzzy logic is suitable for a coordination scheme that allows all behaviours to

contribute sim ultaneously to the control o f the system in a co-operative rather than a

com petitive manner. This is therefore the solution proposed in th is research for

behaviour coordination in the context o f dynam ic target tracking.

2 5

W hen the output o f a behaviour is represented by a fuzzy set, the problem o f

com m and fusion can be seen as an instance o f the problem o f com bining individual

preferences. Fuzzy operators can be used to com bine the preferences o f different

behaviours into a collective preference, and finally to choose a com m and based on

this collective preference. A ccording to this view, com m and fusion is decom posed

into two steps: (1) preference com bination and (2) decision. Fuzzy logic offers many

different operators to perform a com bination and m any defuzzification functions to

select a decision. It is im portant to note that the decision taken from the collective

preference can be different from the result o f com bining the decisions taken from the

individual preferences. Figure 2.4 graphically illustrates this point in the case o f two

behaviours B1 and B2 both controlling the steering angle o f a m obile robot. This

explains why fuzzy com m and fusion is fundam entally different from vector

sum m ation.

Several proposals that use fuzzy logic to perform comm and fusion have appeared in

the literature. Curiously enough, the first such proposal was m ade, in a naive form , by

two roboticists who were unaw are o f fuzzy logic but were frustrated by the p itfalls o f

existing on-off arbitration schem as (R osenblatt and Payton, 1989). T heir suggestion

was later restated in term s o f fuzzy logic by Yen and Pfluger (1995). O ther authors

have proposed sim plified form s o f fuzzy com m and fusion. For instance, G oodridge

and Luo (1994) used weighted singletons as fuzzy outputs and the centre o f gravity

(COG) m ethod for defuzzification and Pin and W atanabe (1994) used sym m etric

rectangles and COG.

2 6

Defuzzify0.0

-10behaviour B1

Defuzzify

behaviour B2

-10I Defuzzifybehaviour B1

7.5

-10behaviour B2

Figure 2.4: Tw o approaches to com m and fusion.Top: com bining individual decisions. B ottom : com bining individual preferences.

Even though there is not m uch w ork on behaviour coordination based on fuzzy logic,

fuzzy logic is w idely used in the controlling and learning m echanism s o f m obile

robots (H u and Gu, 2005; Larsson, 2005; Lin and M on, 2004; A bdessem ed et al.,

2004; Lin and M on, 2004; D em irli and M olhim , 2004; Saffiotti and W asik, 2003,

W asik and Saffiotti, 2002; C oradeschi et al., 2001; B uschka et al., 2000; Sossai, 2000;

H offm ann and Pfister, 1997; Surm an et al., 1995; Pan, et al., 1995).

2.5 Awareness Effect on M obile Robot Co-operation

M uch existing work in the area o f robo t aw areness addresses the problem o f global

coherence and efficiency by designing robotic team s that use sensor inform ation to

glean im plicit inform ation on the activ ities o f o ther robot team m em bers and/or the

current state o f the w orld (D eneubourg et al., 1990; Kube and Zhang, 1992). W ith

these approaches, no explicit com m unication am ong robots is utilised. A m ore

difficult approach requires the robots to use passive action recognition to observe the

actions o f their team -m ates and m odify their ow n actions accordingly (H uber and

Durfee, 1993). A third, quite com m on, approach involves explicit co-operation among

team m em bers by em ploying direct com m unication betw een robots to relay

inform ation on robot goals and/or actions to o ther team m em bers (A sam a et al., 1992;

Parker, 1994; 1995; 1996; 1999). T hese three approaches define a continuum in the

degree o f aw areness o f a robot o f the actions or goals o f its team -m ates, from im plicit

awareness through the effect o f a team -m ate on the world, to passive observation o f

its actions or goals, to explicit com m unication o f actions and/or goals. These

approaches raise interesting questions concerning the im pact o f the aw areness o f the

robot team m em bers o f the actions and/or goals o f its team -m ates.

2 8

For im plicit co-operative system s and those using passive action recognition, the

question is: W hat is the im pact o f a lim ited ability to sense the effect o f robot actions

on the w orld? For explicit com m unication system s, the question is: ‘W hat is the

im pact o f com m unication failure, w hich leads to the lack o f aw areness o f team

m em ber actions/goals?’ or, conversely: ‘W hat benefits can be gained by using explicit

com m unication to increase robot aw areness o f team m em ber actions/goals?’ Previous

research concerning the effect o f robo t aw areness, or recognition, o f team m em ber

actions w as usually described in term s o f the effect o f com m unication in co-operative

robot team s (Balch and Arkin, 1994). H ow ever, Parker (2000) has used the phrase

“robot awareness, or recognition, o f team m em ber actions” to describe precisely the

issue o f interest (aw areness o f team -m ate actions), rather than the accessing o f

inform ation that could possib ly be com m unicated betw een team m em bers. For

exam ple, the bid o f a robot for an activ ity in a negotiation system m ay depend on the

current local state o f the environm ent near a given robot, or the sensed location o f an

intruder, etc. This shows that a robo t m ay becom e aware o f the actions o f a team

m em ber w ithout the use o f explic it com m unication.

M acLennan (1991) investigated the evolu tion o f com m unication in sim ulated worlds

and concludes that the com m unication o f local robot inform ation can result in

significant perform ance im provem ents. B alch and A rkin (1994) exam ined the

im portance o f com m unication in robotic societies perform ing forage, consum ption,

and grazing tasks. They found that som e com m unication could significantly im prove

perform ance for tasks, and that com m unication o f the current robot state was alm ost

as effective as com m unication o f robo t goals. Their research w as perform ed

prim arily on real robots, rather than sim ulated robots.

2 9

D eveloping team s o f robots that are able to perform their tasks over long periods

requires the robots to be aware o f and responsive to continual changes in the

capabilities o f the robot team m em bers and to changes in the state o f the environm ent

and m ission. Parker (1997; 2000; 2001; 2002) described the L-ALLIA NCE

architecture, which enables team s o f robots dynam ically to adapt their actions over

tim e. This architecture, w hich is an extension o f earlier w ork on A LLIA N C E (Parker,

1994), is a distributed, behaviour-based architecture aim ed at applications consisting

o f a collection o f independent tasks. The key issue addressed in L-A LLIA N C E is the

determ ination o f which task robots should select to perform during their m ission, even

w here there are m ultiple robots w ith heterogeneous, continually changing capabilities

present on the team. The L -A L L IA N C E architecture is im plem ented on a team o f

heterogeneous real robots perform ing proof-of-concept box pushing experim ents.

Due to the unreliability o f the sensors and actuators em ployed and uncertainties in the

environm ent, the approach o f Parker o f using a predefined tim e for each behaviour

resulted in inconsistency, even i f the behaviours are repeated and the values are

averaged. This is because there is no guarantee that each robot will repeat the same

behaviour at the same tim e. In th is respect, it m ay be preferable to propose an

architecture which does not rely on explicit com m unication or passive recognition and

generate autom atically the required tim e for a particu lar behaviour by accessing an

on-line know ledge-base, updated by the use o f neuro-fuzzy techniques, as w ill be

discussed later.

3 0

2.6. Fuzzy Logic System s (FLSs) Basic Structure and Design

Elements

The basic structure o f a FLS com prises four basic com ponents (Lee, 1990a). They are

the fuzzification interface, know ledge base, decision-m aking logic and defuzzification

interface. Each com ponent is responsible for a certain function in a FLS. In the

follow ing sections, the function and design param eters o f each o f these com ponents

are presented.

2.6.1. Fuzzification interface

Fuzzification is related to the vagueness and im precision in natural languages. It is a

m apping that transform s m easurem ents into a subjective value, and hence it could be

defined as a m apping from an observed m easurem ent space into a subjective feature

space. In fuzzy control applications, the observed data is usually crisp. Since the

processed data in FLSs are based on fuzzy set theory, fuzzification is necessary during

the early stages to transform the observed crisp data into fuzzy sets. A com m only used

fuzzification approach is to transform this crisp data into fuzzy singletons w ithin a

certain universe o f discourse. T he transform ation process begins w ith the

norm alisation or scaling o f the crisp m easurem ents to certain bounded range say

[ - l ,+ l] using suitable scaling factors. The purpose o f the norm alisation process is to

m ap the crisp input data into a un iverse o f discourse with a finite range. Subsequently,

the fuzzification interface transform s the norm alised crisp input x0 into a fuzzy set A

in universe X with the m em bership function Ha (x) equal to zero for all x e X except at

31

the point x0, at w hich p.a(x0) equals one. In general, the role o f the fuzzification

interface can be sum m arised as follow s (K eller et al., 1992):

a) It observes the crisp input values to a FLS.

b) It perform s a scale transform ation (norm alisation) from the m easurem ent

space into the corresponding universe o f discourse.

c) It perform s the fuzzification function that converts the scaled input data into

fuzzy sets.

2.6.2. Knowledge base

The know ledge base (Lee, 1990a) com prises knowledge concerning the application

dom ain and the desired control objectives. It consists o f a data base and a linguistic

(fuzzy) control rule base. The data base provides necessary definitions, w hich are

em ployed to define linguistic control rules and fuzzy data m anipulation in FLSs. The

rule base characterises the control ob jectives and the control policy o f dom ain experts

by m eans o f a set o f linguistic control rules.

2.6.2.1. Data base

The definitions associated w ith the data base are em ployed to characterise fuzzy

control rules and fuzzy data m anipulation in FLSs. These definitions are subjective in

nature, which reflects engineering experience and judgem ent. These definitions

com prise the norm alisation o f a fuzzy universe o f discourse, the partition o f a fuzzy

universe o f discourse and the defin ition o f m em bership functions associated with

3 2

fuzzy sets. In w hat follow s, im portant definitions relating to the construction o f the

data base in FLSs are discussed.

A. Normalisation o f a fuzzy universe o f discourse

The norm alisation o f a universe o f d iscourse involves a priori know ledge o f the input-

output universe o f m easurem ents. The norm alisation process is a scale transform ation

o f the input-output universe o f m easurem ents into a norm alised closed interval

universe. For exam ple, i f the m easured input data ranges from -7.0 to +3.5, the

universe o f the input m easurem ents can be norm alised by a scale transform ation into a

norm alised closed interval universe [-1, + i] ,

B. Fuzzy partition of the input-output universe

A linguistic variable in the an tecedent o r consequent o f a fuzzy rule form s a fuzzy

input or output feature space respectively. The input or the output feature space o f

each input or output linguistic variable is defined over a certain universe o f discourse.

Each feature space is internally partitioned into a num ber o f clusters o r fuzzy sets that

define the term set o f the input or ou tput linguistic variables. Each fuzzy set is defined

by a certain linguistic term , and usually has a m eaning such as negative big (NB),

negative small (NS), positive big (PB), etc. The num ber o f partitions o f the input and

output feature spaces determ ines the m axim um num ber o f fuzzy rules that can be

generated. Therefore the selection o f the num ber o f partitions influences the generated

num ber o f rules o f FLSs. In m ost applications o f FLSs, experience and engineering

judgem ent are em ployed to choose the num ber o f partitions o f the fuzzy feature space.

3 3

C. Definition o f the membership functions o f fuzzy sets

There are two com m only used m ethods w hich define the m em bership functions o f

fuzzy sets depending on w hether the universe o f discourse is discrete o r continuous

(Lee, 1990b). The first m ethod is a num erical definition where the grade o f

m em bership in a fuzzy set is represented as a vector o f num bers. In this case, the

m em bership function o f each fuzzy set can be written as follows:

P a ( x ) = [ p A (Xo) / x 0 + |I a ( x i ) / x i + ...........................+ p A (x „ ) / x n ] ( 2 .2 )

• • *liw here n is the num ber o f supports o f the discrete universe o f d iscourse, xn is the n

support o f the discrete universe o f d iscourse, and p A (xn) is the m em bership grade o f

the n th support in fuzzy set A. The second m ethod is a functional definition, which

expresses the m em bership function o f a fuzzy set in a functional form , typically a

bell-shaped, triangle-shaped, trapezoid-shaped function, etc. For exam ple the

functional definition o f the bell-shaped m em bership function can be w ritten as

follows:

P a ( x 0 ) = exp[-(x0 - u )2 / a 2 ] (2.3)

w here u and a are respectively, the centre (or m ean) and the width (or variance) o f the

bell-shaped function.

3 4

2.6.2.2. Rule base

A FLS is characterised by a set o f linguistic statem ents based on expert knowledge.

The expert know ledge is usually in the form o f IF - TH EN rules, w hich are easily

im plem ented by fuzzy conventional statem ents in fuzzy logic. The collection o f fuzzy

rules that are expressed as fuzzy conditional statem ents forms the rule set or the rule

base o f a FLS. In this section, the fo llow ing factors which influence the design and the

im plem entation o f a fuzzy rule base are discussed: the choice o f the FLS input-output

variables, the approaches em ployed to generate fuzzy rules, and the functional

im plem entation o f fuzzy rules.

A. Choice o f the FLS input-output variables

It is im portant to choose suitable input and output variables for FLSs, because they

influence the num ber o f rules and the perform ance o f FLSs. In several applications o f

FLSs, the selection o f input-output variables relies on experience and control

engineering judgem ent (Sugeno and N ish ida, 1985). In som e other applications, the

selection is based on a determ inistic m ethod (Sugeno and Yasukawa, 1993).

B. Derivation o f fuzzy rules

In general there are two com m on approaches to deriving fuzzy rules. These two

approaches are not m utually exclusive, and it seem s likely that a com bination o f them

would be necessary to construct an effective m ethod o f deriving fuzzy rules. The first

approach is to generate fuzzy rules based on expert experience and control

3 5

engineering knowledge. This approach is m ainly suitable for generating fuzzy rules

for diagnosis system s including fault d iagnosis and m edical diagnosis system s. This

approach is a heuristic approach, in w hich the fuzzy rules are obtained m ainly from

hum an experience. A hum an expert has to interpret his experience as linguistic

relations betw een the input and output variables o f the FLS. This approach can be

successful if the hum an expert can perform this interpretation. How ever, i f the hum an

expert cannot express his experience linguistically , then the second approach w hich is

based on the observed input-output da ta can be em ployed. This approach can be used

to generate fuzzy rules for FLC s and for fuzzy process m odels. In the case o f FLCs,

the fuzzy rules can be generated based on observations o f the hum an expert's control

actions in terms o f input-output data. In the case o f fuzzy process m odels, the fuzzy

rules are generated based on the process input-output data (Sugeno and N ishida, 1985;

Takagi and Sugeno, 1983, 1985; W ang and M endel, 1992).

C. Functional implementation o f fuzzy rules

A rule base o f a FLS consists o f a set o f fuzzy rules. For exam ple, consider the

following rules:

R i : IF x is Ai and y is Bi T H E N z is Ci

also R 2: IF x is A 2 and y is B 2 T H E N z is C 2

also Rn: IF x is A n and y is B n T H E N z is Cn

3 6

w here x, y and z are linguistic variables and Aj, Bj and Cj are linguistic term s (fuzzy

sets) o f the linguistic variables x, y and z in the universes o f discourse U , V and W

im plication (fuzzy relation) Rj. This fuzzy relation is a fuzzy set in U x V x W and is

defined for all u e U , v e V and w e W a s follows:

and its m em bership function is g iven by:

jaRi (u , v, w) = ( A i and B i -» c i ) (u , v, w ) = [ p Aj(u) a n d P B i(v ) ] -» Mci(w) (2.5)

where “ Aj and Bj” is a fuzzy set in the C artesian product space U x V w hich can be

defined based on the interpretation o f the sentence connective "an d " and, Rj = (Ai and

Bj) -> Cj is a fuzzy im plication (relation) in the Cartesian product space U x V x W

w hich can be defined based on the in terpreta tion o f the sentence connective "and" and

the definition o f the fuzzy im plication function —

The im plication functions can be c lassified into two com m only used categories

(K eller et al., 1992). The first category is the fuzzy conjunction that is defined for all

u e U and v e V as follows:

respectively, w ith i = 1, 2 ,.. . . , n. T he ith fuzzy rule is im plem ented by a fuzzy

Ri = f(u , v, w ) ,n R.(u ,v ,w ) j |(u, v, w ) e (U xV xW )) (2.4)

(2 .6)uxv

3 7

where A and B are fuzzy sets in the universes o f discourse U and V respectively,

A -» B is a fuzzy im plication in the Cartesian product space U x V and * is an

operator that represents a triangular norm . The second category is the fuzzy

disjunction that is defined for all u e U and v e V as follows:

where A and B are fuzzy sets in the universes o f discourse U and V respectively,

A —>B is a fuzzy im plication in the C artesian product space U x V and + is an

operator that represents a triangular co-norm . In general, using the fuzzy conjunction

along with the intersection and algebraic product triangular norm s, the two com m only

used fuzzy im plication functions can be w ritten as follows:

w here jlxa ( u ) a |aB(v) = m in [p A( u ) ,p B(v)] is the intersection triangular norm .

where p A(u) p B(v) = p A( u ) p B(v) is the algebraic product triangular norm.

In m ost existing FLSs, the sentence connective "and" is usually im plem ented as a

fuzzy conjunction in a Cartesian product space (Lee, 1990b). As an illustration, for

(2.7)uxv

(2 .8)uxv

(2.9)uxv

3 8

two fuzzy sets A and B in the universes o f discourse U and V respectively, “A and B ”

is defined by a fuzzy set A x B in the C artesian product space U x V. I f the sentence

connective "and" is interpreted using the intersection triangular norm , the m em bership

function o f this fuzzy set is expressed as follows:

A lternatively, i f the sentence connective "and" is interpreted using the algebraic

product triangular norm , the m em bersh ip function o f th is fuzzy set is expressed as

follows:

On the other hand, the in terpretation o f the sentence connective "also" is based on the

fact that different orders o f fuzzy ru les in the rule base should not influence the

overall behaviour o f a FLS. T his requires that the sentence connective "also" should

have the properties o f com m utativ ity and associativity. It has been reported in (Lee,

1990b) that the operators in triangu lar norm s and co-norm s (intersection, algebraic

product, union, algebraic sum , etc) possess these properties and thus qualify as

candidates for the in terpretation o f the connective "also". H ow ever, several

investigations have been reported in (Lee, 1990b). These investigations studied FLS

characteristics using different in terpretations o f triangular norm s and co-norm s. Based

on these investigations, it has been concluded that the com m on interpretation o f the

connective "also" as the union opera to r U yielded the best results. The union operator

U is a triangular co-norm defined using the m ax function (Lee, 1990b). Subsequently,

H A x b ( u x v ) = m i n M U ) ’ H B ( V )] (2 .10)

!^a x b ( u x v ( = H a ( u ) 1̂ b ( v ) (2 .11)

3 9

considering the rule base o f Subsection 2.1.2.2, the overall fuzzy relation R is defined

as a fuzzy set in U x V x W for all u e U, v e V and w e W as follows:

R = { ((u ,v ,w ),pR (u ,v ,w )) |(u, v , w ) < e (UxVxW )} (2.12)

and its m em bership function is given by:

| i R ( u , v , w ) = m a x fiRi (u, v , w ) (2.13)i=i

w here U, V, and W are universes o f d iscourse, Rj is the ith fuzzy relation o f the ith rule

in the rule base and p R (u ,v , w ) is as defined in Equation (2.4).

2.6.3. Decision making logic

FLSs m ay be regarded as a m eans o f em ulating a skilled hum an operator through an

inference engine. M ore generally, the FLS inference engine m ay be view ed as another

step tow ards m odelling the hum an decision m aking process w ithin the conceptual

fram ew ork o f fuzzy logic and approxim ate reasoning. The function o f the FLS

inference engine is to infer recom m ended solutions from fuzzy rules relevant to given

inputs based on the em ployed inference strategy. Generally, there are two im portant

inference strategies in approxim ate reasoning (Lee, 1990b). They are generalised

m odus ponens (GM P) and generalised m odus tollens (GM T). Specifically, consider

the following rule:

IF x is A TH EN y is B

4 0

where x and y are linguistic variables and A and B are linguistic term s o f the

linguistic variables x and y in the universes o f discourse U and V respectively. The

GM P strategy can be defined as "given x is A' and the fuzzy relation R o f the fuzzy

rule then infer y = B' This inference strategy is a data-driven or forw ard chaining

strategy, w hich is particularly useful in FLC s. On the other hand the GM T strategy is

defined as "given y is B' and the fuzzy relation R o f the fuzzy rule then infer x = A '".

This inference strategy is a goal-driven o r backw ard chaining strategy, w hich is

com m only used in expert fault d iagnosis system s.

2.6.4. Defuzzification strategies

M ost practical control applications require crisp control actions to drive the controlled

process. M oreover, the output o f m ost m odelling and prediction system s has to be

crisp. D efuzzification is the m apping from the linguistic fuzzy output defined over an

output universe into a crisp ou tpu t space. There are three com m only used

defuzzification strategies (Shankir, 2001). The first strategy is the m axim um criterion.

The m ax criterion produces the po in t w 0 in the output universe W that has the

m axim um degree o f m em bership in the output fuzzy set m ax M-(w) = |i (w 0) • Aw eW

problem arises with this m ethod w hen m ore than one elem ent o f W possesses this

m axim al value and thus w 0 is no t un iquely determ ined. The second strategy is the

M ean O f M axim a (M OM ). I f there is m ore than one elem ent in W possessing the

m axim al m em bership value, then M O M produces the average value o f the m axima.

M ore specifically, let Z denote a set o f Wj for w hich an output fuzzy set in a universe

W attains m axim um m em bership values that is Z = I w ; : m ax q (w j) = M-(wi) andI j=l,2,.... J

41

assum e that the cardinality equals r; that is card(Z) = r . Then the defuzzified output

w0 is w ritten as follows:

W 0 = Z (2.14)Wiew r

H ow ever, M OM does not take account o f rules fired below the m axim um level

(Saade, 1996). The third and the m ost com m only used strategy is the Centre O f A rea

(COA) strategy. COA attem pts to co rrec t the draw back o f M O M by considering rules

that can be fired below the m axim um level. C O A generates the centre o f gravity w 0 o f

the possibility distribution o f a con tro l action as follows:

wfZP-(wj).W j

= t l ________n

Z H(wj)j=l

(2.15)

where n is the num ber o f quantisation levels o f a universe W and wj is the point in the

th • •j quantisation level in a universe W at w hich p(w ) achieves its m axim um value

4 2

2.7. Summ ary

This chapter review ed background literature relevant to the w ork presented in this

thesis. The literature on co-operating m obile robots was exam ined from different

perspectives. First, it was surveyed w ith a focus on the classification o f co-operating

m obile robots. Second, the focus shifted to the collective behaviour o f social insects

and the connection o f the functioning principles o f social insect colonies w ith the

design principles o f artificial system s. Third , the literature related to the action

selection problem (ASP) and behav iour coordination was reviewed. Fourth, previous

w ork on robot awareness and its effect on the perform ance o f co-operating m obile

robots was exam ined. Fifth, a b r ie f rev iew for the basic com ponents o f the fuzzy logic

system was given.

4 3

Chapter 3

Biologically Inspired Collective Behaviour and Co-operating

Mobile Robots

3.1. Preliminaries

Social insects such as ants and bees co-operate to achieve com m on goals with

rem arkable success. H ow ever, ind iv idually these insects are very sim ple creatures.

For exam ple, a bee is unlikely to have a global understanding o f the collective task

being perform ed. Instead, com plex behav iours can em erge from a swarm o f bees to

provide solutions through the in teractions o f individual bees sensing and acting

locally on the basis o f sim ple rules. C ollective tasks can thus be perform ed by the

swarm even though some individuals m ight fail or the environm ent they operate in

m ight change.

Studying this phenom enon m ight enable biologists to understand how living

organism s work and engineers to develop new robust and adaptive technologies for

dealing w ith com plex problem s that have defied conventional solution m eans.

This chapter focuses on the developm ent o f in telligent m ulti-agent robot team s that

are capable o f acting autonom ously and o f collaborating in a dynam ic environm ent to

achieve team objectives. It p roposes a biologically-inspired collective behaviour for a

44

team o f co-operating m obile robots. This behaviour is achieved by controlling the

local interactions am ong a set o f identical robots w ith sim ple behaviours with the aim

o f tracking a dynam ic target. The subsum ption architecture is taken as the starting

point for im plem enting the control o f individual robots. W ith this architecture,

behaviours are arranged in o rder o f priority . W hen different behaviours are applicable

sim ultaneously, the behaviour w ith the h ighest priority is activated. This so-called

“com petitive” architecture is adopted because it is com putationally inexpensive and

potentially suitable for low -level reactive and reflexive behaviours.

The rem ainder o f the chapter is o rgan ised as follows. Collective dynam ic target

tracking is discussed in section 2. Section 3 describes a m odified subsum ption

architecture. The sim ulation tool developed to test the proposed architecture is

presented in section 4. Section 5 describes the experim ents conducted using the tool

and the results obtained.

3.2 Collective Dynamic Target Tracking

An im portant issue that arises in the autom ation o f m any security, surveillance and

reconnaissance tasks is that o f m onito ring and tracking the m ovem ents o f targets

navigating in a bounded area o f interest.

The collective dynam ic target track ing task investigated here is based on the

em ergence o f collective strategy in p rey-predator behaviour, w here the predators co­

operate to catch the prey or the prey co-operate to defend them selves. The term

collective is used in the sense o f the collective m otion o f defence or attack. The

45

dynam ics o f predator-prey in teractions w here the predators surround the prey to catch

it using local sensor-based in teractions am ong them have been im plem ented in the

task o f dynam ic target tracking. The prey-capture task is a special case o f pursuit-

evasion problem s, w hich consist o f an environm ent w ith one or m ore prey and one or

m ore predators. Pursuit - evasion tasks are interesting because they are ubiquitous in

the natural world, and offer a c lear ob jective that requires com plex coordination with

respect to the environm ent, and w ith respect to other agents w ith the sam e goal. They

are therefore challenging for even the best learning system s, requiring accurate

success m easurem ent and good analysis and visualisation o f the strategies that evolve.

D ynam ic target tracking involves the fo llow ing and capturing, by a group o f m obile

robots, o f a m oving object w ithin a c lu ttered w orkspace, while avoiding collision with

obstacles and w ith each other.

The target is unpredictable; i.e., its tra jecto ry is no t know n in advance. It is how ever

assum ed to move with a bounded velocity tha t is com parable w ith the velocity o f the

tracking robots.

The research objective w as to identify ho w these robots co-operate to search for,

pursue, surround, and finally capture the target even though some individuals m ay fail

to carry out their tasks. A nother aim w as to dem onstrate how individual m em bers o f

the collective team can perform the task in a distributed fashion so that the collective

team as a whole m eets its goal.

46

From the perspective o f an individual robot, the task consists o f searching for the

target, broadcasting m essages to o ther robots, receiving m essages from other robots

and approaching and capturing the target, as depicted in the state diagram o f figure

3.1(a). In order to decide the d irection from w hich it should approach the target, the

robot is required to be aware o f the actions o f its partners.

The target-tracking task o f the robot team , from a group perspective, can be described,

at a high level, by the state d iagram o f figure 3.1(b). The accom plishm ent o f th is task

is a function o f the effective co-operation betw een the robots.

3.3. M odified Subsumption A rchitecture

Early strategies for controlling robots involved building a representation o f their

environm ent, and then p lanning the ir actions accordingly. Those strategies are

expensive and cannot react w ell to changes in a dynam ic environm ent. In the

subsum ption architecture (figure 3.2) there is no world m odel. Instead, a robot

responds directly to inform ation it receives through its sensors; the robot acts ‘by

reflex’ and in this way the contro l softw are can be very sim ple. The architecture

im plem ents several independent 'behaviours' that react to sensory inputs, and provide

control signals to the actuators (m otors) o f the robot. These ‘behav iours’ have

different priorities, such that only one behaviour is allow ed to com m and the m otors at

any one time.

47

Not found / Not received

Search for target /

Listen for message

Lost

Captured

FoundSend

message

Capturegoal

M essagereceived

Within reachFollow

messagesender

Approachgoal

Figure 3.1(a): State d iagram from the perspective o f an individual robot

48

EnclosedPursue the goal

Within reach

EscapedSurround the goal

Capture the goal

Found

Lost

Lost

Search

Figure 3.1(b): State d iagram from the perspective o f the group

49

4

input output

Layer 2

Layer 1

Layer 3

Layer 0

Figure 3.2: T he Subsum ption A rchitecture

50

The subsum ption architecture com prises a h ierarchy o f controller layers where each

layer is capable o f instantly overrid ing all low er layers and taking control o f the robot

for as long as it w ishes. It then relinquishes control to w hatever low er layer was

previously in com m and o f the robot. T he architecture control system is determ ined by

the structure o f the behaviours and their interconnections. The layer that subsumes

control o f the robot at any poin t needs no know ledge o f w hat is currently controlling

the robot at that point, and sim ilarly the usurped m odule does not require any

inform ation about the subsum er.

As previously m entioned, one o f the prob lem s associated with subsum ption is that

only a single behaviour, one behav iour per layer, is active at any tim e. W hile this is

satisfactory in m any situations, there are tim es w hen a com bination o f m ore than one

behaviour is required. In practice, it has also been found that this very loose coupling

betw een layers is not sustainable (P irjan ian , 1998). Layers often need to pass

inform ation back and forth. Take, for exam ple, the task o f m oving tow ards a target

and avoiding obstacles. Each o f these sub-tasks could be im plem ented as a single

behaviour. So long as no obstacles are detected, the robot will gracefully head

tow ards its target. I f an obstacle is detected , how ever, the obstacle avoidance

behaviour becom es active and steers the robot away from the obstacle. The problem

with this is that the obstacle avoidance behaviour has no know ledge about the target,

and therefore will not necessarily s teer in a d irection that takes the robot closer to its

desired path.

51

In this work, the subsum ption architecture is m odified to com prise m ore than one

behaviour m odule w ith in one layer (figure 3.3). Those m odules run in parallel and

have the sam e priority. B ased on inform ation from the sensors, the activated

behaviour m odule subsum es the others.

The design o f the target-tracking con tro ller begins by specifying the sensing

requirem ents for the task. C ollision free m ovem ent will require an obstacle sensor; to

follow other robots needs a robo t sensor; track ing the target w ill require a target or

goal sensor.

To accom plish the task o f track ing a dynam ic target, each robot was given four m ain

behaviours. The lowest priority defau lt behaviours are the “search” and “listen for

m essages” behaviours. “Search” d irec ts the robot to advance along its current path.

Sim ultaneously, “listen for m essages” m akes the robot receptive to m essages sent by

other m obiles. N o sensors are requ ired to activate these behaviours.

The above default behaviours can be suppressed by the “follow m essage sender”

behaviour i f a m essage has been received from another robot (by m eans o f the robot

sensor on the current robot). “F o llow m essage sender” causes the robot to m ove to its

nearest sensed neighbour. The “send m essage” and “approach goal” behaviours are

activated by the goal sensor. “ Send a m essage” m akes the robot issue a “target

intercepted” m essage to the o ther m obiles and “approach goal” directs them towards

the target. “A pproach goal” causes the robots to turn a num ber o f degrees tow ards the

target while the goal sensor is active. The task is accom plished once several robots

collectively have captured the target.

52

Sensors Behaviours

S2 Obstacle Avoid

S1 Goal Send a Approachmessage goal <5>

SO Robot Follow messagesender

Listen for messages

■ < }

i > -

Search

• 0

Figure 3.3: The behav iour architecture o f a target - tracking robot

Motors ►

53

The “search” , “follow m essage sender” and “approach goal” behaviours can create

m otion resulting in collisions. To p reven t them , the “avoid” behaviour is added. This

highest priority behaviour becom es active and rem ains active as long as the obstacle

sensor has detected an obstacle. T urn ing the robot a fixed num ber o f degrees away

from the sensed obstacles at each sim ulation tim e step prevents collisions.

3.4 Simulation

The objective o f the developed sim ulation tool is to test the proposed architecture

based on the context o f the co-operative task o f dynam ic target tracking. For this, a

sim ulated environm ent has been designed to m odel a large population o f robots (a few

thousand), different obstacles (e.g. in shape and size), and m ultiple dynam ic targets.

A ccom plishing tasks using a decen tralised system o f autonom ous robots requires the

control algorithm s o f each robot to m ake use o f local inform ation. This inform ation is

acquired by the on-board robot sensors and m ust be sufficient to ensure that the entire

system o f robots converges tow ards the desired goal.

Two kinds o f sensors were sim ulated: obstacle detection sensors and target detection

sensors.

The purpose o f the obstacle detection sensors was to provide obstacle distance

inform ation to the robot. Three ultrasonic sensors were m odelled to provide

inform ation on obstacles to the left and the right, and in front o f the robot. The same

m odels were used for the ultrasonic sensors fitted to the m oving target.

54

Target detection w as sim plified by using an infrared source at the centre o f the target

and infrared target sensors m ounted on the robots. The signal received by the sensors

depended on the distance and the o rien tation betw een the robot and the target: the

closer the distance the stronger the signal; sim ilarly, the more directly the source and

sensor were aligned, the m ore pow erfu l the signal.

Two actuators were m odelled , one for each m otor (left and right)- Steering o f the

robot was achieved by d ifferen tia lly tu rn ing the motors.

The behaviours m apped inputs from sensors to outputs to actuators to define a

stim ulus-response relationship. Sensors provided inform ation to the behaviour

m odules, which then processed the da ta to provide com m ands to actuators.

D uring a sim ulation tim e step, each behaviour m odule read its related sensors and

calculated an appropriate response, w ith the resulting com m and sent to a behaviour

arbitration m odule, w hich decided the overall response.

3.5. Experiments and D iscussion

M any factors determ ine the effectiveness o f a co-operative m ultK obot system for

dynam ic target tracking. E xperim ents w ere run w ith d ifferent numbers o f robots and

different obstacle densities. Each experim ent on a collection o f robots was perform ed

thirty tim es and the results w ere averaged. Several sets of experim ents were

conducted to analyse the effects o f various factors on perform ance. The first

experim ent analysed how varying the num ber o f robots affected the tim e required to

55

track (capture) the target. This experim ent took place in a lim ited arena containing

one small target (requiring at least tw o robots to track and capture it) and no obstacles.

The average tracking tim e (m easured as the num ber o f steps by w hich the target has

m oved before being captured) versus the num ber o f robots was exam ined. The second

experim ent differed from the first on ly by the addition o f obstacles in the arena.

Again, perform ance was analysed relative to the num ber o f robots perform ing the

task. Figure 3.4(a) show s one o f the sim ulated environm ents before the experim ent

started. This contained fifteen robots (eleven in one com er and four in another

com er), one target in the opposite com er, and different kinds o f obstacles random ly

distributed. Figure 3.4(b) dep icts an interm ediate stage o f target tracking. The

developed technique enabled the robots to com plete their m issions successfully even

though in some trials som e o f them failed (becam e stuck for a long tim e to avoid an

obstacle) to carry out their tasks as show n in figure 3.4(c). Figure 3.4(d) show s the

final stage where the robots have captured the target. Figure 3.5(a-c) show s that

increasing the num ber o f robots reduced the tim e required to track the target.

H ow ever, robot collision and in terference tended to degrade the perform ance. A dding

m ore robots did therefore produce a proportional increase in perform ance. A dding a

very large num ber o f robots causes the environm ent to be full o f robots. Then, the

target cannot move very far and track ing tim e does not change.

The third experim ent w as conducted by changing the environm ent com plexity

(obstacle density) w ith d ifferen t target sizes (a small target that requires at least two

robots to capture it, a m edium -sized target needing at least four robots and a large

target necessitating at least six robots). F igure 3.6 show s the results obtained. A dding

obstacles in the environm ent increased the tim e required for capturing the target.

56

Furtherm ore, the robots took longer to capture the larger target because m ore robots

were required for this.

Experim ent four concerns m ultip le targets distributed in the environm ent. Again,

perform ance was analysed relative to the num ber o f robots perform ing the task in

environm ents with different obstacle densities. Figure 3.7 shows that the robots co­

operated to track the targets even in a cluttered environm ent. Com pared w ith the

results for tracking one target only, there is no significant difference in perform ance,

except that the num ber o f tim e steps is reduced as show n in figure 3.8. This is because

m oving m ore than one target in the env ironm en t reduces the tracking area available to

the robots and hence increases the chance for successful capturing. H ow ever, in

several trials the robots could not track all the targets as show n in figure 3.9. This is

because there is no coordination am ong the behaviours w ithin one robot and betw een

the robots them selves. A lso, there is no task allocation o r task assignm ent techniques

to allocate a suitable num ber o f robo ts for each target.

57

iiJBSATM

file Obstacle fto of Robot g u n Iechniques Ia rgef £aram eteis

I ' ‘J R O B O T E N V IR O N M E N T , C A R D IF F S C H O O L O F E N G G , S Y S T E M S DIVISIOI

Obstacle 1

Obstacle2

Robots

Obstacle3

Figure 3.4(a): Initial environm ent (one target, different obstacles,and fifteen robots)

58

E8e Qbstado No of Robot fiun Xechniques Iaiget P«iame(««

15

H H

Figure 3.4 (b): Interm ediate stage with robots tracking the target

59

File Qbstacle [jo ol Robot Bun Jectw ques I« g e t Earnneleis

15

UslM

Figure 3.4(c): Robots having captured the target even though som e o f them failed to carry out their tasks

60

itaSATMFile Qbstade tJo ol Robot flixi lechm ques la tg e t Paam eters

15

Figure 3.4(d): Final stage with all robots having captured the target

J S jx J

61

40

cluttered environment uncluttered environment

30-

Q. 25 -

1 0 -

No. of robots

Figure 3.5(a): M obile robots tracking a dynam ic target in cluttered and uncluttered environm ents

62

40 -1

cluttered environment35 -

- uncluttered environment30 -

25 -

353015 20

No. of robots

Figure 3.5(b): E ffect o f behav iou r conflic t on the perform ance o f dynam ic target tracking.

63

cluttered environment uncluttered environmer

1200800

No. of robots1000200 400 600

Figure 3.5(c): M obile robots track ing a dynam ic target in cluttered and uncluttered environm ents w ith varying num bers o f robots

64

small target- - medium target large target

Obstacle density

Figure 3.6: Effect o f changing the obstacle density on dynam ic target-tracking perform ance

65

Ete Qtniade [|o ol Robol Bun Iechmques large! Parameter 10

Baa

m

Figure: 3.7: Tracking dynamic multiple targets in a cluttered environm ent

6 6

16 i

high obstacle density

- - medium obstacledensity

— small obstacle density

Q. 10-

No. of robots

Figure 3 .8 .M ulti-targets (two targets) tracking w ith varying the num ber o f robots in d ifferent environm ents

67

........ I£ie Qbstade [Jo of Robot Qun lechrnquet I» g e t Paiameteis

8

rinfx]

o

Figure 3.9: Robots could not track all the targets

6 8

3.6. Summary

This chapter has proposed an approach to controlling m ultiple robots that involves the

use o f collective behaviour resulting from the sensor-based behaviours o f individual

robots. The approach was inspired by the study o f sim ple creatures that exhibit

collective task achieving behaviours in the way they collaborate and interact w ith one

another.

The control o f each robot in the collective team is based on a m odified subsum ption

architecture. The m odularity o f the subsum ption architecture m akes the control o f the

robot readily adaptable to another task.

The sim ulation results obtained show ed that the robots successfully m anaged to track

and capture the target under different environm ental conditions.

The influence o f environm ental factors (e.g., num ber o f obstacles and target size) and

the num ber o f robots on the perform ance o f the group in a dynam ic target-tracking

task has been analysed. As expected, increasing the num ber o f robots reduced the

tim e required to track the target. H ow ever, robot collision and interference tended to

degrade the perform ance. Continually adding m ore robots produced a proportional

increase in perform ance.

For m ultiple targets, increasing the num ber o f robots reduced the tracking tim e

because the search space was reduced. How ever, the lack o f coord ination and task

allocation algorithm s caused some targets to escape.

69

Chapter 4

Fuzzy-Logic-Based Behaviour Coordination in Multi-Robot

Systems

4.1 Preliminaries

Behaviour-based system s have proved to be useful in enabling robots to cojewith the

dynam ics o f real-w orld environm ents. The behaviour repertoire defines the skills

available to a robot to enable it to react to situations encountered in its environment. A

robot can exhibit m ultiple behaviours. Each behaviour is responsible for achieving or

m aintaining a particular objective. H ow ever, the objective o f one behaviour might be

in conflict w ith those o f o ther behaviours and it is necessary to reach a compromise

betw een conflicting objectives. This highlights the solution o f actions to achieve the

required trade-off as a m ajor issue in the design o f system s for control and co ­

ordination o f m ultiple behaviours in a robot. For this purpose, a fuzzy logic technique

for behaviour coordination is proposed. Fuzzy logic has been adopted as tire basis o f

the technique because o f its ability easily to com bine the d ifferent individual

behaviours in a robot w ith a m odified subsum ption-based control architecture

The rem ainder o f this chapter is organised as follows. Section 2 outlines the fuzzy

logic technique for coordinating behaviours in each robot including command fusion

and dynam ic target tracking. The results obtained are presented and discussed in

Section 3.

7 0

4.2 Fuzzy Behaviour Coordination

4.2.1 Command Fusion

The desired outcom e can be achieved by integrating the outputs o f the applicable

behaviours, a process referred to as com m and fusion. For exam ple, the outputs from

the target following behaviour and the obstacle avoidance behaviour are com bined to

produce a heading that takes the robot tow ards its target w hilst avoiding obstacles. A n

approach based on fuzzy sets operations is proposed here that takes into account the

recom m endations o f all applicable behaviour m odules. B ehaviour coord ination is

achieved by weighted decision-m aking and rule-based (behaviour) selection. The

w eights used for weighted decision-m aking are the degrees o f confidence p laced on

the different behaviours. They are em pirical m easures o f applicability o f particu lar

behaviours.

To illustrate the process o f behaviour coordination, assum e there are ju s t two

behaviours B1 and B2 as shown in figure 4.1. The degrees o f confidence for B1 and

B2 are oq = 0.25 and ot2 = 0.75 respectively. The contribution o f individual

behaviours, each represented by a fuzzy set, is weighted by the corresponding degree

o f confidence. Thus, B1 and B2 are activated to degrees oq and ot2 respectively.

Behaviour activation is accom plished via scalar m ultiplication o f the output fuzzy sets

by the appropriate degrees o f confidence oq and ot2 in this exam ple).

M ultiplication o f an output fuzzy set by a scalar oq is equivalent to the conjunction o f

a set o f uniform m em bership degree oq w ith that output fuzzy set. The resulting fuzzy

sets are then aggregated using an appropriate t-conorm operator (such as the ‘M ax ’

operator), and defuzzified to yield a crisp output u* that is representative o f the

intended behaviour.

In th is procedure, the scalar oq represents the w eight o f a behaviour in the aggregated

control decision and m ultiplication by oq expresses the applicability o f the behaviour

to the current situation. It is not necessary that the sum o f the oq’s is equal to 1. This

hypothetical exam ple, illustrated in figure 4.1, reveals that the output o f the system is

influenced more by its dom inant behaviour B2 as intended. Control recom m endations

from each applicable behaviour m odule are considered in the final decision. In

general, the resultant control action can be thought o f as a consensus o f

recom m endations offered by m ultiple experts.

In som e instances, it m ay be evident from current sensory data that only one particu lar

behaviour is fully applicable (oq associated w ith that behaviour is equal to 1). In this

case, coordination sim ply reduces to behaviour selection, a process also referred to as

sw itching coordination since behaviours are alternately sw itched on and off.

7 2

O u t p u t o f B 1 O u t p u t o f B 2

0.80.7

Turning TurningDegree

Output o f B1

0.2

Degree

Output o f B2

0.53

TurningDegree

TurningDegree

UMulti-valuedoutpuU

0.530.2

TurningDegree

Figure 4.1: Fuzzy coordination o f behaviours

7 3

4.2.2. Fuzzy-Logic-Based Dynam ic Target Tracking Behavioural

Architecture

As w ith other behavioural approaches, the fuzzy-logic-based architecture for m obile

robots, in the context o f a dynam ic target tracking system , consists o f several

behaviours, such as target follow ing and obstacle avoidance. Each behav iour relates

sensor data and robot status to control recom m endations, as show n in figure 4.2, and

has two com ponents: a set o f fuzzy rules and a fuzzy inference m odule. The fuzzy

rules o f a behaviour explicitly represent its control strategy in the form o f linguistic

statem ents. As illustrated in figure 4.2, m ultiple behaviours could share a com m on

fuzzy inference m odule. Fuzzy control recom m endations generated by all behaviours

are fused and defuzzified to generate a final crisp control com m and.

The basic algorithm executed in every control cycle by the architecture consists o f the

follow ing four steps: ( 1) the target follow ing behaviour determ ines the desired turning

direction; (2 ) the obstacle avoidance behaviour determ ines the d isallow ed turning

directions; (3) the com m and fusion m odule com bines the desired and disallow ed

directions and (4) the com bined fuzzy com m and is converted into a crisp com m and

through a defuzzification process. The desired and disallow ed d irections are

m aintained in fuzzy set form to reduce possible loss o f inform ation during com m and

fusion. Figure 4.3 shows an exam ple o f a situation in w hich the target follow ing

behaviour suggests that the robot should turn left, but the robot m ust continue straight

on a little longer to avoid the obstacle on the left (i.e., obstacle A). This exam ple is

used in the following subsections to describe and dem onstrate each step o f the

algorithm .

7 4

Fuzzy Rules for B1

Fuzzy Conclusion from B1

Fuzzy Conclusion from B2

Command

FusionCrisp Sensors Data

Fuzzy Rules for B2 and

Crisp Control CommandsDefuzzification

Fuzzy Conclusion from Bn

Fuzzy Rules for Bn

B s Behaviour

Fuzzy Inference

Fuzzy Inference

Figure 4.2: F uzzy log ic based behavioural architecture

75

T arget

IN

A

O bstacle A

R obot

ObstacleB

Figure 4.3: A n exam ple o f a s itua tion w here a robot w ants to m ove tow ards a target in the p rox im ity o f obstacles

7 6

4.2.2.1. Target Following Behaviour

The target follow ing behav iour genera tes a desired turning direction based on the

current location o f the target and its cu rren t bearing. The target follow ing behaviour

determ ines the desired steering d irec tion in three steps. First, it senses the target.

Second, the behaviour com putes the target angle, w hich is the angle betw een the

current direction o f the robo t and a vecto r from its current location to the target. For

the exam ple given, w ith the robo t head ing N orth , the target angle 0 is -3 0 degrees.

Third, the behaviour uses a set o f fuzzy ru les to change the specific target angle into a

general desired direction, w h ich g ives the robo t m ore flexibility in avoiding obstacles

while still follow ing the target. F igu re 4.4 show s the two fuzzy rules R1 and R2

em ployed in the exam ple by the targe t fo llow ing behaviour. The fuzzy inference

m odule o f the target fo llow ing b ehav iou r com bines the desired directions

recom m ended by all target fo llow ing b ehav iou r fuzzy rules using w eighted decision­

m aking as explained previously . T he p rocess is illustrated in figure 4.5 for a target

angle o f -3 0 degrees using R1 and R2 from figure 4.4. The antecedent m em bership

functions (i.e., “around 0 deg rees” and “around - 4 5 degrees”) are designed to overlap

such that the sum o f th e ir m em bersh ip values in that region is 1 .0 .

7 7

'around -45°"

If Target Angle is "around - 45°"

— 1

0°-90°

Then Desired Direction is "left-forward'

-4 5 ° 45°-1 3 5 ° 0°

(a) Fuzzy rule R1

"around 0°"

If Target Angle is "around 0°"

0° 45°

Then Desired Direction is "forward1

90°0°-9 0 °(b) Fuzzy rule R2

Figure 4 .4: F uzzy ru les for target follow ing

7 8

"around - 45°" "around 0°"

0.8

.0,2

-90° -45° -3 0° 0° 45°

If target angle = - 30°

"left forward" multiplied with 0.8

0.8

45°-135° 0°

i {

"forward"multiplied with 0.2

-----------------0.2

-90° 0° 90°

0.8

0.2

45° 90°-9 0° “ 45°- 135°

desired direction

Figure 4.5: An example of computing desired direction

4.2.2.2. Obstacle Avoidance Behaviour

The obstacle avoidance behav iour uses u ltrasonic data to generate a fuzzy set that

represents the d isallow ed d irections o f travel (i.e., d irections that lead, in the short

term , to or near an obstacle). T he beh av io u r operates by first com paring the distance

o f the closest obstacle de tec ted by each direction sensor to a fuzzy set, “N ear” ,

associated w ith the sensor. B ased on the resu lt o f the com parison, the behaviour

determ ines the degree to w hich the general direction o f each sensor is considered

disallow ed. Exam ples o f fuzzy ru les u sed by the obstacle avoidance behaviour are

show n in figure 4.6. The m em bersh ip functions o f disallow ed turning directions

associated w ith a sensor have been designed such that: (1) they partially overlap those

o f neighbouring sensors and (2) they have a m ajor influence on the direction o f the

sensor.

Once all the fuzzy rules associated w ith the obstacle avoidance behaviour have been

fired, their fuzzy conclusions are com bined using the M ax operator. Figure 4.7 shows

an exam ple o f th is com bination w ith senso r inputs (three u ltrasonic sensors) based on

the situation in figure 4.3. H ere, the fuzzy inference m odule o f the behaviour uses the

M ax operator instead o f o th er t-conorm operators (such as the arithm etic sum)

because it is consistent w ith the in tu itive idea that the degree to w hich a direction is

disallow ed should be determ ined by the sensor source that has the strongest opinion

about it.

8 0

I f 90°" S e n s o r D is t a n c e to N e a r e s t O b s t a c le is "Near"

0 10 d is t a n c e ( m m )

T h e n D is a l lo w e d D ir e c t io n is "left'

- 1 3 5 ° - 9 0 ° - 3 0 ° 0 °

If "0°" S e n s o r D is t a n c e to th e N e a r e s t O b s t a c le is "N ear'

d i s t a n c e ( m m )0 10

T h e n D is a l lo w e d D ir e c tio n is "forw ard'

►4 5 °-4 5 °

Figure 4.6: F uzzy ru les u sed for obstacle avoidance

81

0.80.6

-180° -90° 180°90°

(a) Disallowed direction

0.7

0.4

0° 180°-180°

(b) Allowed direction (complement of (a))

F igure 4 .7 : R o b o t tu rn ing directions

8 2

4.2.2.3. Fuzzy Command Fusion

The third com ponent o f the m obile ro b o t con tro ller com bines the fuzzy conclusions

about the desired d irection and the d isa llow ed direction into a single fuzzy control

com m and. Since the final robo t d irec tion should be both desired from the target

fo llow ing v iew poin t and n o t d isa llow ed by obstacle avoidance considerations, the

com m and fusion m odule uses the M in o p era to r in fuzzy logic to form a conjunction o f

the output o f the tw o behav iours as fo llow s:

^T um ing-D irec tion (x ) = ^ D esired (x ) A N D N o t D isallow ed(x)

m *n ̂ D e s i r e d (x)’ ^ N o t D isallow ed (x)^

m in { p D esired (x )’ ^ A llow ed(x)^

For convenience, the negated D isa llow ed D irec tion w ill be referred to as the Allowed

D irection o f travel.

W ith reference to the exam ple in figure 4 .3 , figure 4.8 illustrates the com m and fusion

and defuzzification steps un d er consideration .

4.2.2.4. Defuzzification

For the case show n in figures 4.3 and 4 .8 , by using the centre o f area (CO A ) strategy,

the crisp turn ing d irection is found to be - 20 degrees. This angle com bines the

recom m endations o f both the target fo llow ing and obstacle avoidance behaviours (see

figure 4.4 and 4 .7b) and enab les the robo ts to approach the target w ithout colliding

w ith obstacles.

83

-180° -90° 0° 180°(a)

0.8

0.2

o-9 0 ° o o-4 5 450 ° 90(b)

o-9CP o. 0 180°90°45-4 5 0 °- 135

(c)

o0,0 0 45°20 ° 180-135° 0° 90-180 -90

(d)

Figure 4.8: S teps to de te rm ine the proper turning direction(a) A llow ed d irec tion(b) D esired d irec tion(c) F usion be tw een allow ed and desired directions(d) D efuzzifica tion by using centre o f area

8 4

4.3. Experiments and Discussion

The experim ents reported in chap te r th ree, concerning tracking a dynam ic target in a

lim ited arena w ith a d ifferen t num ber o f robots and different obstacles, were repeated

to dem onstrate the fuzzy logic techn ique for behaviour coordination. In addition,

situations in which the robots face conflic ting behaviours, such as obstacle avoidance

and target tracking, have been illustrated , to prove the reliability o f the technique.

W ithout coordination (as in ch ap te r th ree), w hen the robots face obstacles while

tracking a target, as show n in F igu res 4 .9a - 4.1 la , obstacle avoidance has the highest

priority. Even though the robo ts subsequen tly lose the target, the robots avoid the

obstacles and one another as show n in F igures 4 .9b - 4.11b. Subsequently, the robots

start searching again, w asting m uch tim e. H ow ever, w ith behaviour coordination (the

resolution o f conflicts be tw een con trad ic to ry behaviours), obstacle avoidance and

target tracking are achieved by se lec ting an action that represents the consensus

am ong the behaviours and th a t b est sa tisfies the decision objectives that they encode.

As shown in Figures 4 .9c - 4 .11c the robo ts avoid the obstacles and m oved directly

towards the target to fo llow it. A fter these specific instances show ed im provem ent in

the system perform ance, the experim en ts described in chapter three were repeated and

evidence o f the successful app lication o f fuzzy logic for behaviour coordination is

shown in figure 4.12. W here conflic t am ong contradictory behaviours is correctly

m anaged by behav iour coord ination , the tracking tim e reduces w ith increasing

num bers o f robots.

8 5

fie Obstacle f l o d Robot Qun Iechniques la ig e t Parameters

I2) ROBOT ENVIRONMENT, CARDIFF SCHOOL OF ENGG, SYSTEMS DIVISIOI

Figure 4.9(a): Initial env ironm en t (one target, one obstacle, and two robots)

86

BKfiSIHIiHIHHHHBHIHMiFie Obstacle No ol Robot 0un .Techniques Tat get Patamelers

2

LM*J

o

Path o f robot 1

Path o f robot2

Figure 4.9(b): M issing o f target w hen there is no coordination

8 7

■B&AE#e Obstacle fjo of Robot Bun Iecbm ques la rg e t Parameters

WM

Figure 4.9(c): Successful capturing o f target w hen there is coordination

lliSATM£le Qbjtade Hoof Robot Qun Iechniquet Ia tge t Eaiam etm

(2> R O B O T E N V IR O N M EN T. C A R D IFF SC H O O L OF EN G G . S Y S T E M S Dl

Figure 4.10(a): Initial env ironm ent (one target, tw o obstacles, and tworobots)

8 9

pie Qbilade Ho of Robot Bun Iechnique* Iaigol E«amelef«

2

Path o f robot2

Path o f robotl

Figure 4.10(b): M issing o f ta rge t w hen there is no coordination

9 0

h h h h h h h h h ih h h h h h hEie Qbjt»de >Jo o( Robot 8un lechraoues 1*3*1 P*am #t« t

2EsL*]

Figure 4.10(c): Successful capturing o f target w hen there is coordination

91

FIs Qbstacle No o( Robot g i n Tecbnguei T aget Paiametefs

R O B O T E N V IR O N M E N T . C A R D IF F S C H O O L O F E N G G , S Y S T E M S DIVISION ■ ■ ■ ■ ■ ■ ■ I

Figure 4 .1 1(a): Initial env ironm ent (one target, one obstacle, and four robots)

9 2

iiM SATME*e Qbjtade g o o f Robot Bun Ie c b n q u e t Iar^et £««mete»t 4

oPath o f robot 1

*

Path o f robot4

Z L -

l i l

Path o f robot3

ocro

Figure 4 .1 1(b): M issing o f target w hen there is no coordination

-=iSJ2Ll

9 3

f i e Q bitade Ho at Robot Bun Ie c h n q u e t I« g e t Eaam eters

4

e a s

#

Figure 4 .1 1(c): Successful capturing o f target when there is coordination

40

35

cluttered enviroment

unclutteredenvironment

30

25

CL

20

1000 1 2 0 0800

No. of robots6004002 0 0

T rack ing a dynam ic target w ith varying num bers o f robotsFigure

9 5

4.4. Summary

This chapter has dem onstrated ho w a fuzzy logic technique enables the resolution o f

conflicts betw een con trad icto ry robo t behav iours by selecting an action that represents

the consensus am ong the behav iours and that best satisfies the decision objectives that

they encode. The results show an im provem ent in the global perform ance o f a

m ultiple robot system .

9 6

Chapter 5

Knowledge-Based Software Architecture for Adaptive

Co-operative Mobile Robots

5.1 Preliminaries

M ulti-robot teams can increase the re liab ility , flexibility, robustness and efficiency o f

autom ated solutions by ta k in g advan tage o f the redundancy and parallelism o f

m ultiple team m em bers. B efo re m u lti-robo t team s can becom e w idely used in

practise, it is necessary to d ev e lo p au tom ated techniques that enable robot team

m em bers autom atically to a d a p t th e ir ac tions over tim e in response to changes in their

environm ent or in the robo t te a m itself.

A chieving adaptive co -o p e ra tiv e robo t behav iou r is m ore challenging. M any issues

m ust be addressed in o rd e r to develop a w orking co-operative team ; these include

action selection, task a llo c a tio n , coherence, com m unication, resource conflict

resolution, and aw areness. A w areness o f o ther m em bers o f the robot team is a

necessary com ponent o f c o -o p e ra tio n ; how ever th is causes an increase in the search

space dim ension (Touzet, 2 0 0 0 ) .

A know ledge-based so ftw a re a rch itec tu re is proposed to enable robot agents to

accom plish collective b e h a v io u rs and adapt their perform ance during the specified

tim e o f the m ission. The im p ro v e m e n t in team perform ance is achieved by updating

the control o f the robots b a s e d o n know ledge acquired on-line.

9 7

The rem ainder o f the chap ter is o rgan ised as follow s. Section 2 outlines the proposed

adaptive co-operative action se lec tion architecture. Section 3 explains the

perform ance evaluation and m on ito ring m odules o f the architecture. The control

strategy, com prising off-line and on -line learning phases, is described in Section 4.

The feed-forw ard neuro-fuzzy techn ique and param eter learning algorithm s are

described in Section 5 and Section 6. Section 7 presents sim ulation results for a box

pushing exercise using ex isting sim u la tion softw are.

5.2 Adaptive C o-operative A ction Selection A rchitecture

The m ajor design goal in the d ev e lo p m en t o f th is architecture is to address the real-

w orld issues o f behav iour coo rd in a tio n , fau lt tolerance and adaptiv ity w hen using

team s o f fallible robots equ ipped w ith no isy sensors and effectors. T he architecture

m ust also allow the bu ild ing o f ro b o t team s able to cope w ith failures and uncertain ty

in action selection and action execu tion , and w ith changes in a dynam ic environm ent.

Furtherm ore, in o rder to m ain tain a p u re ly d istribu ted co-operative con tro l schem e

w hich affords an increased degree o f robustness, ind iv idual agents m ust alw ays be

fully au tonom ous, w ith the ab ility to pe rfo rm useful actions even am idst the failure o f

the o ther robots.

The architecture is developed to be fully d istributed, and giving all robots the

capability to determ ine their ow n actions based upon th e ir curren t situation, the

activities o f o ther robots and the cu rren t environm ental cond itions. N o centralised

control is u tilised , so that it is possib le to investigate the p o w e r o f a fu lly d istributed

robotic system to accom plish g roup goals.

9 8

The com ponents o f th is a rch itec tu re (F igure 5.1) w ill be explained in th is chapter,

w ith the excep tion o f the fuzzy -log ic -based action selection arbiter w hich was

explained in C hap ter 4.

5.2.1. Assum ptions

In this architecture, it is no t requ ired th a t a robo t be able to determ ine the actions o f its

team -m ates through passive o b se rv a tio n , w hich can be d ifficu lt to achieve. Instead,

robots are enabled to learn abou t the ac tions o f th e ir team -m ates through an explicit

com m unication m echanism , w hereby the robots broadcast in form ation concern ing

their curren t activ ities to the rest o f the team .

Furtherm ore, it is assum ed that the robo ts are bu ilt to w ork as a team , and are neither

in direct com petition w ith one ano ther, n o r a ttem pting to subvert the actions o f their

team -m ates, although con flic t m ay arise at a low level due to, for exam ple,

the sharing o f com patib le goals, (N o te , how ever, that som e m ulti-robo t team

applications, such as robo t soccer and m ilita ry battles, m ay require the ab ility to deal

w ith adversarial team s).

It is further assum ed in the arch itec tu re that robots do not have access to som e

centralised store o f know ledge, and that no centralised agent is available that can

m onitor the state o f the en tire robo t env ironm en t and m ake control decisions based

upon this inform ation.

9 9

sensorydata

receivedm essage

Actions

B ^B ehav iou r

C om m ands

B2

Bn

D ata B ase

A ctuators

Monitoringand

Perform anceEvaluation

C ontroller

ActionSelection

A rbiter

Figure 5.1: A dap tive co -opera tive action selection architecture

1 0 0

5.2.2. Architecture Mechanism

At all tim es during the m ission , the m otivation for each robot to activate a certain

behaviour set is based on receiv ing sensory input and in ter-robot com m unication.

W hen these inputs are valid , the behav io u r set becom es active.

Intuitively, the m otivation o f ro b o t q to activate any given behaviour set is initialised

to 0. O ver tim e, the m otivation increases qu ick ly as long as the task corresponding to

that behaviour set is no t be ing accom plished , as determ ined from sensory feedback.

H ow ever, robots also have to be responsive to the actions o f o ther robots, adapting

their task selection to the ac tiv ities o f team m em bers. Thus, i f robot q is aw are that

another robot r^ is w orking on a ce rta in task T l , then q should be satisfied for som e

tim e (based on know ledge learned on -line) tha t the task w ill be accom plished even

w ithout its ow n participation , and thus go on to som e o ther applicable actions. Its

m otivation to activate the b eh av io u r set (addressed by another robot) still increases,

but at a slow er rate. T his charac te ris tic p reven ts any robo t from replicating the actions

o f the o thers and w asting energy. O f cou rse , detecting and in terpreting the actions o f

the o ther robots (som etim es ca lled action recognition) is no t a trivial p roblem , and

often requires perceptual ab ilities tha t are no t yet possible w ith curren t sensing

technology. T hus, to enhance the percep tua l abilities o f the robots, the architecture

utilises a sim ple form o f b ro ad cast com m unication to allow robots to inform other

team m em bers o f th e ir cu rren t ac tiv ities, rather than relying to tally on sensory

capabilities. A t a p re-specified rate , each robo t q broadcasts a statem ent o f its current

101

action, w hich o ther robots m ay listen to o r ignore as they w ish. Tw o-w ay

com m unication is no t em ployed in th is arch itecture .

Each robot is designed to be som ew hat im patien t, in that a robot q is only w illing for

a certain am ount o f tim e to a llo w the m essages from ano ther robot to affect its own

m otivation to activate a g iven behav iour. C ontinued sensory feedback indicating that

a task is not accom plished thus o verrides the statem ents o f another robo t perform ing

that task. T his characteristic a llow s robo ts to adapt to failures o f o ther robots, causing

them to ignore a robot that is no t successfu lly com pleting its task.

A com plem entary characteristic in these robots is acquiescence (com pliance). Just as

the im patience characteristic o f a ro b o t reflects the recognition that o ther robots m ay

fail, the acquiescence characteristic recogn ises that the robot itse lf m ay fail. This

feature operates as follow s. A s a ro b o t q perfo rm s a task, its w illingness to give up

that task increases over tim e p rov ided th a t the sensory feedback indicates that the task

has not been accom plished. A s soon as som e o ther robot r^ signals it has begun that

sam e task and q feels that it (q ) has a ttem p ted the task for an adequate length o f tim e,

the unsuccessful robot q g ives up its task in an attem pt to find an action at w hich it is

m ore productive. A dditionally , even i f ano ther robot q< has no t taken over the task,

robot q m ay give up its task anyw ay i f it is no t com pleted w ithin a tim e lim it. This

allow s q the possib ility o f w o rk ing on ano ther task that m ay prove to be m ore fruitful

rather than attem pting in vain to perfo rm the original task forever. W ith this

acquiescence characteristic , therefo re , a robo t is able to adapt its actions to its ow n

failure.

1 0 2

As a sim ple illustrative exam ple, co n sid e r a team o f tw o robots A and B unloading

boxes from a truck and p lac ing them on one o f tw o conveyor belts, depending upon

the labelling on the box. B oth robo ts have the ability to unload boxes from the truck

to a tem porary storage location , and the ability to m ove them from the tem porary

storage location to the appropria te co n v ey o r belt, (it is assum ed that, due to the way

the loading dock is designed , the robo ts canno t m ove boxes im m ediately from the

truck to the conveyor belt). A t the beg inn ing o f the m ission, say robo t A elects to

unload the boxes from the truck . R obo t B is then satisfied that the boxes w ill be

unloaded, and proceeds to m ove the boxes from the tem porary location to the correct

conveyor belt. As the m ission p rog resses , it is assum ed that the m echanism o f robot A

for un loading the truck fails. S ince no m ore boxes are arriv ing at the tem porary

location, robot B becom es increasing ly im patien t to take over the task o f unloading

boxes, even though robot A is still a ttem p ting to accom plish that task - unaw are that

its sensor is returning fau lty read ings. Fo llow ing a predeterm ined num ber o f

unsuccessful attem pts at un lo ad in g b o x es and receip t o f a signal from robo t B that it

has begun the unloading task , be ing com plic it, robo t A abandons that task and turns

its attention to the task o f load ing the conveyo rs instead.

5.3. M onitoring and P erform ance Evaluation

O ne item o f central im portance to the learn ing m echanism used is the requ irem ent for

robots to m onito r and evaluate the perfo rm ance o f team m em bers in executing tasks.

W ithout this ability , a ro b o t m ust rely on hum an-supplied m easurem ents o f the

perform ance o f robot team m em bers tha t are unlikely to be responsive to changes

10 3

occurring over tim e. In e ither case , once these perform ance m easurem ents are

obtained, the robot team m em bers have a basis for determ ining the preferential

activation o f one behav iour over ano ther, e ithe r for the sake o f efficiency and long­

term adaptation, o r to determ ine w hen a robo t failure has occurred.

The m onitor function, im plem ented w ith in each robot, is responsible for observing

and evaluating the perform ance o f any robo t team m em ber (including itself) w henever

it perform s a behaviour. Thus each robo t m onito rs the perform ance o f o ther robots for

tasks that it itse lf is able to accom plish , record ing inform ation on the tim e o f task

com pletion. R obots do not m o n ito r all tasks by all robots - only tasks that they

them selves have the ability to perform .

D uring a live m ission, each robo t chooses the m ost suitable action to execute based on

sensory feedback and the cu rren t system situation . It then broadcasts that action to its

team m ates to avoid dup lication . T he m on ito r function w ill observe its progress. I f

there is no progress during the expected tim e, the robo t m ust e ither leave th is task, or

request help from the o ther robots. A t the sam e tim e, its team m ates also m onitor its

perform ance w hen they receive its b roadcast m essage. I f there seem s to be no

progress in task perform ance, they start to negotia te w ith that robot to help it or to

take over the task. This p rocedure is repeated until the task is com pleted.

5.4. Architecture C ontrol Strategy

The degree to w hich robot team m em bers can actively pursue know ledge concerning

team m em ber abilities depends on the type o f m ission in w hich they are engaged. If

1 0 4

they are on a train ing m ission, w hose sole purpose is to allow robots to become

fam iliar w ith them selves and w ith the ir team m ates, then the robots have more

freedom to explore their capab ilities w ithou t concern for possib ly not com pleting the

m ission. On the o ther hand, i f the robo ts are on a live m ission that m ay continue for a

long tim e, then the team has to ensure that the m ission is com pleted as efficiently as

possible, w hile continuing to adap t the ir perform ance over tim e as the capabilities o f

their team m ates change. D uring tra in ing m issions, the robots w ill be in an off-line

learning m ode w hereas during live m issions, they w ill be in an on-line learning m ode.

5.4.1 Off-line Learning Phase

The best way that the robots can independen tly leam about their ow n abilities and

those o f their team m ates is by ac tiva ting as m any o f the ir behaviours as possible

during a m ission, and m onito ring th e ir ow n progress and the progress o f team

m em bers during task execution . O n any given m ission, no t all o f the available

behaviour sets m ay be appropriate , so it is usually not possib le to leam com plete

inform ation about the capabilities o f the robo ts from ju s t one m ission scenario.

H ow ever, this learning phase allow s the team to obtain as m uch inform ation as

possible to allow each robot to se lec t its nex t action properly. T his action is one o f

the actions that is currently incom plete , as determ ined from the sensory feedback, and

not being executed by ano ther robo t, as determ ined from the broadcast

com m unication m essages. A ll th is in fo rm ation is used as a com m on know ledge base

from w hich fuzzy m les are generated . T hese rules are then fine-tuned using a feed­

forw ard neuro-fuzzy techn ique exp la ined later in this chapter.

105

5.4.2. On-line Learning Phase

W hen a robot team is on a m ission , it canno t afford to allow m em bers to attem pt to

accom plish tasks for long periods w ith little o r no dem onstrable progress.

The team m em bers have to m ake a concerted effort to accom plish the m ission with

w hatever know ledge is availab le abou t team m em ber abilities, and m ust no t tolerate

long episodes o f robot actions that do not contribute to the task execution. How ever,

each robot continues to observe robo t perform ance during this phase, and to update

the com m on know ledge base (built during the off-line phase) i f required. For

exam ple, due to the unreliability o f the sensors and actuators and uncertain ties in the

environm ent, there m ight be a sm all, bu t acceptable, variation in the tim e required for

robots to im plem ent the sam e behav iour. A ccordingly , the know ledge base has to be

updated. Furtherm ore, i f a n ew situation occurs, a suitable action w ill be executed,

m onitored, evaluated and, i f app ropria te , added to the know ledge base.

5.5. Feed-Forward N euro-fuzzy Technique

One m ajor disadvantage o f fuzzy approaches is that there are no clear guidelines as to

how to fine-tune the fuzzy m em bersh ip functions. H ow ever, learning techniques are

being developed that can help in th is process.

U pdating the know ledge-base affects the curren t rules and hence the system outputs.

Therefore, a neuro-fuzzy techn ique has been proposed to fine-tune these rules and

m inim ise the total error betw een the desired output and the fuzzy controller output.

1 0 6

M am dani-m odel-based fuzzy neural ne tw orks (FNN s) represent m ore transparent

neurofuzzy system s com pared w ith T akagi Sugeno-m odel-based FN N s (Shankir,

2001). The reason is that the rule base o f the M am dani-m odel is m ore understandable

to hum an users. A lso, it is m ore general in term s o f how its rule base is created,

because the latter can be constructed using hum an experience and num erical data.

H ow ever, a d isadvantage o f th is m odel is that it does not allow easy m athem atical

analysis due to the logical natu re o f its inference functions, e.g., the logic m in/m ax

functions. A lso, it does not a llow the sim ple application o f BP as one o f the m ost

pow erful learning algorithm s, due to the non-differentiable m in/m ax functions

em ployed.

In this chapter, a M am dani-m odel-based FN N w ith D ifferentiable A ctivation

functions (D A -FN N ) is described . A d ifferen tiab le alternative to the logic m in and

logic m ax functions term ed so ftm in a n d so ftm ax (Shankir, 2001) are presented. These

two differentiable functions (.so ftm in and so ftm ax ) are em ployed instead o f the two

non-differentiable functions (logic m in and logic m ax) to im plem ent the decision­

m aking m echanism o f D A -FN N . U sing these d ifferentiable functions allow s the

effective application o f B ack p ropaga tion (BP) for the param eter learning o f DA-

FNN.

Figure 5.2(a) presents the structure o f the proposed neuro-fuzzy system . The structure

is a six-layer feed-forw ard connection ist representation o f a M am dani-m odel-based

fuzzy logic system (FLS) (M am dani, 1974). In general, a node in any layer has some

finite "fan-in" o f connections represen ted by w eight values from other nodes and a

"fan-out" o f connections to o ther nodes (see Figure 5.2(b)). A ssociated w ith the fan-in

1 0 7

o f a node is an aggregation f u n c t i o n , / , that serves to com bine information,

activation, o r evidence from o ther nodes. U sing the same notations as in (Lin and Lee,

1992), the function that prov ides the ne t input to such a node is w ritten as follows:

net input = / k (u!< k k k , Up ; W, , W 2 , , W (5.1)

w here p is the num ber o f fan-ins o f the node, w is the link w eight associated w ith each

fan-in, u is an output o f a node in the p reced ing layer associated w ith the fan-in and

the superscrip t indicates the layer num ber. A second action o f each node is to output

an activation value as a function o f its n e t input,

w here a k denotes the ac tivation function in layer k. The functions o f the nodes in

each o f the six layers o f the p roposed structure are described next.

L a y e r 1: N odes in Layer 1 are input nodes that represent input linguistic variables.

Layer one contains N nodes, each o f w hich receives a crisp input

v e c to rx = (x , , . . . . ,x N). The nodes in th is layer sim ply transm it input values to the next

layer directly. T ha t is,

output = 0jk = a k ( / k) (5.2)

(5 .3 )

The link w eights in L ayer 1 are fixed at unity.

10 8

Layer 6defuzzificationnodes

Layer 5pre-defuzzificationnodes

Layer 4output term nodes (softm ax function)

Layer 3 rule nodes (softm in function)

Layer 2 input term nodes

Layer 1 input nodes

Figure 5.2(a): S tructu re o f the proposed neuro-fuzzy system

Layer k

wWj

u

Figure 5.2(b): B asic structure o f a node in the netw ork

1 0 9

Layer 2: N odes in L ayer 2 are input term nodes w hich act as m em bership functions.

An input linguistic variable x in a un iverse o f discourse U is characterised by

T(x) = { t 1x,T x v ,T?} and M (x) = { m 'x > M x v ?Mx}, w here T(x) is the term set o f x;

that is the set o f nam es, e.g., (sm all, m ed ium , large), o f the linguistic values o f x and

M(x) is the m em bership function, e.g ., (triangular, trapezoidal, bell-shaped), defined

on a universe U. The bell-shaped function is chosen because it is differentiable

function. The function o f each node j in a term set i is to calculate the degree o f

m em bership o f input Xj w ith respect to the m em bership function M JX , j = l ,2 , . . . . ,N j ,

associated w ith the term set T (x j) acco rd ing to the fo llow ing bell-shaped function:

f \ 2( w j j* a j ) - m i j

f \ - M^ (m ij, oij) = -------------- j------------- anc* a,2 = e" A (5.4)Gij

w here my and ay are, respectively , the cen tre (or m ean) and the w idth (or variance) o f

the bell-shaped function o f the j th term o f the ith inpu t linguistic variable Xj.

Layer 3: The nodes in L ayer 3 are ru le nodes, w here each node associates one term

node from each term set to form a cond ition part o f one fuzzy rule. In th is structure,

the softm in function (B erenji and K hedkar, 1992) and its com plem ent softm ax

function (Shankir, 2001) are used.

X a i e - k a isoftm in (a j , i = 1,2,..., n ) = (5.5)

Z e " k a ii - \

1 1 0

J]aie ^aisoftm ax ( a i ,i = l,2 ,...,n ) = so ftm in (a j,i = l , .2 ,..,n ) = 1 -

Z e ka i (5.6)1 = 1 _ V '

w here a i = p , aj = 1 - aj and the pa ram ete r k controls the “hardness” o f the softmin

function.

w here r = 1 R, and R is the n u m b er o f ru les o r rule nodes in layer three. H ow ever,

in th is layer, there are no link w eigh ts to be ad justed because all the link w eights are

fixed at unity.

Layer 4: The nodes in th is layer are o u tpu t term nodes w hich act as m em bership

functions to represent the ou tpu t term s o f the respective L linguistic ou tput variables.

The nodes in Layer 4 should in tegrate the fired ru les that have the sam e consequent.

The softm ax function is used to perfo rm the in tegration. Therefore, the function o f

each term node j in the ou tpu t te rm set i can be w ritten as follows:

w here p is the num ber o f ru les sharing the sam e consequent (the sam e output term

node). H ence, the link w eigh ts in L ayer 4 are fixed at unity.

Therefore, the function o f the rth ru le node u sing softm in can be w ritten as follows:

f \ = so ftm in(\i\ , ,....... , u !N) and a] = f ] (5.7)

f* j = so ftm a x ( , m = l,...,p j and a \ = f (5.8)m

111

Layer 5: The num ber o f nodes in layer 5 is 2L , w here L is the num ber o f output

variables, i.e. there are tw o nodes for each ou tpu t variable. The function o f these two

nodes is to calculate the denom ina to r and the num erato r o f a quasi Centre O f Area

(CO A ) defuzzification value for each o u tpu t variable. The functions o f the two nodes

o f the ith ou tput variable are :

/ ^ i = Z a fJ * m i j* o ij and a 5nj = / ^ (5.9)

/d i = ZaU*0ij and a d i ^ d i (5-10>

w here / ^ . and / are, respec tive ly , the node functions o f the num erato r and the

denom inator nodes o f the ith o u tp u t variab le .

Layer 6: The nodes in layer 6 are defuzz ifica tion nodes. The num ber o f nodes in this

thlayer equals the num ber o f o u tpu t lingu istic variables. The function o f the i node

|L

corresponding to the i ou tpu t variab le can be w ritten as follow s:

r t 31111 a i = / ^ and * = a ? ( 5 1 1 )w d i * a di

w here w ^- and are layer 6 link w eights associated w ith each output variable

node.

In order to build a neuro-fuzzy system based on the above descrip tion, three m ain

steps have to be considered. T he first step is to specify the input and output variables

o f the netw ork. The second step is to d iv ide the input-output universes into a suitable

num ber o f partitions (fuzzy sets) and to specify a m em bership function for each

1 1 2

partition. A linguistic term has to be assigned to each m em bership function and the

param eters o f the m em bership function (centre and w idth) have to be specified

initially. The third step is to generate fuzzy ru les to perform the input-output m apping

o f the FLS. A fter the construction o f the netw ork, a param eter learning phase has to

be conducted. The algorithm for tha t phase is explained next.

5.5.1. Parameter Learning A lgorithm

Follow ing the construction phase, the netw ork then enters the param eter learning

phase to adjust its free param eters. T he ad justab le free param eters w ere selected to be

the centres (rm s) and w idths (ct̂ s) o f the term nodes in layer 4 as w ell as the link

w eights in layers 2 and 6. A superv ised learn ing technique is em ployed along w ith the

back propagation (BP) learning a lgo rithm (B erenji and K hedkar, 1992) to tune these

param eters. The problem can be stated as: ‘G iven n input patterns Xi(t), i = 1,...., n, m

desired output patterns y;(t), i = 1,..... , m , the fuzzy partitions, and the fuzzy rule base,

adjust the free param eters o f the ne tw ork o p tim ally ’. In the param eter learning phase,

the goal is to m inim ise the fo llow ing e rro r function:

w here y ( t ) is the desired ou tpu t, and y n e t ( t ) ls the current netw ork output. For

each training data set, starting at the input nodes, a forw ard pass is m ade to com pute

the activity levels o f all the nodes in the netw ork. Then, starting at the output nodes, a

backw ard pass is follow ed to com pute the rate o f change o f the error function with

respect to the adjustable free param eters for all the h idden nodes. A ssum ing that w is

(5.12)

113

an adjustable free param eter in a node, th en the general learning rule can be w ritten as

follows:

Aw = r| -aE

&w

w (t + l ) = w (t) + A w

(5.13)

(5.14)

w here rj is the learning rate. U sing the chain rule, the partial derivative can be defined

thus:

dE dE d (net - input) dE d f dE da d f

dw d ( n e t - input) dw d f dw da d f dw(5.15)

The calculation o f the b ack -p ropagated erro rs as w ell as the updating o f the free

param eters can be described starting at the ou tpu t nodes as follows.

Layer 6: U sing Equation (5 .15) and E quation (5.11), the adaptive rule for the Layer 6

w eights is derived below :

d w n i da*j d f d w n i na

5ni

w di a di

W ni ( t + l ) = W n i ( t ) + Tl6\ 5 W niy

(5.16a)

(5.16b)

^ w di a a f d f $ ^ w di (w di)2a^ j

w di(t + l ) = w d i ( t ) + r |6k 5 w di J

(5.17a)

(5.17b)

1 1 4

w here rjt is the learning rate o f the link w eigh ts in layer 6. T he propagated errors from

Layer 6 to the num erato r and the den o m in a to r nodes in layer 5 are derived as follows:

(5.18a)

(5.18b)

Layer 5: An adjustm ent is requ ired for the link w eights w „y w hich represent the

centres my o f the output m em bersh ip functions. A lso , an ad justm ent is required for the

free param eter oy that rep resen ts the w id th o f the output m em bership functions.

C onsequently , using E quation (5 .9) and E quation (5.15), the adaptive rule to tune the

free param eters in Layer 5 is derived . F irst, the adaptive rule to tune the centres o f the

output m em bership functions can be ob tained as follow s:

8E 8E d&5ni . g / „ j(5 .19a)

5 m ij d f sn ■ d m i]

(5 .19b)

Second, the adaptive rule to tune the w id ths o f the output m em bership functions is

UdE

0 G ijy(5.20b)

w here r/5 is the learn ing rate o f the ad justab le param eters Gij and mjj in layer 5. The

propagated error from layer 5 to the j th node in the ith term set in layer 4 is:

da*5- d f 5 .+ m + J nid r 4d f 5 . ni J ni

+d E 8 a 5di d f *.di

di- (s6ni * mij * o ij)+ (s^i * o ij)(5 -21)

Layer 4: No ad justm ent is requ ired for the link w eights o f layer 4. O nly the error

signals 54 are required to be ca lcu la ted and propagated to a ru le node r in layer 3.

Each one o f these erro r signals is a sum m ation o f L p ropagated error signals 54j , one

error signal from a specific node j o f each term set i, w here i = 1 ,.. . . , L and L is the

num ber o f output variab les (o r term sets). U sing E quation (5.15), the error signal 5?

is then:

8? = Z5?i = Id a $ 8 /5 .

S5 * _ y * _ y'J 5 /5 . 3 a?

(5.22)

From Equations (5.8) and (5 .5)

<7a-y

5 / 5■ ij

= 1 (5.23)

and

1 1 6

£ Z-4 e ^ a r^ ij

3Car

,l _ k a ? ) Z e k ( u u-l + k I u ^ e km = l in =1

—4 U ijm

P

Z eV'n = l

:th

r _ . ^ u

(5.24)

- k ! _4Mjm

J

i f the j term node at the i term se t in layer 4 is connected to the rl rule node in

L ayer 3. O therw ise,

8 f t J ‘J

^ 3Car

= 0 (5.25)

w here p is the num ber o f ru les sha ring the sam e j m ou tpu t term node, and u j*m is the

com plem ent o f the rnm input to the j th o u tp u t term node at the iin term set in L ayer 4th

L a y e r 3: As w ith layer 4, no ad ju stm en t is requ ired for link w eights in layer 3. O nly

the error signals are requ ired to be ca lcu la ted and propagated from the rl ru le node

in layer 3 to the j th term node at the ith te rm set in layer 2. Each one o f these error

signals is a sum m ation o f p p ro p ag a ted erro r signals §fjm from layer 3, w here

m = l , . . . ,p, and p is the n u m b er o f ru les w hich share the sam e j th term node at the

sam e ith input term set in layer 2. U sing E quation (5.15), the error signal 5? can be

calculated as follow s:

8 j = I 8 i im = Sm m

■3m

m i a 2d f 3 d a i\V J m •' J

(5 .26)

From Equations (5.7) and (5 .6)

da

d f= 1 (5.27)

1 1 7

and

O f3 e k a ijm

l - k a s f E e kumi + k 2 u ^ e “ k “mii=l i = l

P 2<?a„(5.28)

2 e - k “miM

i f the j th term node at the i,h in p u t term set in layer 2 is connected to the rule node m in

layer 3; otherw ise,

a / 3J m^ 2 tfa.i

= 0 (5 .29)

w here N is the num ber o f inpu t term sets and u^i *s the ith input to the rule node m in

Layer 3.

L a y e r 2: U sing E quation (5 .13 ) and E quation (5.4), the adaptive ru le to tune the

w eights in layer 2 is g iven by:

d E d E d a ij d f \ _ 3 f 2 - 2 a { ( a j w , j - m i j )

dw,] d a ?; d f l y J y

* - — y = 5?j * e^j *

wj-( t + l ) = wjj ( t ) + ri2

f ^d E

d w ?

(5 .30a)

(5 .30b)

w here 772 is the learn ing rate o f the link w eigh ts in layer 2. The p ropagated erro r from

L ayer 2 to the ith inpu t node in layer 1 is derived as:

1 1 8

5^ = ZS?: = Zj j

dE dajj

5 / «

d / zj

g ? . * *ij y i J

Wy(a}w y - m ij)(5.31)

ay

Follow ing the construction phase and the learn ing phase, an op tim ally tuned FLS is

developed to perform a specific m app ing function. T his m apping function m ay

represent a function o f a dynam ic sy stem o r a con tro l function.

5.5.2. Pattern and Batch M odes o f T raining

In the practical application o f the back p ropagation a lgorithm to the m ulti-layer

perceptron , learning resu lts are o b ta ined from m any presen tations o f a p rescribed set

o f tra in ing exam ples to the netw ork . O ne com plete presen tation o f the entire training

set during the learning p rocess is ca lled an epoch. T he learning process is m aintained

on an epoch-by-epoch basis un til the synaptic w eigh ts and threshold levels o f the

netw ork stabilise and the average squared e rro r over the entire tra in ing set converges

to som e m inim um value. It is good p rac tice to random ise the o rder o f p resen tation o f

train ing exam ples from one epoch to the next. F o r a given tra in ing set, back-

propagation learning m ay p ro ceed in one o f tw o basic w ays, pa ttern o r ba tch m ode.

In the pattern m ode o f back p ro p ag a tio n learn ing , w eigh t updating is perfo rm ed after

the presentation o f each tra in ing exam ple. Each exam ple in the epoch is presented to

the netw ork, and a sequence o f fo rw ard and backw ard com putations is perform ed

resulting in certain ad justm en ts to the synaptic w eights and threshold levels o f the

netw ork.

1 1 9

In batch m ode, w eigh t u p da ting is p e rfo rm ed after the p resentation o f all training

exam ples that constitu te an epoch .

The pattern m ode w as adop ted in th is w ork because it is sim pler to im plem ent and it

can still give tra ined ne tw orks p ro d u c in g o u tpu ts very c lose to the desired outputs.

5.6. Experim ents and Discussion

T his arch itecture has been app lied to a sim u la ted team o f m obile robots perform ing a

p roof-o f-concep t co -opera tive b o x p u sh in g task . T hat task w as chosen because it

enabled the features o f the p ro p o se d a rch itec tu re to be dem onstrated . T he objective o f

the co-operative box -p u sh in g task is to find a box, random ly p laced in the

environm ent o f the robots, and pu sh it ac ro ss the ‘ro o m ’. The box is heavy and long

and one robot alone canno t (co n tin u o u sly ) p u sh the box to m ove it across the room . It

is necessary to synchron ise the p u sh in g o f the box by robots at the tw o ends, so that

the task is defined in term s o f tw o recu rrin g subtasks. T hese subtasks are: push the left

end a little and push the righ t end a little - ne ither o f w h ich can be activated (except

for the first tim e) un less the o p p o site side has ju s t been pushed.

The W ebots s im ulation shell (M iche l, 1998) w as used to im plem ent th is task. T his is a

th ree-d im ensional s im ulation too l w ith a good g raph ical interface to d isplay the

sim ulation results. U sing W ebo ts, robo ts equ ipped w ith actuators and sensors for

detecting the box and o b stac les and a set o f behav io u r m odules that m ap sensor inputs

to actuator ou tpu ts at an e n v iro n m en t con ta in ing a long box and obstacles have been

m odelled.

120

The tw o experim en ts pe rfo rm ed in (P arker, 2001) w ere repeated in th is investigation

for com parison . In the first ex p e rim en t, tw o robots co-operate to find a box and push

it across the room w ith no o b s tac les in the environm ent. The second experim ent

differs by add ing an obstac le , loca ted at one o f the com ers, that obstructs one o f the

robots to study how the o th e r robo t d y nam ically reselects its actions in response to

changes in the m ission situa tion .

At the start, the robots are s itu a ted random ly in the env ironm ent (figure 5.3(a)) and

they begin to locate the box . A fte r b o th o f th em have reached the box (figure 5.3(b)),

it is assum ed that the robo t at the left s tarts to push first (figure 5.3(c)). T he box then

needs to be pushed from the rig h t s ide , so the robo t on the righ t starts to push and

broadcasts that action to the ro b o t o n the left. D uring the expected tim e for that action,

the robot on the left m o n ito rs the pe rfo rm an ce o f its team m ate. T he p rocedure then

repeats itself. F inally , the robo ts co m p le te the task (figure 5.3(d)). In the second

experim ent, one o f the ro b o ts is s tu ck b eh ind an obstacle added to the environm ent

w hile the o ther reaches the b o x (figu re 5 .3(e)). B ecause there is no con tribu tion from

the o ther robo t (sensors read in g s are u nchanged and no m essages are received), the

robot that reached the box s ta rts to p u sh it a t one end (figure 5.3(f)). It th en m oves to

the o ther end to push (figure 5 .3(g)). It co n tin u es its back and forth m ovem ents (figure

5.3(h)), execu ting the task s o f p u sh in g the left end o f the box and push ing the right

end o f the box for as long as it fails to h ear that ano ther robo t is perform ing the

pushing task at the o ther end o f the box .

121

Figure 5.3(a): Initial env ironm ent (two robots random ly situated in an environm ent w ith a long box)

1 2 2

Figure 5.3(b): T he robots reach the box

123

' — \

Fita EiM Simulation

+ * *

Figure 5.3(c): T he left robot starts to push the box

12 4

Figure 5.3(d): B oth robots successfully com plete their m ission

125

Fto £ « Simulation________________________________

O a U O ► M (uoi Bm« _l

rrtaT inHtlp |

Obstacle

* ♦ *| Salect Object]

Figure 5.3(e): O nly one robot reached the box because o f theobstacle

126

FiW E<M Simula! ion Hold I

[ J Q H 0 I M » |ooi 8 m,

+ # * FF O *

Figure 5.3(f): T he robot starts to push from the left side

1 2 7

Fit Ed* Simulation

Figure 5.3(g): T he robot m oves to the other end to push

12 8

Figure 5.3(h): The robot continues the back and forth m ovem ent at the left-right alternating pushing

129

5.7. Summary

The know ledge-based arch itec tu re p resen ted in th is chapter w as used to create a robot

team that can perform m issions o v e r long periods, even w hen the env ironm ent or the

robot team itse lf changes. A n im p o rtan t com ponent o f these robotic system s is a

control strategy that enab les the robo ts to adapt their actions throughout the m ission

w ithout hum an in tervention .

Since the robot team m em bers are con tinually m onitoring the perform ance o f their

team m ates and updating the perfo rm an ce m easurem ents accordingly, the response to

im proved o r degraded capab ilities is au tom atic regard less o f the m ission length. The

results o f sim ulation experim en ts sh o w that the robot team is able to achieve adaptive

co-operative control desp ite dynam ic changes in the env ironm ent and variation in the

capabilities o f the team m em bers.

1 3 0

Chapter 6

Mobile Robot Hardware and Experiments

6.1 Preliminaries

The original objective w as to develop a research platform , including both softw are

and hardw are com ponents, for the eva lua tion o f control algorithm s for m ulti robot

system s. The developed softw are shou ld enable users to develop their ow n control

algorithm s. These algorithm s cou ld th en be tested both in sim ulation environm ents

and also on real m obile robots. T he deve loped hardw are should enable the creation o f

m ultiple autonom ous m obile robo ts w ith the fo llow ing requirem ents:

• Extendibility - the robot hardw are m ust a llow future expansion.

• C om patibility - the robo t design m ust be based on com m ercially available

standard com ponents.

The rem ainder o f the chap ter is o rgan ised as follow s. The construction o f a team o f

m obile robots is covered in S ection 2. E xperim ents are discussed in section 3. The

experim ents w ere devised to dem onstra te practical im plem entations o f som e o f the

ideas proposed in prev ious chapters.

131

6.2 Construction of a team of mobile robots

Sm all radio-controlled toy cars and a sm all rad io-contro lled toy tank w ere adapted to

provide the m echanical struc tu res for the m obile robots and m oving target,

respectively (see F igure 6.1). T he toy cars and the tank w ere driven by sm all dc

m otors and w ere capable o f a m axim um speed o f 3 m /s and 2 m /s respectively. B oth

types o f toys are inexpensive (costing under £10 each) and readily available. This

contributes to achieving the com patib ility and cost requirem ents.

The control system for the robo ts and ta rg e t w as purpose designed for th is application

as the existing rad io-operated con tro llers in the toy cars and toy tank w ere not

suitable. The control system for the ro b o ts com prises a m ain board and sensor board.

The heart o f the m ain board is a P rio rity In terrup t C ontroller (PIC) o f type 18F252,

used to control the robo t/ta rget ac tua to rs and p rocess the sensor signals. A circuit

diagram for the sensor board is show n in F igure 6.2. The sensor board contains

circuitry to enable the robo ts to de tec t the targe t and other obstacles. A n em itter and

four sensors w ere designed to be inco rporated in the sensor board (F igure 6.3).

H ow ever, due to the size o f the robo ts, the d im ensions o f the control system (and thus

the main and sensor board) are very lim ited. This does not perm it the fitting o f m any

sensors. Hence, the rear facing senso r w hich w as not considered essential w as

om itted. Also, there w as no room to fit a system for the robots to detect or

com m unicate w ith one ano ther and tha t w as w hy only the dynam ic target tracking

task could be im plem ented (F igure 6 .3). H ow ever, as the em itter o f a robot is front

facing, it cannot d ifferen tiate be tw een ano ther robot and an obstacle in front o f it.

1 3 2

'UrZ-

Figure 6.1 (a): The mobile robots

133

Figure 6.1 (b): The target

1 3 4

Back o f the robot

T r a n s m it t in g Led

Infra-RedReceiver

Front o f the robotSensor board

Direction o f movement

F igure 6 .2 : R obo t sensors board

1 3 5

Ddto1 /D

7 «sr

Motors correctoroc

( } l ’

icn

7 P M n « rt> W W » M <

F igure 6.3: S chem atic d iag ram for the sensors board

1 3 6

H ow ever, a robo t can d e tec t an o th e r ro b o t approaching it from either side o f it

because one o f the side sen so rs o f the first robo t will be activated by the signal

em itted by the approach ing robo t. A robo t can also d istinguish betw een a target and

ano ther robot located on e ither side o f it because o f the d ifferen t signals they em it.

This help the robots to co o rd in a te th e ir m o v em en ts based on the d irec tion o f m otion

o f their team m ates and that o f the targe t.

Tw o factors restricted the ch o ice o f th e in frared rece ivers and transm itting LED s: the

size o f the robots and the cu rren t con su m p tio n . B oth the transm itting LED and

receivers operate at the sam e freq u en cy signal chosen at 38 kHz. For obstacle

detection, the robot uses the tra n sm itte r to send a signal w hich w ould be reflected and

detected by one o f the rece iv e rs i f th e re is an obstacle near by. O bstacles can be

detected as far as 30 cm and the an g le o f the de tec tion is approxim ately 45 degrees.

The target could be sensed from a d istan ce o f lm .

To d ifferen tia te be tw een ta rg e t and ob stac le , tw o d ifferen t codes are used in the

em itted signal one for ta rg e t d e te c tio n and the o ther for obstacle detection. F igure

6.4(a) show s the signal e m itted by the ta rg e t. A square w ave at 38 kH z is em itted for a

duration o f 0.8 m s and th en s to p p e d fo r 8 m s. F igu re 6 .4(b) show s the signal em itted

by the robot for obstac le d e tec tio n . T h is is a lso a square w ave signal at 38 kH z, but

em itted for a du ration o f 1.4 m s and th en stopped for 18 ms. These values w ere

determ ined experim en tally . T o find the range for obstacle and target detection,

1 3 7

0 8ms8 m s

(a) S ignal from T arg e t

<----- 18ms ►I 4ms

(b) S ignal from ro b o t to de tec t obstacles

F igure 6.4: T he em itted signal (a) fo r the target (b) for the obstacles

1 3 8

experim ents w ere conducted under d iffe ren t conditions and the output o f the PIC

con tro ller m easured. The d e tec tion ranges w ere used by the control program s to

enable the robo ts to m ove safely and p e rfo rm their tasks successfully.

6.3 Experiments and Discussion

E xperim ents w ere run w ith tw o o r th ree robo ts, d ifferent obstacles and one target.

Each experim en t w as perfo rm ed six tim es and the resu lts are averaged. The first set o f

experim en ts analysed how vary ing the n u m b er o f robo ts affected the tim e required for

track ing the target. This experim en t to o k p lace in a lim ited arena containing one target

and no obstacle. T he average track in g tim e versus the num ber o f robots w as noted.

T he second set o f experim en ts d iffe red from the first set only by the addition o f

obstacles in the arena. A gain , p e rfo rm an ce w as analysed relative to the num ber o f

robo ts perform ing the task.

F igures 6 .5(a) show s one o f the e n v iro n m en ts before the experim ent started. This

con tained tw o robots in one co rner, and one target in the opposite corner. Figure

6 .5(b) dep icts an in term ediate stage o f ta rg e t tracking. Figure 6.5(c) show s the final

stage w hen the robo ts have co o p e ra ted and cap tured the target. F igures 6 .6(a), 6.6(b)

and 6.6(c) dem onstra te the sam e scenario for three robots. Figures 6.7 (a), 6 .7(b) and

6 .7(c) show the sam e scenario w ith the add ition o f obstacles.

Even though the perfo rm ance o f the ro b o ts in som e trials w as not as expected (the

robo ts spent too long to track and cap tu re the target), this m ight be due to the narrow

beam angle o f the em itters , so that it w as difficult for the robots to find the target.

1 3 9

H ow ever, the robo ts m anaged in som e tria ls to track and capture the target even in

c lu ttered env ironm ents. It w as found th a t the tim e required to track and capture the

target using th ree robots w as app ro x im ate ly 2 m inutes. W ith only tw o robots, the

required tim e w as abou t 4 m inutes. In the case w here obstacles w ere included, the

tim e w as 7 m inutes. The track ing tim e can be reduced by increasing the num ber o f

robo ts o r the speed o f the robo ts. H o w ev er the speed o f the robo ts w as kept slow

(0.5 m /s) to give them tim e to respond to the signal em itted from the infrared em itters.

D ue to the lim ited capab ilities o f the ro b o ts as p rev iously m entioned in addition to the

robo ts are light and does not have en o u g h fo rce, the box push ing task could not be

im plem ented in real.

1 4 0

Figure 6.5(a): Initial environm ent (one target and two robots)

141

Figure 6.5(b): In term ediate stage w ith robots tracking the target

1 42

Figure 6.5(c): Final stage w ith the robots having captured the target

143

Figure 6.6(a): Initial env ironm en t (one target and three robots)

1 44

Figure 6.6(b): In term ediate stage w ith robo ts tracking the target

145

m

Fieure 6.6Tc^: Final staee w ith the robots havine cantured the tareet

1 4 6

1 . fc—■ .1 *'.

Figure 6.7(a): Initial environm ent (one target, different obstacles,and three robots)

147

Figure 6.7(b): Interm ediate stage with robots tracking the target

148

Figure 6.7(c): Final stage w ith the robo ts having cap tured the target

1 4 9

6.3 Summary

This chap ter has focused on the d es ig n and construc tion o f a team o f m obile robots

for tracking and cap tu ring a d ynam ic targe t.

E lectronic contro l and sensing c ircu its for d riv ing th ree sm all rad io -con tro lled cars

and a toy tank w ere adap ted to act as th ree m obile robo ts and a m oving target,

respectively . E lectronic c ircu itry w as fitted to these veh ic les to enable them to detect

obstacles, signal their p resence (in the ca se o f the targe t and robo ts approach ing the

sides o f ano ther robot) and d e tec t the ta rg e t (in the case o f the robots).

E xperim ents show ed that the ro b o ts su ccessfu lly m anaged to track and cap tu re the

target. H ow ever, due to the lim ited n u m b er o f em itte rs installed , the perfo rm ance o f

the robots in som e trials w as no t sa tisfac to ry . A s expected , the m ore robo ts that

tracked a dynam ic target, the sh o rte r w as the tim e requ ired to cap ture it.

1 5 0

Chapter 7

Conclusions and Further Work

T his chap ter sum m arises the m ain c o n trib u tio n s m ade and the conc lu sions reached in

this w ork and proposes top ics for fu rth e r investiga tion .

7.1. Contributions

1 - A m od ifica tion to the su b su m p tio n robo t con tro l a rch itec tu re has been p roposed to

enab le the control o f m u ltip le ro b o ts u sin g the co llective behav io u r resu lting from

indiv idual sensor-based b eh av io u rs .

2- A fuzzy logic techn ique has been d ev e lo p ed to enab le the reso lu tion o f conflic ts

betw een con trad ic to ry b e h a v io u rs by p ro p o sin g an ac tion that represen ts the

consensus am ong the b eh av io u rs and th a t best sa tisfies the decision ob jec tives that

they encode.

3- A know ledge-based so ftw are a rc h itec tu re has been im plem ented for coopera ting

m obile robots to update th e ir b e h a v io u rs based on know ledge acqu ired on-line.

4- A group o f low cost m in ia tu re m ob ile robo ts has been developed to enab le som e

o f the p roposed ideas to be d em o n stra ted .

151

7.2. Conclusions

1. T his w ork has p roposed an ap p ro ach to con tro lling m ultip le robots tha t involves

the use o f co llective b eh av io u r resu ltin g from several sensor-based behaviours.

The influence o f en v ironm en ta l factors and the nu m b er o f robo ts on the

perfo rm ance o f the group in a d y n am ic targe t-track ing task has been analysed . As

w ould be expected , increasing the n u m b er o f robots reduced the tim e requ ired to

track the target. H ow ever, ro b o t co llis io n and in terference tended to deg rade the

perfo rm ance. C ontinually ad d in g m ore robo ts therefore d id n o t p roduce a

p roportional increase in p e rfo rm an ce .

2. The use o f fuzzy logic enab led the reso lu tion o f conflic ts be tw een con trad icto ry

behav iours by se lec ting an ac tion th a t rep resen ts the consensus am ong the

behav iours and that best sa tisfies the d ec is io n ob jec tives encoded in them .

3. The p roposed co-operative robo t a rch itec tu re has been show n to a llow robot team s

to perform real-w orld m issio n s o v e r long periods, even w hile the env ironm en t o r

the robotic team itse lf chan g es. A n im portan t com ponen t is the con tro l strategy

that enab les the robots to ad ap t th e ir ac tions th roughou t a live m ission w ithout

hum an in tervention. T he im p ro v em en t in team perfo rm ance w as achieved by

updating the con tro l o f the robo ts based on know ledge acquired on-line. Since the

robo t team m em bers con tin u a lly m o n ito r the perfo rm ance o f the ir team -m ates and

update the perfo rm ance m easu res accord ing ly , the response to im proved o r

degraded capab ilities is au tom atic , regard less o f m ission length. The resu lts show

that the robot team is ab le to ach ieve adaptive co-operative contro l despite

1 5 2

dynam ic ch an g es in the e n v iro n m en t and variation in the capab ilities o f the team

m em bers.

7.3. Further W ork

1- T he o u tpu t o f the fuzzy log ic ru les is no t optim al because the rules and

m em bersh ip functions are d ev e lo p ed heuristically . T he rules and m em bership

functions cou ld be o p tim ised by lea rn ing based on som e advanced search m ethods

such as genetic a lgo rithm s.

2- T he deve loped robo t team is o n ly ab le to track one target at a tim e. For m ultiple

targe ts track ing , a task a llo ca tio n a lgo rithm is required to d irect an appropriate

num ber o f robo ts to track each targe t.

3- T he idea o f hav in g robo ts to learn h o w to accom plish a task , ra ther than being told

exp lic itly is an ap pealing one. It seem s easie r and m uch m ore intu itive for the

p rog ram m er to specify w h a t the ro b o t shou ld do, and then let it leam the fine

details o f h o w to do it. A tec h n iq u e such as rein forcem ent learning is required for

op tim ising the in te rac tion w ith an e n v iro n m en t o r contro l o f a system .

4- The robots deve loped in th is w ork are few in num ber and very sim ple due to cost

constra in ts . I f a la rg e r b u d g e t is ava ilab le , larger team s o f m ore advanced robots

cou ld be b u ilt to enab le m ore co m p lex tasks to be attem pted.

153

Publications

1. Pham , D. T. and A w adalla , M . H ., (2002). Fuzzy logic based behaviour

coo rd ina tion in m u lti-ro b o t sy stem for dynam ic target tracking. The Third CIRP

Int. S em in ar on In te lligen t C o m p u ta tio n in M anufacturing E ngineering (IC M E

2002), Isch ia (N ap les), Italy , pp . 551-556 .

2. Pham , D. T. and A w ad alla , M . H ., (2002). N euro-F uzzy B ased A daptive

C oopera tive M obile R obo ts . In the Proc. o f IE C O N ’02. Sevilla, Spain, pp. 2962-

2967

3. Pham , D. T. and A w ad alla , M . H ., (2004). Fuzzy logic based behaviour

coord ina tion in m u lti-ro b o t sy stem . Proc. Instn M ech. Engrs Vol. 218 Part B: J.,

p p . 583-598.

1 5 4

References

A bdessem ed , F., B en m ah am m ed , k. and M onacelli, E. (2004). F uzzy-based reactive

co n tro lle r for a n o n -h o lo n o m ic m obile robot. In R obotics and A utonom ous

S ystem s, V ol. 47 , N o. 1, pp . 31-46 .

A costa, C .A . and H u, H ., (2003a). G o a ls and A ctions: L earn ing by Im itation. Proc. o f

A ISB 2003 C onven tion : C o g n itio n in M achines and A nim als, U niversity o f

W ales, A berystw y th , U K , pp . 112-115.

A costa, C.A . and Hu, H ., (2 003b ). R obo tics Societies: E lem ents o f L earn ing by

Im itation , Proc. o f IA S T E D Int. C onf. on A rtificial In telligence and

A pp lica tions, Innsb ruck , A u stria , pp . 315-320.

A lam i, R., F leury , S., H errb , M ., In g ran d , F., and R obert, F., (1998). M ulti-robo t co ­

opera tion in the M A R T H A p ro jec t. IE E E R obotics and A utom ation M agazine,

V ol. 5, N o . l , pp 36-47 .

A ndrew , H., L o tspeich , D ., H off, J ., and G eorge, A., (1996). R apid rein forcem ent

learn ing for reac tive co n tro l p o licy design . Proc. o f the W orld C ongress on

N eural N e tw o rk s , San D iego , C alifo rn ia , pp. 18-26.

A ram bula , F. and C astaneda , M .A . (2004). A utonom ous robot navigation using

adap tive po ten tia l fie lds. M athem atica l and C om puter M odelling , Vo. 40,

Issues 9 -10 , pp . 1141-1156.

1 5 5

A rkin , R .C . and M ackenzie , D ., (1994). T em poral coord ination o f perceptual

a lgo rithm s for m ob ile robo t nav iga tion . IEEE T ransactions on R obotics and

A u tom ation , V o l.10, N o .3, pp. 276 -286 .

A rkin , R .C ., (1990). In teg ra ting beh av io u ra l, percep tual and w orld know ledge in

reactive nav igation . Jou rna l o f R o b o tics and A utom ation System s, V ol. 6,

pp. 105-122.

A rkin , R .C ., (1999). B eh av io u r-b ased R obo tics . M IT Press.

A rkin , R .C ., B alch , T „ and N itz , E ., (1993). C om m unica tion o f behav ioural state in

m ulti-agen t retrieval tasks. P roc. o f 1993 IEEE Int. Conf. on R obotics and

A utom ation. A tlan ta , G A , V ol. 3, pp . 588-594 .

A sam a, H ., O zak i, H ., M a tsu m o to , A ., Ish ida , Y., and Endo, I., (1992). D evelopm ent

o f task assig n m en t sy stem u s in g com m un ica tion for m ultip le autonom ous

robots. Jou rnal o f R o b o tics and M echatron ics, V ol. 4 , N o. 2, pp. 122-127.

B alch, T. and A rkin , R .C ., (1994). C om m un ica tion in reactive m ulti-agen t robotic

system s. A u to n o m o u s R o b o ts , V o l . l , N o . l , pp. 1-25.

Bay, J. S. and U nsal, C ., (1994). G roup dynam ics in m any-robo t system s. IEEE

S ym poium on In te llig en t C on tro l, C o lum bus, O hio, pp . 249-254.

1 5 6

B eckers, R ., H o llan d , O .E ., and D eneubourg , J.L ., (1994). F rom local actions to

g lobal tasks: s tigm ergy and co llective robotics. In Proc. o f the Fourth

W orkshop on A rtific ia l L ife , B oston , pp. 181-189.

B erenji, H. R. and K hedkar, P., (1992). L earn ing and tun ing fuzzy logic controllers

th rough re in fo rcem en ts. IE E E T ransac tions on N eural N etw orks, V ol.3 , N o.5,

pp. 724-740 .

B oehringer, K ., B row n , R ., D onald , B ., Jenn ings, J., and R us, D., (1995). D istributed

robo tic m an ipu la tion : E x p e rim en ts in m inim isation . In Proc. o f the Fourth Int.

S ym poium on E x p erim en ta l R obo tics , S tandford , pp. 11-252.

B onabeau , E., D origo , M ., and T h e rau laz , G ., (1999). Sw arm intelligence: from

natural to a rtific ia l sy stem s. S an ta Fe Institu te S tudies in the Sciences o f

C om plex ity . O xfo rd U n iv e rs ity P ress.

B rooks, R .A ., (1986). A R o b u st L ayered C on tro l System for a M obile R obot. IEEE

Journal o f R o b o tics and A u to m atio n , V ol. R A -2, N o. 1, pp. 14-23.

B uschka, P., S affio tti, A. and W asik , Z ., (2000). Fuzzy L andm ark-B ased L ocalization

for a L egged R obot. P roc. o f the IE E E /R SJ Intl. Conf. on In te lligen t R obots

and S ystem s (IR O S ), T ak am atsu , Japan , pp. 1205-1210.

1 5 7

C aloud , P., C h o i, W ., L a tom be , J .C , Pape, C .L ., and Y im , M ., (1990). Indoor

au to m atio n w ith m any m ob ile robo ts. In the Proc. o f the IEEE Int. W orkshop

on In te llig en t R obo ts and S ystem s, T suchiura , Japan , pp. 62-72.

C han tem argue , F., and H irsb runner, B., (1999). A collective robotics application

based on em ergence and se lf-o rgan isa tion . In Proc. o f Fifth Int. Conf. for

Y oung C o m p u ter S c ien tis ts , N an jin g , C hina, pp. 1-8.

C oradesch i, S., D riankov , D ., K a rlsso n , L., and Saffiotti, A., (2001). Fuzzy

A nchoring . Proc. o f the IE E E Intl. C onf. on Fuzzy System s, M elbourne,

A ustra lia , pp. 111-114.

C ragg, L. and H u, H. (2005). A p p lica tio n o f re in fo rcem en t learning to a m obile robot

in reach ing rech arg in g s ta tio n opera tion , Pro. o f IPR O M S 2005 V irtual

C onference , h ttp ://c o n fe ren c e .ip ro m s.o rg .

D em irli, K. and M olh im , M . (2004 ). F uzzy dynam ic localization for m ob ile robots.

Fuzzy Sets and S ystem s, V ol. 144, Issue 2, pp. 251-283.

D eneubourg , D ., G oss, S., S and in i, G ., Ferrari, F., and D ario, P., (1990). Self-

o rg an is in g co llec tio n and tran spo rt o f ob jectives in unpredictab le

env ironm en ts . In Japan -U .S .A . S ym posium on F lexible A utom ation , K yoto,

Japan , pp. 1093-1098.

1 5 8

D eneubourg , J .C ., G oss, S., F ranks, N ., Sendova, A ., F ranks, A ., D etrin , C., and

C hatier, L .. (1991). T he dynam ics o f co llective sorting: R obots-like ant and

an t-lik e robo t. In J.A . M ay er and S.W . W ilson, editors, S im ulation o f

A dap tive B ehav iour: from A n im als to A nim ates, pp. 356-365, M IT Press.

D jakov, D .B ., (1996). A n in te llig en t senso r for d irec tion detection. M Sc thesis,

In te lligen t sy stem s labo ra to ry , M anufac tu ring E ngineering C entre , C ard iff

U niversity .

D origo , M ., M an iezzo , V ., and C o lo m i, A ., (1996). T he ant system : O ptim isation by a

co lony o f co -o p e ra tin g agen ts . IE E E T ransactions on System s, M an, and

C y b em etics-P art B , p p .2 9 -4 1 .

D rogoul, A. and Ferber, J., (1992). F rom to m thum b to the dockers: som e experim ents

w ith fo rag ing robo ts. In P roc. o f the Second Int. C onf. on S im ulation o f

A dap tive B ehav iour, H o n o lu lu , H aw aii, pp . 451- 459.

D uhaut, M ., A sam a, H ., an d S uzuk i, S., (1998). The R oboC up physica l agent

cha llenge : P hase I. A p p lied A rtific ial In telligence (A A I), V ol. 12,

pp. 251-265 .

E lfw ing, S., U ch ine , E., D oya, K ., and C hristensen , H. (2004). M ulti-agen t

re in fo rcem en t learn ing . In IR O S -04 , IE E E /JSR , pp. 3164-3169.

1 5 9

Fong, T ., N o u rb ak h sn , I. and D au ten h ah n K ., (2003). A survey o f socia lly interactive

robots. R obo tics and A u to n o m o u s System s, V ol. 42 , pp. 143-166.

Franks. N ., (1989). A rm y an ts: A co llec tiv e in te lligence. A m erican Scientist, V ol. 77,

p p . 139-145.

G asser. L., R ouquette N ., H ill, R. and L ieb, J., (1989). R epresen ting and using

o rgan isa tional k now ledge in D A I system s. In L es G asser and M ichael N.

H uhns ed ito rs, D is tribu ted A rtif ic ia l In te lligence , P itm an, V ol. 2, pp. 55-78.

G hanea-H ercock , R. and B arn es, D .P ., (1996). A n evo lved fuzzy reactive control

system for co -o p era tin g au to n o m o u s robots. In Proc. o f the Fourth Int. Conf.

on S im ulation o f A d ap tive B ehav iou r, (S A B ), C ap C od, pp. 599-607.

G oodridge, S.G . and L uo, R .C ., (1994). Fuzzy behav io u r fusion for reactive con tro l o f

an au tonom ous m obile robo t: M A R G E . In Proc. o f the IEEE Int. C onf. on

R obo tics and A u to m atio n , San D iego , C A , pp . 1622-1627.

Hoff, J. and B ekey, G ., (1995). A n a rch itec tu re for behav iour coord ination learning.

In IEEE Int. C onf. on N eu ra l N etw orks, Perth , A ustralia , V ol. 5,

pp. 2375-2380 .

H offm ann, F. and P fister, G ., (1997). E vo lu tionary D esign o f a Fuzzy K now ledge

B ase for a M ob ile R obot. Int. Journal o f A pproxim ate R eason ing (E lsevier,

N o rth H o lland ),vo l. 17, no . 4 , pp. 447- 469.

1 6 0

H u. H. and G an , J .Q ., (2005). Senso rs and da ta fusion algorithm s in m obile robotics,

R esearch rep o rt C S M -422 , U n ivers ity o f E ssex , ISSN 1744-8050.

Hu, H., K elly , I. and K eating , D., (1998). C oord ination o f m ultip le m obile robots via

co m m un ica tion . In P roc. o f S P IE ’98, M obile R obots XIII C onf., B oston,

pp .94 -103 .

Hu, H. and G u, D. (2005). H ybrid L earn ing A rchitecture for Fuzzy C ontrol o f

Q uadruped W alk ing R obo ts , Int. Jou rnal o f In te lligen t System s, Vol. 20, No.

2 , p p . 131-152.

H u, H. and G u, D. (2004). T ea ch in g robo ts to coord inate their behaviours, Proc. o f

IEEE Int. C onf. on R o b o tics and A u tom ation , R iverside H ilton & T ow ers,

N ew O rleans, LA.

H uber, M . and D urfee , E., (1993 ). O bservational uncertain ty in p lan recognition

am ong in te rac ting ro b o ts . In P roc. o f the 1993 IJC A I W orkshop on

D ynam ica lly In te rac tin g R obo ts , C ham bery , France, pp. 68-75.

H um phrys, M ., (1997). A c tio n se lec tion m ethods using rein forcem ent learning. PhD

thesis, C o m p u ter L abo ra to ry , U n ivers ity o f C am bridge.

H uosheng , 1L, K eating , I.D . and V inagre, D. (1998). “C oord ination o f m ultip le

m ob ile robo ts v ia c o m m u n ica tio n ” . In Proc. o f SPIE. M obile R obots XIII and

In te llig en t T ran sp o rta tio n System s, B oston, M assachusetts, pp. 94-103.

161

Ijspeert, A .J., M artin o li A ., B illa rd A. and G am bardella , L .M ., (2001). C ollaboration

th ro u g h the ex p lo ita tio n o f local in teractions in au tonom ous collective

robo tics: the s tick pu lling experim en t, A utonom ous R o b o ts .V o l.i l , N o.2 , pp

149-171, K luw er A cadem ic .

K eller. J. M ., Y ager, R. R. and T ahan i, H. (1992). N eural N etw ork Im plem entation

o f F uzzy L ogic. F uzzy Sets and System s, V ol.45, p p .1-12.

K hatib , O ., (1986). R e a l- tim e o b stac le avo idance for m anipu la to rs and m obile

robo ts. T he Int. Jo u rn a l o f R o b o tics R esearch , V ol.5 , N o .l , pp. 90-98.

K irchner, W .H . and T o w n e, W .F ., (1994 ). T he sensory basis o f the ho n ey b ee’s dance

language. Scien tific A m erican , V ol. 270 , N o. 6, pp. 74-80.

K orf, R., (1992). A sim p le so lu tion to pu rsu it gam es. In w ork ing papers o f the 11th

Int. W orkshop on D is trib u ted A rtific ia l In te lligence , B oston , pp. 183-194.

K osecka, J. and B ajcsy , R ., (1993 ). D iscre te even t system s for au tonom ous m obile

agen ts. In P roc. o f In te llig en t R obo tic System s ’93, Z akopane, pp. 21-31.

K ristensen , S., (1996). S en so r p lan n in g w ith bayesian decision analysis. PhD thesis,

D epartm en t o f M ed ical In fo rm atics and Im age A nalysis, A alborg U niversity .

1 6 2

K ristina, L. and A ram , G ., (2002). M athem atica l M odel o f Foraging in a G roup o f

R obots: E ffec t o f In te rfe rence . A u tonom ous R obots, V o l.13, N o. 2, pp. 127-

141,2002.

K ube, C. R. A nd Z hang , H ., (1992). C o llec tive robotic in telligence. In Proc. o f the

Second Int. W o rk sh o p on S im ula tion o f A daptive B ehaviour, H onolulu ,

H aw aii, pp. 460 -468 .

K ube, C. R. and Z hang , H ., (1994 ). S tagnation recovery behav iours for co llective

robotics. In 1994 IE E E /R S J/G I Int. C onf. on In telligen t R obots and System s,

V ol. 3, p p .1883-1890.

K ube. C. R. and B onabeau , E., (2000 ). C o opera tive transport by ants and robots.

R obo tics and A u to n o m o u s S ystem s, V ol. 30, pp. 85-101.

L arsson , J., B roxvall, M . and S affio tti A. (2005). A N av iga tion System for A utom ated

L oaders in U nderg ro u n d M ines. P roc. o f the 5th Int. Conf. on Field and

Serv ice R obo tics (F S R -05 ). P o rt D oug las, A ustralia , pp . 112-124.

Lee, C. C. (1990a). F uzzy L og ic in C on tro l System s: Fuzzy Logic C ontro ller Part I.

IEEE T ran s. S ystem s, M an . and C ybem , V ol.20, N o.2 , pp .404-417 .

Lee, C. C. (1990b). F uzzy L ogic In C on tro l System s: Fuzzy Logic con tro ller Part II.

IEEE T ran s. S ystem s, M an. and C ybem , V ol.20, N o.2 , p p .514-527.

1 63

L evy, R. and R osenschein , J., (1992). A gam e theoretic approach to the pursu it

p ro b lem . In w ork ing papers o f the 11th Int. W orkshop on D istributed

A rtific ia l In te llig en ce , B oston , pp. 195-213.

Lin, C. and M on, Y., (2004). H ybrid adap tive fuzzy con tro llers w ith application to

robo tic system s Fuzzy S ets and System s, V ol. 1 3 9 ,N o .l , pp. 151-165.

Lin, C .T . and Lu, Y .C ., (1996 ). A N eura l fuzzy system w ith fuzzy superv ised

learn ing , IEEE T ran sac tio n s . S ystem s, V ol. 26 , N o. 5, pp .745-753.

M ackenzie , D .C , A rkin , R .C . and C am ero n , J., (1997). Specification and execu tion o f

m ulti-agen t m issions. A u to n o m o u s R obots, V ol. 4, N o. 1, pp. 276-292.

M acL ennan , B ., (1991). S y n th e tic e tho logy : A n approach to the study o f

com m un ica tion . In P roc. o f the Second in terd iscip linary w orkshop on

syn thesis and s im u la tio n o f liv in g sy stem s, R edw ood C ity, pp . 631-658.

M aes, P., (1989). H ow to do the rig h t th ing . C onnection Science Journal, V ol. 1, No.

3, pp. 291 -323 .

M am dani, E. H. (1974 ), A p p lica tio n o f F uzzy A lgorithm s for C ontro l o f S im ple

D ynam ic P lan t, P roc. o f IE E , V ol. 121, N o. 12, p p .1585-1588.

1 6 4

M artinoli A., Ijspeert A .J., and G am barde lla L .M ., (1999b). A probab ilistic m odel for

u n d ers tan d in g and co m p arin g co llective aggregation m echanism s. Proc. o f the

Fifth E u ropean C onf. on A rtific ia l L ife (E C A L 99), B erlin , pp. 575-584.

M artino li, A. and M ondada, F., (1995). C ollective and co-operative behaviours:

b io log ica lly in sp ired ex p e rim en ts in robotics, P re-prin ts o f the fourth Int.

S ym posium on E x p erim en ta l R obo tics, IS E R ’95, S tanford , CA , pp. 2-7.

M artino li, A. and M ondada, F., (1998 ). P robab ilistic m odelling o f a b io -insp ired

co llective ex perim en t w ith real robo ts. In Proc. o f the Fourth Int. Sym posium .

O n D istribu ted A u to n o m o u s R obo tic System s D A R S-98, K arlsruhe, G erm any,

pp. 289-308.

M artino li, A., (1999). S w arm in te llig en ce in au tonom ous co llective robotics: from

too ls to the ana ly sis and syn thesis o f d istribu ted contro l strategies.

U npub lished docto ra l d isse rta tio n , PhD T hesis N o 2069, EPFL .

M artino li, A ., F ranzi, E., and M atthey , O . (1997a). T ow ards a reliable set-up for b io ­

insp ired co llec tive ex p e rim en ts w ith real robots. In Proc. o f the F ifth Int.

S ym posium . O n E x p erim en ta l R obo tics ISER -97, B arcelona, C atalonia, pp.

597-608.

M artino li, A ., Ijspeert, A. J. and M ondada, F., (1999 a). U nderstand ing collective

ag g reg a tio n m echan ism s: from probab ilistic m odelling to experim ents w ith

1 6 5

real robo ts . Inv ited paper, S pecial Issue on D istributed A utonom ous Robotic

S ystem s, Jou rna l o f R obo tics and A u tonom ous System s, V ol. 29 , pp. 51-63.

M artino li, A ., Y am am oto , M ., and M ondada, F., (1997b). O n the m odelling o f b io ­

insp ired co llec tive ex p erim en ts w ith real robots. In Proc. o f the fourth

E uropean C onf. on A rtific ia l L ife E C A L '97 , B righton, U K , pp .432-441.

M ataric , J., (1992). D esig n in g em erg en t behaviours: from local in teractions to

co llective in te lligence . In th e second Int. C onf. on S im ulation o f A daptive

B ehav iour, H ono lu lu , H aw aii, pp . 432-441 .

M ataric , M ., N ilsson , M ., and S im sarian , K., (1995). C o-operative m ulti-robo t box

pushing . In the Proc. o f the 1995 IE E E /R S J Int. Conf. on In telligen t R obots

and System s, IE E E C o m p u te r S oc ie ty , pp . 556-561.

M ataric , M .J., (1996). L earn in g in m u lti-ro b o t system s. In A daptation and L earn ing in

M u lti-A gen t S ystem s, G erh ard W eiss and Sandip Sen, editors. L ecture N otes

in A rtific ial In te llig en ce (L N A I), 1042, S pringer-V erlag , p p .152-163.

M ataric , M .J., (1998). C o o rd in a tio n and learn ing in m ulti-robot system s. IEEE

In te lligen t System s, pp .6 -8 .

M elhu ish , C ., H o lland , O ., and H oddell, S., (1998). C ollective sorting and segregation

in robo ts w ith m in im al sensing . In Proc. o f the Fifth Int. C onf. o f S im ulation

o f A d ap tiv e B ehav iour, Z u rich , pp. 345-363.

M ichel, O ., (1998 ). W ebots: S ym biosis be tw een virtual and real m obile robots. In

Proc. o f the first Int. C onf. on V irtual W orlds, V W ’ 98, San M ateo , CA , pp.

254 -2 6 3 , M organ K aufm ann .

M inguez, J. and M ontano , L. (2005). Senso r-based robot m otion generation in

unknow n , dynam ic and tro u b leso m e scenarios. R obotics and A utonom ous

S ystem s, V ol. 52, Issue 4 , pp . 290-311 .

M urrie ta-C id , R ., S arm ien to , A. an d H u tch inson , S. A. (2003). D istribu ted m otion

coord ina tion s tra tegy for m u ltip le nonho lonom ic m obile robots in co-operative

hun ting opera tions. R o b o tics and A u tonom ous System s, V ol. 43 , N o. 4 , pp.

234-242.

N oreils , F.R ., (1993). T o w ard a ro b o t arch itec tu re in tegrating cooperation betw een

m obile robots: ap p lic a tio n to indoor env ironm ent. Int. Journal o f R obotics

R esearch , V o l .12, N o . l , pp . 79-98 .

Pan, J., and Pack, D .J., K o sak a , A ., an d K ak, A .C ., (1995). FU Z Z Y -N A V : A v ision-

based robo t n av ig a tio n a rch itec tu re using fuzzy inference for uncertain ty

reason ing . In P roc. o f the W orld C ongress on N eural N etw orks, W ashington ,

D C , pp. 602-607 .

Parker, L. E., (2002). D is trib u ted a lgo rithm s for m ulti-robo t observation o f m ultip le

m ov ing targe ts . A u to n o m o u s R obots, V ol. 12, N o. 3, pp. 231-255.

1 6 7

Parker, L .E ., (1994). A L L IA N C E : A n arch itectu re for fault to lerance , co-operative

co n tro l o f h e te ro g en eo u s m obile robots. In Proc. o f the 1994 IE E E /R SJ/G I Int.

C onf. on In te lligen t R obots and System s (IR O S ’94), M unich , G erm any,

pp. 776-783 .

Parker, L .E ., (1995). T he e ffec t o f ac tion recognition and robot aw areness in co ­

opera tive robotic team s. Proc. o f the 1995 IEEE/R SJ Int. Conf. on In telligent

R obo ts and S ystem s (IR O S ’95 ), V ol. 1, pp . 212-219.

Parker, L .E ., (1996). O n the desig n o f behav iou r-based m ulti-robot team s. A dvanced

R obotics, V ol. 10, N o .6 , p p .547-578 .

Parker, L .E ., (1997). L -A L L IA N C E : task -o rien ted m ulti-robo t learning in behaviour-

based system s. A d v anced R o b o tics , Special Issue on Selected Papers from

IR O S '96 , V ol. 11, N o . 4 , pp . 305-322 .

Parker, L .E ., (1999). A d ap tiv e h e te ro g en eo u s m ulti-robo t team s. N euro-com puting ,

special issue o f N E U R A P '98 : N eural N etw orks and T heir A pplications, Vol.

28 , pp .75 -92 .

Parker, L .E ., (2000). L ife long adap ta tion in heterogeneous m ulti-robo t team s:

R esponse to C o n tin u a l V aria tion in Individual R obot Perform ance.

A u to n o m o u s Robots, V ol. 8, N o. 3, pp . 239-267.

1 6 8

Parker, L .E ., (2001). E valuating success in au tonom ous m ulti-robo t team s:

ex p e rien ces from A L L IA N C E architecture im plem entations. Journal o f

T h eo re tica l and E xperim en ta l A rtific ia l In telligence, V ol. 13, pp. 95-98.

Parker, L .E ., S chneider, F. and S chu ltz , A. (2005). M ulti-R obot System s: From

S w arm s to In te lligen t A u tom ata . V ol. Ill, Springer, 2005 (ISB N : 1-4020-

3388-5).

Pham , D. T. and D ayal R. P arh i, (2003 ). N av iga tion o f m ultip le m obile robots using a

neural ne tw ork and a Petri N e t m odel. Journal o f R obotica, V ol. 21, N o. 1, pp.

79-93.

P in . F.G . and W atanabe, Y ., (1994 ). N av ig a tio n o f m obile robots using a fuzzy

b eh av io u ris t app roach and c u s to m -d es ig n ed fuzzy in ference boards. R obotics,

V ol. 12, pp. 491 -503 .

P irjan ian , P. and M ataric , M ., (2000). M u lti-ro b o t target acqu isition using m ultip le

ob jec tive b eh av io u r c o o rd in a tio n . In P roc. o f the 2000 IEEE Int. C onf. on

R obo tics and A u to m atio n , San F rancisco , CA , pp. 101-106.

P irjan ian , P. and C h risten sen , H .I., (1997). B ehav iour coord ination using m ultip le-

ob jec tive d ec is io n -m ak in g . SPIE C onf. on In telligent System s and A dvanced

M anu fac tu rin g , P ittsb u rg h , P ennsy lvan ia USA, pp .78-89 .

169

Pirjan ian , P., ( 1 9 9 8 / M ultip le ob jec tive action selection & behav iour fusion using

vo ting . Ph .D thesis , F acu lty o f T echn ica l Sciences, A alborg U niversity ,

D enm ark , pp . 257-282 .

P irjan ian , P., (1999). B eh av io r C oord ination M echanism s - S tate-of-the-art.

T ech -rep o rt IR IS -99 -375 , Institu te for R obotics and In te lligen t System s,

School o f E ng ineering , U n iversity o f Southern C alifornia.

R osenb latt, J. and T horpe , C ., (1995). C om bin ing m ultip le goals in a behaviour-based

arch itec tu re IE E E C o n fe ren ce on In te lligen t R obots and System s, H um an

R obot In teraction and C o o p e ra tiv e R obots (IR O S '95), P ittsburgh , PA , V ol. 1,

pp. 1 3 6 -1 4 1 .

R osenb latt, J., (1997). T he D istrib u ted A rch itec tu re for M obile N avigation . In Journal

o f E xperim en ta l and T h eo re tica l A rtific ia l In telligence, V ol. 9, N o. 2/3,

pp .339-360 .

R osenb latt, J.K . and Pay ton , D .W ., (1989). A F ine-G rained A lternative to the

S ubsum ption A rch itec tu re for M obile R obot C ontrol. In Proc. o f the

IE E E /IN N S In te rn a tio n a l Jo in t C onference on N eural N etw orks, W ashington

D C , V ol. 2 , pp. 317-324 .

R uspin i, E .H ., (1991). O n the sem an tics o f fuzzy logic. Int. Journal o f A pproxim ate

R eason ing , V ol. 5, pp. 45-88 .

170

R ussell, R .A ., (1999). A nt trails - an exam ple for robots to fo llow ? In Proc. 1999

IEEE Int. C onf. on R obotics and A utom ation , pp. 2698-2703.

Saffio tti, A. and W asik , Z ., (2003). U sing H ierarchical Fuzzy B ehav iours in the

R oboC up D om ain. C. Z hou , D. M aravall and D. R uan (E ds) A utonom ous

R obotic System s. Springer, B erlin , pp. 235-262.

Saffio tti, A ., (1997). T he uses o f fuzzy logic in au tonom ous robot navigation : a

cata logue raisonne. T echn ica l report 2 .1 , IR ID IN A , U n iversite L ibre de

B ruxelles, 50 av. F. R osseve lt, C P 194/6, B -1050 B russels, B elgium .

Saffio tti, A ., Z um el, N .B . and R usp in i, E .H ., (2000). M ulti-robo t team coord ination

using desirab ilities. Proc. o f the six th Int. Conf. on In telligen t A utonom ous

System s (IA S), V en ice, Italy , pp . 107-114.

Schultz , A ., Parker, L. E. and S chneider, F., (2003). M ulti-R obot System s: From

Sw arm s to In telligen t A u tom ata , V ol. 2, K luw er, 2003 (ISB N : 1 -4020-1185-

7).

Sen, S., Sekaran, M ., and H ale, J., (1994). L earn ing to coord inate w ithout sharing

inform ation . In Proc. o f the T w elfth N ational Conf. o f A rtificial In telligence,

Seattle, W A, pp. 426-431 .

171

Shankir, Y „ (2001). Fuzzy logic system s and fuzzy neural ne tw orks for dynam ic

system s m odelling and contro l. PhD thesis, In telligent system s laboratory,

M anu fac tu rin g E ngineering C en tre , C a rd iff U niversity .

Sharpe, T. and W ebb, B ., (1998). S im ulated and situated m odels o f chem ical trail

fo llow ing in ants. In Proc. o f the Fifth Int. Conf. o f S im ulation o f A daptive

B ehav iour, C am bridge , L ondon , pp. 195-204.

Sossai, C ., C hem ello , G ., and Saffio tti, A ., (2000). A N ote on the R ole o f Logic in

Fuzzy Logic C ontro llers. P roc. o f the 8th Int. Conf. on Inform ation Processing

and M anagem ent o f U n certa in ty , M adrid , SP, pp. 717-723.

Steels, L., (1990). C oopera tion b e tw een d istribu ted agents th rough self-organisation .

In Proc. o f the IEEE Int. W o rk sh o p on In telligen t R obots and System s '902

V ol. 2, pp. 8-14, A m sterdam : E lsev ie r Science (N orth-H olland).

S tephens, L. and M erx, M. (1990). T he effec t o f agen t control stra tegy on the

perfo rm ance o f a D A I p u rsu it p rob lem . In Proc. o f the 1990 D istribu ted AI

W orkshop, B andera, T exas, pp . 324-331.

Stilw ell, D. and Bay, J., (1993). T o w ard the developm ent o f a m aterial transport

system using sw arm s o f an t-like robots. In Proc. o f IEEE Int. Conf. on

R obotics and A u tom ation , A tlan ta , G A ., pp. 766-771.

172

Sugeno, M. and N ish ida , M. (1985). Fuzzy C ontrol o f M odel Car. Fuzzy Sets and

System s, V o l.16, p p .103-113.

Sugeno, M. and Y asukaw a, T. (1993). A F uzzy-L ogic-B ased A pproach to Q ualitative

M odelling . IEEE T rans. F uzzy System s, V o l.1, N o .l , p p .7-31.

Surm an M .. Juser J. and Peters L., (1995). G uid ing and contro lling m obile robots w ith

a fuzzy controller. In P roc. o f the fourth IEEE Int. Conf. on Fuzzy System s

Y okoham a, Japan , p p .83-88.

T akag i. T. and M. Sugeno (1985), F uzzy Identification o f S ystem s and Its

A pplications to M odelling and C ontro l, IEEE Trans. Syst., M an. and C ybem .,

V ol.SM C -15, N o .l , p p .l 16-132.

T an, M ., (1993). M ulti-agen t re in fo rcem en t learning: Independen t vs. C o-operative

agents. In proc. o f the T en th Int. C onf. on M achine L earning, A m herst, M A,

pp. 330-337.

T arasew ich , P., and M cM ullen , P. R ., (2002). Sw arm In telligence. C om m unications

o f the A C M , Vol. 45 , N o. 8, pp. 62-67.

Touzet, C., (2000) R obot A w areness in C ooperative M obile R obo t Learning.

A utonom ous R obots, V ol. 8, N o. 1, pp. 93-105.

V aughan, R., S toey, K ., S ukhatm e, G ., and M ataric, M ., (2000a). B laz ing a trail:

insect-insp ired resource transporta tion by a robot team . In Proc. o f the fifth

173

Int. S ym posium on D istribu ted A utonom ous R obotic System s (D A R S),

K noxville , T N , pp. 111-120.

V aughan, R .T ., stoy, K.. Sukhatm e, G .S. and M ataric, M .J., (2000b). W histling in the

dark: C o-opera tive trail fo llow ing in uncertain localisation space. In Proc. o f

the Fourth Int. C onf. on A u tonom ous A gents. B oston, pp. 187-194.

W agner, I. A. and B ruckste in , A. M ., (1995). C o-operative c leaners -

a study in ant robo tics, In the Proc .o f the third F rench-Israeli Sym posium on

R obotics. H erzlia, pp. 289-308 .

W agner, I. A., L indenbaum , M . and B ruckste in , A. M ., (1998). E ffic ien tly exp loring a

continuous unknow n dom ain by an an t insp ired process. T he second Int.

w orkshop on R andom isation and A pprox im ation T echn iques in C om puter

Science, B arcelona, Spain , pp . 116-130.

W ang, L. X. and M endel, J. M ., (1993). F uzzy A daptive F ilters w ith A pp lica tion to

N onlinear C hannel E qualisa tion . IE E E T rans. Fuzzy System s, V o l . l , N o.3 ,

p p .161-170.

W asik, Z. and Saffio tti., A ., (2002). A F uzzy B ehav ior-B ased C ontro l System for

M anipulation. Proc. o f the IE E E /R S J Intl. Conf. on In te lligen t R obo ts and

System s (IR O S), L ausanne, C H , pp. 1596-1601.

174

W eiss, G., Sen, S., and E ditors, (1996). A daptation and learning in m ulti-agent

system s. In the e leven th Int. W orkshop on D istributed A rtificial In telligence,

p p . 183-194.

Y am ada, S. and Saito , J., (1999). A dap tive action selection w ithou t explicit

com m unication for m u lti-ro b o t box-pushing . 1999 IE E E /R SJ Int. Conf. on

In telligent R obots and S ystem s (IR O S ’99), pp. 1444-1449.

Y en, J., and Pfluger, N. (1995). A fuzzy logic based ex tension to P ayton and

R osenb latt’s com m and fusion m ethod for m obile robo t nav igation . IEEE

Transactions. O n S ystem s, M an , and C ybernetics, V ol. 25, N o. 6, pp. 971-978.

1 75