obstacle perception by scorpions and robotsneurokybernetik/...force on the ground to move the animal...

Obstacle perception by scorpions and robots

From biology to robotics via physical simulation

and evolving neural networks

Arndt von Twickel

Diploma Thesis

submitted in partial fulfilment of the

requirements for the degree of Diploma of Biology

at the Faculty of Mathematics and Science

University of Bonn

December 2004

Supervisors:

Prof Dr. Hans-Georg Heinzel

Department of Neurobiology

Institute of Zoology

University of Bonn

Prof Dr. Frank Pasemann

Team Intelligent Dynamics (INDY)

Fraunhofer Institute for

Autonomous intelligent Systems (AiS)

Sankt Augustin

Locomotion has not been understood well enough to build robots that autono-mously navigate through rough terrain. The current understanding of locomotionimplies a highly decentralized and modular control structure. Two experimentalapproaches, each addressing a different level of control, have been made to gain in-sight into the mechanisms of obstacle perception as an integral part of rough terrainlocomotion. In the first approach controllers were developed for single, morpholog-ical distinct legs through artificial evolution and physical simulation. The resultsshowed reflex-oscillators which inherently relied on the sensori-motor loop and ahysteresis effect. Successful coupling of six controllers, exclusively by means ofthe sensori-motor loop, showed the applicability of the modular concept. In a sec-ond approach a behavioural experiment was conducted with scorpions (PandinusCavimanus (POCOCK)) walking on a locomotion compensator and making con-tact with obstacles of different heights. The experiment showed that the scorpionsemployed their pedipalps (especially the obstacle facing one) for rhythmic grop-ing movements. No coupling of the pedipalp rhythm to the leg movement couldbe found. Further prolonged phases of exclusive hair-contact and “hair-brushing”behaviours have been observed, suggesting an important role of the pedipalps andtheir hairs in the process of obstacle detection and therefore in rough terrain loco-motion. Taken together, the results establish a basis for future integration of thetwo approaches.

Contents

1 Introduction 1

1.1 Levels of locomotion control . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Approach I: Development and analysis of locomotion controllers . . 5

1.3 Approach II: Obstacle contact in scorpions . . . . . . . . . . . . . . 6

1.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Materials and Methods 10

2.1 Simulation of walking . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.1.1 Physical simulation . . . . . . . . . . . . . . . . . . . . . . . 10

2.1.2 Morphology of the simulated robot . . . . . . . . . . . . . . 13

2.1.3 Neural control . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.1.4 Evolutionary tools and techniques . . . . . . . . . . . . . . . 19

2.1.5 Evaluation and analysis . . . . . . . . . . . . . . . . . . . . 24

2.2 Behavioural experiments with scorpions . . . . . . . . . . . . . . . . 26

2.2.1 Animals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.2.2 Locomotion compensator . . . . . . . . . . . . . . . . . . . . 28

2.2.3 Obstacles and obstacle holding device . . . . . . . . . . . . . 29

2.2.4 Lighting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.2.5 Camera setup . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.2.6 Image processing and analysis . . . . . . . . . . . . . . . . . 34

3 Results 39

3.1 Neural control of forward walking (Simulation) . . . . . . . . . . . . 39

3.1.1 Joint coordination in single legs . . . . . . . . . . . . . . . . 40

3.1.2 Single leg controllers: Examples . . . . . . . . . . . . . . . . 45

3.1.3 Single leg controllers: Mechanisms . . . . . . . . . . . . . . . 48

3.1.4 Single leg controllers: Adaptability . . . . . . . . . . . . . . 53

3.1.5 Coupling of six legs . . . . . . . . . . . . . . . . . . . . . . . 58

3.2 Obstacle detection in scorpions . . . . . . . . . . . . . . . . . . . . 59

3.2.1 Walking and turning movements on flat ground . . . . . . . 59

3.2.2 Obstacle contact and exploration . . . . . . . . . . . . . . . 60

4 Discussion 72

4.1 Simulation of neural locomotion control . . . . . . . . . . . . . . . . 72

4.2 Obstacle detection by scorpions . . . . . . . . . . . . . . . . . . . . 77

4.3 How do the results fit together? . . . . . . . . . . . . . . . . . . . . 82

5 Acknowledgements 84

6 References 85

Appendix 92

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Coordinate Transformation and calculations . . . . . . . . . . . . . . . . 94

Fold-out net to simulator coupling . . . . . . . . . . . . . . . . . . . . . . 96

Fold-out motion tracking parameters . . . . . . . . . . . . . . . . . . . . 97

Affirmation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

1 Introduction 1

1 Introduction

Numerous hexapod and octapod walking machines have been developed up to date.

Most of them are equipped with controllers that allow them to walk on an even

or simple structured terrain. As the environment becomes rougher the number of

robots that are able to traverse this new environment decreases rapidly. As far

as the author is aware of no walking machine exists that is capable of traversing

highly uneven and unstructured terrain with e.g. rocks and ditches. But exactly

here the potential advantage of legged machines can be found when compared to

wheeled ones. Partly the lack of function can be accounted for by the drawbacks

of the hardware but especially the lack of suitable control mechanisms is apparent.

Together with the unsatisfactory solutions brought forward by classical engineering

this led to an increased interest in locomotion strategies in animals where efficient

walking control under changing environmental conditions has been a prerequisite

of survival since millions of years.

Behavioural as well as electro-physiological studies on model organisms like stick-

insects and cockroaches but also studies on other arthropods like scorpions revealed

interesting mechanisms with respect to the control of locomotion. One of the most

important findings, namely the distributed and modular nature of locomotion con-

trol, was already proposed by Wendler [Wendler, 1966] who found a gliding co-

ordination between the legs of the stick insect Carausius morosus. He assumed

that one oscillator exists for each leg and that the co-ordination of all legs results

from the coupling of the single oscillators1. Further Wendler discussed the impor-

tance of sensory inputs from leg internal sense organs as well as their integration

with external sensory signals. In the following a hypothesis is presented that is

based on the points already made by Wendler: The ability of legged animals to

traverse rough terrain and therefore the ability to survive heavily depends on the

adequate perception of obstacles and the subsequent decision of how to deal with

the obstacle. Hereby the term adequate obstacle perception is seen in the sense that

an animal constantly gains enough information about the environment to make

decisions whether to avoid certain obstacles or to overcome them. This adequate

1 By bringing forward this hypothesis he heavily drew on the work done earlier by von Holstwho described interactions of multiple oscillators in fish (magnet effect and superposition)[von Holst, 1939].

1 Introduction 2

perception of obstacles requires (active) perception to take place on different levels

and in a modular manner, ranging from the low-level walking control of single joints

and legs to the integration of high-level, non locomotory sensory systems into the

generation of locomotory behaviour (see Fig. 1). To support this hypothesis two

different experimental approaches were made: The first experiment involved the de-

velopment of locomotion controllers by means of an artificial evolution of artificial

neural networks which were evaluated in a physical simulation. This approach had

the goal to explore possible mechanisms of single leg control and its applicability

to multi-legged controllers through coupling of the single controllers as well as its

implications in rough terrain locomotion. The second experiment was conducted

with living scorpions (Pandinus cavimanus) to clarify the role of their specialized

appendages, namely the pedipalps, in the detection of obstacles. However before

the focus is shifted to these experiments the existing knowledge on mechanisms

of locomotion and obstacle detection is summarized with a focus on arthropods.

Hereby the path from bottom to top in Fig. 1 is followed.

Environment

Single Leg Controller Single Leg Controller

"Higher" Brain

?Coupling

Body (skeleton, muscles, sensors etc.)

Other sensory/motor systems(e.g. antennae)

"Low Level"

"High Level"

"Low Level"

"High Level"

Figure 1: Different modules interact to cause the emergent behaviour of locomotion. The modulestreated in this work are depicted in grey.

1 Introduction 3

1.1 Levels of locomotion control

At a first glance the control of locomotion seems to be simple, consisting of rhyth-

mic movements with two distinct phases: A stance phase during which a leg exerts

force on the ground to move the animal along the ground surface and a swing

phase during which the leg is returned to the position from which the next stance

phase starts. These two phases are formally divided by two distinct points: Dur-

ing forward walking the posterior extreme position (PEP) marks the end of the

stance phase and the anterior extreme position (AEP) marks the end of the swing

phase. Contrary to the first impression the mechanisms underlying legged loco-

motion are quite complex. Locomotion emerges through the interactions between

muscular, skeletal, nervous, respiratory and circulatory systems and involves the

control of many degrees of freedom. Both feed-forward motor patterns and neu-

ral and mechanical feedback interact [Dickinson et al., 2000]. The neural motor

signals serve only as a suggestion [Raibert and Hodgins, 1993], the reaction of the

mechanical system depends on its actual state and its interaction with the environ-

ment. Concerning the neural control of locomotion large progress has been made

in recent years but the information still remains incomplete [Orlovsky et al., 1999].

One of the best understood organisms in this respect is the stick insect: As was

already assumed by Wendler (see above) the neural locomotion controller is com-

posed of six individual pattern generators or as later referred to single leg controllers

[Orlovsky et al., 1999].

Single Leg Controllers Every single controller consists of a central neu-

ral network and sensors local to the leg and can in turn be decomposed into

at least three central rhythm generating networks controlling the three main

leg joints by alternatively exciting and inhibiting the antagonistic motor neu-

ron pools [Schmidt et al., 2001]. Hereby sensory organs of the leg constitute

an integral part of the central rhythm generating networks (for a review see

[Bässler and Büschges, 1998], see also [Pearson, 1993] and [Duysens et al., 2000]).

A set of neural rules which govern the control of single joints as well the coordina-

tion between joints has been discovered over the last few years which is sufficient

to simulate middle leg stepping movements and with a few additional hypotheti-

cal sensory influences also fore- and hind leg stepping movements (for review see

1 Introduction 4

[Ekeberg et al., 2004]).

Inter leg coordination Contrary to the coordination of the joints com-

posing one leg the coordination in between legs is less well understood in

terms of neural control (see [Blümel, 2004]). On a behavioural level gait

patterns (e.g. tripod gait and tetrapod gait) respectively simple control al-

gorithms that produce these gaits (e.g. [Bowerman, 1975a], [Wendler, 1966],

[Wilson, 1966], [Delcomyn, 1981] and [Cruse, 1990]) have been extensively de-

scribed. Further the adaptations of locomotion behaviour to other forms of

walking, e.g. curve walking (see [Cruse and Saavedra, 1996]), to amputations

([Bowerman, 1975b] and [Wendler, 1966] and to changing environmental con-

ditions ([Watson et al., 2002] and [Blaesing and Cruse, 2004]), including reflexes

(see e.g. [Pearson and Franklin, 1984] for insects and [Gorassini et al., 1994] and

[Hiebert et al., 1994] for cats) were investigated.

Influences of non locomotory sensory systems Despite the sensory, mechan-

ical and neural mechanisms local to the legs and their coupling structure external

body systems take part in shaping the locomotor behaviour. For example ani-

mals that are active during the night or under other low light conditions often

use mechanical senses to detect obstacles and/or to follow structures in the en-

vironment. They often do this by employing mechanoreceptors that are located

on some kind of filamentous structure anterior of the body, e.g. arthropod anten-

nae or mammalian vibrissae. In the cockroach rapid antennal wall-following was

studied [Camhi and Johnson, 2002] and the discovered principles applied to a robot

[Cowan et al., 2003]. In the stick insect the role of the antennae in obstacle and gap

detection were investigated (see [Dürr and Krause, 2001], [Krause and Dürr, 2004]

and [Blaesing and Cruse, 2004]). Further in the stick insect studies the antennae

were seen as an additional pair of legs and in this context also non-locomotory move-

ments of normal legs, e.g. leg searching movements [Dürr, 2001], have to be men-

tioned. A special structure, the pectines of scorpions and its involvement in reflec-

tory body height regulation, has recently been investigated (see [Schneider, 2002]

and [Kladt, 2003]).

1 Introduction 5

Higher control Apart from the legs themselves and sensory structures on other

body parts the locomotory behaviour can be influenced by higher brain centres

[Schaefer and Ritzmann, 2001]. One example is goal-oriented behaviour (see e.g.

[Arbas et al., 1993] and [Böhm et al., 1991]). Another is that of command neurons

[Bowerman and Larimer, 1974].

1.2 Approach I: Development and analysis of locomotion

controllers

Given the incomplete (though constantly increasing) knowledge on the structure

and function of locomotion controllers in biology (see above) some questions remain

concerning the control of highly non-linear legged locomotion with many degrees of

freedom (DOFs): What are the structural and functional dynamic principles of the

control of legged locomotion? How do Central Pattern Generators (CPGs) and re-

flex control work together? How does a controller adapt to a changing environment?

To at least give partial answers to these questions the following approach was taken:

First controllers for three morphological distinct legs (fore-, middle- and hind-leg)

with 3 DOFs each were independently developed as recurrent dynamical neural net-

works based on standard additive artificial neurons with tanh as transfer function.

Although they are very simple, these artificial neural elements are known to show

complex dynamical effects like hysteresis, oscillation and chaos [Pasemann, 2002],

and are therefore suited to build up controllers for complex non-linear tasks. The

controller development took place by means of an artificial evolution. Controllers

were evaluated on a physically simulated robot (a “one leg preparation” mounted

on a rail) with a defined morphology, a defined set of sensors and motors. The

employed fitness function mainly consisted of the total forward movement achieved

per trial. Subsequently variation and reproduction was applied to the best per-

forming (selected) controllers to finally start the “evaluation-variation-selection”

loop anew [Huelse et al., 2004]. During the evolutionary process special care was

taken by means of a cost function, punishing large networks and high connectivities

to keep the networks as small as possible. To contribute to the robustness of the

controllers the evaluation took place in the simulator under different environmental

conditions (gaps, obstacles etc.). The so obtained controllers were then analysed

to reveal their mechanisms of structure and function. Finally the six leg controllers

1 Introduction 6

were connected via sensory inputs from the other legs to form a controller for a

six-legged walking machine.

By utilising this approach several distinct structured and extremely small con-

trollers for each of the three legs were developed. All of the controllers were roughly

doing an equal good job in propelling the body forwards by generating similar mo-

tor patterns which were characterised by bi-stability and marked phase relations.

Analysis revealed that most of the controllers functioned without an inherent neural

oscillator and were therefore dependent on sensory inputs. All controllers developed

either contained a neural hysteresis element or a hysteresis phenomena arising from

the interaction of motors, environment and sensors (Hysteresis occurs in neurons

with a positive self-feedback). The hysteresis elements explained the motor patterns

observed, which suggested the existence of a bistable element within the controller

network. Analysis of the controllers under “rough terrain” conditions further re-

vealed some interesting effects of a dynamical interaction with the environment,

which e.g. enabled a controller to lift the foot higher upwards during obstacle

contact than during normal walking and which enabled another controller to find

support for the foot during gap crossing. Finally the coupling of six controllers and

their application to a hexapod robot resulted in an almost perfect tripod gait on

even terrain and a highly irregular walking pattern on rough terrain. The robot

was able to climb over a wide range of obstacles but whether this was due to the

irregular walking pattern is not clear yet.

Altogether these findings suggest a major role of sensory inputs for the process

of pattern generation in locomotion. It would be interesting to know if pattern

generators as simple as the ones found in this work and as dependent on sensory

inputs exist in biology. The simplicity and the environmental interactions seem

to fit well with the hypothesis of Brooks, namely that the “world [serves] as its

own model” (sensor-motor loop) and that by employing this environmental loop

the degree of explicit control is decreased (small size) (see [Brooks, 1991a] and

[Brooks, 1991b]).

1.3 Approach II: Obstacle contact in scorpions

Scorpions are nocturnal animals and therefore they rely predominantly

on tactile senses to detect disturbances in their local environment

1 Introduction 7

[Brownell and Farley, 1979a]. The mechanical senses are extremely well de-

veloped, including the detection of wind currents and ground vibrations, sensitivity

to “ordinary” contact stimuli, cuticular stresses and several proprioceptive influ-

ences [Root, 1985]. Previous studies (see [Köpke, 2001] and [Schneider, 2002] on

different scorpion species) have suggested a role of the pedipalps (the pincers, see

Fig. 2) in the active perception of obstacles. Turning movements shortly after the

first pedipalp contact with an obstacle as well as groping movements in proximity

to the obstacle were observed. In the majority of the cases turning was found

to occur in the direction contra-lateral to the side that made the first contact.

[Schneider, 2002] found no significant change in behaviour in blinded scorpions

thereby supporting the view that the visual system in scorpions mainly serves as

an extremely sensitive detector of faint light spots for navigation and for timing

of circadian rhythms (see [Fleissner, 1968] and [Fleissner and Fleissner, 2001]).

Despite these observations no detailed knowledge on the role of the scorpion

pedipalps exists. For example it is known that specialized hairs, so called

trichobothria, which (in scorpions) are exclusively found on the pedipalps, play a

role in the orientation towards weak air currents (see [Linsenmair, 1968]). Further

the structure of the trichobothria has been studied in detail [Hoffmann, 1967] and

they have been used for classification purposes (the so called trichobothriotaxie –

see [Vachon, 1973]). But no study to date focused on the mode of contact during

obstacle detection, nor on the role of the trichobothria or the other “cuticular”

hairs in the detection of obstacles. Because of this lack of knowledge and because

of the promising applicability of active perception, using pedipalp like structures,

to robotics, first efforts were made to unravel the mechanisms involved.

Scorpions of the species Pandinus cavimanus (POCOCK)2 were filmed while

making obstacle contacts on a locomotion compensator. Two cameras were used:

The first filmed the contact in the very proximity of the obstacle front to make hair

contacts visible (which required a special lightening, see “Materials and Methods”)

and the second filmed from above to enable motion tracking of the pedipalp and

leg movements during obstacle contact. Data obtained from the motion tracking

and the analysis of contact was then plotted and analysed. A typical sequence

of events from the first contact to the decision to either climb over or avoid the

2 For an overview of the morphology of Pandinus see Fig. 2.

1 Introduction 8

obstacle was found to be as follows: Upon first contact the pedipalp moved slightly

backwards when compared to normal walking and seemed to act as kind of a shock

absorber. At the same time the pedipalp tips moved apart laterally and a second

(or affirmative) touch was made at a position on the obstacle slightly apart from

the first one. Either a pause or a turn in front of the obstacle followed whereby

the pause is interpreted as the scorpion waiting for ground vibrations to occur to

distinguish between a solid obstacle and another animal. The turn on the other

hand involved an inward bend of the obstacle facing pedipalp and resulted in a wall

following behaviour together with a tactile exploration of the obstacle. This tactile

exploration consisted of rhythmic movements of the pedipalps with either repetitive

cuticle contact with the obstacle or “hair brushing” of the obstacle or both. During

this tactile exploration the height of the contact points on the obstacle increased

in time. Once the rim of the obstacle was reached a climb was very likely. On the

other hand if first the the lateral rim of the obstacle was reached the scorpion almost

certainly walked around the obstacle instead of climbing over it. Altogether the

findings strongly support the role of the pedipalps in the perception of obstacles

through active touch. Further the observations suggest a major role of the hair

contact (“brushing” movements) in the obstacle exploration.

Coxa Femur

Patella

Tibia

TarsusBasitarsus

Trochanter

Ophistosoma

Pro− Meso− Metasoma

Optic tubercle withmedian eyes

Telson(with sting)

I IIIIIVIVIV VII XIXIXVIII

1

3

24

Pedipalp

Chelicera

XII Anus

Lateral eyesTibiaTarsus

Patella

Chela (pincers)

Figure 2: The scorpion Pandinus cavimanus (POCOCK) depicted from above. Nomenclaturefollows [Snodgrass, 1952].

1 Introduction 9

1.4 Outline

Due to the two diverse experimental approaches each of the following chapters

is divided into two parts: First the “low level” part, i.e. the development and

simulation of leg controllers, is treated followed by the experiments on the role of

scorpion pedipalps in obstacle detection, the “high level” part. Additionally there

is a third part in the “Results” chapter in which the results of the two approaches

are related.

In the “Materials and Methods” chapter the tools and experimental techniques

used are presented for both approaches. Concerning the simulation part the phys-

ical simulation, including the morphology of the simulated robot and different en-

vironmental scenarios, the type of controller (artificial neural networks), the evo-

lutionary concept and techniques of analysis for evolved controllers are explained.

As for the behavioural experiments with scorpions the experimental animals, the

locomotion compensator, the obstacles, camera and lighting setup as well as the

image processing, motion tracking and analysis techniques are revealed.

In the “Results” chapter data from both experimental series is summarized. The

section “Neural control of walking (Simulation)” gives an overview of the motor

signals needed to drive simulated fore-, middle- and hind-legs, it presents several

controller structures which do an equal good job in driving the simulated single leg

to subsequently analyse their mechanisms, it gives examples for the adaptability of

the controllers and finally shows the performance of a controller for a hexapod robot

resulting from the coupling of single leg controllers. The section “Obstacle detection

in scorpions” depicts the behavioural elements found in scorpions during obstacle

contact, like rhythmic movement of the pedipalps and their postures, frequent

pauses and “hair brushing” movements.

Finally in the “Discussion” chapter conclusions are drawn considering both ap-

proaches independently to then show possible connections between the two ap-

proaches as well as their relevance for future research.


2 Materials and Methods

As in the remaining chapters this chapter is divided into two main parts: First

the tools and techniques employed in the development and simulation of walking

controllers are discussed to later describe the experimental setup and the methods

used for the analysis of the obstacle detection experiments with scorpions.

2.1 Simulation of walking

Several decentralized control structures, consisting of controllers for individual legs

as well as a coupling structure between them were developed and evaluated for

a walking robot. Various tools and techniques (see Fig. 3) have been employed

during this process and these techniques are dealt with in detail throughout the

next section: First, the physical simulator used is introduced together with the

morphology of the simulated robot (including sensory and motor systems) and the

simulation environment. Subsequently, the focus is switched to artificial recurrent

neural nets as the control architecture of all controllers developed in the course

of this work. Their general properties are shortly discussed and their interface

to sensory and motor systems explained. Then, artificial evolution in general, as

well as its actual implementation and application in the development of locomotion

control structures, is presented. Concluding this section tools and strategies to

analyze and evaluate the controllers are explained.

2.1.1 Physical simulation

To simulate walking machines, the Open Dynamics Engine (ODE)3[Smith, 2004],

a free library for simulating articulated rigid body dynamics, was used in conjunc-

tion with the proprietary program YARS (Yet Another Robot Simulator)4, which

provided a simplified interface to ODE including a defined set of geometries, joints,

motors and sensors.

Basic concepts The term “rigid body dynamics” refers to mechanical systems

having rigid bodies (solid objects), joints (e.g. hinge joints, slider joints, ball and

3 Version 0.5, see http://ode.org/ode.html.4 http://www.ais.fraunhofer.de/INDY/, see menu item TOOLS.


Connector

SimulatorExecutor

Analyser

Fitness values

Net

Sensor values

Motor values

NetsNeural

Robot

Environments

NetEvaluator

Evolution

Evolution

Plotting/Stimulation

Figure 3: Overview of the tools used in the artificial evolution and simulation process. Denotedis the flow of information between the evolution program (which performs a selection,variation and reproduction on a population of artificial neural nets), the processing ofthe neural nets (Executor) and the physical simulator. The tools in solid black arecommented on in detail in the following section. Tools denoted in grey are only shortlydescribed.

socket joints), contacts, collisions and friction. ODE provides all of these and is

therefore suited to simulate objects like ground vehicles and walking machines. On

the other hand, it cannot be employed to simulate non rigid body dynamics like par-

ticle effects or waves. That means that no deformation of objects is possible. ODE

is very fast (of which the importance will become clear later on – see evolution part

p. 19) in contrast to commercial products like MSC.ADAMS5. Since a trade-off has

always to be made between stability, accuracy and real-time performance, the speed

comes at the cost of a lowered accuracy6 but this caused no major drawbacks in the

context of this work. The higher accuracy is needed in e.g. car crash simulations

in the automotive industry. A further advantage of ODE is its stability: Physical

simulations often suffer from instabilities caused e.g. by forces being much to high

and result in “explosions” (or singularities). ODE is not immune to these effects

but because of its integrator makes simulations highly stable. To make the sim-

ulations as stable as possible the following guidelines were used: Time-steps were

not chosen too large, very large masses were not mixed with very small ones, forces

were tried to be kept under a certain limit and frictions were limited. Generally

there is a recommended range for lengths and mass values (0.1 . . . 10) [Smith, 2004]

which guarantees the highest precision possible in calculations. Since no size de-

pendent factors are present in the simulation (e.g. aerodynamic resistance), it is

5 http://www.mscsoftware.com.au/products/software/msc/adams/.6 Only a first order integrator is used by ODE.


possible to scale all units accordingly. For this reasons all parameters are given in

dimensionless numbers: unit length [ul], unit mass [um], unit force [uf ] and unit

velocity [uv]. One could interpret them as meters [m], kilograms [kg], Newtons [N]

and meters per second [ms].

The simulators were not directly programmed in C++ with the ODE-library. In-

stead, the program YARS was used which dynamically parsed a XML-configuration

file into C++-code and provided a built-in interface for communication with the

neuro-controller. It is based upon the concept that segments are connected by

joints to form segment chains which in turn may be interconnected by joints to

form a vehicle or robot. For example, a walking robot would consist of several

segment chains (legs), connected to a central segment chain (body). The legs and

the body themselves would consist of single segments, e.g. upper leg, lower leg,

foot etc. Implemented geometries for the single segments included spheres, boxes

and capped cylinders. Joints could be of one of the following types: fixed joint,

hinge joint, a combination of two hinge joints (e.g. wheel suspension) or a slider

joint. Joints could be either passive (spring like) or active (e.g. servo motor). Fur-

thermore a defined set of sensors was made available through the XML-file and

an arbitrary number could be attached to existing segments/joint: angle-position-,

angle-velocity-, infrared-distance-, light-intensity- and force-feedback-sensors. En-

vironments could be created in two ways: Either a fixed environment, consisting

of the geometries listed above, could be defined in the XML-configuration file or

another configuration file was used to specify mean values and probabilities for

variations in a random environment. It was even possible to have multiple envi-

ronmental scenarios stored in multiple configuration files which were run through

consecutively.

Concept of the single-leg simulation Inspired by work of the Büschges-group

(see [Blümel, 2004] and [Ekeberg et al., 2004]), locomotion controllers were devel-

oped for single legs to later couple these controllers to provide a controller for the

whole robot. This approach has already been successfully employed by other groups

before (see e.g. [Seys and Beer, 2004], [Brooks, 1989], [Klaassen et al., 2004] and

[Schmitz et al., 2001]). To be able to simulate just one leg some assumptions /

constraints had to be made and some auxiliary attachment to be developed, since

one leg alone cannot constantly support the body during walking. Therefore a rail-


like structure was developed in the simulator (see Fig. 4) which consisted of two

parallel panels with a narrow space in between. Three spheres7, connected by fixed

joints and therefore forming a triangle, exactly fitted into this space. Equipped with

carefully chosen masses and frictions the triangle could be pulled/pushed along the

rail by the legs without being tilted. The body of the simulated robot was fixed to

the uppermost sphere by means of a slider joint which allowed the robot to move

up- and downward. Altogether the robot was able to move forward/backward and

upwards/downwards but not to move sideways or to turn.

(slider joint)up and down

"rail"

back and forth (rail)

platform

Figure 4: Single Leg Simulator with the body connected to a a rail like structure via a slider-jointhere shown with a hind leg.

2.1.2 Morphology of the simulated robot

The development of the simulator was closely coupled to the development of the

actual hardware robot. Therefore several restrictions applied regarding the mor-

phology of the simulated machine: Servomotors had to be used which caused highly

non biological dimensions and weight distributions as well as limitations concern-

ing the maximum forces and velocities which themselves caused a limited range

of possible segment lengths. Also the choice of sensors was limited (see below).

The exact dimensions and weights are shown in Fig. 5 and Tab. 1. The robot was

constructed with the idea that different legs (fore-leg, middle leg and hind leg) have

7 Rationale: Spheres have the least amount of contact points possible when coming in contact withplanar surfaces and therefore have the lowest computational cost.


to fulfill different tasks and therefore need to have a distinct morphology. Limited

by the constraints outlined above, the only morphological differences were the at-

tachment points on the body and the initial orientations at the body as well as

the angle ranges of the joints: The fore-legs had a working range in front of the

shoulder joint, the middle legs around the shoulder joint and the hind legs behind

the shoulder joint. The working range of the fore-leg shoulder joint was also chosen

larger to allow for groping movements.

walkingdirection

60.0°

30.0°23.0°

20.0°

30.0°

20.0°23.0°

30.0°

75.0°35.0°

30.0°

10.0°

60.0°

2

1

3

1 + 2 + 3

x

y y

z

0.05 ul 0.05 ul

longitudinal axis

forward (−) / backward (+) − joint (Cx−Tr) up (+) / down (−) − joint (Tr−Fe) andinside (−) / outside (+) − joint (Fe−Ti)

Figure 5: Dimensions of all segments and angle ranges of all joints of the right side of the robotwith the left side being symmetric.

Segment Replacement for Length [ul] Mass [um]Body Body and Coxa see Fig. 5 0.800

Trochanter Trochanter 0.025 0.065Femur Femur + Patella 0.050 0.065Tibia Tibia, Basitarsus + Tarsus 0.085 0.016

Table 1: Segment lengths and masses. The replacement column refers to the morphology ofscorpions.


Motor System Due to the hardware constraints, the motor system of the (sim-

ulated) robot solely consisted of servo motors of one type (max. force = 0.3 uf ],

max. velocity = 1.2 [uv] for parameters). In contrast to an arthropod muscular

motor system, motors e.g. only allow activation of movement in one or the other di-

rection, no co-activation is possible. Also, the employment of a servo motor means

that a desired angle is given to the motor, which in turn uses all its resources to

position itself according to the command given. The forces and accelerations can-

not be influenced directly. To account for later use on the hardware robot, artificial

noise of 2% (gaussian distribution) was constantly added to the motor signal.

Sensory System Only two types of sensors were employed in each leg: Angle

sensors in the joints and a contact sensor underneath the foot. Analogous to the

motor signals, artificial noise of 2% (gaussian distribution) was added to each sensor

signal. Values for the angle sensors were on a continuous and linear range between

the minimum and maximum angles (corresponding to the motor values). The foot

contact sensor used was not a real binary contact sensor but rather an infrared

distance sensor with a continuous range.8 To simulate a contact sensor the scope

of the infrared sensor was limited to the very proximity of the foot (length of 0.008

[ul]). To give a contact signal for different orientations of the foot to the ground

(e.g. obstacle climbing) the opening angle of the sensor was chosen to be quite

large (125◦). Additionally, experiments with a force sensor as a replacement for the

contact sensor measuring the force between the foot and the tibia were conducted.

Due to technical problems (artifacts in the signals) this work was not continued.

Nevertheless, for future research, this type of sensor promises a high potential in

terms of reliability and provided additional (e.g. load-) information.

Motor and sensor mapping To allow for neural nets with tanh as the transfer

function (see below) to control the robot, the motor and sensor signals had to be

mapped to values in the range [−1; 1]: The motor values and angle sensor valueswere mapped in such a way that either the maximum or the minimum angle possible

corresponded to a value of one and the other to minus one. For the contact sensor

this was different: A value of zero corresponded to no contact and a value of

8 Of which the reason is that the infrared sensor was implemented in the simulator whereas thebinary contact sensor was not.


−1 −1

0 0

1 1N

euro

n−O

utpu

t

Neu

ron−

Out

putCx−Tr−Joint Fe−Ti−Motor

Motor

MotorSensor

Sensor

−1 −1

0 0

1 1

400 400450 450500 500

Neu

ron−

Out

put

Neu

ron−

Out

put

time [steps] time [steps]

Tr−Fe−Joint Foot−Contact

Motor Sensor

onoff

Sensor

Figure 6: Motor- and corresponding Sensor-Signals of the simulated robot under walking condi-tions.

approx. 0.5 to maximal contact. Sample motor and corresponding sensor signals

under real simulation conditions (walking with ground contact) are depicted in Fig.

6. Mapping conventions, i.e. which sign corresponds to which movement direction,

are shown in Fig. 7.

up = +walking forwards = −

down = −

inside = − outside = +backwards = +

direction

Figure 7: Mapping conventions of motor- and sensor-neurons.

2.1.3 Neural control

Artificial discrete-time dynamical neural networks were developed and applied to

the control of a walking machine. On account of their major role in this work, at

first some of their properties are shortly described in the following section. Then,

their linkage to the walking machine’s hardware (motors and sensors) is explained

and initial network configurations are given.


Activationfunction

Transferfunction

net_k

Outputy_k

w_k0

w_k1

w_k2

w_km

x_0 = +1

x_1

x_2

x_m

w_k0 = b_k (bias)Fixed input

Inpu

ts

−1

−0.5

0

0.5

1

−4 −2 0 2 4

tanh(x)

Figure 8: A technical neuron (left) and its transfer (activation) function (right). Modified from[Haykin, 1999]

Artificial neurons All technical neuron referred to later on will have the same

properties as the neuron described here (see Fig. 8): It receives inputs from the units

xi (0 ≤ i ≤ m) with x0 being the formally introduced bias unit with a stationaryoutput of +1. These inputs are weighted by means of a multiplication with their

corresponding weights (synaptic strengths) wki (0 ≤ i ≤ m). Therefore, activity akof neuron k is equal to the sum of the inputs multiplied by their weights:

ak =m∑

i=0

wkixi (1)

The output of the strictly linear activation function is then subjected to the non-

linear sigmoid transfer function (see Fig. 8 on the right)

f(ak) = tanh(ak) =eak − e−akeak + e−ak

(2)

which is a bounded (] − 1,+1[) and strictly monotone differentiable function. Al-though technical neurons are far from being a realistic model of biological neurons

they share some interesting properties: The bounds of the nonlinear transfer func-

tion can be interpreted as an analog to the bounded firing rate of biological neurons

and the activation function can be interpreted as the summation of synaptic inputs

at the dendrites and soma level in biological neurons. In summary, the output yk


of neuron k may be calculated as follows (combining 1 and 2):

yk = tanh

(m∑

i=0

wkixi

)(3)

If wkk 6= 0, the neuron does have a self-connection, otherwise it does not.

Recurrent neural networks Often, neural networks are constructed in such a

way that the neurons can be grouped into layers, starting with the input neurons

getting external (e.g. sensory) input and ending with the output neurons which

send signals out of the network (e.g. motor signals). The neurons in between are

called hidden or internal neurons. If the architecture of the network is strictly

layered and only forward connections (no lateral nor backward connections) exist,

it is called a feed-forward network (see Fig. 9, left side). Otherwise, i.e. if loops

exist within the network, the network is called a recurrent network (RNN) (see

Fig. 9, right side). Throughout this work, generally no restriction was imposed

on the architecture of the neural networks (unless specified otherwise) therefore

explicitly allowing RNNs. One exception is that input neurons never receive any

input from other neurons. Another exception was the number of input(sensory)-

and output(motor)-neurons which was determined by the number of sensors and

motors used. In special cases restrictions were applied, e.g. to develop a controller

without sensory feedback. In this case, inputs from sensory neurons to hidden and

output neurons were prohibited.

��

��

��

Input−

Hidden−

Output−

��

��

��

��

��

��

laye

r of n

euro

ns

Figure 9: Examples of a pure feed-forward net (left side) and a recurrent net (right side).

Linkage of neural controllers and the simulator Since all simulated legs

contained three motors and four sensors, all single leg controllers consisted of at


least four input- and three output-neurons. The connections are depicted in detail

in Fig. 10. Note that a mapping had to be performed by the simulator to map all

motor/sensor values to the tanh interval ]− 1; 1[ (see above).

Output−

Input−

(Hidden−)

Neu

rons

1 2 3 4

765

SensorFe−Ti Joint

Cx−Tr Joint

Motor

MotorSensor

SensorMotor

Tr−Fe Joint

SensorFoot

Figure 10: Linkage of the sensors and motors of the simulator to the sensor and motor neuronsof the neural net.

Initial Net structures The following three neural networks were taken as seeds

to develop single leg controllers by subjecting them to an artificial evolution pro-

cedure (see below):

A net solely consisting of input- and output-neurons without connections inbetween them (see Fig. 11 on the left).

A net inspired by the finite state model of a single leg found in [Blümel, 2004]and [Ekeberg et al., 2004] (see Fig. 11 on the right).

A net with a central two-neuron oscillator [Pasemann et al., 2003] and nosensory feedback (see Fig. 12 with the output of the oscillator shown on the

right side).

2.1.4 Evolutionary tools and techniques

Artificial evolution was employed as a tool to develop neural control structures for

the locomotion of a legged robot. The flow of data during an artificial evolutionary

process is depicted in Fig. 13 and is shortly being elucidated here: Neural nets


5 6 7

1 2 3 4

Motor−Neurons

Sensor−Neurons

?Hidden−Neurons

��

��

��

��

��

��

��

��

Motor−Neurons

12

8

1

13

9

2

14

10

3

15

11

4

765

Sensor−Neurons

Figure 11: Initial structures for evolution. On the left side an empty net is depicted and on theright a biologically (stick-insect) inspired network.

Hidden−Neurons

�� 9!�!�!�!!�!�!�!!�!�!�!"�"�""�"�""�"�"8

5 6 7

1 2 3 4

Motor−Neurons

Sensor−Neurons

1.40.35

−0.35

1.6

−1

−0.5

0

0.5

1

0 40 60 80 100 20

Neuron 8

Neuron 9

time[steps]

Neu

ron−

Out

put

Figure 12: Start-net with a constructed two-neuron oscillator: On the left the structure of thenet and on the right the output of the oscillator neurons (eight and nine) are shown.

generated by the evolution program EvoSun are successively being send to the ex-

ecution program Hinton. Hinton processes one net at a time and communicates

with the simulator to exchange sensor and motor data. The simulator processes

a certain number of steps (in this case 10 corresponding to a total of 100Hz with

net updates are 10 Hz) without communicating with the executor. Then commu-

nication takes place and the net is updated according to the sensory data received

from the simulator and the internal state of the net and a new motor output is

generated which in turn is send to the simulator. This net-update-simulation-loop

is continued for a specified number of cycles and if desired, the loop itself is run

through several times, each time with a different environment. In the course of the

simulation a fitness value is constantly calculated. After the simulator/net-update

process has completed the specified number of cycles, the final fitness value is send

back to EvoSun. EvoSun continues to send nets to Hinton for evaluation until all


nets of one generation have been evaluated. EvoSun then selects a certain number

of nets (selection process) according to their fitness values generated during evalu-

ation. The selected nets are reproduced and the “offspring” undergoes a variation

process. A new generation is then ready to be evaluated. This variation-evaluation-

selection loop [Huelse et al., 2004] is run through repeatedly until the evolutionary

process is stopped by the user. During evolution the user has the possibility to

change several parameters influencing the variation-evaluation-selection loop, e.g.

population size, weighting of fitness terms, number of evaluation cycles, mutation

probabilities, the type of evolution (structural/parameter evolution), etc.

Clie

nt

Clie

nt Hinton

Variation

SelectionEvaluation

Ser

ver

ENS^3EvoSun

Neural Net

Fitness Values

Ser

ver

YARS

Evolution Executor Simulator

Neural NetNeural Nets EnvironmentsRobot

Sensory Data

Motor Data

TCP/IP UDP

Figure 13: The Evolutionary Concept. Adapted from [Mahn, 2003].

Evolutionary algorithm The ENS3-algorithm is used as the evolutionary strat-

egy [Huelse et al., 2004]: ENS3 is an implementation of a variation-evaluation-

selection loop operating on a population of n neuromodules. The algorithm works

on a population which is divided into parents and offspring. Several operators are

put to work on the neuro-modules:

The evaluation operator which consists of a fitness function that measuresthe performance of each neuro-module. If the desired number of neurons and

connections can be negatively added to the fitness function by means of a

cost function to keep the size of the evolved networks within limits.


The selection operator is of stochastic nature. It determines the number ofoffspring for each neuro-module by means of a rank process, based on the

results of the evaluation operator, and by means of a Poisson distribution.

Each neuro-module with a number of offspring greater than zero is passed on

to the next generation. User definable parameters determine the mean size

of the new population as well as the selection pressure (e.g. elitism can be

forced).

The reproduction operator creates a certain number of copies (offspring) ofeach individual neuro- module, whereby the number of copies is determined

by the selection operator.

The variation (or mutation) operator realizes both a combinatorial and areal-valued variation in a stochastic manner. On one hand the combinatorial

variation accounts for insertions and deletions of hidden neurons and connec-

tions which are determined by per-neuron and per-connection probabilities

(random variable [0, 1]). On the other hand the real-valued variation is re-

sponsible for the variation of the weight and bias terms. The probability of

variation is determined by another random variable [0, 1], its magnitude by a

Gaussian distributed random variable.

The algorithm has no formal stop criterion – it is rather assumed that the user

determines the “right” time to end the evolutionary process by monitoring relevant

parameters.

Techniques used Numerous parameters have to be set before/during evolution.

On one hand, there exists no standard common procedure but on the other hand,

the setting of parameters is not performed arbitrarily. Rather, some general strate-

gies exist which may serve as a guideline. Some of the strategies employed during

this work are introduced here:

The weightings of different terms of the fitness functions were adapted tothe state of evolution: E.g. during the evolution of locomotion controllers

first a high reward was given for smooth and slow oscillatory movements and

only a low or no reward for forward movement. Then, as some individuals

arose in the evolutionary process that made smooth forward movements an


increasingly higher reward was given for the forward movement while at the

same time the weighting of the oscillatory term was decreased.

It was tried to keep the evolved networks small to avoid a large parameterspace because the larger the parameter space the less likely new/better solu-

tions were found by the evolutionary algorithm. Additionally it was easier to

analyse smaller nets later on. The small size was achieved by first allowing

arbitrary growth of the neuronal structure to just introduce costs for neu-

rons and synapses when the behaviour met the demands defined prior to the

experiment.

If a functional principle was discovered in a net (e.g. a special connectionstructure) the neuro-module was manually edited according to the princi-

ple discovered, deleting neurons and connections not required, and then re-

subjected to parameter evolution.

Environments were (randomly) changed in the physical simulation in everygeneration to obtain maximal robust controllers. To even carry this idea

further a defined set of environmental scenarios could be successively put in

place during each generation to make sure the nets have at least an average

performance in each of these environments (see below for example scenarios).

The evolution started several times with varied seed nets, to obtain differentstart-points in the parameter space.

Mutation probabilities and amplitudes were adapted to the state of evolution,e.g. evolution was started with high mutation probabilities and amplitudes,

which corresponded to large leaps on the weight-space-landscape, and these

high amplitudes and probabilities were decreased as soon as functional con-

trollers arose (fine-tuning by successively smaller steps on the weight-space-

landscape).

Structure evolution was followed by parameter evolution: Once a networkperformed sufficiently good, its structure was fixed and the parameters were

optimized.


Parallel Evolution of three leg controllers and a coupling structure In

order to couple six controllers it was assumed that contra-lateral leg controllers

represent copies of each other. The single leg controllers were expanded to include

two additional “interface” (hidden) neurons and then three controllers and a cou-

pling structure were evolved in parallel making a total of four artificial evolutions

running in parallel. Because the single controllers could not be evaluated separately

the “Connector” tool was developed (see Fig. 3) which combined the four separate

nets into a control structure for a six legged walking machine in the following way:

The Connector tool made a copy of each of the three leg controllers to obtain a total

of six leg controllers. Following the six nets were interconnected by connecting their

sensor outputs to the interface neurons of the other controllers. This connection

structure was defined in the fourth net. The connection structure could be fixed

(e.g. to only allow ipsi-lateral coupling and no contra-lateral coupling) or arbitrary.

The combined net was then evaluated via the above explained Executor-Simulator

loop and the resulting fitness value was passed back to all four evolution runs.

2.1.5 Evaluation and analysis

After several evolution runs had been conducted the performances of the best nets

of all evolution runs were compared with each other and, if available, with some

reference controllers. The overall best performing nets or nets having a particular

interesting structure were afterwards subjected to further analysis. The process of

this analysis is described hereafter.

Evaluation A special program was developed for evaluation purposes: The evo-

lutionary concept, described in Fig. 13, was adjusted by replacing the evolution-

module EvoSun with an evaluation-module (see “NetEvaluator” in Fig. 3). This

NetEvaluator fed the Executor Hinton with nets for 20 generations without the nets

being subjected to mutation and selection. The fitness values returned by Hinton

were saved in a table together with the names of the nets being evaluated. Be-

cause one important requirement regarding the controllers was robustness, all nets

were evaluated in seven different environmental scenarios (see Fig. 14) with the

fitness obtained in each environment being added to the total fitness. Therefore,

the total fitness value was a good measure for the generalisation performance of the


controller. Poor or especially high fitness values for single environmental scenarios

were good indications for specialisations.

0

0

0

0

00

0

"RAIL"

"RAIL"

"RAIL"

"RAIL"

"RAIL"

"RAIL"

"RA

IL"

"RAIL"

z

x

z

x

z

x

z

x

z

x

z

x

z

x

x

y

−1−

−2−

−3−

−7−−4−

−5−

−6−

−7−

Figure 14: Environments used to test the robustness and generalisation ability of the differentcontrol architectures.

Analysis As a first step in the analysis of the structure-function-relations of a

controller, its behaviour was described qualitatively as well as quantitatively (see

evaluation p. 24). A tool showing the activities of the neurons and the strengths

and signs of the synapses during the robot-environment interaction (simulation) in

form of an animated neural net gave first visual clues. Using the tool in single step

modus together with its plotting capabilities (all neuron outputs could be plotted)

allowed further inspection. Also, the activities of all neurons could be arbitrarily

set (stimulation/lesion experiments) during the simulation to examine the influence

of certain connections and sensor inputs. Finally, the neuro-modules were analysed

as dynamical systems (loops, hysteresis effects etc.).


2.2 Behavioural experiments with scorpions

Freely walking scorpions were observed on a walking compensator while making

contact with obstacles of different heights. Special attention was paid to the mode

of contact – whether the contact was made with the cuticle or only with the hairs.

To be able to differentiate between the two situations, a custom apparatus was de-

veloped that made use of “backlight” and “streak of light” effects in close proximity

to the obstacle. The contact situations were recorded on video tape and processed

with a motion tracking program on a personal computer. After the resultant co-

ordinate data was transformed into a reference coordinate system and important

parameters like distances, velocities and angles had been calculated, the data was

plotted for visual inspection, both as an x/y-plot and as a stick figure animation.

2.2.1 Animals

Sixteen adult female scorpions of the species Pandinus cavimanus (Pocock)

[Gaban, 1997], also called “Red Claw scorpions”, were obtained from a national

dealer9 and kept for the experiments. Out of these 16 animals, six were chosen for

the final experiments. One of these six animals was later observed to have an injury

and the data obtained in experiments with this particular animal was discarded.

Since no qualitative differences were observable among the individuals regarding

the reaction upon obstacle contact, no reference will be made to the individual

animals in the course of this document.

Number Sex Length 5th M.-Seg. [mm] Body Length [mm] Weight [g]1 ♀ 10.8 52 10.92 ♀ 8.1 49 4.73 ♀ 9.7 48 8.64 ♀ 8.6 49 7.65 ♀ 11.4 55 13.6

Table 2: Animal data.

Care and housing The natural environment of the species Pandinus cavimanus

is the rain-forest of east-african states like Tanzania. To (at least partly) mimic the

9 Zoo Center Gaidzik, Hochstr. 88, 47228 Duisburg.


natural conditions the animals were kept in a warm and humid terrarium (see Fig.

15). To prevent cannibalism, all individuals were kept separately. Under natural

(40 W)lightbulb

soil,bark mulch+ sand

ventilationopenings

stone

water

300 mm

200 mm

200 mm

rooftile

Figure 15: Terrarium of a single scorpion.

conditions scorpions mostly spend their time under some sort of a rock or similar

object, often dug deep into the substrate. To provide the animals with a suitable

shelter the terrariums were filled with approximately 6 cm of soil and sand, covered

with a thin layer of bark mulch. A roof tile served as hiding place. Heating of the

terrarium was ensured by a 40 W light bulb which at the same time functioned to

set the day/night cycle (see below). 24h-temperature measurements showed that

the temperatures in proximity of the light bulb ranged from 23◦C to 38◦C and under

the roof tile at a more equated 23◦C to 28◦C. This fits well within the range of tem-

peratures measured for the scorpion Pandinus imperator [Mahsberg et al., 1999]

(that has a habitat comparable to that of Pandinus cavimanus) under natural con-

ditions. The nocturnal animals were kept at an inverted 12/12 h day/night cycle

(light: 6:00 PM to 6:00 AM) to ensure maximal activity during the time of experi-

ments [Fleissner and Fleissner, 2001]. Concerning the humidity, a trade-off had to

be made between quasi-natural conditions and a lowered probability of infections

with mites and fungi: Every two to three days the terrarium was lightly moistened

by means of a plant sprayer. Every 10 to 14 days the scorpions were fed with

alive insects: house crickets (Acheta domesticus), flour worms (Zophobas larvae),

migratory locusts (Locusta migratoria) or field crickets (Gryllus campestris) were


supplied to ensure that all necessary nutrients were available. In case the food

animals had not been eaten after a period of 48 hours they were removed from the

terrarium. A shallow bowl of water was permanently provided and refilled either

daily or every second day.

Preparation Prior to the experiments on the locomotion compensator a circular

(�5 mm), self-adhesive reflex-marker (Scotchlite High Gain RP 7610) was fixed onthe Prosoma (in between the median eyes and the cheliceres) of each animal.

2.2.2 Locomotion compensator

CameraI

InfraredSensor

CameraII

LightSource II Light

Source I

Walking compensator

ControlVideo-Mixer

Video-Recorder

FP1

FunctionGenerator

Motors

r = 250 mm

Obstacle

Walkingcompensator

01.1 A

Ampere-Meter

Oscillos-cope

Figure 16: Overview of the experimental setup. Shown is a scorpion making contact with anobstacle in the obstacle holding device while walking on the locomotion compensatorand the technical apparatus, e.g. lighting and video recording.

Throughout the experiments the scorpions were free to walk on top of the sphere

of a locomotion compensator ([Kramer, 1975],[Wendler and Scharstein, 1986]). Hereby

the word “compensator” is to be seen in regard to translational movements only,

rotational movements were unaffected by the apparatus. More generally speaking,


animals walking on top of the sphere were free to choose their own course relative

to the environment but were kept on the apex of the sphere by means of a control

loop: The sphere (�50 cm, painted matt black) was mounted in such a way that itcould be rotated around two orthogonal horizontal axes by servomotors. A position

sensor fixed above the centre of the sphere permanently evaluated the deviation of

the animal (the reflex marker fixed on top of the animal) from the upper pole (area

approx. �3.5 cm) lighted with infrared light from a sampling camera by an Osram6 V/15 W bulb (type 8017), filtered through an infrared edge filter, KODAK Wrat-

ten No. 87 (50% transmission at 790 nm). Every time the reflex marker moved out

of the lighted area, the two positioning motors were controlled in such a way that

the sphere was turned until the reflex marker was moved back into the center of

the sampling area.

2.2.3 Obstacles and obstacle holding device

Obstacles While walking on the locomotion compensator, the scorpions were

presented with obstacles of changing height. The obstacles had a curved shape

with the radius equivalent to that of the sphere. They were made out of the

thermoplastic Trovidur10 and were used with the following dimensions: Width 150

mm, Depth 10 mm and Heights of 6, 8, 10, 12, 14, 16 and 20 mm. To maximize

the tactile stimulus given by the obstacles, their front and top surfaces were coated

with sandpaper. A simple scale and three circular markers were painted on the

sandpaper (see Fig. 17). Each experimental run consisted of the animal walking

without obstacle contact for a couple of minutes followed by 45 obstacle contacts

with the scorpion either climbing over or avoiding the obstacle and concluded by a

couple of minutes walking without obstacle contact. Out of the 45 obstacle contacts

15 were made with 12 mm and 20 mm obstacles each and three with 6, 8, 10, 14 and

16 mm obstacles each. The presentation of obstacles followed a random pattern.

Obstacle holding device As stated above, an objective of this thesis is to eluci-

date the mode of contact of the scorpion with the obstacle. In order to visualize the

hairs of the scorpion pedipalps during obstacle contact on the locomotion compen-

sator, quite some effort had to be put in the development of a suitable apparatus.

10 Röchling Engineering Plastics KG.


exchangeable Obstaclecoated with sandpaper(heights: 6, 8, 10, 12, 14, 16 and 20 mm)

heatsink

(5W Luxeon LED)Light Source

bundle of mattblack straws

power +signal

CCD−Camera (S/W)

Sphere of locomotion compensator

Reflector

150 mm

Figure 17: Obstacle mount shown with obstacle, camera and light source on the locomotion com-pensator.

The solution proposed here is an obstacle mount which fixates the obstacle, the light

source and the camera with respect to each other (see Fig. 17). This set-up allows

a fixed camera view of the obstacles and the same lighting of the area around the

obstacle independent of the actual location and orientation of the obstacle on the

sphere. The obstacle mount consisted of two brass blocks close to the sphere with

two acute feet each in order to maximize the grip on the sphere. It provided a slot

which could accommodate obstacles (see above) of different heights and allowed

for a rapid exchange. Further on, it had two continuously adjustable clamps to

hold the CCD-camera and the led light-source on opposite sites of the obstacle. To

prevent the animals from crawling under these two clamps, which would result in

the reflex marker getting out of the light beam of the sensor, two boundaries made

out of matt black cardboard were fixated in front of the obstacle: one on the left

side and the other on the right side. Finally, all potentially reflective parts of the

obstacle mount were either painted matt black or covered with black cardboard to

minimize reflections of the infrared sensor as well as reflections of the light sources.


2.2.4 Lighting

The lighting was the most crucial part in making the hairs of the scorpions visible.

Employing two light sources, one used as back light and the other as incident

light, was found to be an appropriate solution. The back light was mounted to

the obstacle mount and consisted of a white 5W Luxeon LED.11 Apart from its

small size an advantage of the LED was the well defined wavelength range (peaks

at 440nm and 550nm) which particularly did not interfere with the sensor of the

locomotion compensator. During tests with e.g. a halogen lamp the locomotion

compensator showed an unpredictable behaviour. One drawback of using a LED

with such a huge power to size ratio was that it emitted a non negligible amount

of heat and therefore required some sort of a cooling system – in this case it was

fixed to a heat sink (Without the LED would not have survived longer than a

couple of minutes at full power operation). To focus the beam of light and to

prevent unnecessary reflections on surfaces (e.g. the sphere) a reflector (FHS-

HNB1-LL01-H) was mounted in front of the LED and a bundle of matt black

painted straws was mounted in front of the reflector minimizing light scatter. To

further improve the heat dissipation problem and to decrease the effective shutter

time of the camera (which was not manually controllable – see below) the LED

was pulsed at a frequency of 50 Hz with a 2.5 ms on and a 17.5 ms off interval.

The resultant shutter time was 1/400 s. Pulsing the LED was realised by means

of chained pulse- (PG1) and function- (FP1) generators of which the FP1 had an

integrated amplifier. The output of the function generator was monitored with an

oscilloscope (Tektronix TDS 210) and the brightness of the LED was controlled via

a slider potentiometer (1 – 7#

) which permitted a regulation of the current passing

through the LED which was in turn monitored by an ampere-meter (Voltcraft M-

3650D). Secondary, a cold light source (Schott KL 1500 electronic) with two fibre

glass arms, each terminated by a focusing lens, was employed. In contrast to the

first light source this one remained fixed with respect to the walking compensator

and not to the obstacle. The two light beams were focused on the apex of the

sphere from above at slight (opposing) angles from the vertical axis.

11 http://www.lumileds.com.


2.2.5 Camera setup

Cameras Two cameras were utilized to film the scorpion during obstacle contact,

one b/w CCD-Camera (Monacor TVCCD-30MA, fixfocus lens), fastened to the

obstacle holding device and a DV-Camcorder (Sony DCR-PC110E) mounted on

a tripod and therefore fixed with respect to the locomotion compensator. The

problem of attaching a camera close to the obstacle was a poor depth of focus

which had to be counteracted by a decreased aperture (custom made out of matt

black cardboard). This in turn entailed a prolonged shutter time that could not

be directly influenced since the chip camera employed an automatic shutter time

regulation not accessible by the user. As an indirect solution, the shutter time was

regulated by pulsing the LED so that it functioned as a stroboscope (see above for

details).

Video Mixing The signals of both cameras were fed into a video mixer (Pana-

sonic Digital AV Mixer WJ-AVE5) where the images of both cameras were combined

into one (see Fig. 18 for details). The resulting video stream was recorded on an-

other DV-Camcorder (Sony DCR-TRV900E) serving solely as a recording device.

Three quarters of the pixels were allotted to the camera fixed to the obstacle and

one quarter to the camera filming from above. This distribution was necessary to

have an image resolution high enough to visualize the pedipalp hairs of the scor-

pion. A shortcoming of the video mixer was only noticed after the experiments

had already taken place: Usually it is possible to regain a temporal resolution of 50

Hz out of the 25 Hz video recordings by means of deinterlacing making use of the

fact that every second horizontal line is shifted in time by half the time between

two frames. Since the image in the top right corner had its resolution decreased

by half regarding the horizontal and vertical lines the information necessary for the

deinterlacing process was lost. This problem is dealt with later on (see below). On

the contrary, the resolution of the other video stream was not decreased – only one

fourth of the picture was overlaid with the other image – and therefore it retained

the 50Hz information.

Distortion, Calibration and other sources of error There are many factors

influencing the accuracy of measurements in video images, starting with the objects


100 mm (162 Pixel)(1 Pixel ~ 0.62 mm)

100 mm (249 Pixel)(1 Pixel ~ 0.40 mm)

576

Pix

el

720 Pixel

288

Pix

el

360 Pixel

Figure 18: Video Mixing, resolutions and scalings of the two video images.

being filmed continuing with the optical system of the camera and its viewpoint

ending with A/D conversion and post processing on the computer. Most of these

factors have been neglected i.e. they have not been compensated for. Instead only

an error estimation was performed. In detail that means that the small image (cam-

era above the sphere) which was later used for a quantitative analysis was calibrated

via ordinary scale paper on the apex of the sphere prior to the experiment. Distor-

tions caused by the perspective, by the optical system of the camera, the curvature

of the sphere and different distances of the objects observed to the camera were

neglected. An estimation of the error via measurements of angles and lengths of

identical objects at different positions and in different orientations showed that the

errors of distances and angles were typically in the range of ±10%. On the contrary,the large image was mainly used for qualitative analysis, e.g. to determine whether

contact took place and in case of contact the type of contact. Due to its proximity

to the filmed object, the distortion of this camera was considerably higher. Again

scale paper was filmed at discrete positions prior to the experiment to get an indi-

cation of the dimensions for the qualitative analysis. Since the argumentation later

on mainly relies on movements relative to each other and/or qualitative aspects,

the above mentioned errors were tolerated and not being compensated for (like

possible with e.g. a 3D computer calibration tool). For the few plots where the

height of the contact point on the obstacle was measured this was done manually

and in this case the magnitude of the error was again around ±10%.


2.2.6 Image processing and analysis

Stills

Videos

coordinatedata

distances,

anglesvelocities +

Kino:

editing + export

Transcode +Avimerge:Batch mergingof still frames(scale) andvideos + batchdeinterlacing

PC

DV−VCRIEEE 1394

Prepared Videos

DV−capture,

OpenOffice calc:

angles and velocities

coordinate transformation,calculation of distances

coordinatedata

Winanalyze:Motion tracking +export of coordinate data

Stick FigureAnimation

Scilab:

Gnuplot:Plotting

Figure 19: Image processing and analysis flow chart.

In the following paragraph the path depicted in Fig. 19 is followed and commented

on in detail:

Video editing First of all, the video was copied to a Personal Computer via

IEEE 1394 (Firewire interface) employing the non-linear DV editor Kino12 and cut

into small sequences where obstacle contact took place (1s before first contact to 1s

after decision13). Additionally, snapshots were taken from the calibration images.

Subsequently, the still images were prepended before the video sequences to ensure

that each sequence had its own calibration information. The frame rate of the

combined video was then subjected to a process which doubled the frame rate by

segmenting the single images into single horizontal lines and taking every second

line for one new image. Following this, motion-based deinterlacing was applied to

each image to obtain the original width to height proportion. Prepending, double

frame output and deinterlacing were performed by a custom script which made

use of the two programs Avimerge and Transcode14 (especially the filters doublefps

and smartbob were used. Videos obtained by this procedure were then saved in an

12 http://kino.schirmacher.de/.13 Time of decision is defined as follows: 1. If the obstacle is climbed the time of first tarsus contact

with top surface of obstacle 2. If the obstacle is evaded the time of the first pedipalp reachingoutside boundary range.

14 http://www.theorie.physik.uni-goettingen.de/˜ostreich/transcode/.


uncompressed format to allow post-processing with the commercial motion tracking

program WinAnalyze v1.415.

Number Name Position1 B1 Border Meso- and Metasoma2 B2 Reflex Marker Prosoma3 PL1 Joint between Patella and Tibia of left Pedipalp4 PL2 Tarsal Endpoint of left Pedipalp5 PR1 Joint between Patella and Tibia of right Pedipalp6 PR2 Tarsal Endpoint of right Pedipalp7 LL Tarsus of 3rd left Leg8 LR Tarsus of 3rd right Leg9 OC Central point of obstacle10 OL Point on the left front of obstacle11 OR Point on the right front of obstacle

Table 3: Markers on the scorpion that were tracked in the image sequence. For an explanation ofthe terms see Fig. 2.

OL OR

OC

PR1PL

PL1

LL

LR

B1

B2

PL2

O

PR

B

PR2

Body coordinatesystem

(0,0)

Obstacle coordinate system

y

x(0,0)

x_by_b

Figure 20: Markers on the scorpion and definition of coordinate systems. For an explanation ofthe abbreviations see Tab. 3.

15 http://www.winanalyze.com/.


Motion Tracking At first, the images were calibrated in the most simple way

– a certain distance on a scale paper recorded on video was set as scale – and

subsequently a defined set of points on the scorpion was manually motion-tracked

(see Table 3 and Fig. 20) for approx. 200-800 frames, depending on the time to

decision. Doing so revealed a major drawback of this program: There was no mode

which allowed to track one marker manually and step one frame forward with every

mouse click. Instead Winanalyze always placed the markers on the old (manual)

or a new position based on a motion-tracking algorithm (automatic). This might

work well if one uses explicit markers like coloured discs on the knees and ankles

of humans. But for the purpose of tracking points on an unmarked animal this

decreases the accuracy for several reasons: On the one hand the program makes

one think that it is not a problem to track several points at once, it already proposes

a new position for each marker (therefore making it hard to follow the real motion)

and it is not possible to click through an image series very fast (therefore interfering

with a “sense of motion”). The loss in accuracy is visible as soon as one compares

the velocity curves of image sequences which have been tracked multiple times.

By adjusting the position of the marker only every nth frame the velocity curve

eventually shows a rhythmic motion where there had been none in the original

image sequence. In future experiments it would be advisable to explore other

possibilities of motion tracking like the open source ImageJ16 in combination with

e.g. the Manual-Tracking Plugin.17 Yet another disadvantage of WinAnalyze has

been its unpredictable order of export data output – a nightmare for everyone who

wants to automatically process the data. On account of this, only the coordinate

data – which was always put out in a predictable order – was used but not the

velocity, distance, angle and acceleration data as would have been theoretically

possible. The additional data was rather calculated by hand (see below) which had

the additional advantage of being transparent in terms of the formulas used.

Coordinate Transformation and Calculations Since the scorpion was free

to choose its orientation on the Locomotion compensator every image sequence

(camera from above) showed the obstacle contact in a different orientation. To

obtain comparable data the following coordinate transformations were performed:

16 http://rsb.info.nih.gov/ij/.17 http://rsb.info.nih.gov/ij/plugins/manual-tracking.html.


Firstly to the obstacle coordinate system and secondly to the bodily coordinate

system (see Fig. 20). On this basis, further parameters were calculated: velocities,

distances and angles (see Fig. 20 and Tables 4 and 5). For details concerning the

coordinate transformation and the calculation of the other parameters, see appendix

page 94.

Name line fromB B1 to B2

PL PL1 to PL2PR PR1 to PR2O OL to OR

D(x) specified point to O parallel to x axisD(y) specified point to OC parallel to y axisD(xb) specified point to B2 parallel to xb axisD(yb) specified point to B2 parallel to yb axis

Table 4: Formally defined connectors and distances which are used to e.g. depict angles betweencertain parts of the body. See also Fig. 20.

Name –$

+$

B-O right turn left turnPL-O right turn left turnPR-0 right turn left turnPL-B inward turn outward turnPR-B inward turn outward turn

Table 5: Conventions regarding the algebraic signs and movement directions of angles. See alsoFig. 20.

Visualization For all experiments, the data in the normalized coordinate system

was visualized in two different ways: First, x/y plots were made of all important

parameters including contact data (cuticle/hair/no contact) in every x/y plot. Then

the movement of the scorpion was visualized in the obstacle coordinate system by

means of a stick figure animation (custom solution, Scilab18) where, depending

on the data, all or only selected points/lines were visualized, in either a sequence

or as an overlay (as displayed in this document). Sometimes height information

18 http://scilabsoft.inria.fr/.


was included in the overlaid animation plot by comparing data from the two videos

(obstacle frame of reference and Locomotion compensator frame of reference). Since

the image sequences filmed sidewards from the obstacle showed large distortions

due to the cameras proximity to the filmed objects, quantitative data (e.g. height

information) could only be seen as an approximation.

3 Results 39

3 Results

In the following chapter mechanisms of obstacle detection and perception are ap-

proached from two sides. First diverse walking controllers developed in an artificial

evolution process are presented. Their performance in different environments (e.g.

flat ground, obstacles, gaps) is analysed to conclude general mechanisms. In the

second part, more complex obstacle perception behaviours are looked at from the

viewpoint of biology. Experiments with scorpions making contacts with obstacles

of different heights are presented. The observed behaviours are described and clas-

sified with a focus on the role of the pedipalps and their hairs as active sensory

systems.

3.1 Neural control of forward walking (Simulation)

As the lowest level in the process of obstacle- perception and avoidance/climbing

the control of walking was investigated. To this end controllers for single legs with

three joints were developed through artificial evolution of neural networks and their

evaluation in a physical simulator. Paying to the fact that the single legs of multi-

legged animals, amongst others those of scorpions, at least partly serve different

purposes controllers for three different types of legs (fore-, middle- and hind-legs)

were developed. Consecutively the results of this study are presented. First the

behaviours of all three leg-types and the motor signals necessary to produce these

obstacle perception by scorpions and robotsneurokybernetik/...force on the ground to move the animal...

Documents