the coevolution of robot controllers (”brains”) and
TRANSCRIPT
![Page 1: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/1.jpg)
1
The coevolution of robot controllers (”brains”) and morphologies (”bodies”) – challenges and opportunities
Stefano Nichele
February 18, 2015
![Page 2: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/2.jpg)
2
Outline • Evolutionary Robotics
– The basics – Why Evolutionary Robotics?
• Evolution of Controllers • Co-evolution body/brain
– Why co-evolve? – How co-evolve? – State of the art
• (Major) Challenges and Opportunities – Simulation vs Real – Making it work! – Artificial Life approach
![Page 3: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/3.jpg)
3
Evolutionary Robotics
![Page 4: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/4.jpg)
4
Evolutionary Robotics
”Evolutionary robotics is a new technique for automatic creation of autonomous robots.”
S. Nolfi and D. Floreano (2000). Evolutionary Robotics: The Biology, Intelligence and Technology of Self-Organizing Machines. The MIT Press
• Seminal book on the subject • EvoRobotics ~ 20 years old
![Page 5: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/5.jpg)
5
• Yet, surprisingly the number of real, physical robots with evolved bodies can be counted on the fingers of one hand
• 95+% [1]: evolution of robot brains (only the controllers)
• 4-%: evolution of robot morphologies in simulation (co-evolution body/brain)
• <1%: co-evolution of body/brain in real robots
[1] Stefano Nichele, rough estimate.
![Page 6: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/6.jpg)
6
Why Evolutionary Robotics?
![Page 7: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/7.jpg)
7
Because...
• More complex (dynamic) environments • More complex robots (morphologies and brains) • Difficult to design/program (top-down vs bottom-up)
• Suggest/test hypothesis on behavior and cognition • Help understand emergence of life/intelligence
![Page 8: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/8.jpg)
8
Why is it difficult?
• Behavioral physical systems are difficult to design – Much more than computer programs, they also depend on
interactions with the environment § Dynamic, unpredictable § Not fully accessible to robots
• Robot + environment = dynamic system – Robot’s sensory state is function of environment and its own
actions – Not easy to know a priori what internal mechanism results in which
behavior (or the other way around)
![Page 9: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/9.jpg)
9
Evolution of controllers
![Page 10: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/10.jpg)
10
Evolution of robot controllers
M. Waibel, D. Floreano, L. Keller (2011). A Quantitative Teat of Hamilton’s Rule for the Evolution of Altruism. PLoS Biol 9(5): e1000615
What is evolved:
• ANN weights • Connections • Learning rules • Nodes • Archietectures • ...
Nolfi & Floreano, 2004 Evolutionary Robotics
![Page 11: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/11.jpg)
11
Evolutionary substrate • Evolve network controllers in simulation (artificial agents) • Co-Evolve morphology and control in simulation
– Early work by Cliff (1993) and Sims (1994)
• Evolve HW for control – Thompson (1995): evolved networks of logic getes on FPGA (simulation)
• Evolve in simulation, test on real robot – Firmly the state of the art. – Why? Simulated robots can easily change & evolution takes time! – Problem: Reality gap
• Evolution entirely on robots M, Matatic and D. Cliff (1996) . Challenges in evolving controllers for physical robots. Robotics and Autonomous Systems 19(1996): 67-83
(individuals X generations X evolved period)
![Page 12: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/12.jpg)
12
Co-evolution of controllers and morphologies
![Page 13: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/13.jpg)
13
Why evolving robot morphology?
• Both control circuit and body plan co-evolve in true evolvable hardware
• Body/brain = chicken & egg problem (Funes & Pollack, 1997)
• A brain does not do much without a body, while a body cannot do much without a brain to control it
• The course of natural evolution shows a history of body, nervous system and environment all evolving simultaneously in cooperation with, and in response to, each other
H. Lund, J. Hallam, W. Lee (1997), Evolving Robot Morphology. Proceeding of the 4th International Conference on Evolutionary Computation
![Page 14: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/14.jpg)
14
Why (and how) co-evolve • How to co-optimize morphology/control:
1. Direct methods: 1-to-1 genome to parts mapping 2. Recursive methods: L-system/grammar (a la Sims) 3. Growth / development: more biological a. Optimize parameters of a fixed body plan b. Optimize body plan itself
• Why, beyond the vague argument that together is better: – Best body plan for a given task not known a priori by designer – Simpler control if morphology is optimized – Body parts can specialize (more objectives, e.g. object manupulation) – Generalization to unseen environmental conditions (robust)
J. Bongard,The Utility of Simulated Robot Morphology Increases with Task Complexity for Object Manipulation. Artificial Life 16(2010): 201-223. MIT Press.
![Page 15: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/15.jpg)
15
Co-evolution, more objectives • Bongard, 2010
G=grasping, A=active categorical perception, L=lifting LN=phalanges’ lengths, RD=phalanges’ radii, SP=spacing between fingers
![Page 16: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/16.jpg)
16
The GA for robot bodies
A. Winfield and J. Timmis (2015), Evolvable Robot Hardware. Evolvable Hardware, eds M. Trefzer and A. Tyrrell, Springer, in press.
![Page 17: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/17.jpg)
17
Relevant work and State of the art
![Page 18: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/18.jpg)
18
Sims (1994)
• 1st body / brain co-evolution with GA • 3D simulated physical world • Morphology: shape, size, joints, ... • Controller: #nodes, connections, type of functions, ... • L-system for development
K. Sims (1994), Evolving Virtual Creatures. SIGGRAPH 1994 Proceedings.
![Page 19: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/19.jpg)
19
Morphology space (body plan)
• Robot body plan (Lund, 1997) should adapt to task: – Type, number, position of sensors – Body size – Wheel radius, wheel base – Motor time constants
• Small changes in morphology = large changes in performance
• Some body plans are impossible or impracticable
![Page 20: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/20.jpg)
20
Lund et. al (1997) - cricket
The Khepera robot with ears. The ears have programmable amplifiers, synthesizers, and mixers.
Gryllus Bimaculatus
• Male produces a species-specific song • Female has complex delay-based ears • Recognize sound, navigate towards source • Co-evolution of controller and ear morphology
• Include body plan in genome as Hox genes o to control body plan o (in nature, growth of body parts)
![Page 21: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/21.jpg)
21
LEGO robots
P. Funes and J. Pollack (1998). Evolutionary Body Building: Adaptive Physical Design for Robots. Artifical Life 4(1998): 337-357
H. Lund (2003). Co-evolving Control and Morphology with LEGO Robots. In Hara and Pfeifer (eds). Morpho-functional Machines, Springer Verlag.
• Demonstrate that evolving morphology for the real world is possible • Conservative and efficient simulator
• GA body (string) • 3 wheels, 25 wheel bases, 11 light sensor positions = 825 morphologies • GP brain (tree-like) • linear perceptron (evolution of 6 weights) • simulate and transfer to reality • line following task
![Page 22: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/22.jpg)
22
GOLEM (2000)
H. Lipson and J. Pollack (2000). Automatic Design and Manifacture of Robotic Lifeforms. Nature, 406(2000): 974-978
• Simulated control and mechanics • Direct encoding of anatomic parts / connectivity • 3D printed (except motors)
Limitations: • No sensors / interactions with environment • Not automatic feedback from real world • Reality gap
![Page 23: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/23.jpg)
23
Morphology Self-Modeling
J. Bongard, V. Zykov and H. Lipson (2006). Resilient Machines Through Continuous Self-Modeling. Science, 314(2006): 1118-1121
• Indirectly infer own structure • Create compensatory behavior in case of failure • Modeling, testing, prediction... • Development of cognition in machines • Consciousness
![Page 24: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/24.jpg)
24
Robogen
J. Auerbach, D. Aydin, A. Maesani, P. Kornatowski, T. Cieslewski, G. Heitz, P. Fernando, I. Loshchilov, L. Daler and D. Floreano. RoboGen: Robot Generation through Artificial Evolution. Artificial Life 14: International Conference on the Synthesis and Simulation of Living Systems, New York, NY, USA, July 30-August 2, 2014.
• Integrated open-source environment
• Co-evolve robot controllers (fully-connected recurrent ANNs) and bodies (represented as GP trees)
• Predefined 3D printable body parts
• Educational project
![Page 25: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/25.jpg)
25
State of the art (co-evolution)
• Hornby, Pollack (2001) * • Bongard (2009) • Auerbach, Bongard (2010) • Aydin (2010)
Developmental or generative processes *
![Page 26: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/26.jpg)
26
Challenges and opportunities
![Page 27: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/27.jpg)
27
What HW can we evolve ?
• Sensing: cameras, IR sensors, touch sensors • Signaling: LEDS, speakers, WiFi • Actuation: Motors • Energy: Electrical Power Source • Control: Microcontrollers • Physical Structure: metal or plastic structure for chassis, legs,
wheels, grippers etc. • Interconnect: Wires connecting electronics
...a realistic subset
![Page 28: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/28.jpg)
28
The Reality Gap
• Difference between real reality and simulated reality • Consequence: real robot performance does not match evolved
robot in simulation
K. Glette, G. Klaus, J. Cristobal Zagal, J. Tørresen. Evolution of Locomotion in a Simulated Quadruped Robot and Transferral to Reality Proceedings of the 17th International Symposium on Artificial Life and Robotics, 2012
![Page 29: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/29.jpg)
29
Open issues
• Designer’s influence is still strong: fitness function, genotype-to-phenotpe mapping, environment, population size, robot body, control architecture
– Active research in all those issues
• Relevance for the understanding of natural systems – Evolutionary Robotic models are extremely simplified... – ...but useful to understand general principles
• Co-evolution • Interaction learning / evolution
![Page 30: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/30.jpg)
30
Challenges
• Hardware design (many degrees of freedom) • Scalability of controllers • With brain-body co-evolution, search space quickly
becomes very large (scalability) • Application (identify specific application domains where
benefits can be demonstrated) • Real time on real HW takes time
– Battery lifetime – Robot lifetime (maintenance, repairs)
• Noise and errors in simulation – Interleaving evolution in simulation and testing on-line – Feedback
![Page 31: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/31.jpg)
31
Grand Challenges
• ”Making it work” (Winfield, ICES 2014) Limited progress in last 15 years
• Genotype-Phenotype mapping: ontogenetic developmental principles must be incorporated – Variable length genomes – simmetry / modularity / reuse
• Big systems, e.g. more than 1000 parts • Automatically synthesize more complex behaviors than
those designed by hand • Co-evolve morphology and control (not in simulation)
![Page 32: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/32.jpg)
32
The Triangle of Life
A. Eiben, N. Bredeche, M. Hoogendoorn, J. Stradner, J. Timmis, A. Tyrrell and A. Winfield. The Triangle of Life: Evolving Robots in Real-time and Real-space. Proceedings of the European Conference on Artificial Life 2013. The MIT Press.
• Fully embodied evolution in real-time and real-space • General conceptual framework • Co-evolution body/brain • Partly implemented in EU Symbrion
To date, no system that implements a complete artificial ecology with physically reproducing robots, where robot morphology and controllers co-evolve in an open-ended process
![Page 33: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/33.jpg)
33
Modular Self-Reconfigurable Robot Systems
• Fixed morphology, but • Self-assembly • Co-evolved robot controllers / morphology
(disassemble and reassemble machines to form new morphologies)
• Versatility, robustness, low cost
Molecube system from Cornell
![Page 34: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/34.jpg)
34
Opportunities
• Artificial life approach
Here there is no infrastructure: the co-evolving body/brain machinery is completely embodied within the robotic population
![Page 35: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/35.jpg)
35
References S. Nolfi and D. Floreano (2000). Evolutionary Robotics: The Biology, Intelligence and Technology of Self-Organizing Machines. The
MIT Press
M. Waibel, D. Floreano, L. Keller (2011). A Quantitative Teat of Hamilton’s Rule for the Evolution of Altruism. PLoS Biol 9(5): e1000615
M, Matatic and D. Cliff (1996) . Challenges in evolving controllers for physical robots. Robotics and Autonomous Systems 19(1996): 67-83
H. Lund, J. Hallam, W. Lee (1997), Evolving Robot Morphology. Proceeding of the 4th International Conference on Evolutionary Computation
J. Bongard,The Utility of Simulated Robot Morphology Increases with Task Complexity for Object Manipulation. Artificial Life 16(2010): 201-223. MIT Press.
A. Winfield and J. Timmis (2015), Evolvable Robot Hardware. Evolvable Hardware, eds M. Trefzer and A. Tyrrell, Springer, in press.
K. Sims (1994), Evolving Virtual Creatures. SIGGRAPH 1994 Proceedings.
P. Funes and J. Pollack (1998). Evolutionary Body Building: Adaptive Physical Design for Robots. Artifical Life 4(1998): 337-357
H. Lund (2003). Co-evolving Control and Morphology with LEGO Robots. In Hara and Pfeifer (eds). Morpho-functional Machines, Springer Verlag.
H. Lipson and J. Pollack (2000). Automatic Design and Manifacture of Robotic Lifeforms. Nature, 406(2000): 974-978
J. Bongard, V. Zykov and H. Lipson (2006). Resilient Machines Through Continuous Self-Modeling. Science, 314(2006): 1118-1121
J. Auerbach, D. Aydin, A. Maesani, P. Kornatowski, T. Cieslewski, G. Heitz, P. Fernando, I. Loshchilov, L. Daler and D. Floreano. RoboGen: Robot Generation through Artificial Evolution. Artificial Life 14: International Conference on the Synthesis and Simulation of Living Systems, New York, NY, USA, July 30-August 2, 2014.
K. Glette, G. Klaus, J. Cristobal Zagal, J. Tørresen. Evolution of Locomotion in a Simulated Quadruped Robot and Transferral to Reality. Proceedings of the 17th International Symposium on Artificial Life and Robotics, 2012
A. Eiben, N. Bredeche, M. Hoogendoorn, J. Stradner, J. Timmis, A. Tyrrell and A. Winfield. The Triangle of Life: Evolving Robots in Real-time and Real-space. Proceedings of the European Conference on Artificial Life 2013. The MIT Press.
![Page 36: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/36.jpg)
36
![Page 37: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/37.jpg)
37
Competitive Co-evolution
• CCE (e.g. predator-prey) may enhance artificial co-evolution of body/brain and optimize controller/morphology combination
• Competing populations can reciprocally drive each other to incrementally increasing levels of behavioral complexity
G. Buason, N. Bergfeldt and T. Ziemke (2005). Brains, Bodies, and Beyond: Competitive Co-Evolution of Robot Controllers, Morphologies and Environments. Genetic Programming and Evolvable Machines, 6, 25-51, 2005
![Page 38: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/38.jpg)
38 Ziemke et al, 2005
Exp. 1 Preconditions: 1- same max speed in both robots 2- speed in predator constrained the view angle
Exp. 2 – adding a constraint Preconditions: 1- same max speed in both robots 2- speed in both robots constrained by the view angle
Morphological space Prey ”choose” speed over vision
![Page 39: The coevolution of robot controllers (”brains”) and](https://reader034.vdocument.in/reader034/viewer/2022052013/62860b5c5e8bc4277c2b83b1/html5/thumbnails/39.jpg)
39
Artificial neuron (McCullock and Pitts, 1943) Artificial Neural Networks EvoDevo
Prerequisites
Encodings: - Direct (limited: scalability, no morphogenesis, limited regularity/modularity, no reuse) - Matrix rewriting - Cellular encoding - Axon growth - HyperNeAT (regularity without development using CPPN) - Indirect encodings