Nathan F. Lepora · Anna MuraMichael Mangan · Paul F.M.J. VerschureMarc Desmulliez · Tony J. Prescott (Eds.)

123

LNAI

979

3

5th International Conference, Living Machines 2016Edinburgh, UK, July 19–22, 2016Proceedings

Biomimeticand Biohybrid Systems

Lecture Notes in Artificial Intelligence 9793

Subseries of Lecture Notes in Computer Science

LNAI Series Editors

Randy GoebelUniversity of Alberta, Edmonton, Canada

Yuzuru TanakaHokkaido University, Sapporo, Japan

Wolfgang WahlsterDFKI and Saarland University, Saarbrücken, Germany

LNAI Founding Series Editor

Joerg SiekmannDFKI and Saarland University, Saarbrücken, Germany

More information about this series at http://www.springer.com/series/1244

Nathan F. Lepora • Anna MuraMichael Mangan • Paul F.M.J. VerschureMarc Desmulliez • Tony J. Prescott (Eds.)

Biomimeticand Biohybrid Systems5th International Conference, Living Machines 2016Edinburgh, UK, July 19–22, 2016Proceedings

123

EditorsNathan F. LeporaUniversity of BristolBristolUK

Anna MuraUniversitat Pompeu FabraBarcelonaSpain

Michael ManganUniversity of LincolnUK

Paul F.M.J. VerschureUniversitat Pompeu FabraBarcelonaSpain

Marc DesmulliezHeriot-Watt UniversityEdinburghUK

Tony J. PrescottUniversity of SheffieldSheffieldUK

ISSN 0302-9743 ISSN 1611-3349 (electronic)Lecture Notes in Artificial IntelligenceISBN 978-3-319-42416-3 ISBN 978-3-319-42417-0 (eBook)DOI 10.1007/978-3-319-42417-0

Library of Congress Control Number: 2016944482

LNCS Sublibrary: SL7 – Artificial Intelligence

© Springer International Publishing Switzerland 2016This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of thematerial is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,broadcasting, reproduction on microfilms or in any other physical way, and transmission or informationstorage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology nowknown or hereafter developed.The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoes not imply, even in the absence of a specific statement, that such names are exempt from the relevantprotective laws and regulations and therefore free for general use.The publisher, the authors and the editors are safe to assume that the advice and information in this book arebelieved to be true and accurate at the date of publication. Neither the publisher nor the authors or the editorsgive a warranty, express or implied, with respect to the material contained herein or for any errors oromissions that may have been made.

Printed on acid-free paper

This Springer imprint is published by Springer NatureThe registered company is Springer International Publishing AG Switzerland

Preface

These proceedings contain the papers presented at Living Machines: The 5th Inter-national Conference on Biomimetic and Biohybrid Systems, held in Edinburgh, UK,during July 19–22, 2016. The international conferences in the Living Machines seriesare targeted at the intersection of research on novel life-like technologies inspired bythe scientific investigation of biological systems, biomimetics, and research that seeksto interface biological and artificial systems to create biohybrid systems. The confer-ence aim is to highlight the most exciting international research in both of these fieldsunited by the theme of “Living Machines.”

Biomimetics is the development of novel technologies through the distillation ofprinciples from the study of biological systems. The investigation of biomimetic sys-tems can serve two complementary goals. First, a suitably designed and configuredbiomimetic artifact can be used to test theories about the natural system of interest.Second, biomimetic technologies can provide useful, elegant, and efficient solutions tounsolved challenges in science and engineering. Biohybrid systems are formed bycombining at least one biological component—an existing living system—and at leastone artificial, newly engineered component. By passing information in one or bothdirections, such a system forms a new hybrid bio-artificial entity. The theme of theconference also encompasses biomimetic methods for manufacture, repair, and recy-cling inspired by natural processes such as reproduction, digestion, morphogenesis, andmetamorphosis.

The following are some examples of living machines as featured at this and pastconferences:

– Biomimetic robots and their component technologies (sensors, actuators, proces-sors) that can intelligently interact with their environments

– Active biomimetic materials and structures that self-organize and self-repair– Nature-inspired designs and manufacturing processes– Biomimetic computers—neuromimetic emulations of the physiological basis for

intelligent behavior– Biohybrid brain–machine interfaces and neural implants– Artificial organs and body parts including sensory organ–chip hybrids and intelli-

gent prostheses– Organism-level biohybrids such as robot–animal or robot–human systems

Five hundred years ago, Leonardo da Vinci designed a series of flying machinesbased on the wings of birds. These drawings are famous for their beautiful, lifelikedesigns, created centuries before the Wright brothers made their first flight. Thisinspiration from nature that Leonardo pioneered remains as crucial for technologytoday as it was many centuries ago.

Leonardo’s inspiration was to imitate a successful biological design to solve ascientific problem. Today, this subject area is known as biomimetics. The Americaninventor Otto Schmitt first coined this term in the 1950s while trying to copy how nervecells function in an artificial device. He put together the Greek words bios (life) andmimetic (copy) and the name caught on.

Why is nature so good at finding solutions to technological problems? The answerlies in Charles Darwin’s theory of evolution. Life, by the process of natural selection, isa self-improving phenomenon that continually reinvents itself to solve problems in thenatural world. These improvements have accumulated over hundreds of millions ofyears in plants and animals. As a result, there are a myriad natural design solutionsaround us, from the wings of insects and birds to the brains controlling our bodies.

Biomimetics and bio-inspiration has always been present in human technology, forexample, making knives akin to the claws of animals. An exciting development,however, has been the dramatic expansion of the biomimetic sciences in the newmillennium. The Convergent Science Network (CSN) of biomimetic and biohybridsystems, which organized the first Living Machines conference, has also completed asurvey on The State of the Art in Biomimetics (Lepora, Verschure and Prescott, 2013).As part of the survey, we counted how much work on biomimetics is published eachyear. This revealed a surprising answer: from only tens of articles before the millen-nium, it has exploded since then to more than a thousand papers each year.

This huge investment in research inspired by nature is producing a wide variety ofinnovative technologies. Examples include artificial spider silk that is stronger thansteel, super-tough synthetic materials based on the shells of molluscs, and adhesivepatches mimicking the padded feet of geckos. Medical biomimetics is also leading toimportant benefits for maintaining health. These include bionic cochlear implants forhearing, fully functional artificial hearts, and modern prosthetic hands and limbs aimedat repairing the human body.

Looking to the future, one of the most revolutionary applications of biomimeticswill likely be based on nature’s most sophisticated creation: our brains. From oursurvey of biomimetic articles, we found that a main research theme is to take inspi-ration from how our brains control our bodies to design better ways of controllingrobots. This is for a good reason. Engineers can build amazing robots that haveseemingly human-like abilities. But so far, no existing robot comes close to copying thedexterity and adaptability of animal movements. The missing link is the controllingbrain.

It is often said that future scientific discoveries are hard to predict. This is not thecase in biomimetics. There are plenty of examples surrounding us in the natural world.The future will produce artificial devices with these abilities, from mass-producedflying micro devices based on insects to robotic manipulators based on the human handto swimming robots based on fish. Less certain is what they will do to our society,economy, and way of life. Therefore the Living Machines conference also seeks toanticipate and understand the impacts of these technologies before they happen.

The main conference, during July 20–22, took the form of a three-day single-trackoral and poster presentation program that included five plenary lectures from leading

VI Preface

international researchers in biomimetic and biohybrid systems: Antonio Bicchi(University of Pisa) on robotics, haptics, and control systems; Frank Hirth (KingsCollege London, Institute of Psychiatry) on evolutionary neuroscience; YoskikoNakamura (University of Tokyo) on biomimetics in humanoids; Thomas Speck(Albert-Ludwigs-Universität, Freiburg) on plants and animals as concept generators forbiomimetic materials and technologies; and Barbara Webb (University of Edinburgh)on perceptual systems and the control of behavior in insects and robots. There werealso 20 regular talks and a poster session featuring approximately 40 posters. Sessionthemes included: biomimetic robotics; biohybrid systems including biological-machineinterfaces; neuromimetic systems; soft robot systems; active sensing in vision andtouch; social robotics and the biomimetics of plants.

The conference was complemented with a further day of workshops and symposia,on July 19, covering a range of topics related to biomimetic and biohybrid systems:Our Future with Living Machines: Societal, Economic, and Ecological Impacts (JoseHalloy and Tony Prescott); Living Machines That Grow, Evolve, Self-Heal andDevelop: How Robots Adapt Their Morphology to the Environment (Barbara Mazzolaiand Cecilia Laschi); and The Emergence of Biological Architectures (EnricoMastropaolo, Naomi Nakayama, Rowan Muir, Ross McLean, Cathal Cummins).

The main meeting was hosted at Edinburgh’s Dynamic Earth, a five-star visitorattraction in the heart of Edinburgh’s historic old town, next to the Scottish Parliamentand Holyrood Palace. Dynamic Earth is a visitor experience that invites you to take ajourney through time to witness the story of planet Earth through a series of interactiveexhibits and state-of-the-art technology. Satellite events were held nearby at Universityof Edinburgh’s School of Informatics in George Square. The Dynamics Earth expe-rience, with its seamless integration of nature and technology, provided an ideal settingto host the 5th Living Machines Conference.

We wish to thank the many people that were involved in making LM2016 possible:Tony Prescott and Marc Desmulliez co-chaired the meeting; Nathan Lepora chaired theProgram Committee and edited the conference proceedings; Paul Verschure chaired theinternational Steering Committee; Michael Mangan and Anna Mura co-chaired theworkshop program; Anna Mura and Nathan Lepora co-organized the communications;Sytse Wierenga, Carme Buisan, and Mireia Mora provided additional administrativeand technical support including organizing the website; and Katarzyna Przybcien andLynn Smith provided administrative and local organizational support. We would alsolike to thank the authors and speakers who contributed their work, and the membersof the Programme Committee for their detailed and considered reviews. We are gratefulto the five keynote speakers who shared with us their vision of the future.

Finally, we wish to thank the sponsors of LM2016: The Convergence ScienceNetwork for Biomimetic and Neurotechnology (CSNII) (ICT-601167), which isfunded by the European Union’s Framework 7 (FP7) program in the area of FutureEmerging Technologies (FET), and Heriot Watt University in Edinburgh, UK. Addi-tional support was also provided by the University of Sheffield, the University ofBristol, the University of Pompeu Fabra in Barcelona, and the Institució Catalana de

Preface VII

Recerca i Estudis Avançats (ICREA). LM2016 was supported via a Santander MobilityGrant. Living Machines 2016 was also supported by the IOP Physics journal Bioin-spiration & Biomimetics, who will publish a special issue of articles based on the bestconference papers.

July 2016 Nathan F. LeporaAnna Mura

Michael ManganPaul F.M.J. Verschure

Marc DesmulliezTony J. Prescott

VIII Preface

Organization

Conference Chairs

Marc Desmulliez Heriot Watt University, UKTony Prescott University of Sheffield, UK

Program Chair

Nathan Lepora University of Bristol, UK

Satellite Events Chairs

Michael Mangan Lincoln University, UKAnna Mura Universitat Pompeu Fabra, Spain

International Steering Committee Chair

Paul Verschure Universitat Pompeu Fabra and ICREA, Spain

Communications Chairs

Anna Mura Universitat Pompeu Fabra, SpainNathan Lepora University of Bristol, UK

Technical Support

Katarzyna Przybcien Heriot Watt University, UKLynn Smith Heriot Watt University, UKCarme Buisan Universitat Pompeu Fabra, SpainMireia Mora Universitat Pompeu Fabra, SpainSytse Wierenga Universitat Pompeu Fabra, Spain

Program Committee

Andrew Adamatzky UWE, Bristol, UKRobert Allen University of Southampton, UKYoseph Bar-Cohen JPL, USAFederico Carpi Queen Mary University of London, UKAnders Christensen University Institute of Lisbon, PortugalFrederik Claeyssens University of Sheffield, UKAndrew Conn University of Bristol, UK

Jorg Conradt TU München, GermanyHolk Cruse University Bielefeld, GermanyMark Cutkosky Stanford University, USADanilo De Rossi Research Centre E. Piaggio, ItalyMarc Desmulliez Heriot Watt University, UKSanja Dogramadzi University of the West of England, UKStéphane Doncieux ISIR, FranceVolker Dürr Bielefeld University, GermanyWolfgang Eberle Imec, BelgiumMaria Elena Giannaccini University of Bristol, UKBenoît Girard CNRS and UPMC, FranceSabine Hauert University of Bristol, UKHelmut Hauser University of Bristol, UKIvan Herreros Universitat Pompeu Fabra, SpainKoh Hosoda Osaka University, JapanIoannis Ieropoulos University of the West of England, UKCecilia Laschi Scuola Superiore Sant’Anna, ItalyNathan Lepora University of Bristol, UKMichael Mangan Lincoln University, UKUriel Martinez-Hernandez University of Leeds, UKBen Mitchinson University of Sheffield, UKVishwanathan Mohan Italian Institute of Technology, ItalyAnna Mura Universitat Pompeu Fabra, SpainMartin Pearson Bristol Robotics Laboratory, UKHemma Philamore University of Bristol, UKAndrew Philippides University of Sussex, UKTony Pipe Bristol Robotics Laboratory, UKTony Prescott University of Sheffield, UKRoger Quinn Case Western Reserve University, USASylvian Saighi University of Bordeaux, FranceThomas Schmickl Karl-Franzens University Graz, AustriaCharlie Sullivan University of the West of England, UKLuca Tonin University of Padova, ItalyPablo Varona Universidad Autonoma de Madrid, SpainEleni Vasilaki University of Sheffield, UKBenjamin Ward-Cherrier University of Bristol, UKStuart Wilson University of Sheffield, UKHartmut Witte Technische Universität Ilmenau, Germany

X Organization

Contents

Full Papers

The Natural Bipeds, Birds and Humans: An Inspiration for Bipedal Robots. . . . 3Anick Abourachid and Vincent Hugel

Retina Color-Opponency Based Pursuit Implemented Through SpikingNeural Networks in the Neurorobotics Platform . . . . . . . . . . . . . . . . . . . . . 16

Alessandro Ambrosano, Lorenzo Vannucci, Ugo Albanese,Murat Kirtay, Egidio Falotico, Pablo Martínez-Cañada, Georg Hinkel,Jacques Kaiser, Stefan Ulbrich, Paul Levi, Christian Morillas,Alois Knoll, Marc-Oliver Gewaltig, and Cecilia Laschi

A Two-Fingered Anthropomorphic Robotic Hand with Contact-AidedCross Four-Bar Mechanisms as Finger Joints . . . . . . . . . . . . . . . . . . . . . . . 28

Guochao Bai, Jieyu Wang, and Xianwen Kong

Living Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Rina Bernabei and Jacqueline Power

iCub Visual Memory Inspector: Visualising the iCub’s Thoughts . . . . . . . . . 48Daniel Camilleri, Andreas Damianou, Harry Jackson, Neil Lawrence,and Tony Prescott

A Preliminary Framework for a Social Robot “Sixth Sense” . . . . . . . . . . . . . 58Lorenzo Cominelli, Daniele Mazzei, Nicola Carbonaro,Roberto Garofalo, Abolfazl Zaraki, Alessandro Tognetti,and Danilo De Rossi

A Bio-Inspired Photopatterning Method to Deposit Silver Nanoparticlesonto Non Conductive Surfaces Using Spinach Leaves Extract in Ethanol . . . . 71

Marc P.Y. Desmulliez, David E. Watson, Jose Marques-Hueso,and Jack Hoy-Gig Ng

Leg Stiffness Control Based on “TEGOTAE” for Quadruped Locomotion . . . 79Akira Fukuhara, Dai Owaki, Takeshi Kano, and Akio Ishiguro

Wall Following in a Semi-closed-loop Fly-Robotic Interface. . . . . . . . . . . . . 85Jiaqi V. Huang, Yilin Wang, and Holger G. Krapp

Sensing Contact Constraints in a Worm-like Robot by Detecting LoadAnomalies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Akhil Kandhari, Andrew D. Horchler, George S. Zucker,Kathryn A. Daltorio, Hillel J. Chiel, and Roger D. Quinn

Head-Mounted Sensory Augmentation Device: Comparing Hapticand Audio Modality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Hamideh Kerdegari, Yeongmi Kim, and Tony J. Prescott

Visual Target Sequence Prediction via Hierarchical Temporal MemoryImplemented on the iCub Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Murat Kirtay, Egidio Falotico, Alessandro Ambrosano, Ugo Albanese,Lorenzo Vannucci, and Cecilia Laschi

Computer-Aided Biomimetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131Ruben Kruiper, Jessica Chen-Burger, and Marc P.Y. Desmulliez

A Neural Network with Central Pattern Generators Entrained by SensoryFeedback Controls Walking of a Bipedal Model . . . . . . . . . . . . . . . . . . . . . 144

Wei Li, Nicholas S. Szczecinski, Alexander J. Hunt, and Roger D. Quinn

Towards Unsupervised Canine Posture Classification via Depth ShadowDetection and Infrared Reconstruction for Improved Image SegmentationAccuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Sean Mealin, Steven Howell, and David L. Roberts

A Bio-Inspired Model for Visual Collision Avoidance on a HexapodWalking Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

Hanno Gerd Meyer, Olivier J.N. Bertrand, Jan Paskarbeit,Jens Peter Lindemann, Axel Schneider, and Martin Egelhaaf

MIRO: A Robot “Mammal” with a Biomimetic Brain-Based ControlSystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

Ben Mitchinson and Tony J. Prescott

A Hydraulic Hybrid Neuroprosthesis for Gait Restoration in Peoplewith Spinal Cord Injuries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Mark J. Nandor, Sarah R. Chang, Rudi Kobetic, Ronald J. Triolo,and Roger Quinn

Principal Component Analysis of Two-Dimensional Flow Vector Fieldson Human Facial Skin for Efficient Robot Face Design . . . . . . . . . . . . . . . . 203

Nobuyuki Ota, Hisashi Ishihara, and Minoru Asada

Learning to Balance While Reaching: A Cerebellar-Based ControlArchitecture for a Self-balancing Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Maximilian Ruck, Ivan Herreros, Giovanni Maffei, Martí Sánchez-Fibla,and Paul Verschure

Optimizing Morphology and Locomotion on a Corpus of ParametricLegged Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Grégoire Passault, Quentin Rouxel, Remi Fabre, Steve N’Guyen,and Olivier Ly

XII Contents

Stick(y) Insects — Evaluation of Static Stability for Bio-inspired LegCoordination in Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

Jan Paskarbeit, Marc Otto, Malte Schilling, and Axel Schneider

Navigate the Unknown: Implications of Grid-Cells “Mental Travel”in Vicarious Trial and Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Diogo Santos-Pata, Riccardo Zucca, and Paul F.M.J. Verschure

Insect-Inspired Visual Navigation for Flying Robots . . . . . . . . . . . . . . . . . . 263Andrew Philippides, Nathan Steadman, Alex Dewar,Christopher Walker, and Paul Graham

Perceptive Invariance and Associative Memory Between Perceptionand Semantic Representation USER a Universal SEmantic RepresentationImplemented in a System on Chip (SoC) . . . . . . . . . . . . . . . . . . . . . . . . . . 275

Patrick Pirim

Thrust-Assisted Perching and Climbing for a Bioinspired UAV . . . . . . . . . . 288Morgan T. Pope and Mark R. Cutkosky

The EASEL Project: Towards Educational Human-Robot SymbioticInteraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

Dennis Reidsma, Vicky Charisi, Daniel Davison, Frances Wijnen,Jan van der Meij, Vanessa Evers, David Cameron, Samuel Fernando,Roger Moore, Tony Prescott, Daniele Mazzei, Michael Pieroni,Lorenzo Cominelli, Roberto Garofalo, Danilo De Rossi,Vasiliki Vouloutsi, Riccardo Zucca, Klaudia Grechuta, Maria Blancas,and Paul Verschure

Wasp-Inspired Needle Insertion with Low Net Push Force . . . . . . . . . . . . . . 307Tim Sprang, Paul Breedveld, and Dimitra Dodou

Use of Bifocal Objective Lens and Scanning Motion in Robotic ImagingSystems for Simultaneous Peripheral and High Resolution Observationof Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

Gašper Škulj and Drago Bračun

MantisBot Uses Minimal Descending Commands to Pursue Prey asObserved in Tenodera Sinensis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

Nicholas S. Szczecinski, Andrew P. Getsy, Jacob W. Bosse,Joshua P. Martin, Roy E. Ritzmann, and Roger D. Quinn

Eye-Head Stabilization Mechanism for a Humanoid Robot Testedon Human Inertial Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

Lorenzo Vannucci, Egidio Falotico, Silvia Tolu, Paolo Dario,Henrik Hautop Lund, and Cecilia Laschi

Contents XIII

Towards a Synthetic Tutor Assistant: The EASEL Project and itsArchitecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

Vasiliki Vouloutsi, Maria Blancas, Riccardo Zucca, Pedro Omedas,Dennis Reidsma, Daniel Davison, Vicky Charisi, Frances Wijnen,Jan van der Meij, Vanessa Evers, David Cameron, Samuel Fernando,Roger Moore, Tony Prescott, Daniele Mazzei, Michael Pieroni,Lorenzo Cominelli, Roberto Garofalo, Danilo De Rossi,and Paul F.M.J. Verschure

Aplysia Californica as a Novel Source of Material for Biohybrid Robotsand Organic Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

Victoria A. Webster, Katherine J. Chapin, Emma L. Hawley,Jill M. Patel, Ozan Akkus, Hillel J. Chiel, and Roger D. Quinn

A Soft Pneumatic Maggot Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375Tianqi Wei, Adam Stokes, and Barbara Webb

Short Papers

On Three Categories of Conscious Machines . . . . . . . . . . . . . . . . . . . . . . . 389Xerxes D. Arsiwalla, Ivan Herreros, and Paul Verschure

Gaussian Process Regression for a Biomimetic Tactile Sensor . . . . . . . . . . . 393Kirsty Aquilina, David A.W. Barton, and Nathan F. Lepora

Modulating Learning Through Expectation in a Simulated Robotic Setup. . . . 400Maria Blancas, Riccardo Zucca, Vasiliki Vouloutsi,and Paul F.M.J. Verschure

Don’t Worry, We’ll Get There: Developing Robot Personalities to MaintainUser Interaction After Robot Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409

David Cameron, Emily Collins, Hugo Cheung, Adriel Chua,Jonathan M. Aitken, and James Law

Designing Robot Personalities for Human-Robot Symbiotic Interactionin an Educational Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413

David Cameron, Samuel Fernando, Abigail Millings, Michael Szollosy,Emily Collins, Roger Moore, Amanda Sharkey, and Tony Prescott

A Biomimetic Fingerprint Improves Spatial Tactile Perception . . . . . . . . . . . 418Luke Cramphorn, Benjamin Ward-Cherrier, and Nathan F. Lepora

Anticipating Synchronisation for Robot Control . . . . . . . . . . . . . . . . . . . . . 424Henry Eberle, Slawomir Nasuto, and Yoshikatsu Hayashi

MantisBot: The Implementation of a Photonic Vision System. . . . . . . . . . . . 429Andrew P. Getsy, Nicholas S. Szczecinski, and Roger D. Quinn

XIV Contents

Force Sensing with a Biomimetic Fingertip . . . . . . . . . . . . . . . . . . . . . . . . 436Maria Elena Giannaccini, Stuart Whyle, and Nathan F. Lepora

Understanding Interlimb Coordination Mechanism of Hexapod Locomotionvia “TEGOTAE”-Based Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441

Masashi Goda, Sakiko Miyazawa, Susumu Itayama, Dai Owaki,Takeshi Kano, and Akio Ishiguro

Decentralized Control Scheme for Myriapod Locomotion That ExploitsLocal Force Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449

Takeshi Kano, Kotaro Yasui, Dai Owaki, and Akio Ishiguro

TEGOTAE-Based Control Scheme for Snake-Like Robots That EnablesScaffold-Based Locomotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

Takeshi Kano, Ryo Yoshizawa, and Akio Ishiguro

Modelling the Effect of Cognitive Load on Eye Saccades and Reportability:The Validation Gate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

Sock C. Low, Joeri B.G. van Wijngaarden, and Paul F.M.J. Verschure

Mutual Entrainment of Cardiac-Oscillators Through Mechanical Interaction . . . 467Koki Maekawa, Naoki Inoue, Masahiro Shimizu, Yoshihiro Isobe,Taro Saku, and Koh Hosoda

“TEGOTAE”-Based Control of Bipedal Walking . . . . . . . . . . . . . . . . . . . . 472Dai Owaki, Shun-ya Horikiri, Jun Nishii, and Akio Ishiguro

Tactile Vision – Merging of Senses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480Nedyalka Panova, Alexander C. Thompson,Francisco Tenopala-Carmona, and Ifor D.W. Samuel

Tactile Exploration by Contour Following Using a Biomimetic Fingertip . . . . 485Nicholas Pestell, Benjamin Ward-Cherrier, Luke Cramphorn,and Nathan F. Lepora

Towards Self-controlled Robots Through Distributed Adaptive Control . . . . . 490Jordi-Ysard Puigbò, Clément Moulin-Frier, and Paul F.M.J. Verschure

Discrimination-Based Perception for Robot Touch. . . . . . . . . . . . . . . . . . . . 498Emma Roscow, Christopher Kent, Ute Leonards, and Nathan F. Lepora

On Rock-and-Roll Effect of Quadruped Locomotion: From Mechanical andControl-Theoretical Viewpoints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

Ryoichi Kuratani, Masato Ishikawa, and Yasuhiro Sugimoto

Hydromast: A Bioinspired Flow Sensor with Accelerometers . . . . . . . . . . . . 510Asko Ristolainen, Jeffrey Andrew Tuhtan, Alar Kuusik,and Maarja Kruusmaa

Contents XV

Developing an Ecosystem for Interactive Electronic Implants . . . . . . . . . . . . 518Paul Strohmeier, Cedric Honnet, and Samppa von Cyborg

Gait Analysis of 6-Legged Robot with Actuator-Equipped Trunk and InsectInspired Body Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526

Yasuhiro Sugimoto, Yuji Kito, Yuichiro Sueoka, and Koichi Osuka

Quadruped Gait Transition from Walk to Pace to Rotary Gallopby Exploiting Head Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532

Shura Suzuki, Dai Owaki, Akira Fukuhara, and Akio Ishiguro

Exploiting Symmetry to Generalize Biomimetic Touch . . . . . . . . . . . . . . . . 540Benjamin Ward-Cherrier, Luke Cramphorn, and Nathan F. Lepora

Decentralized Control Scheme for Centipede Locomotion Based on LocalReflexes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545

Kotaro Yasui, Takeshi Kano, Dai Owaki, and Akio Ishiguro

Realization of Snakes’ Concertina Locomotionby Using “TEGOTAE-Based Control” . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548

Ryo Yoshizawa, Takeshi Kano, and Akio Ishiguro

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553

XVI Contents

Full Papers

The Natural Bipeds, Birds and Humans:An Inspiration for Bipedal Robots

Anick Abourachid1 and Vincent Hugel2(B)

1 National Museum of Natural History, Paris, Franceabourach@mnhn.fr

2 University of Toulon, Toulon, Francevincent.hugel@univ-tln.fr

Abstract. Despite many studies, the locomotion of bipedal leggedrobots is still not perfect. All the current robots are based on a humanoidmodel, which is not the unique one in Nature. In this paper we comparethe natural bipedies in order to explore new tracks to improve roboticbipedal locomotion. This study starts with a short review of the histori-cal bases of the biological bipedies to explain the differences between thestructures of the human and bird bodies. The observations on the kine-matics of bird walking describe a modular system that can be reproducedin robotics to take advantage of the bird leg versatility. For comparisonpurposes, a bird model is scaled up to have the same mass and the sameheight of the center of mass as a humanoid model. Simulation resultsshow that the bird model can execute larger strides and stay on course,compared with the humanoid model. In addition the results confirm thefunctional decomposition of the bird system into the trunk and the thighsfor the one part, and the distal part of the leg for the other part.

Keywords: Human bipedy · Bird bipedy · Kinematics · Roboticsmodeling

1 Introduction

All current operational bipedal robots are humanoid robots. The lower part iseither a mobile wheeled base, or is composed of legs with classical kinematicsinspired by human legs. Despite progress in locomotion ability, they are quitefar from the performances of human locomotion. In this paper, we explore thebipedal systems used in nature in order to find new tracks for improving robots’bipedal locomotion.

The first part presents a short review of the natural bipedal systems, humansand birds, with a short summary of the history and a description of their structure.The second section focuses on the kinematics and dynamics of walking. Section 3is dedicated to the robotic modeling of both bipeds that are tested on simulations,which are described in Sect. 4. Results are analyzed and discussed in Sect. 5.

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 3–15, 2016.DOI: 10.1007/978-3-319-42417-0 1

4 A. Abourachid and V. Hugel

The Natural Bipeds

Humans and birds are the only bipedal animals totaling 10001 strictly bipedalspecies: one species of primate and large diversity of birds. Primates are basicallyquadruped mammals. Apart from humans, some of them can walk bipedally,but typically in a transitory way, when carrying objects for instance, or fordisplay. Even if humans are able to swim, it is a exceptional behavior, andhuman are basically specialized for one locomotor behavior: walking. Birds arespecialized for the flight, but are also bipedal species. The wings are only usedfor flying, but the legs are versatile, used for hopping, walking, swimming, butalso for take-off and landing. The differences between the natural bipeds arisefrom different histories. In the primate clade, the permanent bipedal postureappeared 4 million years ago, with the origin of the genus Australopithecus.Since then, a bipedal gait was used by seven species of Homo over the past2 million years but is today used exclusively by the species Homo sapiens. Incontrast, there are 10000 extant representatives of a group of cursorial bipedaldinosaurs, the theropods, that developed the ability to fly during the Mesozoicabout 200 million years ago [1]. Some of these flying theropods survived the massextinction of the dinosaurs and gave rise to the current diversity of birds. Allthe birds have inherited the same attributes typical of a flying animal and showstick-like forelimbs supporting feathers, a rigid trunk, and a bony short tail [2].They all have three long bones in the legs: the femur, the tibiotarsus, and thetasometatarsus, and all of them are digitigrade, walking on their toes. The mainvariability in relation with the locomotor behaviors concerns the developmentof the sternal keel and the lengths of the wings for flight and the proportions ofthe leg bones. Depending on the species and their way of life, they vary from 10to 32 %, 37–56% and 13–45% of the total leg length for the femur, tibiotarsusand tarsometatarsus respectively [3]. On the contrary, all the humans belong tothe same species thus the leg proportions are very consistent with the tibia andfemur varying up to 6 % of the total leg length among extant populations ofhumans [4].

The human and bird bodies do not share the same design and geometry(Fig. 1). The human body posture is fully erected, the trunk vertical above thehip on the straight legs. The avian posture is crouched, the joints are flexed, thehips are located on the back, and the trunk is usually hanging horizontally betweenthe thighs. These differences lead to differences in the walking kinematics.

2 Walking in Human and Birds

2.1 Kinematics

The kinematics of human walking can be approximated by an inverted pendulumgait with the moving thigh and the lower leg powering the forward motion ofthe upper body [5]. The center of mass (CoM) path is guided by adjusting thepelvic rotation, pelvic tilt, lateral pelvic displacement, the knee and hip flexion,and the knee and ankle interaction [6]. The motions of the trunk and the head

The Natural Bipeds, Birds and Humans: An Inspiration for Bipedal Robots 5

Fig. 1. CoMparison between the two biological bipedal systems. (A) a theropod, closeto the birds’ ancestor, and (B) a primate, close to the human ancestor, are representedbackward. Parts of the body are shared between the birds (C) and the human (D),but the orientation of the segments are different, erected in the humans and crouchedin the birds. The 11 bones of the plantar sole, tarsus and metatarsus, included in thehuman foot, are fused and form the tarsometatarsus bone of the leg in birds. The CoMis at the hip level in humans and at the knee level in birds.

are small and play a minor role in the kinematics compared to the pelvis andlegs [6,7]. The arm swing balances the trunk torques induced by the legs [8].

The kinematics of the bird are more versatile as they use more than one loco-motor mode, depending on the species. During walking, the contribution of thethigh to the forward body motion (femur motion) is far lower than that of themore distal segments, the tibiotarsus and the tarsometatarsus [9–11]. The thighmuscles that insert on the pelvis are mainly used to move the trunk that repre-sents 80 % of the body mass, and to adjust the path of the CoM. In terrestrialbirds such as the quail (Coturnix coturnix ), during walking the trunk oscillatesaround the CoM, pitching, rolling and yawing being guided by the muscles of thethigh that cross the hip [10]. When a duck like the teal (Callonetta leucophrys)swims, the part distal to the knee is used as the paddle to propel the system. Thethigh does not move at all, anchoring the leg on the trunk similar to a paddle onthe hull of a boat. The hip is less flexible compared to that in terrestrial birdssuch as the quail. Consequently, the thigh is capable of moving the trunk lesswhen walking. The foot is put medially on the ground, allowing medio-lateraltranslation of the trunk, keeping the CoM above the support polygon. Thesemotions give the typical waddling of ducks [12]. When taking off, the birds hopbefore using the wings. All the acceleration is provided by the legs, the trunkbeing propelled first by an extension of the hip joints when the legs push on thesubstrate [13].

Thus, the birds bipedy may be considered as the association of two morpho-functional modules, the trunk plus thighs guiding the CoM in a different waydepending on the species and the locomotor behavior, and the part of the legsdistal to the knee, propelling the system. This functional modularity could par-ticipate in the versatility of the pelvic locomotor system.

6 A. Abourachid and V. Hugel

2.2 Dynamics

The comparison of birds with humans shows that in both cases the CoM is at thelevel of the third joint from the ground up: the hip in humans, the knee in birds.Above the CoM the human trunk is erected and has no significant functionin the control of the path of the CoM. In birds, the hip is positioned abovethe CoM and the thigh muscles can move the trunk to adjust the path of theCoM depending on the mechanical demands. However, during walking, the samemechanism is used in humans and birds to minimize energy expenditure, with akinetic-gravitational energy transfer during each stride [14]. This mechanism canbe modeled by the trajectory of the CoM during locomotion, successively goingup and down at each stride like in an inverted pendulum. The relation betweenthe metabolic energy consumption and the mechanical work, as function of speedand body size, is similar in birds, humans and all other mammals [15,16].

Since the two natural biped systems use a dynamic inverted pendulum modelduring walking, it is possible to use this property to model and to compare bothsystems. The next sections are dedicated to robotics modeling to test whether abird-like bipedal robot could be an alternative to a humanoid robot.

3 Robotic Modeling

In order to compare the bird system with the human system, two robotic modelswere designed for simulation (Fig. 2). The objective is to make these models walkforward inside a dynamic simulator and to check the advantages of the birdsystem over the human system.

The bird model is inspired by the quail. The leg is composed of three segments– femur, tibiotarsus, and tarsometatarsus, and one main flat toe. The humanoidrobotic model is the model used to design the ROMEO humanoid manufacturedby the Aldebaran-Robotics company. The model has two segments – femur andtibia–, and one foot per leg. For both models, the CoM, namely G, is placedat the same height hG = 0.66 m (humanoid in flexed position, see simulationSect. 4). In the bird model, the CoM is located between the knees. The toelength is made larger than the humanoid foot. Actually the toe length was fixedas twice the longitudinal distance between the heel and the knee to ensure theprojection of the CoM in the middle of the toe.

Both models have the same mass m = 40.5 kg. The mass distribution for thebird model was drawn from a real quail that was frozen and cut into pieces. Eachpiece was weighed and its CoM was calculated by the double suspension method[17]. In the humanoid model, the CoM is above the hips. The humanoid mass dis-tribution was drawn from the mass distribution of a real human being (Table 1).

The size of the humanoid model is 1.4 m. The size of the bird model is 1.1 m.This size was obtained after scaling up the quail dimensions of the different partsusing hG.

The Natural Bipeds, Birds and Humans: An Inspiration for Bipedal Robots 7

Table 1. Mass rate distribution. For the bird the foot includes the tarsometatarsusand the toes.

Quail Humanoid

Head/Neck 9 7.6

Body 63 51

Wings/arms 4× 2 5.3× 2

Femurs 5× 2 9.8× 2

Tibias 4× 2 4.3× 2

Feet 1× 2 1.3× 2

Total 100 100

Fig. 2. Robotic models of bird and humanoid.

3.1 Leg Kinematics

The leg of the bird model features two or three rotary joints at the hip, one atthe knee, one at the ankle and two joints between the tibiotarsus and the toe,named the foot joint. The hip joints include a roll joint (longitudinal axis), ayaw joint (vertical axis), and a pitch joint (lateral axis) that can be inhibitedwhen necessary to check specific inverse kinematics. The knee joint is a pitchjoint (lateral axis) that is slightly rotated in the horizontal plane. The anklejoint is also a pitch joint but with a slight inclination about the longitudinal axisaccording to the analysis of the quail walk [18]. The foot joint includes a rolljoint and a pitch joint.

The humanoid leg has a classical kinematics with three orthogonal rotaryjoints at the hip – yaw, roll, and pitch joints in this order starting from thepelvis –, a pitch joint at the knee, and a pitch and roll joint at the ankle in thisorder from the knee [19].

Table 2 summarizes the main characteristic differences between the bird modeland the humanoid model. The number of DoF for the bird is set to 6 here sincethe pitch joint is considered to be inhibited (see simulations). The noticeable dif-ferences include the position of the CoM with respect to the hips, the shape of thelegs and the distribution of the joint degrees of freedom along the leg.

8 A. Abourachid and V. Hugel

Table 2. Main characteristic differences between bird and humanoid models.

Characteristics Bird Humanoid

Nb of segments/leg 3 2

DoF 6 6

Distribution of DoF 2-1-1-2 3-1-2

(from pelvis)

Orientation of joint axes inclined (knee+ankle) straight

Legs in standing position flexed and inclined vertically stretched

(knees abducted / feet adducted)

Location of CoM middle of knees above hips

Center-of-mass vertical through middle of knees close to frontal plane

Toe/Foot 1 flat main toe (rigid) 1 flat foot (rigid)

3.2 Dynamic Model and Walking Algorithm

The dynamic model adopted for the quail robot and the humanoid robot isthe model of the 3D linear inverted pendulum (3D-LIP) where the total robot’smass m is concentrated at the top of a massless stick that joins the Zero MomentPoint (ZMP) on the ground to the suspended mass [20,21]. The ZMP can beconsidered as the center of pressure (CoP) below the supporting foot. The stickis telescopic, i.e. the stick length varies according to the variation of the leg jointangles. The height of the CoM is kept constant during the walk. This means thatthe humanoid has to flex knees before starting to walk since feet are rigid andremain horizontal during the walk. The bird robot can walk directly withoutinitial configuration. Its feet also remain horizontal. Actually the robotic feetused here are rigid and cannot roll unlike bird toes and human soles.

All the walks have instantaneous double support phases.The walking algorithm adopted for both models is decomposed into the

following steps:

Planning of Forward Foot/Toe Steps. A step is defined as the longitudinaldistance between two consecutive feet touching down. The step length is thesame along the walk and the first stride is a half step. For all models, thestep length was set to 0.25 m for the first set of simulations, then it was setto 0.33 m. The ratio of knee flexion for the humanoid was set to 0.9, thismeans that the flexion motion leads to the humanoid vertically going down10% of the initial hip height. The step duration is the same for all the walks,ts = 0.4 s.

Planning of ZMP. For each support leg, the ZMP is fixed in the middle of thefoot/toe when the CoM velocity is constant (no acceleration). The acceler-ation phase at the beginning of the walk and the deceleration phase at theend were managed separately to calculate the ZMP automatically [21]. Atstart the ZMP is shifted to the heel, whereas at stop the ZMP is shifted tothe front of the foot/toe. This is due to the longitudinal acceleration of the

The Natural Bipeds, Birds and Humans: An Inspiration for Bipedal Robots 9

CoM, which can be tuned by reducing the lengths of the first steps or byincreasing the time to execute them.

CoM Trajectory. The horizontal coordinates of the CoM, namely xG and yG, aredrawn from the equation that govern the linear inverted pendulum model [20]:

xG =g

hG(xG − xP ), yG =

g

hG(yG − yP )

where g is the gravity and xP , yP are the horizontal coordinates of the ZMP.The analytic solution leads to a hyperbolic shape of the CoM trajectory.

Inverse Kinematics. This step is used to calculate the joint commands to besent to the dynamic simulator. Here the CoM is assumed to be fixed withrespect to the trunk of the model, which is a reasonable approximation. Theinverse kinematics use the classical pseudo-inverse of the Jacobian matrixthat gives the variations of the foot trajectory as a function of the joint anglevariations.

4 Simulations

The simulator used is the Adams MSC-Software environment coupled with Mat-lab. Joint commands are calculated in Matlab and sent to Adams throughSimulink using a periodic timer. Joint position and velocity commands are usedas desired inputs to PID control blocks inside Adams to control the motors thatdrive the leg joints. There is no internal stabilizer, i.e. there is no closed-loopcontrol with any inertial measurement unit feedback, because we want to checkthe suitability of the biped model for the walk using a classical walking algorithmused in robotics.

The interaction model of feet/toes with the ground is achieved through littlespherical caps located below the soles of feet/toes. The contact of those caps withthe ground result in normal forces that are modeled using penetration depth,penetration velocity, stiffness and damping coefficient [22].

Three kinds of simulations were carried out:

1. humanoid model flexing knees with a flexion ratio of 90%, then walking 1 mforward with 0.25 m steps lasting 0.4 s each (Fig. 3A).

2. bird model walking 1 m forward with 0.25 m steps lasting 0.4 s each. All hipjoints are controlled, and the inverse kinematics process tends to minimizethe variations of all joints, and therefore to “distribute” the angle amplitudeamong all joints (Fig. 3B).

3. bird model walking 1 m forward with 0.25 m steps lasting 0.4 s each. The pitchjoint at the hip was inhibited. Consecutively most of the thrusting is achievedby the part of the leg below the knee (Fig. 3C).

The same series of simulation is reproduced with a step length of 0.33 m.

Figure 4 presents the results relative to steps of 0.25 m. The top view figuresshow the theoretical and real foot(toe)prints, the theoretical trajectory of the

10 A. Abourachid and V. Hugel

A.

B.

C.

Fig. 3. Successive 0.1 s snapshots of a 1 m forward walking with step length of 0.25 m.A: humanoid model (First 0.7 s of walk after 2 s knee flexion); B: bird model withcontrol of hip pitch joint and C: bird model with hip pitch joint inhibited (First 0.9 sof walk)

CoM, and the centers of pressure and the real CoM obtained from simulation inthe dynamic environment. Figure 5 depicts the results relative to steps of 0.33 m.The CoP was calculated at each frame from the normal force sensors located oneach spherical cap located on the corners of the feet/toes.

5 Results and Discussion

The results relative to walking steps of 0.25 m show that all models, namely thehumanoid, the quail robot with control of the hip pitch joint for walking, andthe quail model with the hip pitch joint inhibited, can follow the trajectory ofthe CoM that was planned offline (Fig. 4).

During the walk the centers of pressure (CoP) of the humanoid model are morescattered than the quailmodel (Fig. 4A).As long as theCoP remain inside the foot-prints this is not a problem, however it appears that it is more difficult to stabilizethe CoP in the case of the humanoid structure. This observation can be related tothe fact that the simulated humanoid can be considered as a double inverted pen-dulum (leg – trunk with CoM above the hips) actually whereas the simulated quailmodel can be considered as an inverted pendulum connected to a standard pendu-lum (leg – trunk with CoM below the hips), which is intrinsically more stable.

The Natural Bipeds, Birds and Humans: An Inspiration for Bipedal Robots 11

Fig. 4. Footprints of humanoid, and toeprints of bird model walking 1m forward withstep length of 0.25 m. (Color figure online)

It is noticeable that the deceleration phase in the case of the quails leads to lit-tle pitching oscillations as a significant number of centers of pressure accumulateon the fore and rear extremities of the toes in the final toeprints (Fig. 4B,C).These oscillations are responsible for a shorter path traveled compared withthe path planned, i.e. 0.92 m instead of 1 m. The pitching oscillations can beexplained by the fore and rear distributions of the mass of the trunk in thequail model than makes it more sensitive to pitching moves. The quail modelwith hip pitch joint inhibition is interesting here because the control of the hippitch joints can be decoupled and used to counteract those pitching oscillations toenhance the walk balance during acceleration/deceleration phases. This matchesthe observation from biologists that consider that the hip pitch joints play therole of controlling the movement of the trunk with respect to the legs to adjustthe posture of the bird.

12 A. Abourachid and V. Hugel

Fig. 5. Footprints of humanoid, and toeprints of bird model walking 1 m forward withstep length of 0.33 m. (Color figure online)

The results relative to walking steps of 0.33 m are really selective. Actu-ally the humanoid model cannot stay on course and suffer from disturbing yawmoves from the beginning (Fig. 5A). Centers of pressure accumulate on the lat-eral sides of the footprints and act as pivots that make the robot turn badly. Thishighlights the role of the arms in the human walk, whose back-and-forth swingcounteracts the disturbing rotation about the vertical. On the contrary the quailmodel regularly stays on course (Fig. 5B and C) except at the end because theacceleration phase is too fast. The fact that bird legs only account for 20% ofthe total mass while human legs account for 30% diminishes the influence ofthe disturbing yaw moment due to leg swing in the case of the bird model. Thisseries of simulations shows the superiority of the quail model to achieve largersteps in comparison with the humanoid model of the same mass, with the sameheight of the CoM. Here it must be noted that the distance between toes in thequail model is smaller than the distance between feet in the humanoid model.

The Natural Bipeds, Birds and Humans: An Inspiration for Bipedal Robots 13

This leads to a reduced amplitude of the oscillation of the CoM in the case ofthe quail model, which helps the quail model stay on course. Moreover, the realquail adopts a kind of catwalk by placing the landing toe in front of the othersupport toe, which constrains the amplitude of the CoM sway even more. Whenscaled up the quail model also features larger toes compared with the humanoidfoot size. This difference also contributes to increase the stride range capabilitiesof the quail model.

The simulation results are interesting but the implementation of the roboticmodels has some limitations that must be taken into account in the analysis.The first limitation is the walking algorithm that assumes that feet and toesremain flat all along the walk. A consequence is that the humanoid leg and theset of tibiotarsus and metatarsus in the quail model tend to get stretched at theend of the stance phase when the step length is increased, which is not efficientas a starting position to raise the leg, and therefore limits the stride length. Thiscan explain why the natural bipeds roll the foot/toe sole in the stance phasebefore raising it in order to avoid stretched legs in the rear position. In the caseof the humanoid model, the rolling of feet can be useful to avoid the flexed-kneewalk, and to set up human-like walks with folding and unfolding legs.

In addition the walking algorithms do not include a specific management ofthe starting and ending phases of the walk. In cruise mode the CoP is planned tostay in the middle of the footprint but in the starting phase the CoP is pushedbackwards to initiate the walk, and in the end phase, the CoP is pushed forwardsto halt the motion. At last the joints modeled in the dynamic simulator are rigid.There is no flexible joint, neither compliance to store/restore energy as occursin the natural biped legs.

6 Conclusion

This study has described the biological differences between the human bipedsand the bird bipeds, from a kinematics and a dynamics point of view. Based onthese observations, a robotics humanoid model and a robotic quail-like modelwere designed to compare the walking capabilities of both structures. Simulationscarried out inside a dynamic environment have highlighted a major capability ofthe quail model over the humanoid model, namely the capability to increase thestride length while maintaining the course. Besides, the control of the hip pitchjoint in the quail model can be decoupled and used to counteract the pitchingoscillations of the trunk in the sagittal plane, the part of the bird leg belowthe knee being dedicated to thrusting the system. This confirms the mechanicaladvantage of the modular organization of the bird’s bipedy with a trunk-thighmodule and a distal module.

Future work will explore the mechanical consequences of the differences inthe body proportions observed in birds. Up to now only the ratios of leg segmentsrelative to the quail were used in the simulation study. The limb ratios of otherbirds could be more suited to the design of a robotic biped. Another researchstudy will focus on the design of rolling toes in order to increase the walkingefficiency of the robotic model.

14 A. Abourachid and V. Hugel

References

1. Hope, S.: The Mesozoic radiation of Neornithes. In: Chiappe, L.M., Witmer, L.M.(eds.) Mesozoic Birds, Above the Head of Dinosaurs, pp. 339–388. University ofCalifornia Press, Berkeley (2002)

2. Norberg, U.M.L.: Vertebrate Flight. Springer, Berlin (1990)3. Gatesy, S.M., Middleton, K.M.: Bipedalism, flight and the evolution of theropod

locomotor diversity. J Vertebr. Paleontol. 17, 308–329 (1997)4. Porter, A.M.W.: Modern human, early human and Neanderthal limb proportions.

Int. J. Osteoarchaeology 9, 54–67 (1999)5. Pontzer, H., Holloway 3rd, J.H., Raichlen, D.A., Lieberman, D.E.: Control and

function of arm swing in human walking and running. J. Exp. Biol. 212, 523–534(2009)

6. Hayot, C.: 3D biomechanical analysis of the human walk: comparison of mechanicalmodels. PhD thesis, Poitiers (in French), France (2010)

7. Winter, D., Quanbury, A., Reimer, G.: Analysis of instantaneous energy of normalgait. J. Biomech. 9, 253–257 (1976)

8. Li, Y., Wang, W., Crompton, R.H., Gunther, M.M.: Free vertical moments andtransverse forces in human walking and their role in relation to arm-swing. J. Exp.Biol. 204, 47–58 (2001)

9. Gatesy, S.M.: Guineafowl hind limb function. I cineradiographic analysis and speedeffects. J. Morphol. 240, 115–125 (1999)

10. Abourachid, A., Hackert, R., Herbin, M., Libourel, P.A., Lambert, F.O., Gioanni,H., Provini, P., Blazevic, P., Hugel, V.: Bird terrestrial locomotion as revealed by3D kinematics. Zoology (Jena) 114, 360–368 (2011)

11. Stoessel, A., Fischer, M.: Comparative intralimb coordination in avian bipedallocomotion. J. Exp. Biol. 215, 4055–4069 (2012)

12. Provini, P., Tobalske, B.W., Crandell, K.E., Abourachid, A.: Transition from legto wing forces during take-off in birds. J. Exp. Biol. 215, 4115–4124 (2012)

13. Provini, P., Goupil, P., Hugel, V., Abourachid, A.: Walking, paddling, waddling:3D kinematics of Anatidae locomotion (Callonetta leucophrys). J. Exp. Zool. 317,275–282 (2012)

14. Cavagna, G.A., Heglung, N.C., Taylor, R.: Mechanical work in terrestrial locomo-tion: two basic mechanisms for minimizing energy expenditure. Am. J. Physiol. 2,243–261 (1977)

15. Taylor, C.R., Heglund, N.C., Maloy, G.M.O.: Energetics and mechanics of terres-trial locomotion I Metabolic energy consumption as a function of speed and bodysize in birds and mammals. J. Exp. Biol. 97, 1–21 (1982)

16. Fedak, M.A., Heglund, N.C., Taylor, C.R.: Energetics and mechanics of terrestriallocomotion II kinetic energy changes of the limbs and body as a function of speedand body size in birds and mammals. J. Exp. Biol. 79, 23–40 (1982)

17. Abourachid, A.: Mechanics of standing in birds: functional explanation of lamnessproblems in giant turkeys. Br. Poult. Sci. 34, 887–898 (1993)

18. Hugel, V., Hackert, R., Abourachid, A.: Kinematic modeling of bird locomotionfrom experimental data. IEEE Trans. Robot. 27(2), 185–200 (2011)

19. Zorjan, M., Hugel, V.: Generalized humanoid leg inverse kinematics to deal withsingularities. In: IEEE International Conference on Robotics and Automation, pp.4791–4796 (2013)

20. Kajita, S.: Humanoid Robot. Ohmsha Ltd, 3–1 Kanda Nishikicho, Chiyodaku,Tokyo, Japan (2005)

The Natural Bipeds, Birds and Humans: An Inspiration for Bipedal Robots 15

21. Hugel, V., Jouandeau, N.: Walking patterns for real time path planning simulationof humanoids. In: IEEE RO-MAN, pp. 424–430 (2012)

22. Zorjan, M., Hugel, V., Blazevic, P., Borovac, B.: Influence of rotation of humanoidhip joint axes on joint power during locomotion. Adv. Robot. 29(11), 707–719(2015)

Retina Color-Opponency Based PursuitImplemented Through Spiking Neural Networks

in the Neurorobotics Platform

Alessandro Ambrosano1(B), Lorenzo Vannucci1, Ugo Albanese1,Murat Kirtay1, Egidio Falotico1, Pablo Martınez-Canada6, Georg Hinkel4,

Jacques Kaiser2, Stefan Ulbrich2, Paul Levi2, Christian Morillas6, Alois Knoll5,Marc-Oliver Gewaltig3, and Cecilia Laschi1

1 The BioRobotics Institute, Scuola Superiore Sant’Anna,Viale R. Piaggio 34, 56025 Pontedera, Italy

alessandro.ambrosano@sssup.it2 Department of Intelligent Systems and Production Engineering (ISPE IDS/TKS),

FZI Research Center for Information Technology, Haidund-Neu-Str. 10-14,76131 Karlsruhe, Germany

3 Blue Brain Project (BBP), Ecole polytechnique federale de Lausanne (EPFL),Campus Biotech, Batiment B1, Ch. des Mines 9, 1202 Geneva, Switzerland

4 Department of Software Engineering (SE), FZI Research Center for InformationTechnology, Haid-und-Neu-Str. 10-14, 76131 Karlsruhe, Germany

5 Department of Informatics, Technical University of Munich,Boltzmannstrae 3, 85748 Garching, Germany

6 Department of Computer Architecture and Technology,CITIC, University of Granada, Granada, Spain

Abstract. The ‘red-green’ pathway of the retina is classically recognizedas one of the retinal mechanisms allowing humans to gather color infor-mation from light, by combining information from L-cones and M-conesin an opponent way. The precise retinal circuitry that allows the oppo-nency process to occur is still uncertain, but it is known that signals fromL-cones and M-cones, having a widely overlapping spectral response,contribute with opposite signs. In this paper, we simulate the red-greenopponency process using a retina model based on linear-nonlinear analy-sis to characterize context adaptation and exploiting an image-processingapproach to simulate the neural responses in order to track a moving tar-get. Moreover, we integrate this model within a visual pursuit controllerimplemented as a spiking neural network to guide eye movements in ahumanoid robot. Tests conducted in the Neurorobotics Platform confirmthe effectiveness of the whole model. This work is the first step towardsa bio-inspired smooth pursuit model embedding a retina model usingspiking neural networks.

1 Introduction

One of the most important characteristics of the primate visual system is rep-resented by the space-variant resolution retina with a high-resolution fovea thatc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 16–27, 2016.DOI: 10.1007/978-3-319-42417-0 2

Retina Color-Opponency Based Pursuit 17

offers considerable advantages for a detailed analysis of visual objects. Whenlight hits the retina, photoreceptor cells transform it in electrochemical signals,which get furtherly processed first by flowing through bipolar cells and thenthrough ganglion cells, whose axons form the optic nerve. Two more types ofcells, horizontal and amacrine, participate in this process by connecting multi-ple photoreceptors and bipolar cells respectively, acting as feedback channels.

All these cells cooperate in a complex “retina circuit”, that can be splitin many different “microcircuits” where around 80 types of identified neuralcell types take part. These microcircuits process simultaneously the light andforward different information to the brain through dedicated pathways. Colorinformation is carried by some of these pathways, combining signals from thecone photoreceptors, which are sensitive to coloured light [1].

There are three types of cone cells in the human retina, each one sensitive toa different wavelength range: S-cones are sensitive to short wavelengths, M-conesto middle wavelengths and L-cones to long wavelengths, corresponding roughlyto blue, green and red light respectively. Both the density of photons on the coneand the color of the light determine the probability that an individual cone willcapture a photon [2]. For this reason, signals from a single cone type can’t carryany color clue, but at least two types of cone must be compared in order to getactual color information.

Two processes of this sort are known to happen inside the retina: the red-greenopponency, where M-cones and L-cones responses are taken into account in orderto get color information in the red-green axis, and yellow-blue opponency, whereall three kinds of cone are considered to get color information in the yellow-blueaxis [3]. They are called opponency mechanisms because in both cases some conetype contribute with a positive weight and some other with a negative weight. Inthe red-green case we have M-cones opposed to L-cones whereas in the yellow-blue case the joint effect of M-cones and L-cones is opposed to S-cones response,though the exact circuitry of both processes is still under investigation.

The space-variant resolution of the retina requires efficient eye movements forcorrect vision. Two forms of eye movements — saccades and smooth pursuit —enable us to fixate the object on the fovea. Saccades are high-velocity gaze shiftsthat bring the image of an object of interest onto the fovea. The purpose of smoothpursuit eye movements is to minimise the retinal slip, i.e. the target velocity pro-jected onto the retina, stabilizing the image of the moving object on the fovea.Retinal slip disappears once eye velocity catches up to target velocity in smoothpursuit eye movements. In primates, with a constant velocity or a sinusoidal targetmotion, the smooth pursuit gain, i.e. the ratio of tracking velocity to target veloc-ity, is almost 1.0. In the last years several models of robotic visual pursuit havebeen developed. Shibata and colleagues suggested a control circuits for the integra-tion of the most basic oculomotor behaviours [4] including the smooth pursuit eyemovement. A similar model of smooth pursuit and catch-up saccade [5] was imple-mented on the iCub robot. Also models based on artificial neural networks weredeveloped for visual tracking tasks [6–9]. In this paper we present a first attemptof embedding a retina model inside a visual pursuit controller suitable for a robotic

18 A. Ambrosano et al.

implementation. The controller, taking as input the output of the retina can gen-erate an appropriate motor command to follow a moving target without needingfor computer vision algorithm in extracting the target position from the incomingvisual image. Moreover, the controller uses biologically inspired Spiking NeuralNetworks in order to implement parts of the controller, thus a proper frameworkthat combines robotics and neural simulations has been used: the NeuroroboticsPlatform.

2 The Retina Simulation Platform

A great number of models have been proposed to reproduce different process-ing mechanisms of the retina [10–13]. However, these are often ad hoc modelsfocused on fitting some specific retina functions rather than providing a gen-eral retina simulation platform. We chose the retina simulator COREM [14,15],which includes a set of computational retinal microcircuits that can be used asbasic building blocks for the modeling of different retina functions. In addition,COREM implements an interface with Spiking Neural Networks (SNNs) thatallows its integration with models of higher visual areas and the NeuroroboticsPlatform.

The computational retinal microcircuits that can be configured withinCOREM consist of one spatial processing module (a space-variant Gaussian fil-ter), two temporal modules (a low-pass temporal filter and a single-compartmentmodel), a configurable time-independent nonlinearity and a Short-Term Plastic-ity (STP) function. The simulation engine allows the user to create custom retinascripts and to easily embed the retina model in the neural simulator.

3 The Neurorobotics Platform

The Neurorobotics Platform (NRP) is developed as part of the Human BrainProject1 to run coupled neural and robotics simulations in an interactive plat-form. Whereas there are multiple neural simulators (e.g. Neuron [16], NEST [17]),robotics and physics simulators (e.g. Gazebo [18]), the NRP aims at offering a plat-form that combines the neuroscientific and robotic fields, by providing coupledphysics and neural simulations. A core part of the NRP is the Closed-Loop-Engine(CLE) that allows to specify the data exchange between the brain simulation andthe robot in a programmatic manner and orchestrates the simulations.

The key concept of the NRP is offering scientists an easy access to a sim-ulation platform using a state-of-the-art web interface. Scientists are relievedfrom the burdensome installation process of scientific simulation software andare able to leverage large-scale computing resources. Furthermore, support formonitoring and visualizing the spiking activity of the neurons or joint states ofthe robot is offered as well as the camera image perceived by the robot.

To give an impression on how the platform looks like, a screenshot of a visualpursuit experiment with the iCub [19] humanoid robot is depicted in Fig. 1.1 https://www.humanbrainproject.eu/.

Retina Color-Opponency Based Pursuit 19

Fig. 1. Screenshot showing a pursuit experiment with the iCub robot in the Neuroro-botics Platform (NRP). A plot at the top shows spiking activity of neurons.

The Closed-Loop-Engine (CLE), depicted in Fig. 2, implements the core ofthe NRP. The CLE controls the neural and the world simulation, and executesa lightweight simulation data interchange mechanism. Neural and robotics sim-ulation are iteratively run in parallel for the same amount of time, after whichthe transfer functions (special functions that translate output from one simu-lation into input for the other) are executed periodically. Communication withthe brain is realized through recording and injecting spike data. Interfacing withthe robot simulation is done using the Robot Operating System middleware(ROS [20]). In order to ensure reproducibility, data exchange is conducted in adeterministic fashion.

3.1 Integrating the Retina Simulation Platform in the NRP

The original retina codebase [15] was implemented as a stand-alone program. Sothe first step towards the embedding in the NRP has been to isolate all the corefunctionalities of the model in a separate module, that could be used as a library.The original executable has been kept as a “frontend” application depending onsuch library.

In order to have a retina model involved in the robot control inside a NRPsimulation, it is necessary to forward the camera image or the robot to the retina,get the retina output information so it can be processed in a SNN, and finallydecode the SNN output to control the robot. All these steps must take placeinside transfer functions.

The whole retina model is written in C++, whereas all the NRP back-end, including transfer function related business logic, is written in Python.

20 A. Ambrosano et al.

NeuronalNetwork

Simula on

Neuron2RobotTransfer Func on

Robot2NeuronTransfer Func on

Robo csSimula on

Spike Sink

Spike Source

Robot Publisher

Robot Subscriber

Fig. 2. A closed loop between a robotics simulation and a neural network

In particular, in order to easy the development of these transfer function mod-ules, the part of the transfer functions that specify the connection to and fromboth simulations are implemented using a subset of python that defines a DomainSpecific Language [21]. Thus, to allow the NRP to send input to the retina andget output from it, the functions in the retina library involved in these processeshave been wrapped with Python bindings, making them accessible from Pythoncode.

4 Retina Based Controller

We tested the retina-NRP interaction by creating a custom experiment on theplatform performing a visual tracking task with a humanoid robot. Since ourfocus is on the red-green opponency, we created an environment in which asimulated iCub [19] humanoid robot will have to track a green ball movinghorizontally on a red background.

In the experiment, image from one of the two cameras mounted on the roboteyes is sent to the retina simulator. Retinal output is then gathered and onehorizontal stripe of the output image (320 pixels from a 320× 240 image), inter-secting the target, is forwarded to a neural network, where the spikes are filteredin order to avoid any undesired false detection. Finally, spike trains coming fromthe SNN are translated in a motion command for the robot eyes, making themfollowing the target. An overview of the controller is shown in Fig. 3.

4.1 Retina Circuit

The retina circuit implemented with the framework described in Sect. 2 is com-posed of two symmetric subcircuits processing independently the input image.In the first layer we have simulated L-cones and M-cones, receiving the initial

Retina Color-Opponency Based Pursuit 21

image. Output from the photoreceptors is then forwarded to the two differentsubcircuits, namely the L+M− circuit and the L−M+ circuit.

The L+M− circuit (L−M+ description can be derived by symmetry) sumsthe output signals from the two cones, giving a positive weight to L-cones anda negative weight to M-cones. Combined cones signal is processed by simulatedhorizontal, bipolar, amacrine and ganglion cells. Every simulated ganglion cell isthen sensible to a red center on green background within its receptive field. Thehigh level result is a sensitivity to borders on a static image, which is slightlyaccentuated around borders between red and green objects for both circuits.In case of moving images instead, ganglion cells in the L+M− are particularlysensitive, due to their temporal characteristics, to green objects appearing inreceptive fields that were earlier impressed by red objects. We will exploit thispeculiarity to infer the position of an object by combining responses from thetwo circuits.

4.2 Brain Model

The spiking neural network used in our experiment comprises 1280 integrate andfire neurons [22], organized in two layers. The first layer acts as a current to spikeconverter, it has been designed for taking current intensities value coming fromthe retina library to neural spikes. In the second layer, every neuron, except the“side” ones, gather information from 7 subsequent neurons on the first layer,acting as a local spike integrator.

Every layer embeds two independent populations of 320 neurons, processingseparately output from the two different ganglion cell layers of the retina circuit,so there is a one to one correspondence between considered pixels and neuronsin a single first layer population. Thus, neurons from 1 to 320 will get inputfrom the first ganglion circuit and forward spikes to neurons from 641 to 960,and neurons from 321 to 640 will get input from the second ganglion circuit andforward spikes to neurons from 961 to 1280.

The summation occurring in the second layer, together with linearly decreas-ing synapse weights from the center of the image to the periphery, serves as afilter for avoiding undesired spikes in peripheral regions of sight.

4.3 Transfer Functions

The robot controller is implemented by means of two transfer functions (TFs),one robot to neuron and one neuron to robot.

In the robot to neuron TF, the image on the eye camera of the robot iscollected and forwarded to the retina library, updating its status. Outcome fromthe ganglion layers of the retina circuit are processed, then current values for astrip of pixel containing the target are transmitted to the two populations in thefirst layer of the brain.

Output spikes from the second brain layer are then processed by the secondTF with the following steps:

22 A. Ambrosano et al.

Fig. 3. Block diagram of the implemented robot controller. (Neuron image from AmeliaWattenberger, released under Creative Commons License 3.0 https://thenounproject.com/term/neuron/79860/)

– Spike counts for every neuron on the two output population are gathered intwo collections SC1 and SC2, that we represent as two functions

SC1, SC2 : [1, 320] → N

where [1, 320] is a discrete set, representing a neural population.– As higher stimulation of ganglions correspond to higher spike frequency in the

brain, we expect one clearly distinguished frequency maximum per population,the first one corresponding to the pixel where the ball enters the receptive fieldin the first ganglion layer and the other to the pixel where the ball leaves thereceptive field in the second ganglion layer. Since we are using one neuron perpixel on an horizontal stripe of pixels, the two maxima will correspond to twocolumn indexes of the original image, which we will call p1 and p2. For eachpopulation we obtain the said column index by computing first the maximumneural spike rate, then taking the first neuron index with that spike rate.In formulas, p1 and p2 are defined as

p1 = min{arg maxx

{SC1(x)}}

p2 = min{arg maxx

{SC2(x)}}– We take as an estimate of the target center the value

p =p1 + p2

2

Retina Color-Opponency Based Pursuit 23

– The estimated position of the ball p is then converted to an estimated anglep with respect to the eye center with the equation

p = − arctan(p − 160

160

)

– The eye position angle command ect+1, depending on the current eye positionet is computed as follows

ect+1 = et + 0.2p

where the constant 0.2 is determined empirically and prevents quick eye move-ments that could result in the target loss from the robot sight.

5 Results

5.1 Target Detection

In this experiment the controller has been validated by testing only its targetdetection capabilities without any actuation on the robot. The experiment was

1 2 3 4 5 6 7 8Time (s)

-0.3

-0.2

-0.1

0

0.1

0.2

Angl

e (ra

d)

Target positionTarget estimated position

(a)

1 2 3 4 5 6 7 8Time (s)

0

320

640

960

Neur

on ID

(b)

Fig. 4. The computed ball position (a) and the brain response (b) for a ball withsinusoidal motion moving at 0.5 Hz, robot eyes still. (Color figure online)

24 A. Ambrosano et al.

started with a slightly different neuron to robot transfer function which didn’tsend any control message back to the robot and with the target ball movingwith a sinusoidal trajectory with a cycle frequency of 0.5 Hz, covering almosthalf the field of vision of the camera. During the experiment, data about bothtarget perceived location and the neural network response was collected and itis reported in Fig. 4.

At the beginning of the simulation, the robot suddenly moves to a defaultposition, while the controller responsible for the upright position starts. For thisreason, the camera image during the first 2 seconds of simulation may be affectedby this movement and thus its elaboration may provide wrong results. In Fig. 4awe can observe how the estimated target position, after the robot stabilization,follows a sinusoidal trend. The target position is plotted with a dashed line as areference. Sensitivity of different ganglion inputs can be noticed in the first halfof the spike plot of Fig. 4b, where the first population spikes in correspondenceto the target entering the receptive field and the second spikes when the target

Fig. 5. Tracking results with sinusoidal (a) and random linear (b) trajectories (Colorfigure online)

Retina Color-Opponency Based Pursuit 25

leaves the receptive field. The filtering action of the second neural layer can beobserved in the second half of the spike plot.

5.2 Target Pursuit

In these experiments we used the controller enabling motion commands with thesame setup of the previous experiment. We tested the controller with a targetmoving with a sinusoidal trajectory (Fig. 5a) and with a random linear trajectory(Fig. 5b), namely a linear trajectory changing direction (left or right) randomlyevery second. For the same reasons explained in the previous test, we wait forthe robot stabilization before starting actuating the eye. It can be observed howthe target estimation is still effective even with a moving eye, which validatesour choice of a retina simulator as a data source for the controller.

Comparing our approach with a purely image-processing based target detec-tion [23], we can easily observe a greater estimate error on our controller, thatcan be noticed easily in Fig. 5b at seconds 5, 10 and 12. A one to one comparison,though, is imprecise, as in the image-processing approach all the pixels of thecamera image are processed by the controller, whereas in our implementationwe consider just one stripe of 320 pixels.

6 Discussion and Future Works

In this paper we propose a retinal red-green opponency based robot controller,processing image from the outside world through a retina model instead of usingclassic image-processing methods. We set up a framework by integrating anexisting retina framework in the NRP, implementing a custom retina circuitand a custom neural network and finally setting up an experiment in the NRPwith suitable transfer functions. Two experiments allowed us to validate theeffectiveness of this setup for both detection and pursuit of a moving green targeton a red background, although with the limitations of a single retina pathwayand a single image stripe processed by the brain model. This work representsa first attempt towards a bio-inspired visual pursuit controller embedding aretina model. In future works, we plan to extend the “field of view” of thecontroller to the whole image, after a careful design of the pursuit controllerin order to simulate a brain function areas involved in the smooth pursuit eyemovement through spiking neural networks. We will improve our retina circuitto simulate also space-variant resolution and try to model and combine moreretinal pathways concurring in color detection or motion detection. In order tointerface this controller model with a real robotic platform, the retina modelcould be interfaced with real-time neural simulations such a the ones providedby neuromorphic hardware like SpiNNaker [24].

Acknowledgements. The research leading to these results has received funding fromthe European Union Seventh Framework Programme (FP7/2007-2013) under grantagreement no. 604102 (Human Brain Project). The authors would like to thank the

26 A. Ambrosano et al.

Italian Ministry of Foreign Affairs, General Directorate for the Promotion of the “Coun-try System”, Bilateral and Multilateral Scientific and Technological Cooperation Unit,for the support through the Joint Laboratory on Biorobotics Engineering project.

References

1. Dacey, D.M.: Primate retina: cell types, circuits and color opponency. Prog. RetinalEye Res. 18(6), 737–763 (1999)

2. Baylor, D., Nunn, B., Schnapf, J.: Spectral sensitivity of cones of the monkeyMacaca fascicularis. J. Physiol. 390, 145 (1987)

3. Dacey, D.M., Packer, O.S.: Colour coding in the primate retina: diverse cell typesand cone-specific circuitry. Curr. Opin. Neurobiol. 13(4), 421–427 (2003)

4. Shibata, T., Vijayakumar, S., Conradt, J., Schaal, S.: Biomimetic oculomotor con-trol. Adapt. Behav. 9(3–4), 189–207 (2001)

5. Falotico, E., Zambrano, D., Muscolo, G., Marazzato, L., Dario, P., Laschi, C.:Implementation of a bio-inspired visual tracking model on the icub robot. In:Proceedings of IEEE International Workshop on Robot and Human InteractiveCommunication, pp. 564–569 (2010)

6. Vannucci, L., Cauli, N., Falotico, E., Bernardino, A., Laschi, C.: Adaptive visualpursuit involving eye-head coordination and prediction of the target motion. In:IEEE-RAS International Conference on Humanoid Robots, pp. 541–546 (2014)

7. Vannucci, L., Falotico, E., Di Lecce, N., Dario, P., Laschi, C.: Integrating feed-back and predictive control in a bio-inspired model of visual pursuit implementedon a humanoid robot. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A., Prescott,T.J. (eds.) Living Machines 2015. LNCS (LNAI), vol. 9222, pp. 256–267. Springer,Heidelberg (2015)

8. Zambrano, D., Falotico, E., Manfredi, L., Laschi, C.: A model of the smooth pursuiteye movement with prediction and learning. Appl. Bionics Biomech. 7(2), 109–118(2010)

9. Falotico, E., Taiana, M., Zambrano, D., Bernardino, A., Santos-Victor, J., Dario,P., Laschi, C.: Predictive tracking across occlusions in the icub robot. In: 9thIEEE-RAS International Conference on Humanoid Robots, HUMANOIDS 2009,pp. 486–491 (2009)

10. Benoit, A., Caplier, A., Durette, B., Herault, J.: Using human visual system mod-eling for bio-inspired low level image processing. Comput. Vis. Image Underst.114(7), 758–773 (2010)

11. Wohrer, A., Kornprobst, P.: Virtual retina: a biological retina model and simulator,with contrast gain control. J. Comput. Neurosci. 26(2), 219–249 (2009)

12. Herault, J., Durette, B.: Modeling visual perception for image processing. In:Sandoval, F., Prieto, A.G., Cabestany, J., Grana, M. (eds.) IWANN 2007. LNCS,vol. 4507, pp. 662–675. Springer, Heidelberg (2007)

13. Morillas, C.A., Romero, S.F., Martınez, A., Pelayo, F.J., Ros, E., Fernandez, E.:A design framework to model retinas. Biosystems 87(2), 156–163 (2007)

14. Martınez-Canada, P., Morillas, C., Pino, B., Ros, E., Pelayo, F.: A computationalframework for realistic retina modeling. Int. J. Neural Syst. (Accepted for publi-cation)

15. Martınez-Canada, P., Morillas, C., Nieves, J.L., Pino, B., Pelayo, F.: First stageof a human visual system simulator: the retina. In: Tremeau, A., Schettini,R., Tominaga, S. (eds.) CCIW 2015. LNCS, vol. 9016, pp. 118–127. Springer,Heidelberg (2015)

Retina Color-Opponency Based Pursuit 27

16. Hines, M.L., Carnevale, N.T.: The NEURON simulation environment. Neural Com-put. 9(6), 1179–1209 (1997)

17. Gewaltig, M.O., Diesmann, M.: NEST (NEural Simulation Tool). Scholarpedia2(4), 1430 (2007)

18. Koenig, N., Howard, A.: Design and use paradigms for gazebo, an open-sourcemulti-robot simulator. In: Proceedings of the IEEE/RSJ International Conferenceon Intelligent Robots and Systems (IROS 2004), vol. 3, pp. 2149–2154. IEEE (2004)

19. Metta, G., Natale, L., Nori, F., Sandini, G., Vernon, D., Fadiga, L., Von Hofsten,C., Rosander, K., Lopes, M., Santos-Victor, J., et al.: The iCub humanoid robot: anopen-systems platform for research in cognitive development. Neural Netw. 23(8),1125–1134 (2010)

20. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R.,Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop onOpen Source Software, vol. 3, p. 5 (2009)

21. Hinkel, G., Groenda, H., Vannucci, L., Denninger, O., Cauli, N., Ulbrich, S.: Adomain-specific language (DSL) for integrating neuronal networks in robot control.In: ACM International Conference Proceeding Series, pp. 9–15, 21 July 2015

22. Brette, R., Gerstner, W.: Adaptive exponential integrate-and-fire model as an effec-tive description of neuronal activity. J. Neurophysiol. 94(5), 3637–3642 (2005)

23. Vannucci, L., Ambrosano, A., Cauli, N., Albanese, U., Falotico, E., Ulbrich, S.,Pfotzer, L., Hinkel, G., Denninger, O., Peppicelli, D., Guyot, L., Von Arnim, A.,Deser, S., Maier, P., Dillman, R., Klinker, G., Levi, P., Knoll, A., Gewaltig, M.O.,Laschi, C.: A visual tracking model implemented on the iCub robot as a use casefor a novel neurorobotic toolkit integrating brain and physics simulation. In: IEEE-RAS International Conference on Humanoid Robots, pp. 1179–1184 (2015)

24. Painkras, E., Plana, L.A., Garside, J., Temple, S., Galluppi, F., Patterson, C.,Lester, D.R., Brown, A.D., Furber, S.B.: SpiNNaker: A 1-w 18-core system-on-chip for massively-parallel neural network simulation. IEEE J. Solid-State Circuits48(8), 1943–1953 (2013)

A Two-Fingered Anthropomorphic Robotic Handwith Contact-Aided Cross Four-Bar Mechanisms

as Finger Joints

Guochao Bai, Jieyu Wang, and Xianwen Kong(✉)

Heriot-Watt University, Edinburgh EH14 4AS, UK{gb9,jw26,X.Kong}@hw.ac.uk

Abstract. This paper presents an anthropomorphic design of a robotic fingerwith contact-aided cross four-bar (CFB) linkages. Anatomical study shows thatfinger joints have a complex structure formed by non-symmetric surfaces andusually produce complex movement than a simple revolute motion. The articularsystem of human hand is firstly investigated. Kinematics of a CFB mechanism isthen analyzed and computer aided design of fixed and moving centrodes of CFBmechanism is presented. Gripping analysis of human hand shows two easilyignored components of a finger, fingernail and soft fingertip. Based on the rangeof motion of the joints of the most flexible thumb finger, a two-joint anthropo‐morphic finger is developed by using contact-aided CFB linkages which can alsobe used for joint design of prosthetic knee. Prototype of a two-fingered hand ismanufactured by using 3D printing technology and gripping of a wide range ofobjects is tested.

Keywords: Robotic finger · Anthropomorphic · Contact-aided · Cross four-bar

1 Introduction

It is known that investigating anthropomorphic robotic hands may help understandhuman hand. The grasping and manipulation abilities of current robotic hand are far lessdexterous than the human hand, even though significant progresses have been made inthe past four decades [1–3]. The design considerations include number of fingers, joints,DOF (degree-of-freedom), range of motion for each joint, speed of movements, andforce generation capability [4]. Design of a versatile and robust robotic hand that hasthe similar dexterity as human hand is an undisputedly challenging task.

A finger of the human hand possesses several biological features which are hard tomimic simultaneously [5]. They include: (1) Unique shape of bones at the joints todetermine DOF of joints; (2) Joint capsule formed by ligaments to limit the range ofmotion of joint; (3) Cartilage and synovial fluid for low-friction contact between bonesurfaces; (4) Non-linear interactions between tendons and bones to dynamically deter‐mine the motion of finger.

In a human hand, the anatomical structure and nervous system have significantcontribution to its ability. Anatomical study shows that finger joints have a complex

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 28–39, 2016.DOI: 10.1007/978-3-319-42417-0_3

structure formed by non-symmetric surfaces and usually produce complex movementthan a simple revolute motion.

Most of the existing robotic hands are connected with revolute joints betweenphalanges in order to achieve the required DOF and kinematic characteristics, such ashinges, gimbals, cables, gears or belts. Compliant materials are recently used as joints.Xu proposed one type of MCP joint whose biomechanics and dynamic properties areclose to human counterparts [6]. The artificial joint is composed of a ball joint, crochetedligaments, and a silicon rubber sleeve which is distinctive to the other finger joints.

Many robotic hands are driven by gears [7, 8] or tendon [9]. There are also smartmotor [10] and air muscle [11] types. The function of these drive approaches is to makethe trajectory of a fingertip similar to the typical human trajectory during reaching andgrasping objects.

In this paper, a novel type of finger is proposed (see Fig. 1). Unlike most of thecurrent finger design, fingernail and soft fingertip are considered for specific gripping.Firstly, the articular system of human hand including bones and joints is presented.Physical parameters of finger skeleton and approximate limits of joints’ motion aresummarized. Secondly, kinematics of cross four-bar (CFB) linkage is analyzed and thena contact-aided CFB mechanism is generated for the design of finger joint and knee.Thirdly, based on the limits of joints’ motion referred in Sect. 2, a two-joint thumb fingeris designed and the mechanism is analyzed. Then the developing process is summarizedand design approach is proposed. Finally, a prototype of artificial finger consideringfunction of fingernail and soft fingertip is fabricated using 3D printing and some graspingtests of finger are conducted to verify the performance of the finger.

Fig. 1. Contact-aided joints and motion of a robotic finger.

A Two-Fingered Anthropomorphic Robotic Hand 29

2 Articular System of Human Hand

In order to mimic human hand for grasping objects, articular system of the human handis investigated. The accurate models of the human finger have been proposed based onanatomical studies [12, 13]. The bone and articular structure of human hand are two keycomponents to be imitated because the structures and their relative movements haveessential effects on grasping objects. By analyzing and learning the articular system,similar mechanisms can be chosen. Human bones and articulations are introducedbriefly. DOF and range of motion of each joint are summarized in literature [14, 15].

We use the subscripts I for the thumb, II for the index, III for the middle, IV four thering, and V for the small fingers. Three types of hand bones, carpals, metacarpals andphalanges are shown in Fig. 2. The names of joints are nominated according to handbones, such as carpalmetacarpal joint (CMC, for short) which locates between carpaland metacarpal. Hand fingers are separated into proximal, middle and distal phalanges,excepting thumb finger which has only two phalanges.

Fig. 2. Bones and joints (left hand anterior view).

30 G. Bai et al.

The definition of movements of each finger in terms of extension/flexion, abduction/adduction and hyper-extension which means the finger over pass the dotted line, shownin Fig. 3.

Fingers play an important role in grasping and manipulation. Research on move‐ments of human fingers gives us targets to mimic. Tables 1 and 2 show the range ofmotion of joints and lengths of phalanges of the fingers.

Table 1. Range of movements of the finger joints(H refers to hyper extension) [16].

Fingers Joints Action Ranges(in degree)Thumb CMC Adduction/Abduction 0(contact)/60

Extension/Flexion 25/35MCP Adduction/Abduction 0/60

Extension/Flexion 10H/55IP Extension/Flexion 15H/80

Index MCP Adduction/Abduction 13/42Extension/Flexion 0/80

PIP Extension/Flexion 0/100DIP Extension/Flexion 10H/90

Middle MCP Adduction/Abduction 8/35Extension/Flexion 0/80

PIP Extension/Flexion 0/100DIP Extension/Flexion 10H/90

Ring MCP Adduction/Abduction 14/20Extension/Flexion 0/80

PIP Extension/Flexion 0/100DIP Extension/Flexion 20H/90

Small MCP Adduction/Abduction 19/33Extension/Flexion 0/80

PIP Extension/Flexion 0/100DIP Extension/Flexion 30H/90

Fig. 3. Flexion/extension, abduction/adduction of middle finger [16].

A Two-Fingered Anthropomorphic Robotic Hand 31

Table 2. Length range of physical parameters of the finger skeleton.

Phalanges Length(in millimeter)Proximal phalanges 38–55Middle phalanges 24–35Distal phalanges 22–30

By analyzing the contained structure and movements of fingers, we intend to replacethem with similar mechanisms to mimic the human hand in the following section.

3 Analysis of CFB Mechanism

Planar four-bar mechanism, which is widely used in many applications, is efficient attransferring motion and power. A planar four-bar linkage is composed of four revolutejoints with parallel axes. Some biomechanics researchers have used four-bar linkage inthe design of human body. For example, a four-bar linkage system is presented in [17]to replicate the polycentric motion of the knee that occurs during passive knee flexion-extension. A CFB mechanism is proposed in [18] for the knee design of bipedal robot.Applications on robotic hands for motion imitation also exists in [19–21]. In this paper,centrodes of the CFB mechanism are investigated and a contact aided CFB mechanismis proposed to mimic the complex rolling characteristics of finger joints. This design hasan overconstrained characteristic to increase the structure stiffness.

3.1 Kinematics of CFB Mechanism

Centrode, as an important characteristic in planar kinematics, is the path traced byan instantaneous center of rotation of a rigid link moving in a plane [22]. The motionof the coupler link with respect to the ground link is pure rotation around the instan‐taneous center. According to Kennedy-Aronhold Theorem, the centrode can be foundat the intersection of the crank link and follower link in a CFB mechanism, whichmeans that CFB mechanism always has centrodes between the coupler link and theground link. The fixed centrode can be found by drawing the trajectory of the inter‐section of crank link and follower link. The process of drawing the two trajectoriesof centrodes is shown in Fig. 4. The trajectories depend on lengths of the associatedfour links.

Partial trajectories of centrodes with the rotational angle of the crank link equalingto 35° are shown in Fig. 4. Random angles are available by using this approach. Themotion of the coupler link with respect to the ground link is duplicated by making thesetwo centrodes roll against one another without slipping. Because of the pure rolling ofthe two curves, they have the same lengths.

32 G. Bai et al.

Fig. 4. CFB mechanism and trajectories of its centrodes. (a) Produced trajectory of fixedcentrodes. (b) Produced trajectory of moving centrodes. (c) Trajectories integrated in one figure.

3.2 Contact-Aided CFB Mechanism

Due to the characteristics of the centrodes, we can design a contact-aided CFB mecha‐nism which has a much higher stiffness than the CFM mechanism, by adding a highkinematic pair between the coupler and the ground link.

Contact-aided mechanism was first introduced in [23]. Cannon and Howell proposeda novel design of compliant rolling-contact element capable of performing the functionsof bearing and a spring [24]. The use of contact surfaces is to enhance the functionalityof compliant mechanism to be capable of performing certain kinematic tasks similar torigid body. Contact surfaces integrating with rigid links as an overconstrained mecha‐nism have certain kinematics and limited motion range. As shown in Fig. 5, a contact-aided CFB mechanism with limited motion range of 90°. An application on artificialjoints will be introduced in Sect. 5.1.

Fig. 5. Contact-aided CFB mechanism. (a) Initial position. (b) Final position. (c) Side view ofthe assembly.

A Two-Fingered Anthropomorphic Robotic Hand 33

4 Gripping Analysis and Design of Finger Joints

4.1 Gripping Analysis

Configurations of human hand are different in gripping variant shapes and sizes of theobjects. Nevertheless, gripping processes are similar, typically including searching,reaching, gripping, and moving [25]. Two gripping examples are now considered andvarious configurations of fingers are observed. The two objects are a compression springwith a diameter of 2.5 mm and a length of 15 mm and a plastic wheel with a diameterof 22 mm and a height of 11 mm, as shown in Fig. 6.

Fig. 6. Gripping process for objects in different sizes.

By analyzing the gripping processes of the two objects, two DOF are adopted withone for force applying and the other for gipping. For small diameter object such as thespring, the fingernails and the fingertips cooperate to enclose and hold with very smallgripping force. For the larger part such as the plastic wheel, soft fingertips that deformduring contact apply a large space of frictional forces and moments [26].

Thus the design of the finger should consider these three items: two DOF, fingernail,and soft fingertip.

4.2 Design Process of a Two-Joint Finger

Taking the thumb finger as an example, the MCP and IP joints of this finger are consid‐ered for design. The action ranges of extension/flexion as to MCP and IP joints are 10H/55

34 G. Bai et al.

and 15H/80 respectively as shown in Fig. 7. Thumb finger is the only one which has hyperextension at MCP joint which means that it is the most flexible one in five fingers.

Fig. 7. Extension/flexion ranges of two joints of thumb finger and equivalent CFB mechanismsof the joints.

In order to mimic the ranges of motion of the two joints, two CFB linkages shouldbe used with one for mimicking IP joint and the other for MCP joint.

According to the approach proposed in Sect. 3, the extension/flexion ranges of thesetwo CFB mechanisms determine the range of motion of the mechanisms, thus the fixedand moving centrodes of the mechanisms can be calcualted. Meanwhile, two-joint fingerwill have a fixed motion range, as shown in Fig. 8.

Fig. 8. Side view of two-joint finger connecting with fixed and moving centrodes.

A Two-Fingered Anthropomorphic Robotic Hand 35

5 Prototype and Testing

5.1 Prototype

The gripping analysis (see Sect. 4.1) of the human hand indicates that fingernails andsoft fingertip play critical roles in specific gripping tasks. A 3D model considering thesedetails are designed in order to mimic human hand’s structure. Two joints of the fingerare decoupled and a tendon is used to drive phalanges of the finger. The finger is dividedby its symmetrical plane into front and back halves which make the manufacturing andassembly efficient. The detailed design of the two-joint finger is shown in Fig. 9.

Fig. 9. 3D model of a two-joint finger.

A 3D printed prototype is shown in Fig. 1. The material used for the outside layerof the phalanges is silicone elastomer while the material for crank link and follower linksof CFB mechanism is PLA. Nylon rope passes through cable slots, between whichcompression spring is placed for rebounding after gripping process.

Fig. 10. Prosthetic leg using contact aided CFB linkage.

36 G. Bai et al.

Based on the above approach and design process, a prosthetic leg using a contact-aided CFB linkage is developed with the bending angle of 90° as shown in Fig. 10.

5.2 Testing

A two-fingered hand is developed for manually gripping test as shown in Fig. 11. Thedifferential palm contains a movable pulley which drives the two fingers with underac‐tuation. The objects used for grasping are of a wide range, including plastic cup, tinyspring and pinecone, etc. as shown in Fig. 12. Fingernail plays a significant role forgripping tiny spring. The cooperation of fingernail and fingertip ensures a successfulgripping of a ball pen. The soft fingertips provide a large friction force and moment for

Fig. 11. Two-fingered hand at open and bending statuses.

Fig. 12. Gripping testing on a variety of objects.

A Two-Fingered Anthropomorphic Robotic Hand 37

the cuboid-shaped battery. The differential drive contributes to the irregular non-centered pinecone gripping. The testing on these universal objects verified the feasibilityof the design.

6 Conclusions and Discussions

The kinematics of contact-aided CFB mechanism was analyzed and a design approachto mimic human finger joints was proposed. The gripping process carried out by humanhand was analyzed and the structures of fingernail, soft fingertip were designed forprototyping. A design example of two-joint thumb finger was conducted by using theproposed design approach. The prototype was manufactured by 3D printing technologywith specific materials. The testing of gripping of a variety of objects verified the feasi‐bility of the biomimetic design.

Acknowledgments. The authors would like to thank the Engineering and Physical SciencesResearch Council (EPSRC), United Kingdom, for the support under grant No EP/K018345/1.

References

1. Salisbury, J., Craig, J.: Articulated hands: force control and kinematic issues. Int. J. Robot.Res. 1, 4–17 (1982)

2. Jacobsen, S., Iversen, E., Johnson, D.R., Biggers, K.: Design of the Utah/M.I.T. dextroushand. In: Proceedings IEEE the International Conference on Robotics and Automation, pp.1520–1532. IEEE Press, New York (1986)

3. Butterfass, J., Grebenstein, M., Liu, H., et al.: DLR-hand II: next generation of a dextrousrobot hand. In: Proceedings IEEE International Conference on Robotics and Automation, pp.109–114. IEEE Press, New York (2001)

4. Deshpande, A.D., Xu, Z., Weghe, M.J.V., Brown, B.H., Ko, J., Chang, L.Y., Wilkinson, D.D.,Bidic, S.M., Matsuoka, Y.: Mechanisms of the anatomically correct testbed hand. IEEE/ASME Trans. Mechatronics. 18, 238–250 (2013)

5. Xu, Z., Kumar, V., Matsuoka, Y., Todorov, E.: Design of an anthropomorphic robotic fingersystem with biomimetic artificial joints. In: 4th IEEE RAS & EMBS InternationalConference, pp. 568–574. IEEE Press, New York (2012)

6. Xu, Z., Todorov, E., Dellon, B., Matsuoka, Y.: Design and analysis of an artificial finger jointfor anthropomorphic robotic hands. In: 2011 IEEE International Conference Robotics andAutomation (ICRA), pp. 5096–5102. IEEE Press, New York (2011)

7. Townsend, W.: The barrett hand grasper—programmably flexible part handling andassemble. Int. J. Ind. Robot. 27, 181–188 (2000)

8. Kyberd, P.J., Light, C., Chappell, P.H., Nightingale, J.M., Whatley, D., Evans, M.: The designof anthropomorphic prosthetic hands: a study of the southampton hand. Robotica 19, 593–600 (2001)

9. Liu, H.: DSP and FPGA-based joint impedance controller for DLR/HIT II dexterous robothand. In: Proceedings IEEE/ASME International Conference on Advanced IntelligentMechatronics, pp. 1594–1599 (2009)

38 G. Bai et al.

10. Yamano, I., Maeno, T.: Five-fingered robot hand using ultrasonic motors and elastic elements.In: Proceedings IEEE International Conference on Robotics and Automation, pp. 2673–2678(2005)

11. Shadow robot company. http://www.shadowrobot.com/hand/12. Buchner, H., Hines, M., Hemami, H.: A dynamic model for finger interphalangeal

coordination. J. Biomech. 21, 459–468 (1988)13. Dennerlein, J., Diao, E., Mote Jr., C.: Tensions of the flexor digitorum superficialis are higher

than a current model predicts. J. Biomech. 31, 295–301 (1998)14. Sancho-Bru, J., Perez-Gonzalez, A., Vergara-Monedero, M.: A 3-D dynamic model of human

finger for studying free movements. J. Biomech. 34, 1491–1500 (2001)15. Buchcholz, B., Armstrong, T., Goldstein, S.: Anthropometric data for describing the

kinematics of the human hand. Ergonomics 35, 261–273 (1992)16. Peña, E., Yang, J., Abenoza, M., Tico, N., Abedel-Malek, K.: Virtual human hand:

autonomous grasping strategy. In: Proceedings of the 17th IASTED International Conferenceon Applied Simulation and Modelling, pp. 609–618. Felice, F. de, Corfu (2008)

17. Goodfellow, J., O’Connor, J.: The Knee, W. Norman Scott Mosby-Year Book, Inc. (1994)18. Hamon, A., Aoustin, Y.: Cross four-bar linkage for the knees of a planar bipedal robot. In:

10th IEEE-RAS International Conference Humanoid Robots (Humanoids), pp. 379–384.IEEE Press, New York (2010)

19. Birglen, L., Gosselin, C.M.: Geometric design of three-phalanx underactuated fingers. J.Mech. Design. 128, 356–364 (2006)

20. Wu, L., Carbone, G., Ceccarelli, M.: Designing an underactuated mechanism for a 1 activeDOF finger operation. Mech. Mach. Theory 44, 336–348 (2009)

21. Moon, Y.M.: Bio-mimetic design of finger mechanism with contact aided compliantmechanism. Mech. Mach. Theory 42, 600–611 (2007)

22. Eckhardt, H.D.: Kinematic Design of Machines and Mechanisms, pp. 249–283. McGraw-Hill, New York (1998)

23. Mankame, N.D., Ananthasuresh, G.K.: Contact aided compliant mechanisms: concept andpreliminaries. In: ASME 2002 International Design Engineering Technical Conferences andComputers and Information in Engineering Conference. American Society of MechanicalEngineers, pp. 109–121 (2002)

24. Cannon, J.R., Lusk, C.P., Howell, L.L.: Compliant rolling-contact element mechanisms. In:ASME 2005 International Design Engineering Technical Conferences and Computers andInformation in Engineering Conference, pp. 3–13. American Society of MechanicalEngineers (2005)

25. Niebel, B. W.: Motion and Time Study, pp. 197–205. Richard D Irwin (1993)26. Akhtar Khurshid, A.G., Malik, M.A.: Robotic Grasping and Fine Manipulation Using Soft

Fingertip. Adv Mechatronics, pp. 155–174 (2011)

A Two-Fingered Anthropomorphic Robotic Hand 39

Living Designs

Rina Bernabei1(✉) and Jacqueline Power2

1 University of New South Wales, Sydney, AustraliaR.Bernabei@unsw.edu.au

2 University of Tasmania, Launceston, AustraliaJacqueline.Power@utas.edu.au

Abstract. This paper will outline why product designers are exploring makingprocesses of the natural world and how this is of benefit to traditional productdesign practice. This type of experimental design work pushes the boundaries ofconventional product design in which mass-manufacture efficiency drives thedesign and production process. Products, which use growth processes as funda‐mental to the making process, are increasingly becoming more feasible for end-user acquisition. This paper will provide two case study examples. These casestudies contextualise these products and how they co-exist and contribute to thewell-established design approaches of digital fabrication and co-creation.

Keywords: Biodesign · Object design · Material technology · Emotional design

1 Introduction

Product designers are increasingly exploring the opportunities afforded by using livingand artificial systems in material technologies and manufacture-related processes. Thistype of experimental work pushes the boundaries of conventional product design, inwhich mass-manufacture efficiency drives the design and production process. This paperprovides two case study examples of bio-design - one is a product, a chair, and one apiece of jewelry, a ring. These designed artefacts demonstrate the use of natural growthprocesses for the creation of a finished product or artefact.

This paper questions how the emerging field of bio-design is of benefit to traditionaldesign practice, particularly product design. This discourse rests on a rich foundation ofdesign theory that has been developed since modernism. Bio-design crosses over andintersects some current paradigms in very interesting and innovative ways. The authors,researchers in the field of both product design and interior design, see the potential ofbio-design to co-exist and contribute to a series of well-established design approachesincluding digital fabrication and co-creation approaches.

2 Defining Bio-design

Within the design disciplines, the term bio-design is used in reference to designed worksthat incorporate living organisms. The term bio-design is a broad one, capturing in itsuse of the word ‘design’ many disciplines including architecture, fashion, and products.

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 40–47, 2016.DOI: 10.1007/978-3-319-42417-0_4

A clarification of the prefix ‘bio-’ is required. William Myers, author of the 2012 publi‐cation BioDesign: nature + science + creativity and curator of the exhibition BioDesign:on the cross pollination of nature, science and creativity, explains that:

“Biodesign goes further than other biology-inspired approaches to design and fabrication. Unlikebiomimicry, cradle to cradle, and the popular but frustratingly vague ‘green design’, biodesignrefers specifically to the incorporation of living organisms as essential components, enhancingthe function of the finished work. It goes beyond mimicry to integration, dissolving boundariesand synthesizing new hybrid typologies. The label is also used to highlight experiments thatreplace industrial or mechanical systems with biological processes.” [1]

Such a definition enables many design approaches to fall under the umbrella of bio-design. Rather than explore products that incorporate living organisms to varyingdegrees and for various purposes, this paper seeks to instead consider products or arte‐facts that use actual growth processes as fundamental to the making process. In otherwords, without the biological processes, there would be no tangible or useable productor artefact – the product is quite literally grown. This requires a rethink of traditionalproduct manufacturing systems.

Bio-design products and artefacts remain in their infancy, many as conceptual worksor prototypes rather than available for user acquisition. The two pieces selected as casestudy examples below are either one-off prototypes or concept designs. [2] The casestudy examples will not be positioned to engage with ethical dilemmas that arise withsuch modes of production and the utilisation of biological processes. This is however,an important aspect of bio-design that demands in the future greater attention in theliterature. Design and art historian Christina Cogdell has commented that, “it appearsthat none of the architects or designers who uphold their bioart as a prototype for biode‐sign has seriously engaged with their critical intent or with the ethical problems”. [3]This is not to say that designers have not considered these issues as part of their practicebut may have instead not articulated them, focusing on other facets of the bio-designprocess, such as material application, aesthetic values and collaborative processes.

3 Bio-design and Digital Fabrication

Digital fabrication is a well-established product design manufacturing practice and newmaking opportunities are presented when coupled with bio-design processes. Accordingto computer scientist Albrecht Schmidt “Historically, the separation of the physical anddigital worlds has been very clear, but digital fabrication is blurring that dividing line.”[4] In design, digital fabrication is definable as:

“a process whereby an object design is created on a computer, and the object is then automaticallyproduced by a machine. Digital fabrication machines can be roughly sorted into two categories:subtractive and additive. Subtractive approaches use drill bits, blades or lasers to remove materialfrom an original material source, thus shaping the three-dimensional object. Additive processesdeposit progressive layers of a material until a desired shape is achieved.” [5]

Digital fabrication includes technologies such as 3D printing, laser cutting andcomputer-numerically controlled (CNC) routing. Additive digital fabrication processes

Living Designs 41

such as 3D printing, is relatively new. Like bio-design, its integration with and overlapwith existing design paradigms, is still being explored.

Eric Klareenbeek’s MyceliumChair (Fig. 3) is a bio-design example integrating bothbiological elements and digital fabrication technologies - “a chair in which 3D printing andgrowing material are combined.” [6] Klareenbeek collaborated with a number of scientistson the project. [7] The final chair, exhibited at Dutch Design Week in 2013, has beenprinted from a mixture of materials, including water, straw, biodegradable plastic andmycelium, which Klareenbeek describes as “the threadlike network in fungi”. [8] Themycelium itself grows inside the chair, giving it strength - the chair is effectively alive.

Fig. 1. Studio Eric Klarenbeek’s, MyceliumChair complete with living oyster mushrooms. http://www.ericklarenbeek.com/

Although the growth of the mycelium within the chair is not visible, remarkably thechair supports a colony of oyster mushrooms actually grown from the chair (Fig. 3).Unlike the presence of the mycelium, which provides strength to the chair, the oystermushrooms themselves are purely decorative. [9] With the mushrooms in full bloom,their continued growth was halted by drying the structure, and coating it in biodegradableplastic. [10] The materials used in the project have a raft of implications for production.For instance, Eric Klareenbeek has said:

42 R. Bernabei and J. Power

“its all living material; meaning its growing and multiplying, so if you treat the organism well,you’ll never run out of ‘glue’. In other words, it’s theoretically inexhaustible!!”

Fig. 2. Digital fabrication of Studio Eric Klarenbeek’s, myceliumchair. http://www.ericklarenbeek.com/

Bio-design when used with digital fabrication, such as 3D printing, presents diversepossibilities for material use and its assembly. In the case of the MyceliumChair, 3Dprinting facilitated the application of a new composite material that could generate adesired complex form (Fig. 4). Digital fabrication technologies enable designers toexperiment with new materials and with increasing home-based platforms, have thecapacity in the future to extend the making into the hands of the users themselves.

4 Bio-design and Co-creation

Bio-design involves designing with nature. In this way, it could be considered a formof co-creation, which is defined by Elizabeth Sanders and Pieter Jan Stappers who areleading practitioners in the field of co-creation and co-design, as;

“any act of collective creativity, i.e. creativity that is shared by two or more people. Co-creationis a very broad term with applications ranging from the physical to the metaphysical and fromthe material to the spiritual…” [11]

We propose that bio-design presents an important new avenue for undertaking co-creation. Not only does bio-design require collaboration between different disciplines,including those beyond the design disciplines, but it also facilitates the creation and useof multi-, cross-, and inter- disciplinary knowledge. [12]

Living Designs 43

A bio-design artefact example that reveals alignments with co-creation is the Bioje‐wellery project (Fig. 1). This project brought together researchers and designers TobieKerridge and Nicki Stott at the Royal College of Art, and Ian Thompson a bioengineerfrom the Kings College London. This project arguably provided a groundbreaking newapproach to jewellery design. The dedicated project website provides the followingoverview of the project;

“Biojewellery started out by looking for couples who wanted to donate their bone cells. Theircells were seeded onto a bioactive scaffold. This material encouraged the cells to divide andgrow rapidly, and the resulting tissue took on the form of the scaffold, which was a ring shape.The couple’s cells were grown at Guy’s Hospital, and the final bone tissue was taken to a studioat the Royal College of Art to be made into a pair of rings. The bone was combined with tradi‐tional precious metals so that each has a ring made with the tissue of their partner.” [13]

Fig. 3. Final rings grown for two of the Biojewellery project participants. http://www.biojewellery.com/project6.html

The Biojewellery rings were grown from the cells of a loved one, enabling the wearerto adorn their bodies literally with a part of someone else. As described in the exhibitioncatalogue accompanying the project, “each person wears the body of their lover on theirhand.” [14]

The success of the project required interdisciplinary involvement harnessing variousdisciplinary expertise and material knowledge (Fig. 2). In the approach taken to thecoming together of various disciplines for the Biojewellery project:

44 R. Bernabei and J. Power

“instead of attempting to smooth over tensions between the disciplines involved, we activelysought to highlight areas of conflict. Our aim was to provoke debate – not only between special‐isms but amongst the public at large. In other words, by confounding expectations of the natureof this kind of multi-disciplinary work, we hoped to surprise both ourselves and others.” [15]

In this project the users were also significantly involved, donating cells to grow therings and providing designs of decorative patterns for the metalwork. The ability to growthe rings from bone provided an additional layer of potency to the meaning embeddedin these rings. It also forms a strong emotional bond between the product and the user.The emotional connection differs from that of mass produced products, as the bio-product described is both co-created, and is made up from tangible user bio-products,forming an intimate extension of user with product.

Designer Toby Kerridge has said that “my aim was to reflect on how we (Ian, Nikki,myself) executed project work, including working with organisations and the functionof these designs once they get out into the world, for example in accomplishing someform of public engagement.” [16] This further suggests that the co-creation aspect ofthe project was extremely important, as well as the dialogue that exists in relation to theproject outcomes.

Fig. 4. Examples of some of the biomedical materials used in the Biojewellery project. http://www.biojewellery.com/project6.html

Living Designs 45

5 Conclusion

Bio-design continues to grow and attract interest in the product design discipline.However little seems to have been said regarding how bio-design fits within the existingestablished design paradigms. Through co-creation, bio-design has the potential tocreate strong emotional connections between products and users, by having users inti‐mately involved in the creation of the product. Digital fabrication processes and inparticular 3D printing, when teamed with bio-design presents the opportunity to increasethe designer’s palette of materials and open new form possibilities.

Bio-design presents more than the promise of product manufacturing splinteringfrom industrial production processes of old and incorporating biological processes. Thecase studies here of Tobie Kerridge and Eric Klareenbeek present a tantalising glimpseof how bio-design is being incorporated in the design methodologies of practicingdesigners and researchers.

References

1. Myers, W.: Bio Design, pp. 7–8. The Museum of Modern Art, New York (2012)2. These definitions are provided in line with Christina Cogdell who commented that the

exhibition Design and the Elastic Mind failed to differentiate between; “imagined visions –virtual pieces, if you will, materialized for the exhibition through digitally manipulatedphotographs or videos – one-off prototypes seemingly ready for production, and post-production designs.” Christina Cogdell, ‘Design and the Elastic Mind, Museum of ModernArt (Spring 2008). Design Issues 25(3), 95 (Summer 2009)

3. Cogdell, C.: BioArt to BioDesign. Am. Art 25(2), 28 (2011). Summer4. Schmidt, A.: Where the physical and digital worlds collide. Computer 45(12), 77 (2012)5. Zoran, A., Buechley, L.: Hybrid reassemblage: an exploration of craft, digital fabrication and

artifact uniqueness. Leonardo 46(1), 6 (2013)6. ‘MyceliumChair,’ Eric Klarenbeek. http://www.ericklarenbeek.com/. Accessed 9 Feb 20147. The Wageningen UR Plant Breeding Mushroom Research Group, CNC Exotic Mushrooms

and Beelden op de Berg. ‘MyceliumChair,’ Eric Klarenbeek. http://www.ericklarenbeek.com/. Accessed 9 Feb 2014

8. ‘MyceliumChair,’ Eric Klarenbeek. http://www.ericklarenbeek.com/. Accessed 9 Feb 20149. Mycelium Chair by EricKlareenbeek is 3D-printed with Living Fungus, Dezeen. http://www.

dezeen.com/2013/10/20/mycelium-chair-by-eric-klarenbeek-is-3d-printed-with-living-fungus/.Accessed 7 Jan 2014

10. Mycelium Chair by EricKlareenbeek is 3D-printed with Living Fungus, Dezeen. http://www.dezeen.com/2013/10/20/mycelium-chair-by-eric-klarenbeek-is-3d-printed-with-living-fungus/.Accessed 7 Jan 2014

11. Sanders, E.B.N., Stappers, P.J.: Co-creation and the new landscapes of design. CoDesign:Int. J. CoCreation Des. Arts 4(1), 6 (2008)

46 R. Bernabei and J. Power

12. Julieanna Preston defines multidisciplinary as referring to “knowledge shared by more thanone discipline in which each is tackling a common challenge using its specialized tool kit –essentially, the whole is the sum of its part. Crossdisciplinary refers to one discipline usingthe knowledge set of another, i.e., importation across discipline boundaries.” JulieannaPreston, “A Fossick for Interior Design Pedagogies,” in After Taste: expanded practice ininterior design, ed. Kent Kleinman, Joanna Merwood-Sailsbury and Lois Weinthal, p. 105.Princeton Architectural Press, New York (2012)

13. Biojewellery: designing rings with bioengineered bone tissue. http://www.biojewellery.com/project.html. Accessed 4 Feb 2014

14. Kerridge, T., Stott, N., Thompson, I.: Biojewellery: Designing Rings with BioengineeredBone and Tissue, p. 8. Oral & Maxillofacial Surgery, King’s College London, London (2006)

15. Kerridge, T., Stott, N., Thompson, I.: Biojewellery: Designing Rings with BioengineeredBone and Tissue, p. 9. Oral & Maxillofacial Surgery, King’s College London, London (2006)

16. Kerridge, T.: Personal communication, 12 February 2014

Living Designs 47

iCub Visual Memory Inspector:Visualising the iCub’s Thoughts

Daniel Camilleri(B), Andreas Damianou, Harry Jackson,Neil Lawrence, and Tony Prescott

Psychology Department, University of Sheffield, Western Bank, Sheffield, UKd.camilleri@sheffield.ac.uk

http://www.sheffield.ac.uk

Abstract. This paper describes the integration of multiple sensoryrecognition models created by a Synthetic Autobiographical Memory intoa structured system. This structured system provides high level controlof the overall architecture and interfaces with an iCub simulator based inUnity which provides a virtual space for the display of recollected events.

Keywords: Synthetic autobiographical memory · Unity simulator ·Yarp · Deep gaussian process

1 Introduction

Human episodic and autobiographical (or event) memory can be considered asan attractor network operating in a latent variable space, whose dimensionsencode salient characteristics of the physical and social world in a highly com-pressed fashion [1]. The operation of the perceptual systems that provide inputto event memory can be analogised to a deep learning process that identifiespsychologically meaningful latent variable descriptions [2]. Instantaneous mem-ories then correspond to points in this latent variable space and episodic mem-ories to trajectories through this space. Deep Gaussian Processes (DGP) [3]are probabilistic, non-parametric equivalents of neural networks and have manyattractive properties as models of event memory; for example, the ability to dis-cover highly compressed latent variable spaces, to form attractors that encodetemporal sequences, and to act as generative models [4].

As part of the WYSIWYD FP7 project to develop social cognition for theiCub [5] humanoid robot, we are exploring the hypothesis that an architectureformed by suitably configured DGPs can provide an effective synthetic ana-logue to human autobiographical memory, a system that we call SAM. Work sofar has focused on the development of models for separate sensory modalitiesthat demonstrate useful qualities such as compression, including identification ofpsychologically-meaningful latent variables, pattern completion, pattern separa-tion, and uncertainty quantification. The next phase focuses on the integrationof different sensory modalities using multiple sub-models which are cast within a

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 48–57, 2016.DOI: 10.1007/978-3-319-42417-0 5

iCub Visual Memory Inspector 49

single and coherent framework. However, with the use of multiple sensory modal-ities modelled together, one requires firstly a single point of entry for easy andintuitive communication with all models. This interface controls access to theseparate sub models. Secondly, with the availability of multiple sub models, onealso requires a method for visualising and associating all the recollected eventsin a virtual environment. This offers an important window into the recollectionprocess of multiple models by visually displaying all recollected sensory modali-ties. We refer to this idea as the “visual memory inspector” (VMI) environment,to highlight the fact that we can actively (i.e. in a user-driven manner) interactwith it and explore the memory space. The unique generative properties of theSAM model which we employ significantly facilitate the deployment of the VMI.

The development of this interface is also interesting since studies of humanautobiographical memory indicate that whilst episodic memories are recoveredvia a loop through the hippocampal system the outputs of that system gen-erate activity within primary sensory areas that appears to encode a sensoryexperience of the recollected event [6]. These patterns can then be picked upfor processing elsewhere in the brain, for instance, by systems that plan futureactions, or that reflect on the implications of remembered experiences. Thisactivity also feeds through to hippocampus (as part of the loop) and may playa role in the reconstruction of further memories. Whilst the brain architectureunderlying human autobiographical memory is poorly understood, our hope isthat the development of an integrative architecture for autobiographic memoryin a humanoid robot could provide clues for unravelling the role of different brainareas in human memory, and provide a top-down functional description of howsuch a system could operate.

The rest of this paper first provides an introduction to the operation of SAMin Sect. 2 together with a brief description of the models that have been trainedso far based on DGP in Sect. 3. Section 4 subsequently outlines the need for asupervisory process to interface with multiple SAM Models. Section 5 describesthe implementation of the Visual Memory Inspector crucial to the understand-ing of how the iCub is analysing the situation and finally Sect. 6 provides adescription of the upcoming work on this project.

2 SAM Backend

In this section we first explain the SAM architecture used as a backend and inthe subsections that follow, we outline the SAM-based sub-models developed inour work.

The SAM [2] system is a probabilistic framework which approximates func-tional requirements to Auto-biographical memory as have been identified inprevious studies [1]. In detail, denote the N observed sensory data as D multi-dimensional vectors {yn}Nn=1, i.e. yn ∈ �D. Typically these vectors are noisyand high-dimensional, for example if the robotic agent is perceiving visual sig-nals, each frame yn will be a noisy image with D equal to the number of all thepixels composing it. In SAM, each yn is modelled through yn = f(xn)+ε, where

50 D. Camilleri et al.

ε is Gaussian noise. Here, xn ∈ �Q, Q � D is a low-dimensional vector, and{xn}Nn=1 forms the (compressed) memory space (called a latent space) which islearned through the agents experience by Bayesian inference. Moreover, f is aGaussian process mapping which maps latent points back to the original obser-vation space. This is a generative mapping and plays a key role to the VMI.Furthermore, this mapping is anchored on a user-defined number of “anchor”points U, meaning that any output of the function f will be a combination ofelements from U. [2] explains how the combination of anchor and latent pointsform the final memory space, where high-level analogies to neurons and synapsescan be defined. In SAM, one can stack multiple latent spaces to form a hierar-chical (deep) memory space. Notice that thanks to the Bayesian framework ofSAM, we have access to the (approximate) posterior distribution q(x|y), mean-ing that once the model is trained we can readily obtain the reverse mapping off when new sensory outputs y∗ need to be considered.

We now proceed to outline the specific SAM-based sub-models which handledifferent types of sensory modalities. We will see later how these sub-models canbe handled within a central framework which we call SAM Supervisor.

3 SAM Models to Date

3.1 Face Recognition Model

Face recognition with SAM has been demonstrated in previous work [7] whichused a Viola-Jones face detector [8]. This method has been improved with theapplication of facial landmark identification and tracking through the use of amore robust face tracker called the Cambridge Face Tracker [9]. The output ofthis face tracker, depicted in Fig. 1a provides an outline of the face togetherwith a general direction of looking. This face outline is then extracted from theoriginal image and processed into a rotationally invariant representation alongthe roll axis as shown in Fig. 1b.

(a) (b)

Fig. 1. (a) Cambridge Face Tracker with light blue facial landmarks and red box rep-resenting orientation of the head. (b) Augmented output with roll rotational invariance(Color figure online)

iCub Visual Memory Inspector 51

The image is subsequently resized, vectorised, labelled and then trained uponin the same manner as the previous work. Recollection is then carried out withthe use of labels which returns a face extracted from the latent feature spacethat could either be a past observation or a fantasy face.

3.2 Tactile Model

The tactile model interfaces with the iCub’s skin and collects the pressure readingover all texels of a specific body part, the arm, which are compiled into a singlevector and trained upon to recognise four distinct types of touch which are:

1. Hard Touch2. Soft Touch3. Caress4. Pinch

Fig. 2. Examples of the four types of touch on the iCub forearm classified by the TactileModel

The recognition of these types of touch which can be seen in Fig. 2 is par-ticularly important in social situations. The recollection of a fantasy instance asthe inverse process describes the pressure which is required for the re-enactmentof the recollected touch.

3.3 Emotion from Speech Model

Another important indication which guides social interaction is the detectionof emotional state especially from voice. As such there is currently work beingcarried out on the use of Mel-Frequency Ceptral Coefficients (MFCC) [10] paired

52 D. Camilleri et al.

with a Gaussian Mixture Model (GMM) to construct classification vectors fortraining with SAM.

In the first stage of feature extraction mel-frequency ceptral coefficients(MFCCs) are created for each frame of the utterance. These are a standardin speech processing and have shown great success in a number of tasks includ-ing speaker recognition and emotion recognition - as well as their ubiquitoususe in speech recognition systems. MFCCs are approximations to the Fouriertransform of the power spectrum of a frame of audio.

These MFCCs are then made into supervectors. Firstly, for each speaker,a GMM is trained on every feature from each of their utterances. These makeup the speaker-specific Gaussian mixture models (GMMs). The feature vector isthen generated using the method described in [11] which extracts the MFCC fea-tures for each utterance combined with the posterior probability of the mixtureof Gaussians and this vector is used for training with SAM.

On the other hand, extraction of MFCCs from the raw waveform loses muchof the original information necessary to recreating sound waves, such as intona-tion and other long-term features of speech. Thus the current state of the systemdoes not allow the conversion of a recollection to sound.

However, this will be tackled in future work through the extraction of differ-ent features from sound which do not abstract the original audio signal as highlyas MFCC features. One such feature that is being researched is the use of powerspectra.

4 SAM Supervisor

The current challenge with multiple models of separate sensory modalities is therequirement to launch and interface with each individually. This hampers thedevelopment of more complex hierarchical models that link multiple modalitiesand this issue has led to the development of a streamlined system through theuse of a supervisory process.

The aim of this supervisory process is, as mentioned in the introduction, toprovide a single framework where all models that are developed with SAM caninteract with the rest of the modules developed for WYSIWYD. As such the roleof SAM Supervisor is fourfold:

1. Provide a single point of contact with all external modules accessing themodels which greatly facilitates external interfacing. This exposes two validcommands for each model which are ask modelName label which returnsthe label given an instance of data or ask modelName instance whichreturns a fantasy memory instance given a label.

2. Initialise all models as subprocesses of the supervisor to ensure parallel oper-ation and perform routine checks on the status of the loaded models.

3. Check that all models are up to date with respect to the available data (expe-riences) in the ABMSql database. This ensures that the iCub is current withrespect to the conglomeration of its experiences to date.

iCub Visual Memory Inspector 53

4. Allow for the specification of model configurations that specifically describewhich model configurations are to be loaded thus allowing custom memorylayouts to be design and implemented easily.

Moreover, the presence of multiple models brings to light another challengeand that is understanding what is currently happening within the memory sys-tem which leads to the requirement of a Visual Memory Inspector whose imple-mentation is detailed in the next section.

5 Visual Memory Inspector

The aim of the Visual Memory Inspector, as stated before, is to understandbetter what is currently occurring within the reasoning and recollection processesof the iCub in a visual manner. Thus this requires, first of all, a virtual world thatbehaves similar to the real world as it is understood by the iCub, a model of theiCub himself as the protagonist of this world as well as a means of communicationwith Yarp [12] for the transmission of information and motor commands.

As such this virtual world requires a platform with a physics engine to gov-ern interaction between objects while also offering flexibility in interfacing withexternal libraries, cross-platform execution for both Windows and Linux andfinally an easy way of developing applications within this platform.

The four candidates considered as a platform for the VMI were the Unity GameEngine [13], V-Rep [14], Webots [15] and Gazebo [16]. On one hand, Gazebo offersa versatile environment for the simulation of robots but requires the installationof ROS. V-Rep and Webots are also oriented towards the simulation of robots butare both difficult to interface to with Yarp and also have licensing restrictions anda small niche user base making development challenging.

Unity on the other hand satisfies all the requirements set for the VMI. Ithas an advanced GPU accelerated physics engine for simulating collisions andmotion, allows multiplatform compilation because it derives from a .NET pro-gramming paradigm and furthermore facilitates the inclusion of external librariesin C# and/or JavaScript. Moreover Unity also has a vast user base which facili-tates development and has no licensing requirements for the basic version whichis versatile enough for the requirements of the project. Consequently after thechoice of a development platform, the next step is to set up communication withYarp from within Unity.

5.1 Unity-Yarp Integration

In order to integrate Yarp libraries within Unity, a common language is requiredto bridge the two platforms. Unity on one hand, can be developed using twolanguages, C# or JavaScript, of which JavaScript is easy to use but C# providesa higher level of control over the execution of code. On the other hand, Yarp isdeveloped in C but it also provides language bindings for a variety of languagesthrough the use of SWIG (Simplified Wrapper Interface Generator) [17] of whichone of these languages is C#.

54 D. Camilleri et al.

Thus with C# as the chosen language, the integration of Yarp with Unity iscarried out by adding the Swig generated .dll (Windows) or .so (Linux) and .csfiles that are generated through the compilation of Yarp to the Plugins folder ofa Unity project and it is then imported as a library within the code.

With Unity capable of implementing Yarp ports the following crucial stepfor integration looked into the implementation of bottle and image conversionfunctions that allow decoding and encoding of Yarp information into a moreUnity friendly format. Finally, since a call to yarp read is a blocking call, athreaded class was employed for the communication processes so that they canrun in parallel to the visualisation. This results in higher frame rates and moretime efficient processing of events. The next section describes the implementationof a virtual iCub as the protagonist of the memory inspector.

5.2 iCub Simulation

The VMI is currently targeted towards the visualisation of memories and as suchdoes not require a sensory interface within the virtual environment. Nonetheless,future work will look into expanding the scope of the VMI to a simulation spacewhere future planning can also be virtually carried out.

This is why the current implementation of VMI pairs itself with iCub SIMto allow motion control of the iCub within the VMI and also provides a stereostream of the iCub’s current point of view. This is accessible separately fromthe VMI interface which by itself provides a game like environment with thecapability of walking around the 3D scene in the iCub’s memory to changeviewing angles.

6 Future Work

6.1 Recollection Visualisation

The current state of the project allows the placement of the protagonist withina previously saved environment that could be obtained from a .obj model whichincludes Kinect generated 3D models. The VMI also has the functionality todynamically load pre-existing 3D objects within the environment and assign agiven label and position. An example of this can be seen in Fig. 3 where theVMI has dynamically generated a person and two objects with specific locationswithin the environment.

The next major step in the implementation of the VMI is to integratewith the Language Reservoir developed by INSERM [18]. This module withinWYSIWYD generates a Predicate-Action-Object-Recipient (PAOR) descrip-tion of a previous memory retrieved through ABMSql. Each part of thisconcise description received by the VIM is then transmitted to SAM as anask modelName instance request which returns a fantasy memory from thecorresponding latent space.

iCub Visual Memory Inspector 55

(a) (b)

Fig. 3. Demonstration of VMI dynamic object loading. (a) Depicts the initial state ofthe VMI which starts off with just the iCub and an environment (b) Depicts the stateof the VMI with the dynamic addition of a person and two objects within the loadedenvironment

Subsequently, after all constituents have been parsed and an instancereceived, the information is displayed as a scene within the VMI. A demon-strative example of such an interaction can be seen in Fig. 4 where the faceinstance recovered from SAM for the label ‘Daniel ’ has been embodied within ageneric body.

Fig. 4. Demonstration of the addition of a face recalled from SAM to the generic bodyof a dynamically instantiated person within the VMI

56 D. Camilleri et al.

6.2 Virtual Sensing for Planning

Of the iCub’s four senses, currently only sight is available within the VMI.Upcoming work will focus on the implementation of depth cameras, texels fortouch and binaural microphones for sound which will allow planning and simu-lating the outcome of actions within the VMI.

7 Conclusion

This paper has briefly demonstrated the various applications that have beendeveloped for SAM using different sensory modalities. Moreover, this paper hasdemonstrated the three challenges that arise with the concurrent use of multipleSAM models. These are the management of all models, the ease of interfacingand finally the challenge of visualising what is happening within these memorymodels in an interactive manner. As such we proposed the use of two modules:Sam Supervisor the Visual Memory Inspector (VMI) which provide a solution tothis systems problem. Finally we lay out a plan for the continued developmentof these modules into an easily expandable software system upon which thedevelopment of a synthetic human autobiographical memory can be based.

References

1. Evans, M.H., Fox, C.W., Prescott, T.J.: Machines Learning - Towards a New Syn-thetic Autobiographical Memory. In: Duff, A., Lepora, N.F., Mura, A., Prescott,T.J., Verschure, P.F.M.J. (eds.) Living Machines 2014. LNCS, vol. 8608, pp. 84–96.Springer, Heidelberg (2014)

2. Damianou, A., Ek, C.H., Boorman, L., Lawrence, N.D., Prescott, T.J.: A Top-Down Approach for a Synthetic Autobiographical Memory System. In: Wilson,S.P., Verschure, P.F.M.J., Mura, A., Prescott, T.J. (eds.) Living Machines 2015.LNCS, vol. 9222, pp. 280–292. Springer, Heidelberg (2015)

3. Damianou, A., Lawrence, N.: Deep gaussian processes. In: Carvalho, C., Ravikumar,P. (eds.) Proceedings of the Sixteenth International Workshop on Artificial Intelli-gence and Statistics (AISTATS). AISTATS 2013, JMLR W&CP, vol. 31, pp. 207–215(2013)

4. Damianou, A.: Deep gaussian processes and variational propagation of uncertainty.PhD thesis, University of Sheffield (2015)

5. IIT: iCub: an open source cognitive humanoid robotic platform. http://www.icub.org/. Accessed 1 Mar 2016

6. Gelbard-Sagiv, H., Mukamel, R., Harel, M., Malach, R., Fried, I.: Internally gen-erated reactivation of single neurons in human hippocampus during free recall.Science 322(5898), 96–101 (2008)

7. Martinez-Hernandez, U., Boorman, L., Damianou, A., Prescott, T.: Cognitivearchitecture for robot perception and learning based on human-robot interaction

8. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2),137–154 (2004)

9. Baltrusaitis, T., Robinson, P., Morency, L.P.: Constrained local neural fields forrobust facial landmark detection in the wild. In: Proceedings of the IEEE Interna-tional Conference on Computer Vision Workshops, pp. 354–361 (2013)

iCub Visual Memory Inspector 57

10. Zheng, F., Zhang, G., Song, Z.: Comparison of different implementations of MFCC.J. Comput. Sci. Technol. 16(6), 582–589 (2001)

11. Loweimi, E., Doulaty, M., Barker, J., Hain, T.: Long-Term Statistical FeatureExtraction from Speech Signal and Its Application in Emotion Recognition. In:Dediu, A.-H., Martın-Vide, C., Vicsi, K. (eds.) SLSP 2015. LNCS, vol. 9449, pp.173–184. Springer, Heidelberg (2015). doi:10.1007/978-3-319-25789-1 17

12. YARP: Yet another robot platform. http://wiki.icub.org/yarpdoc/. Accessed 10Feb 2016

13. Unity Technologies: Unity 5.2.2. https://unity3d.com/. Accessed 1 Mar 201614. Copelia Robotics: V-rep. http://www.coppeliarobotics.com/index.html. Accessed

1 Mar 201615. Cyberbotics Ltd.: Webots. https://www.cyberbotics.com/overview. Accessed 1

Mar 201616. Open Source Robotics Foundation: Gazebo. http://gazebosim.org/. Accessed 1

Mar 201617. Swig: Swig. http://www.swig.org/. Accessed 1 Mar 201618. Hinaut, X., Twiefel, J., Petit, M., Bron, F., Dominey, P., Wermter, S.: A recurrent

neural network for multiple language acquisition: Starting with english and french

A Preliminary Framework for a SocialRobot “Sixth Sense”

Lorenzo Cominelli1(B), Daniele Mazzei1, Nicola Carbonaro1,Roberto Garofalo1, Abolfazl Zaraki1, Alessandro Tognetti1,2,

and Danilo De Rossi1,2

1 Faculty of Engineering, Research Center “E. Piaggio”,University of Pisa, Pisa, Italy

lorenzo.cominelli@for.unipi.it2 Department of Information Engineering, University of Pisa, Pisa, Italy

http://www.faceteam.it

Abstract. Building a social robot that is able to interact naturally withpeople is a challenging task that becomes even more ambitious if therobots’ interlocutors are children involved in crowded scenarios like aclassroom or a museum. In such scenarios, the main concern is enablingthe robot to track the subjects’ social and affective state modulating itsbehaviour on the basis of the engagement and the emotional state of itsinterlocutors. To reach this goal, the robot needs to gather visual andauditory data, but also to acquire physiological signals, which are fun-damental for understating the interlocutors’ psycho-physiological state.Following this purpose, several Human-Robot Interaction (HRI) frame-works have been proposed in the last years, although most of them havebeen based on the use of wearable sensors. However, wearable equipmentsare not the best technology for acquisition in crowded multi-party envi-ronments for obvious reasons (e.g., all the subjects should be preparedbefore the experiment by wearing the acquisition devices). Furthermore,wearable sensors, also if designed to be minimally intrusive, add an extrafactor to the HRI scenarios, introducing a bias in the measurements dueto psychological stress. In order to overcome this limitations, in this work,we present an unobtrusive method to acquire both visual and physiolog-ical signals from multiple subjects involved in HRI. The system is ableto integrate acquired data and associate them with unique subjects’ IDs.The implemented system has been tested with the FACE humanoid inorder to assess integrated devices and algorithms technical features. Pre-liminary tests demonstrated that the developed system can be used forextending the FACE perception capabilities giving it a sort of sixth sensethat will improve the robot empathic and behavioural capabilities.

Keywords: Affective computing · Behaviour monitoring · Human-Robot Interaction · Social robotics · Synthetic tutor

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 58–70, 2016.DOI: 10.1007/978-3-319-42417-0 6

A Preliminary Framework for a Social Robot “Sixth Sense” 59

1 Introduction

Nowadays, it is well-known that our emotional state influences our life and deci-sions [1]. Education and learning processes are not excluded from this claim andthe review, written by Bower [2], about how emotions might influence learning, aswell as the more specific research about the effects of affect on foreign languagelearning, done by Scovel [3], are two of several important studies confirmingthis theory. As a consequence, in the last years this topic has triggered alsothe interest of social robotics scientists [4,5]. Indeed, they develop humanoidrobots destined to interact with people, and the emotional and psychologicalstates of these androids’ interlocutors have to be necessarily taken into account.This becomes even more important in case where robots have to interpret therole of synthetic tutors intended for teaching children and conveying pedagogicalcontents in scenarios like musea or schools [6].

In order to interpret the emotional states of the pupils who interact withrobots, social robotics researchers have proposed many different solutions thatcan be divided in two main categories: Visual-Auditory Acquisition and Physio-logical Signals Acquisition. The processing of both kinds of information aims toan estimation of the interlocutors’ affective states, and these social/emotionalinformation is exploited, in turn, by the control architecture of the robots, tomodulate their behaviour, or completely change the actions they were planning.This improves the empathy and facilitates the dialogue between robots who areteaching and children who are learning. In any case, the main problem of thesetwo approaches is that, up to date, there is not a good integration of data deliv-ered by both the acquisitions. In most of the cases, the perception systems aredesigned to use only one kind of acquisition, and this is not sufficient to deter-mine a stable perception of the human emotional state, but just a temporaryassessment that can’t be used in a long-term interaction [7].

The visual-auditory acquisition has the peculiarity to be more stable and wellfunctioning thanks to a lot of available devices, which are easy to use but alsoeasy to mislead (e.g., a smiling face has not to be always interpreted as an happyperson); while the acquisition of physiological parameters has the advantage toreveal hidden emotional states and involuntary reactions that are relevant forhuman behaviour understanding, but this approach has the major outstandingproblem to be an obtrusive, if not even invasive method.

In this paper, we present a novel architecture that supports the acquisitionof both visual-auditory and physiological data. It is composed of the Scene Ana-lyzer, a software for audio and video acquisition and social features extraction [8];and the TouchMePad, consisting of two electronic patches and a dedicated sig-nal processing software for the users physiological monitoring and affective stateinference. TouchMePad is thought to be integrated with the other half of theperception system to become the sixth sense of a social robot. Moreover, the ded-icated software, as well as the whole acquisition framework, has been designed tocollect data in a sporadic and unobtrusive way in order to minimise the stress forthe subjects and reach a high level of naturalness during the HRI. This is highlybeneficial considering that Scene Analyzer is able to identify different persons

60 L. Cominelli et al.

by means of a QR codes reader module. Thanks to this recognition capability,once a subject is recognised, the physiological signals acquired by the Touch-MePad can be stored in a database that will permanently associate a unique IDwith the recognised subject. This information is compared with other meaning-ful social information (e.g., facial expression, posture, voice direction), providingthe robot with the capability to change or modulate the tutoring according todifferent persons and their emotional state both in real time and accordingly topast interactions (e.g., previous lessons).

2 The Perception Framework

The perception framework is composed of two main parts: the Scene Analyzer,our open-source visual-auditory acquisition system that we continuously upgradeand deliver on Github1; the TouchMePad, composed of two hardware patches forphysiological parameters acquisition and a dedicated software for signal process-ing and affective computing.

2.1 Scene Analyzer

We designed Scene Analyzer (SA) as an out-of-the-box human-inspired percep-tion system that enables robots to perceive a wide range of social features withthe awareness of their real-world contents. SA enjoys of a peculiar compatibility,indeed, due to its modular structure, it can be easily reconfigured and adaptedto different robotics frameworks by adding/removing its perceptual modules.Therefore, it can be easily integrated to any robot irrespective of the workingoperating system.

SA consists of four distinct layers (shown in Fig. 1) data acquisition, low-leveland high-level features extraction, structured data creation and communication.SA collects raw visual-auditory information and data about environment and,through its parallel perceptual sub-modules, extracts higher level informationthat are relevant from a social point of view (e.g., subject detection and tracking,facial expression analysis, age and gender estimation, speaking probability, bodypostures and gestures). All this information is stored in a dynamic storage calledmeta-scene. The SA data communication layer streams out the created meta-scene through YARP (Yet Another Robot Platform) middleware [9].

SA, supports a reliable, fast, and robust perception-data delivering enginespecifically designed for the control of humanoid synthetic tutors. It collectsvisual-auditory data through Kinect ONE 2D camera, depth IR sensor, andmicrophone array with the highest level of precision. For example, it detects andkeeps track of 25 body-joints of six subjects at the same time (Fig. 2), whichis a fundamental capability for crowded scenarios. With such an accuracy itis possible to perceive gestures that are very important in educational contexts(e.g., head rubbing, arm crossed, exulting etc.). For the same purpose, extraction

1 https://github.com/FACE-Team/SceneAnalyzer-2.0.

A Preliminary Framework for a Social Robot “Sixth Sense” 61

Fig. 1. Framework of the Scene Analyzer, the visual-auditory part of the perceptionsystem.

of data about joint coordinates of the lower part of the body has been added tothe coordinates of the upper torso. These added coordinates makes the systemcapable to detect also the body pose of a person. Accordingly, we can distinguishbetween a seated and a standing position. For further information about meta-scene data structure and SA visual-auditory acquisition please refer to [10].

In order to endow the robot with a stable and reliable ID recognition capa-bility, a new module has been implemented: the QR Code Module. It works inparallel analysing the image provided by a separated HD camera, and it is con-ceived for detecting and extracting information from QR codes. These bar-codesmatrices can be printed and applied to the user’s clothes or attached to theobjects with which the robot has to interact. SA exploits an SQL database inwhich a list of QR codes are associated with permanent IDs and names. How-ever, any further information can be added to any stored subject or object. Everytime the synthetic agent will be able to recognise in the FOV a QR code that isknown, because saved in the internal database, it will retrieve this informationand automatically assign a known ID and name to that entity. Moreover, assum-ing that the entity is a person who has already interacted with the robot, oncethe subject has been recognised by the means of the QR code, the associationbetween the permanent ID and that subject will continue even if the QR codewill be no more visible by the camera.

This solution for assigning, storing and retrieving permanent informationabout interlocutors is fully automatic and mandatory in order to customise

62 L. Cominelli et al.

Fig. 2. Performance of the Scene Analyzer in a crowded scenario.

teaching methods for different students. The possibility to deliver personalisedlearning has been considered as a requirement according to the last decadestrend in education and the perspectives for the next generation learning envi-ronment [11,12]. Another important feature of SA is processing of environmentaldata streamed by Zerynth Shield2, an electronic board for environmental analy-sis (e.g., light intensity, temperature, audio level), which is useful to determinepotential influences of the environment on interaction and learning engagement.

2.2 TouchMePad

In order to estimate and recognise human emotions from physiological parame-ters, several techniques have been developed in the last years, and most of themexploit wearable sensors (e.g., [13]). Since our system is intended to be used incrowded environments involving pupils or young users, the usage of sensorizedclothes such as gloves or shirts is a considerable complication. Furthermore, acontinuous and permanent contact would invalidate the naturalness of the inter-action which is already difficult enough since a humanoid robot is implicatedin it. Last but not least, an unceasing acquisition of multiple data from manysubjects, including who is not currently involved in a real interaction with therobot, would be useless as well as overwhelming for the data processing phase.

For all these reasons, we opted for a user-centred solution that is non-invasive,unobtrusive and keeps the naturalness of a social interaction. Indeed, it is con-ceived to prevent discomfort for the user who has not to be permanently attachedto sensors. On the contrary, two electronic patches are attached to the synthetictutor shoulders and the subjects are asked to touch sporadically the shoulders ofthe robot in order to acquire physiological signals only in some key moments of

2 http://www.zerynth.com/zerynth-shield/.

A Preliminary Framework for a Social Robot “Sixth Sense” 63

the interaction. This facilitates both the user and the acquisition system, reduc-ing the number of contacts with the sensors, as well as the amount of gathereddata, to the strictly necessary.

Therefore, TouchMePad (TMP) can be considered as the other half of theperception framework providing the robot with a sort of sixth sense. TMP isconceived to monitor the variation of the physiological parameters that are cor-related to human affective state. As shown in Fig. 3 the system is composedof the electronic patches, acting as electrodes, and a central electronic unit forpower supply, elaboration and transmission of user physiological parameters.The electronic unit is designed to conveniently combine ECG analog front-endwith EDA one by means of a low-power micro controller. The developed ECGblock is a three leads ECG system that samples signals at the frequency of512 Hz. The front-end is based on the INA321 instrumentation amplifier andon one of the three integrated operational amplifiers available in the microcontroller (MSP family made by Texas Instruments, MSP430FG439), to reachthe total 1000x amplification. Regarding the EDA circuit, a small continuousvoltage (0.5 V) is applied to the skin and the induced current is measured throughtwo electrodes positioned in correspondence with the middle and the ring fin-ger. The ratio between voltage drop and induced current represents the skinelectric impedance, the EDA signal. An ad-hoc finger electrodes patches weredesigned and developed allowing the acquisition of user physiological parametersin a natural way. Therefore, the user does not have to wear any type of band,shirt etc. but simply touching the patches with the fingers it is possible to cal-culate the Inter-Beat Interval (IBI) parameter and the Electro Dermal Activity(EDA) signal. Finally, the system evaluates the robustness of user contact toidentify physiological signal segments that are affected by artifacts and have tobe discarded in further analysis. The detection of an artifact can also trigger anautomatic request, by the robot to the user, to modify fingers position on theelectrodes or to place them on the patches once again for a better acquisition.

Fig. 3. Schema and photos of the acquisition patches.

64 L. Cominelli et al.

For further information about physiologic signal acquisition and the methodswe use for processing this data, please refer also to our previous works involvingthe acquisition of ECG [14,15] and EDA [16].

3 The ID Tracking Module

Since the perception system is composed of two separated main acquisition sys-tems working in parallel (i.e., SA for visual-auditory acquisition and TMP forphysiological parameters acquisition), we needed to find a solution to unify thedata extracted from subjects in order to have a unique classification associatedwith IDs. The issue about having a link between information delivered by the SAand the one delivered by TMP depended on a very practical problem: every timea subject had to place their fingers on the electrodes, the camera got coveredby that subject, and the SA was no longer able to detect any person in the fieldof view. This entails that the entire perception system, as it was initially con-ceived, was not able to assign the physiologic parameters to any person, becauseno person was detected while signals were acquired. Therefore, we developed adedicated sub-module, which provided a workaround for this ‘identity problem’,ensuring the assignment of both SA and TMP data to a univocal ID.

We named it the ID Tracking Module (IDTm) and it works as a bridgebetween the TMP and the SA. This module continuously calculates the standarddeviation of the pixels extracted by a cropped, re-sized, central part of the imageacquired by the Kinect camera. Every time a subject approaches the patchescovers most of the image field, as well as all the other people present in thescene. Nevertheless, we have another effect: the image becomes almost the same,especially the central part, regardless light conditions, contrast and brightness.Furthermore, the SA delivers information about the distance in meters of anysubject until is detected. Considering that only one subject at a time has thepossibility to put their fingers on the patches, we retrieve the information aboutthe last detected closest subject and, when the standard deviation of the centralimage gathered by the camera comes under a threshold (empirically decided at50), the IDTm saves the ID of the last detected closest subject, assumes thathe/she is the one approaching the pads, and will assign potential physiologicparameters detected by the TMP, to that specific ID. All the information aboutthe physiological parameters is stored in the database, assigned to the IDTouching

provided by the IDTm, ready for an affective state estimation and a possiblecomparison between past and future acquisitions. Finally, the subject is detectedagain and recognised as soon as he leaves the patches and comes back clear inthe gathered image.

This strategy not only solves a system drawback, but also demonstrates thepower of a multi-tasking and modular architecture as the one we are presenting.A schema of this solution is shown in Fig. 4.

A Preliminary Framework for a Social Robot “Sixth Sense” 65

Fig. 4. The ID tracking schema.

4 Evaluation of the Perception System

4.1 Materials and Methods

The use case is based on the installation of the SA and the TMP on the FACERobot body [17].

Data collected by the SA are stored in an SQL database together with theone provided by the TMP, thanks to the ID Tracking module. Once a contacton the TMP is detected, the quality of the signal is analysed and the infor-mation about contact quality is shown on the screen in which the interlocutorcan read instructions. Projected instructions are: “Please put your fingers onthe patches”, “Bad contact! Please place your fingers again” or “Good Contact!Please keep your fingers on the patches”. In this latter case, a 60 s countdownimmediately starts on the screen. This is the duration that we set for havinga reliable acquisition of physiological data. As a consequence, if the contact ismaintained at a sufficient quality level, at the end of the countdown it doesappear the following message: “Thank you, you can remove your hands now”.Acquired data are pre-filtered by the Physio Signal Processing Module shown inFig. 4, which discards unreliable values, then calculates the IBI mean value, theEDA, asks for the IDTouching to the IDTm, then finally sends through yarp allthis information to the database, adding the physio data to the proper ID.

Several tests were performed in order to verify the capability of the TMPto monitor user physiological state. A 27 years old male was selected for theexperiment whose heart beat frequency in rest condition is known and in valueof 80 bpm. This information has been useful for the comparison with the valuecalculated from the IBI values extracted by the sensor patches. The test consistedof different sessions of 1 min in which the subject was asked to maintain a restcondition and to put his fingers of both hands in direct contact with the TMP.

66 L. Cominelli et al.

4.2 The Experiment

The experiment begins with the detection of a social scene involving two subjects.Initially, the image gathered by the SA has a high variance as shown in the firstchart of Fig. 5. After 5 s the robot is able to detect both subject 1, who is a 26years old female (the red line in Fig. 5), and subject 2 (the blue line in the samechart), which is the 27 years old male who is going to touch the TMP. After 8 ssubject 2 approaches the FACE Robot and his detected distance decreases, whilethe female remains standing at 1.6 m from the robot. When subject 1 gets closerto the robot there are two consequences: covering the image, the pixels’ standarddeviation decreases and, at 10 s, subject 1 gets lost by the perception system.After just 1 s, also subject 2 disappears completely from the scene (d = 0), buthis ID is saved by the IDTm as the last closest subject’s ID, assuming that heis the one who is going to touch the patches.

Therefore, the acquisition system starts looking for a potential contact, check-ing and analysing the contact quality of the finger electrodes. This can be noticedin the first 2 s of Fig. 6, that shows the trend of the Contact Quality (CQ) para-meter in correspondence with the two acquired physiological parameters (i.e.,IBI and EDA). CQ allows to distinguish not only between bad and good sensorcontact, but also to determine which fingers are not well positioned by means ofthe elaboration of the relative physiological signal. In fact, each of the six fingersinvolved in the acquisition has some peculiarities: for example the right handmedium finger is used as the reference electrode for the ECG circuit, while thering and the medium finger of the left hand allow to track the EDA signal, aspreviously shown in the schema of Fig. 3. Considering Table 1, in which all thepossible combination of CQ values are reported, we can claim that the systemis working correctly when the CQ parameter is equal to 50, while in other con-dition the validity of physiological data is not guaranteed and fingers should berepositioned.

The physiological parameters gathered during all the 60 s of acquisition areshown in Fig. 6. At the beginning CQ is 60, representing that the circuit forhand detection is activated and ready for the elaboration. Then the IBI andEDA values are tracked and stored. At t = 9s the CQ value decreases to 30,

Table 1. CQ value definition

CQ value Meaning

0 No Contact

10 IBI No Contact, EDA Good Contact

20 IBI & EDA Bad Contact

30 IBI Bad Contact, EDA Good Contact

40 IBI Good Contact, EDA Bad Contact

50 IBI & EDA Good Contact

60 Contact Circuit Active

A Preliminary Framework for a Social Robot “Sixth Sense” 67

Fig. 5. The social robot sixth sense evaluation test - ID tracking module. (Color figureonline)

Fig. 6. Physiological Parameters extracted by the perception framework in real timeduring the 60 s of acquisition highlighted in Fig. 5.

68 L. Cominelli et al.

likely due to a bad placement or a movement of the finger for IBI acquisition. Atthe same instant, in fact, it is possible to notice that the IBI signal presents asudden variation with its value that goes under the minimum level of the session(represented by the green line). The same situation could be found at t = 31s.An other important event is located at t = 40s. At this stage the subject waselicited with an auditory stimulus (i.e., a sudden hand clap) to validate thecontribution of the EDA circuit. As a results, in Fig. 6 the EDA signal leavesthe constant trend with a relative peak that depends on the phasic activity ofthe sweat gland in response to the auditory stimulus. At the end of one minuteof acquisition the subject is asked to remove his hands from the patches. Hemoves away from the robot that detects again him and the female who was stillstanding in the initial position, recognising both of them.

5 Conclusions and Future Works

In this preliminary work, an unobtrusive social scene perception system providedwith a physiological signal perception “sixth sense” given by a sporadic acqui-sition of physiological parameters, has been developed. Such a system has beenproved to endow a social robot with the possibility to infer emotional and psy-chological states of its interlocutors. This information is fundamental for socialrobots aimed at establishing natural and emphatic interactions with humans.This system has been designed in order to be used in robotic enhanced teachingcontexts, where social robots assume the role of synthetic tutors for childrenand pupils. To evaluate the developed perception framework including all itsfeatures, we designed an ad-hoc test that stressed all the functionalities of theacquisition system. The experiment demonstrated the capability of the systemto properly acquire the entire data-set of information assigning gathered datato unique and permanent subjects’ IDs. Moreover, it is important to highlightthat the fusion of the data collected by the two system integrated in the pre-sented setup goes beyond the simple data merging. The enhanced meta-scenewhich results from the system will give to researchers the possibility to betterinfer the subject’s social and affective state by correlating psycho-physiologicalsignals with behavioural data.

The presented perception system will be used as the acquisition system ofseveral humanoid robots involved in the EASEL European Project3 in differenteducational scenarios (e.g., musea, school classrooms). This will validate theportability of the system and will make it a practical framework for testing howto improve the dialogue between robotic tutors and children as well as theirlearning.

Acknowledgment. This work was partially funded by the European Commissionunder the 7th Framework Program projects EASEL, Expressive Agents for SymbioticEducation and Learning, under Grant 611971-FP7- ICT-2013-10. Special thanks toDaniela Gasperini for her fundamental contribution in the experiments organization.

3 http://easel.upf.edu/.

A Preliminary Framework for a Social Robot “Sixth Sense” 69

References

1. Damasio, A.: Descartes’ Error: Emotion, Reason, and the Human Brain.Grosset/Putnam, New York (1994)

2. Bower, G.H.: How might emotions affect learning. Handb. Emot. Mem. Res. Theor.3, 31 (1992)

3. Scovel, T.: The effect of affect on foreign language learning: a review of the anxietyresearch. Lang. Learn. 28(1), 129–142 (1978)

4. Hudlicka, E.: To feel or not to feel: the role of affect in human-computer interaction.Int. J. Hum. Comput. Stud. 59(1), 1–32 (2003)

5. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots.Robot. Auton. Syst. 42(3), 143–166 (2003)

6. Causo, A., Vo, G.T., Chen, I.M., Yeo, S.H.: Design of robots used as educationcompanion and tutor. In: Zeghloul, S., Laribi, M.A., Gazeau, J.-P. (eds.) Roboticsand Mechatronics. Mechanisms and Machine Science, vol. 37, pp. 75–84. Springer,Switzerland (2016)

7. Yan, H., Ang Jr., M.H., Poo, A.N.: A survey on perception methods for human-robot interaction in social robots. Int. J. Soc. Robot. 6(1), 85–119 (2014)

8. Zaraki, A., Mazzei, D., Giuliani, M., De Rossi, D.: Designing and evaluating asocial gaze-control system for a humanoid robot. IEEE Trans. Hum. Mach. Syst.44(2), 157–168 (2014)

9. Metta, G., Fitzpatrick, P., Natale, L.: Yarp: yet another robot platform. Int. J.Adv. Robot. Syst. 3(1), 43–48 (2006)

10. Zaraki, A., Giuliani, M., Dehkordi, M.B., Mazzei, D., D’ursi, A., De Rossi, D.:An rgb-d based social behavior interpretation system for a humanoid social robot.In: 2014 Second RSI/ISM International Conference on Robotics and Mechatronics(ICRoM), pp. 185–190. IEEE (2014)

11. Sampson, D., Karagiannidis, C.: Personalised learning: educational, technologicaland standardisation perspective. Interact. Educ. Multimedia (4) 24–39 (2010)

12. Brusilovsky, P.: Developing adaptive educational hypermedia systems: from designmodels to authoring tools. In: Murray, T., Blessing, S.B., Ainsworth, S. (eds.)Authoring Tools for Advanced Technology Learning Environments, pp. 377–409.Springer, Netherlands (2003)

13. Lisetti, C.L., Nasoz, F.: Using noninvasive wearable computers to recognize humanemotions from physiological signals. EURASIP J. Adv. Sign. Process. 2004(11),1–16 (2004)

14. Tartarisco, G., Carbonaro, N., Tonacci, A., Bernava, G., Arnao, A., Crifaci, G.,Cipresso, P., Riva, G., Gaggioli, A., De Rossi, D., et al.: Neuro-fuzzy physiologi-cal computing to assess stress levels in virtual reality therapy. Interact. Comput.(2015). iwv010

15. Carbonaro, N., Anania, G., Mura, G.D., Tesconi, M., Tognetti, A., Zupone, G., DeRossi, D.: Wearable biomonitoring system for stress management: a preliminarystudy on robust ECG signal processing. In: 2011 IEEE International Symposiumon a World of Wireless, Mobile and Multimedia Networks (WoWMoM), pp. 1–6.IEEE (2011)

70 L. Cominelli et al.

16. Carbonaro, N., Greco, A., Anania, G., Dalle Mura, G., Tognetti, A., Scilingo, E.,De Rossi, D., Lanata, A.: Unobtrusive physiological and gesture wearable acqui-sition system: a preliminary study on behavioral and emotional correlations. In:Global Health, pp. 88–92 (2012)

17. Mazzei, D., Zaraki, A., Lazzeri, N., De Rossi, D.: Recognition and expression ofemotions by a symbiotic android head. In: 2014 14th IEEE-RAS InternationalConference on Humanoid Robots (Humanoids), pp. 134–139. IEEE (2014)

A Bio-Inspired Photopatterning Method to Deposit SilverNanoparticles onto Non Conductive Surfaces Using

Spinach Leaves Extract in Ethanol

Marc P.Y. Desmulliez(✉), David E. Watson, Jose Marques-Hueso,and Jack Hoy-Gig Ng

School of Engineering & Physical Sciences, Nature Inspired Manufacturing Centre (NIMC),Heriot-Watt University, Earl Mountbatten Building, Edinburgh EH14 4AS, Scotland, UK

m.desmulliez@hw.ac.uk

Abstract. Densely packed silver nanoparticles (AgNPs) were produced ascontinuous films on non-conductive substrates in a site-selective manner. Theformation of the AgNPs were directed by blue light using extract from spinachleaves acting as photo-reducing agent. This bio-inspired production of reducedions nanofilms benefit applications where seed conductive layers are required forthe manufacture of metal parts embedded into plastics.

Keywords: Nature-inspired manufacturing · Biomimetism · Biomimicry ·Additive manufacturing · Artificial photosynthesis

1 Introduction

Intense research efforts have focused on the synthesis and deposition of silver nanopar‐ticles (AgNPs) due to their usefulness in plasmonics, electronics, biological sensing andas an antimicrobial agent [1–3]. Some applications require these silver nanoparticles tobe assembled in a reproducible manner as micro- or nano-scale periodical structuressuch as arrays or lines of specific width and space. Usually, AgNPs are synthesized ina colloidal solution and then consolidated onto a substrate using laser sintering [4],microwave [5] or plasma treatment [6]. They can also be prepared using the bottom-upapproach whereby metal ions are molecularly linked to the substrate onto which theyare deposited and then reduced to form nanoparticles within the molecular network ofthe linker layer. The latter approach can easily anchor metal nanoparticles onto a strongelectrostatic adhesive such as dopamine [7] or ion-exchange resins [8].

In that respect, surface hydrolysed polyimide or polyetherimide films are idealsubstrate materials for hosting metal nanoparticle thin films [9, 10]. These metal thinfilms can be produced in situ with a controllable thickness within a surface layer of ion-exchange resins of the R-COO- type obtained as the result of alkaline hydrolysis ofpolyimide forming an amorphous layer of poylamic acid on its surface [8]. In addition,different reduction methods, including thermal, chemical or photo-induced, can bereadily applied to such silver ion-exchange polyimide substrate (Ag+-PI).

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 71–78, 2016.DOI: 10.1007/978-3-319-42417-0_7

Photo-induced reduction is the simplest route for replicating the desired patterns byregulating the shape of the light source such as in laser writing applications or by usinga photomask as in photolithography. To the best of the authors’ knowledge, reportsregarding high-throughput photo-induced patterning of silver nanoparticle thin filmsthat is compatible with large area substrate are still lacking due to the poor quantumyield of the photo-reducing agents or catalysts employed today.

Herein we demonstrate a nature-inspired strategy to reduce metal ions using photo‐system 1 (PS I) found in spinach extract dissolved in ethanol. PS I is thought to be actingas the photo-reducing agent to produce patternable metal nanoparticles that meet therequirements of both production speed and process scalability. PS I comprises of lightharvesting pigment proteins, amongst which chlorophyll is being most commonlyreferred to for photosynthesis.

2 Materials and Methods

For our purpose, the PS I and mineral agents contained in the leaves were simplyextracted from spinach leaves by dissolving them in ethanol solvent, after using a blenderto collapse the chloroplasts in the leaves which house the PS I. In a typical process, 100 gof fresh leaves was mixed with 500 ml of absolute ethanol in a blender and processedfor 1 min. After passing the mixture through a 1:11 μm grade filter paper, 90 % of theliquid was then poured into a second flask and left to stand for some time for sedimen‐tation. The process of filtration was then repeated again to avoid any debris in the extractsolution. Next, 2 ml of the PS I and minerals extract solution were pipetted onto thesurface of the Ag+-PI substrate.

The substrate was originally prepared by a 2-step immersion: first into a 1 M KOHsolution at 50 °C for 5 min, followed by a 0.01 M [Ag(NH3)2]+ aqueous solution at roomtemperature for 5 min. As the preparation process involves only a couple of dippingprocess, substrates of large dimensions can readily be prepared depending on the sizeof the tanks that contain the two solutions described above. The photo-induced reductionstep took up to five minutes using a 460 nm wavelength fibre delivered light emittingdiode with an intensity of around 1270 mW/cm2.

3 Results

Figure 1, top left, shows photographs of circular spots of silver patterns with anapproximate diameter of 1 mm produced upon exposure to the blue light source atthe tip of the optical fibre. The corresponding field emission scanning electronmicroscopy (FESEM) images where the growth of silver nanoparticles can be clearlyobserved in the same figure. Generally, particle size and density of the AgNPsincreased with increasing exposure times. At 30 s exposure, AgNPs as small as 15 nmcan be easily identified across the illuminated area. Some particles aggregated tosizes of about 70–80 nm. Within 60 s of exposure, the whole illuminated area wascompactly covered with AgNPs. The smaller particles have an average size of about30 nm and the larger aggregated particles about 70 nm. At 180 s exposure, a large

72 M.P.Y. Desmulliez et al.

amount of particles had aggregated to 40 nm or bigger, where, noticeably, someparticles as large as 120 nm can be seen.

Fig. 1. Photographs and FESEM images of: (a) control background with no photo-inducedreduction. (b–e) the location selective production of AgNPs on Ag+-PI upon exposure to the lightsource for 30, 60, 180 and 600 s, respectively. Scale bar is 500 nm.

In the latter case the AgNPs are not compactly connected as in the case of the 60 sexposed sample indicating that the silver ion source in the substrate supplying the reac‐tion has been depleted and left a void behind when the smaller particles aggregated toform bigger particles.

At the prolonged exposure of 600 s, a dark mark can be clearly seen in the photographat the centre of the circular pattern in the Fig. 1 (top left). The excess photon energyunder this exposure time perhaps induced some thermal effects on the substrate resultingin polymer restructuring. The corresponding FESEM micrograph in Fig. 1(e) suggeststhat a reaction, as yet not explained, seemed to have taken place, which removed thelarger particles seen in Fig. 1(d). The small amount of AgNPs observed in Fig. 1(a) withno light exposure could be the result of spontaneous reduction of Ag+ ions, which purelyutilize the redox potential difference between impurity entities and the Ag + ions in theabsence of a reducing agent.

A Bio-Inspired Photopatterning Method to Deposit Silver Nanoparticles 73

4 Discussion

4.1 Short Exposure Time

The short exposure times required using a relatively low light intensity irradiation forthe formation of high density AgNPs were surprising. Up to now, irradiation methodsused for metal nanoparticle formation usually produce sparse particles from a dispersionin the liquid phase [11]. The enhancement reported here can be credibly contributed tothe function of PS I extract. It is known that impurity of metal traces is present in theabsolute ethanol solvent, however they did not contribute to the significantly fasterphotoreduction rate. Our previous work with the same Ag+-PI substrate system using asynthetic photoreducing agent methoxy (ethylene glycol) and with the same absoluteethanol solvent required several hours of light exposure to obtain appreciable amountof AgNPs [9]. Although the PS I structure might have been altered from its naturalmembrane arrangements through the extraction procedure, it has definitely enabled asignificant increase of the photoreduction rate. It would be interesting to study theperformance in the photoreduction rate along side purified PS I, and the effects ofdifferent extraction methods and the different types of plant leaves as the source material.

The reason why the photo-induced reduction can be conducted in such a short timewith a PS I coating on top of the Ag+-PI is partly due to the efficient electron transferwithin the PS I protein complex. Upon reception of the incident photon energy, rapidcharge separation occurs within 10–30 ps, releasing an electron down an intra-proteinenergy cascade to an iron-sulfur complex called FB

− [12]. When the PS I is in its naturalenvironment, the soluble, iron-containing protein ferredoxin shuttles the electrons awayfrom FB

− to achieve a nearly perfect quantum yield [13]. In addition, the nucleation and

Fig. 2. Schematics of plasmon field enhancement and charge storage on the surface of AgNPs,which assist the photo-reduction of Ag+ ions in the PS I/AgNPs hybrid system. The figure is aschematic representation of the assumed process with size of the nanoparticles not at the rightscale. The substrate has a prior deposited Ag + ion-exchanged resin.

74 M.P.Y. Desmulliez et al.

growth of the AgNPs in the present system benefit from a self-catalysed mechanism asillustrated in Fig. 2.

During the growth of the AgNPs, the PS I proteins are confined into the nano-cavitiesamongst the Ag aggregates. From this situation, two main driving forces can be attrib‐uted to the long range electron transfer from the PS I coating to the Ag+ ions within theion-exchange resins on the substrate. Firstly, the plasmon enhancement of photon fieldswithin a hybrid system of PS I molecular complex and AgNPs aggregates has beenmodelled [14] and measured [15] to provide a several fold increase in the generation ofexcited state electrons. Secondly, the AgNPs act as ultra large surface area nanostruc‐tured electrodes for storing the excited electrons released from PS I as charges on theAgNPs surfaces [16] and thus minimising charge recombination. More recent studiescarried out by the group seem to indicate that salts from the extract of spinach leavesmight also contribute to the donation of electrons. This new information will bepresented during the conference.

4.2 Three Potential Photoreduction Mechanisms

A cross-section FESEM image prepared by Broadband Ion Beam (BIB) machining is,shown in Fig. 3 and provides an insight into the photoreduction mechanisms. Theprocess, developed and used by MCS Ltd, allows deformation and smear-free cross-sectioning without obscuring the AgNPs-polymer matrix. The BIB system allows repre‐sentative sample sizes (up to 2 mm) to be cross-sectioned without mechanically touchingthe sample enabling absolute confidence that deformation or voids did not occur duringthe cross-sectioning procedures as in the FIB system.

There are three key features to note:

(i) The AgNPs produced by the photoreduction effectively formed a uniform nano‐composite layer within the modified polyimide substrate surface. No gradient ofAgNPs distribution was observed across the layer. The electrons generated fromthe PS-I coating and the natural mireal salts from the leaves were effectively trans‐ferred into the surface-modified polymer substrate. This suggests that some elec‐tron relay mechanisms must be involved as suggested earlier, since there wouldinevitably be an illumination gradient as the light impinges into the surface of thesubstrate due to both the polymer molecules and the in situ growth of the AgNPs.

(ii) The uniform boundary of the AgNPs nanocomposite illustrates the diffusion-controlled and well-defined nature of the 2-step immersion surface modificationprocess where the polyimide molecules were first hydrolysed by KOH followedby Ag+ ion-exchange. The line of shade underneath the AgNPs nanocompositelayer represents a small amount of unreacted Ag+ ions that were not reduced. Anexplanation could be that the rate of Ag+ ion diffusion within the polymer matrixis much slower than the rate of the electron transfer.

(iii) A relatively high thickness of approximately 1.5 μm layer of AgNPs nanocompo‐site was produced. Clusters of AgNPs were created through their nucleation andgrowth, however they were unable to form chains of continuous network withinthe volume of the nanocomposite to render the layer electrically conductive.

A Bio-Inspired Photopatterning Method to Deposit Silver Nanoparticles 75

5 A Scalable Nature-Inspired Manufacturing Process

Our approach in the synthesis and patterning of the AgNPs is easily scalable to largearea industrial production. Regular methods commonly involve producing the AgNPsin a synthesis matrix first, followed by the separation of the particles and their consoli‐dating onto desired locations on a substrate according to the following sequence:

1. Homogeneous mixing of the capping agent and the stabilizing agent with the metalions.

2. Reduction of the capped metal ions.3. Extraction of metal nanoparticles.4. Purification to remove organic residues.5. Consolidation and patterning onto a surface.

This procedure can take hours to perform. By first incorporating the metal ions intothe ion-exchange resin, the molecular networks of the polymer served the role of acapping agent when the AgNPs were grown in situ within the surface layer of thesubstrate. As a result, significant production time is reduced in the present process:

1. Hydrolysis of polyimide by KOH immersion. 5 min.2. Ion-exchange in a silver salt solution. 5 min.

Fig. 3. FESEM image of a cross-section of the AgNPs nanocomposite within the surface-modified layer of the polyimide substrate.

76 M.P.Y. Desmulliez et al.

3. Extraction of PS I and of the mineral salts form the leaves. The extract can beprepared separately and stored frozen prior to reaction. 30 min.

4. Exposure to light source for photo-reduction of silver ions. 1 min or less.

The requirement of the light intensity (around 1270 mW/cm2) is also low enoughsuch that a flood exposure lamp can be used for large area parallel production of AgNPpatterns through a photomask. However ambient room lighting or daylight does notprovide a large enough energy dose for any accelerated photoreduction of Ag+ ions inthe system here.

6 Conclusions

In summary, we have reported in this article a rapid strategy to photopattern macro-scalecompact thin films consisting of AgNPs.

We propose some mechanisms involved in the outstanding electron transfer phenom‐enon, which resulted in the charge separation and transfer within the PS I coating.Furthermore, some electron relay systems must be present within the silver/polymermatrix to allow the electron transfer to proceed further and produce an almost 1.5 μmthickness layer metal matrix composite.

The whole production process itself can be as short as 11 min, if the PS I and mineralsalts extract is prepared in advance. The absence of any toxic reducing agents or complexorganic ligands in the whole production process also makes this patternable AgNPsnanocomposite fabrication method suitable for biological and sustainable applications.

Acknowledgements. The authors of this article would like to acknowledge the financial supportof the UK Engineering & Physical Sciences Research Council (EPSRC) through the funding ofthe two research grants entitled Photobioform I (EP/L022192/1) and Photobioform II (EP/N018222/1).

References

1. Luo, X., Morrin, A., Killard, A.J., Smyth, M.R.: Application of nanoparticles inelectrochemical sensors and biosensors. Electroanalysis 18(4), 319–326 (2006)

2. Cobley, C.M., Skrabalak, S.E., Campbell, D.J., Xia, Y.: Shape-controlled synthesis of silvernanoparticles for plasmonic and sensing applications. Plasmonics 4(2), 171–179 (2009)

3. Sondi, I., Salopek-Sondi, B.: Silver nanoparticles as antimicrobial agent: a case study on E.coli as a model for Gram-negative bacteria. J. Colloid Interf. Sci. 275, 177–182 (2004)

4. Yung, K., Wu, S., Liem, H.: Synthesis of submicron sized silver powder for metal depositionvia laser sintered inkjet printing. J. Mater. Sci. 44(1), 154–159 (2009)

5. Perelaer, J., de Gans, B., Schubert, U.: Ink-jet printing and microwave sintering of conductivesilver tracks. Adv. Mater. 18(16), 2101–2104 (2006)

6. Reinhold, I., Hendriks, C., Eckardt, R., Kranenburg, J., Perelaer, J., Baumann, R., Schubert,U.: Argon plasma sintering of inkjet printed silver tracks on polymer substrates. J. Mater.Chem. 19(21), 3384–3388 (2009)

A Bio-Inspired Photopatterning Method to Deposit Silver Nanoparticles 77

7. Cheng, C., Nie, S., Li, S., Peng, H., Yang, H., Ma, L., Sun, S., Zhao, C.: Biopolymerfunctionalized reduced graphene oxide with enhanced biocompatibility via mussel inspiredcoatings/anchors. J. Mater. Chem. B 1, 265–275 (2013)

8. Akamatsu, K., Ikeda, S., Nawafune, H., Deki, S.: Surface modification-based synthesis andmicrostructural tuning of nanocomposite layers: monodispersed copper nanoparticles inpolyimide resins. Chem. Mater. 15(13), 2488–2491 (2003)

9. Ng, J.H.-G., Watson, D.E.G., Sigwarth, J., McCarthy, A., Prior, K.A., Hand, D.P., Yu, W.,Kay, R.W., Liu, C., Desmulliez, M.P.Y.: On the use of silver nanoparticles for directmicropatterning on polyimide substrates. IEEE Trans. Nano. 11(1), 139–147 (2012)

10. Watson, D.E., Ng, J.H.-G., Aasmundtveit, K.E., Desmulliez, M.P.Y.: In-situ silvernanoparticle formation on surface-modified polyetherimide films. IEEE Trans. Nano. 13(4),736–742 (2014)

11. Balan, L., Schneider, R., Turck, C., Lougnot, D., Morlet-Savary, F.: Photogenerating silvernanoparticles and polymer nanocomposites by direct activation in the near infrared. J.Nanomaterials 2012, 1–6 (2012)

12. Chitnis, P.R.: Photosystem I: function and physiology. Annu. Rev. Plant Biol. 52(1), 593–626 (2001)

13. Noy, D., Moser, C.C., Dutton, P.L.: Design and engineering of photosynthetic light-harvesting and electron transfer using length, time, and energy scales. Biochim. Biophys.Acta 1757(2), 90–105 (2006)

14. Govorov, A.O., Carmeli, I.: Hybrid structures composed of photosynthetic system and metalnanoparticles: plasmon enhancement effect. Nano Lett. 7(3), 620–625 (2007)

15. Carmeli, I., Lieberman, I., Kraversky, L., Fan, Z., Govorov, A.O., Markovich, G., Richter,S.: Broad band enhancement of light absorption in photosystem I by metal nanoparticleantennas. Nano Lett. 10(6), 2069–2074 (2010)

16. Lioubashevski, O., Chegel, V.I., Patolsky, F., Katz, E., Willner, I.: Enzyme-catalyzed bio-pumping of electrons into au-nanoparticles: a surface plasmon resonance and electrochemicalstudy. J. Am. Chem. Soc. 126(22), 7133–7143 (2004)

78 M.P.Y. Desmulliez et al.

Leg Stiffness Control Based on “TEGOTAE”for Quadruped Locomotion

Akira Fukuhara1,2(B), Dai Owaki1, Takeshi Kano1, and Akio Ishiguro1,3

1 Research Institute of Electrical Communication, Tohoku University,2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan

{a.fukuhara,owaki,tkano,ishiguro}@riec.tohoku.ac.jp2 JSPS, 5-3-1 Kojimachi, Chiyoda-ku, Tokyo 102-0083, Japan

3 CREST, Japan Science and Technology Agency,4-1-8 Honcho, Kawaguchi, Saitama 332-0012, Japan

http://www.cmplx.riec.tohoku.ac.jp

Abstract. Quadrupeds exhibit adaptive limb coordination to achieveversatile and efficient locomotion. In particular, the leg-trajectorychanges in response to locomotion speed. The goal of this study is toreproduce this modulation of leg-trajectory and to understand the con-trol mechanism underlying quadruped locomotion. We focus primarily onthe modulation of stiffness of the leg because the trajectory is a resultof the interaction between the leg and the environment during locomo-tion. In this study, we present a “TEGOTAE”-based control scheme tomodulate the leg stiffness. TEGOTAE is a Japanese concept describingthe extent to which a perceived reaction matches the expected reaction.By using the presented scheme, foot-trajectories were modified and thelocomotion speed increased correspondingly.

Keywords: Quadruped locomotion · Autonomous distributed control ·Foot trajectory · Leg stiffness · TEGOTAE

1 Introduction

Efficient and adaptive locomotion of quadrupeds can be realized by flexible inter-limb and intra-limb coordination. For example, quadrupeds exhibit various typesof gait depending on the locomotion speed, morphology, and the environment [1].Although various quadruped robots with particular focus on inter-limb coordina-tion have been reported [2,3], the mechanisms of intra-limb coordination withoutinverse-kinematics are still an open challenge. As regards to intra-limb coordi-nation, cursorial mammals change their stroke length in response to increasinglocomotion speed [4]. In addition, the period of the cyclic motion and the lengthof one step change adaptively with locomotion speed [5]. In order to reveal the

A. Fukuhara—This work was supported by JSPS KAKENHI Grant-in-Aid for JSPSFellows (16J03825).

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 79–84, 2016.DOI: 10.1007/978-3-319-42417-0 8

80 A. Fukuhara et al.

Rotational actuator

Prismatic actuator

Passive rotational spring and damper

Ground reaction force

Fig. 1. Kinematic model of quadruped robot.

mechanism behind efficient and adaptive locomotion, both inter-limb and intra-limb coordination should be investigated in detail.

The goal of this study is to design adaptable and cursorial quadruped robotsas well as to understand the mechanism of adaptive limb coordination. First,we focus on the modification of leg stiffness given that the foot trajectory isaffected by the interaction between the leg and the environment during locomo-tion. Many studies on bio-inspired robots have shown the merits of compliantlegs for stable locomotion [6]. However, the control schemes for adjusting stiff-ness were different depending on each study. In this study, we propose a newcontrol scheme for quadruped locomotion to adaptively modulate the stiffnessof different leg regions, based on the concept “TEGOTAE.” In a 2D simulation,leg-stiffness control based on TEGOTAE leads to a change in the foot trajectory,and locomotion speed increases correspondingly.

2 Model

In order to reproduce the inter-limb coordination, we implemented the simplecentral pattern generator (CPG) model proposed in our previous study [7] intoa simple 2D robot-model, as shown in Fig. 1. The robot is modeled as a mass-spring system. Each leg of the robot has 2 degrees of freedom (DOF) owing tothe presence of rotational and prismatic actuators. These actuators are modeledusing a virtual spring and damper. The stiffness of the virtual springs representsthe leg stiffness. In addition, the body of the quadruped comprises 3 mass points;these mass points are connected using passive rotational spring and damper. Inthis study, we present a new TEGOTAE-based control method for adjustmentof leg stiffness thereby enabling improved intra-limb coordination.

Leg Stiffness Control Based on “TEGOTAE” for Quadruped Locomotion 81

2.1 CPG Model

In our previous study [7], we proposed a CPG model that consists of four decou-pled oscillators with local sensory feedback obtained from only a force sensorplaced on each corresponding leg. Each oscillator of the CPG is described as

φi = ω − σNi cos φi, (1)

where φi is the phase of the oscillator of the i th leg, ω is the intrinsic angularvelocity, σ is a weighting factor for sensory feedback, and Ni is sensory informa-tion about ground reaction force (GRF). Each i th leg tends to be in the swingphase for 0 < φi < π, while it tends to be in the stance phase for π < φi < 2π.When Ni = 0, φi increases with ω and the leg repeats recovery-stroke andpower-stroke motions depending on φi. When Ni > 0, φi is modulated to pulltoward 3π/2. Therefore, the leg supporting the body tends to keep stance phasein response to Ni. In spite of the simple rules of this CPG, our quadruped robotsexhibited a variety of gaits in response to mass distribution and the frequencyof locomotion [7].

2.2 Leg Stiffness Control Based on TEGOTAE

In order to model flexible intra-limb coordination, we deal with leg stiffness as thefirst step. In this study, while the basic foot trajectory is designed using simplesinusoidal curves, the stiffness of the leg is adjusted on the basis of TEGOTAE.The basic mapping of φi to foot trajectory is described as follows:

θi = Camp cos φi, (2)li = Loffset − Lamp sinφi, (3)

where θi and li are target angle and position for rotational and prismatic actua-tors, respectively, which constitute i th leg in Fig. 1, Camp is amplitude value forthe rotational motion, and Loffset and Lamp are offset and amplitude values forthe prismatic motion, respectively. According to Eqs. (2) and (3), each i th legtends to be in recovery-stroke when 0 < φi < π, while it tends to be in power-stroke when π < φi < 2π. Based on the reference trajectory, the rotational andprismatic actuators are controlled by using PD control, which is given by thefollowing equations:

τRoti = KRot

i (θi − θi) − DRoti θi (4)

τPrii = KPri

i (li − li) − DPrii li, (5)

where τRoti is the control torque for the rotational actuator, τPri

i is the controlforce for the prismatic actuator, KRot

i and KPrii are P gains for rotational and

prismatic actuator, respectively, and DRoti and DPri

i are D gains for rotationaland prismatic actuator, respectively. KRot

i and KPrii also reflect the leg stiff-

ness for rotational and prismatic joints. These K are adjusted in response toTEGOTAE.

82 A. Fukuhara et al.

In order to quantify TEGOTAE, we describe a function T as

T (x,S) = f(x)g(S), (6)

where f(x) is expectation of a controlled parameter x and g(S) is a functionof a sensory information vector S measuring the actual reaction from the envi-ronment, respectively. T is described as a product of the expectation f and thereaction g. There are various kinds of feedback to be considered for TEGOTAEdepending on the kind of animal locomotion [8]. For the sake of simplicity, wedefine TEGOTAE with respect to propulsion and body-support. In addition,we regard that the rotational actuator is responsible for propulsion while theprismatic actuator is responsible for body-support and stability of locomotion.Based on this assumption, TEGOTAE for each actuator is described as follows:

TRoti = τRot

i Nnormali , (7)

TPrii = τPri

i Naxiali , (8)

where TRoti and TPri

i are TEGOTAE functions for rotational and prismaticactuators, respectively, and Nnormal

i and Naxiali are normal and axial com-

ponents of GRF, respectively, as shown in Fig. 1. Although, according to Eq.(6), the expectation part is a function of x, the control inputs for actuators(τRot and τPri) replace f(x) in order to simplify TEGOTAE. Based on theseTEGOTAE functions, the stiffness of rotational and prismatic actuators areadjusted as follows:

KRoti = KRot

def + σRotTRoti , (9)

KPrii = KPri

def − σPriTPrii , (10)

0 0.1 0.2 0.3 0.40

100

200

300

2

3

4

5

[

1/N

m]

[1/Nrad]

Dis

tanc

e [m

]

0 0.1 0.2 0.3 0.40

100

200

300

15

20

25

30

0 0.1 0.2 0.3 0.40

100

200

300

30

40

50

(a) = 5 [rad/s] (b) = 10 [rad/s]

[

1/N

m]

[1/Nrad]

Dis

tanc

e [m

]

(c) = 15 [rad/s]

[

1/N

m]

[1/Nrad]

Dis

tanc

e [m

]

Fig. 2. Plots of distance traveled against each ω. (Color figure online)

Leg Stiffness Control Based on “TEGOTAE” for Quadruped Locomotion 83

where KRotdef and KPri

def are positive constants for default value of each joint,and σRot and σPri are the weighting factors for each joint. When TRot

i > 0,KRot

i increases so as to increase the leg stiffness in rotational motion. Withincreasing stiffness of rotational joints, the rotational actuator tends to obtainmore TEGOTAE from environment. However, when TPri

i > 0, KPri decreasedand the prismatic movement became softer; this was attributable to increasedshock-absorption.

3 Preliminary Results

In the 2D simulation environment, we implemented the proposed TEGOTAE-based control into a quadruped robot shown in Fig. 1. Only ω was increased from5 to 10, and then from 10 to 15. The robot moved for 20 s for each ω.

Figure 2 shows the distance traveled against each value of ω. According toFig. 2(a), the effects of the TEGOTAE-based control are not observed withrespect to distance traveled during locomotion with low ω. Whereas for medium

Fig. 3. Measured foot trajectories during the simulation. (Color figure online)

84 A. Fukuhara et al.

and high value of ω (Fig. 2(b) and (c)), the distance traveled were increasedwhere both TRot

i and TPrii affect.

Figure 3 shows the trajectory of mfooti during the simulation. In Fig. 3(a)

and (c), foot trajectory of fore leg is plotted so as to make mchesti the origin,

while the trajectory of the hind leg is plotted so as to make mhipi the origin

in Fig. 3(b) and (d). Without TEGOTAE, the contact point with the groundchanges randomly for high ω. On the other hand, with TEGOTAE, the contactpoint of each stride becomes closer. In addition, the bottom part of trajectorieswith TEGOTAE converged to a flat line because of contact with the ground,especially when ω = 15.

4 Conclusion and Future Work

In this study, we focused on the adjustment of leg stiffness to investigate flex-ible foot trajectories in response to locomotion speed, and also presented aTEGOTAE-based control scheme. The proposed scheme changes the foot trajec-tory at high ω; in addition, the distance traveled also increased, correspondingly.For the sake of simplicity, we consider a simple structure of the leg and sensoryinformation, Naxial

i and Nnormali . Richer sensory information and more complex

leg-structure will be considered in future work.

References

1. Catavitello, G., Ivanenko, Y.P., Lacqaniti, F.: Planar covariation of hindlimb andforelimb elevation angles during terrestrial and qquatic locomotion of dogs. PLoSONE 10(7), e0133936 (2015)

2. Kimura, H., Sakurama, K., Akiyama, S.: Dynamic walking and running of thequadruped using neural oscillator. Auton. Robots 7, 247–258 (1999)

3. Aoi, S., Katayama, D., Fujiki, S., Tomita, N., Funato, T., Yamashita, T., Senda, K.,Tsuchiya, K.: A stability-based mechanism for hysteresis in the walk-trot transitionin quadruped locomotion. J. R. Soc. Interface (2012). doi:10.1098/rsif.2012.0908

4. Shen, L., Poppele, R.E.: Kinematic analysis of cat hindlimb stepping. J. Neurosci.74(6), 2266–2280 (1997)

5. Maes, L.D., Herbin, M., Hackert, R., Bels, V.L., Abourachid, A.: Steady locomotionin dogs: temporal and associated spatial coordination patterns and the effect ofspeed. J. Exp. Biol. 211, 138–149 (2007)

6. Zhou, X., Bi, S.: A survey of bio-inspired compliant legged robot designs. Bioinspir.Biomim. 7, 041001 (2012)

7. Owaki, D., Morikawa, L., Ishiguro, A.: Listen to body’s message: quadruped robotthat fully exploits physical interaction between legs. In: 2012 IEEE/RSJ Interna-tional Conference on Intelligent Robots and Systems, pp. 1950–1955. IEEE Press,New York (2012)

8. Kano, T., Chiba, H., Umedachi, T., Ishiguro, A.: Decentralized control of 1D crawl-ing locomotion by exploiting “TEGOTAE” from environment. In: The First Inter-national Symposium on Swarm Behavior and Bio-Inspired Robotics, pp. 279–282(2015)

Wall Following in a Semi-closed-loopFly-Robotic Interface

Jiaqi V. Huang(&), Yilin Wang, and Holger G. Krapp

Department of Bioengineering, Imperial College London,London SW7 2AZ, UK

j.huang09@imperial.ac.uk

Abstract. To assess the responses of an identified optic-flow processinginterneuron in the fly motion-vision pathway, the H1-cell, we performedsemi-closed-loop experiments using a bio-hybrid two-wheeled robotic platform.We implemented a feedback-control architecture that established ‘wall follow-ing’ behaviour of the robot based on the H1-cell’s spike rate. The analysis ofneuronal data suggests the spiking activity of the cell depends on both themomentary turning radius of the robot as well as the distance of the fly’s eyesfrom the walls of the experimental arena. A phenomenological model that takesinto account the robot’s turning radius predicts spike rates that are in agreementwith our experimental data. Consequently, measuring the turning radius usingon-board sensors will enable us to extract distance information from H1-cellsignals to further improve collision avoidance performance of our fly-roboticinterface.

Keywords: Motion vision � Brain machine interface � Blowfly � Collisionavoidance

1 Introduction

Avoiding collisions with obstacles or other moving objects is a fundamental require-ment for any robotic platform moving in terrestrial, aerial, or aquatic environments.The same is true for most animals. Except for animals such as bats and whales, whichavoid collisions based on self-generated acoustic signals, most species - especiallyflying insects, rely on visual cues [1, 2].

Several insects respond with escape behaviour and/or adjusting steering manoeu-vers when facing retinal image expansion (looming stimuli) as demonstrated forinstance in locusts [3] and hawk moths [4]. Based on models of the underlying neu-ronal circuits, bio-inspired control strategies have been adopted in engineering tosupport collision avoidance capabilities in technical applications [5, 6].

The blowfly, as one of the most manoeuvrable flying insects, is a well-establishedmodel system for studying visual guidance at the behavioural and neuronal level [7], withthe potential to inspire novel technical solutions to collision avoidance in robotic systems.We have chosen to work on one of the blowfly’s individually identified visualinterneurons, the H1-cell, and explore its responses to directional motion as a collisionavoidance sensor on a bio-hybrid robotic system under open- and closed-loop conditions.

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 85–96, 2016.DOI: 10.1007/978-3-319-42417-0_9

The blowfly H1-cell has been studied for more than thirty years [8, 9]. It is excitedby horizontal back-to-front motion and inhibited by motion in the opposite direction. Inthe absence of visual motion, the cell produces a spontaneous spike rate in the range of25–50 Hz. Most previous electrophysiological studies have characterized H1-cellresponses during rotational pattern motion [10–12], while translational motion has beenless frequently applied [13].

Our research is focusing on both rotational and translational movement. Initially,we built and verified a mobile recording platform [14], then investigated a controlstrategy based on the H1-cell’s temporal frequency tuning in lab environment [15], andimplemented an autonomous fly-robot interface to test a temporal frequency tuningbased collision avoidance control algorithm [16]. Here we study the distance-dependentresponse properties of the H1-cell, which will be exploited for collision avoidance.

For collision avoidance, distance estimation is crucial. It provides information onwhen to turn away from the potential obstacles and in which direction. In theory, opticflow induced by pure rotation does not contain any relative distance information.Distance information requires translational self-motion [17]. Blowflies generate sac-cadic rotations followed by translational drift phases during which the activity of twofurther directional selective interneurons, the HSE- and HSN-cell, encodes relativedistance. This information is then used to generate the next saccade away from apotential obstacle [18]. The preferred directions of the HSE/N cells are opposite tothose in the H1-cell, which enables them to respond during forward and slightly lateraltranslations, but their activity cannot be recorded using extracellular recordingtechniques.

By combining both rotational and translational movements, we may be able tostimulate the H1-cell and potentially obtain distance information from its extracellu-larly recorded responses [15]. Any trajectories may be described in terms of a circulartrajectory with a turning radius ranging from zero to infinity. A zero turning radius isequivalent to a pure rotational movement, while an infinite turning radius would beequivalent to straight translational movement – any turning radius in between zero andinfinity results in a circular trajectory.

To test whether the H1-cell responses contain distance information we charac-terised the H1-cell spike rate in an open-loop experiment with different turning radii.We implemented a semi-closed-loop bang-bang control algorithm on ourfly-robot-interface with a turning radius, which was chosen based on earlier open-loopexperiments. The algorithm alternatingly turns the robot towards and away from thewall to keep a certain average distance, and to avoid collision with the wall. Thissemi-closed-loop algorithm requires the control of three parameters: (i) the duration ofthe turn towards the wall, (ii) the duration of the turn away from the wall, (iii) the speedof the wheels, which are related to turning radius. The turns in opposite directions areperformed at the same turning radius, but their execution is triggered either by a timeroverflow interrupt in microprocessor under open-loop control, or by neural signalanalysis, which counts the number of action potentials under closed-loop control. Bothopen-loop and closed-loop controls are outlined in the following sections.

86 J.V. Huang et al.

2 Methods

2.1 Configuration of Experimental System

The fly-robot-interface includes three parts: (i) a blowfly, (ii) a mobile extracellularrecording platform; right-hand side single unit recording only, (iii) a two-wheeled robot(Pololu© m3pi). These three parts were assembled as shown in Fig. 1.

H1-cell spiking activity was recorded using a tungsten electrode and was fed intothe customised preamplifier with a gain of 10000. The amplified signals were fed intothe robot controller for closed-loop control and a PC for off-line data analysis.

The neural data for closed-loop control were sent to the ADC port on the robotcontroller (ARM microprocessor NXP© LPC1768). The neural action potentials weresorted after digitisation and used to calculate the spike rate over a time interval of50 ms as input to the robotic control algorithm.

A copy of the data was recorded to a computer hard disk by a data acquisition card(NI USB-6215, National Instruments Corporation, Austin, TX, USA) at a rate of20 kHz.

Fig. 1. The fly robot bio-hybrid system. The mobile extracellular recording platform with asingle electrode is mounted on top of an m3pi robot. A blowfly is positioned in the centre of thechassis platform. H1-cell action potentials are pre-amplified with a customised amplifier enclosedby an aluminium case for electrical shielding.

Wall Following in a Semi-closed-loop Fly-Robotic Interface 87

2.2 Blowfly Preparation

The preparation of blowfly is the same as in previous series of experiments [14–16],which were performed on the same platform. Female blowflies, Calliphora vicina, aged4–11 days, were chosen for the experiments. Wings were immobilised by adding asmall droplet of bee wax on the wing hinges. Legs and proboscis were removed toreduce any movements that could degrade the quality of the recordings. All woundswere sealed with bee wax. The head of the fly was attached to a custom-made fly holderusing bee wax after adjusting its orientation according to the ‘pseudopupil methods’[19]. The thorax was bent down and attached to the holder to expose the back of thehead. The cuticle was cut and removed along the edges at the back of the head capsuleunder optical magnification using a stereo microscope (Stemi 2000, Zeiss©). Aftercarefully removing fat and muscle tissue with fine tweezers, physiological Ringersolution (for recipe see Karmeier et al. [20]) was added to the brain to preventdesiccation.

The dissected fly was then moved to the recording platform. Tungsten electrodes(3 MΩ tungsten electrode, product code: UEWSHGSE3N1M, FHC Inc., Bowdoin,ME, USA) were mounted on a micromanipulator (MM-1-CR-XYZ, National Aperture,Inc., Salem, NH, USA) and placed in close proximity to the H1-cell. Acceptablerecording quality required a signal-noise-ratio > 2:1, i.e. the peak amplitude of therecorded H1-cell spikes was at least twice as high as the largest amplitude of thebackground noise.

2.3 Experimental Procedure

Experiments consisted of two parts: (i) the neural characterisation under open-loopconditions and (ii) the control algorithm verification under closed-loop conditions.

For the open-loop part of the experiment, the blowfly was mounted on the robot,where the H1-cell responses were recorded while the robot was moving along thepre-programmed trajectory inside a corridor with vertically striped patterns attached tothe walls (see Fig. 4). The pre-programmed trajectory consisted of a series of alter-nating semi-circular turns with increasing turning radius. The sequence of roboticinstructions was programmed as: robot initialisation (left/right 90° spin/return forinfrared sensor initialisation); stop for 2 s; starting semi-circular turns with turning radiiof 5, 10, 15, 20, 25, and 30 cm.

For the closed-loop experiments, the robot was programmed with a combinedopen-loop and closed-loop control algorithm, as only the left H1-cell was used as asensor for detecting the distance to the wall. The control architecture is shown as a flowchart in Fig. 2.

The purpose of the experiment was to test if the robot would adjust its averagedistance towards the wall while moving forward, and keep the distance without col-lision until reaching the end of the experimental arena.

88 J.V. Huang et al.

3 Results

3.1 Open-Loop Neural Recording

The neural activities of left H1-cell were recorded by the NI-DAQ for 10 s and the datawere stored on a computer hard disk.

The off-line data analysis included: spike sorting, ISI (inter-spike interval-based)spike rate calculation, and Kaiser filtering (window length = 10001, alpha = 128) ofthe time-continuous spike rate. Because of the high signal-to-noise-ratio, whichwas >> 2:1, the spikes could be sorted by a single threshold algorithm. The spike

Fig. 2. Control algorithm flow chart of the bio-hybrid robotic firmware for wall following.(ADC: Analog to Digital Converter, ISR: Interrupt Service Routine)

Wall Following in a Semi-closed-loop Fly-Robotic Interface 89

occurrence was recorded whenever the potential exceeded the threshold voltage(magenta band in Fig. 3, top trace).

From the plot, we can see the H1-cell responses when the robot starts to performalternating turns from around t = 4 s (Fig. 3). The spike rate of the H1-cell decreasedwith increasing turning radius, which is in agreement with the result from a model thatpredicts the cell responses as a function of turning radius and wall distance (see nextsection).

3.2 Verification of Closed-Loop Control Algorithm

We recorded a video from a ceiling-mounted camera, which showed that the robot wasable to adjust the distance to the left wall, and avoid collision until the end of thecorridor (Fig. 4).

Fig. 3. Neural recording during an open-loop experiment. The top trace shows the raw H1-cellextracellular recording with its threshold (shown in magenta). The bottom subplot gives the spikerate calculated from the data presented in the top trace (blue curve is the ISI spike rate and redcurve is the ISI rate filtered by Kaiser filtering (window length = 10001, alpha = 128). The robotinitialization generates two activity peaks at the beginning. After 2 s of a stationary period, therobot starts to turn with different radii, which were 5, 10, 15, 20, 25, and 30 cm, shown in thegrey area. Note that the average spike rate decreases with increasing turning radius. (Color figureonline)

90 J.V. Huang et al.

The video can be found at https://www.youtube.com/watch?v=IEgyMUs3_QU.

Fig. 4. The robot trajectory. The experimental arena was a tunnel with vertical stripes (gratings)attached to the walls (spatial wavelength: 30 mm or 16.2° from initial point). The green lines arethe orientations (posterior) of the robot; the red dots are the positions of the robot along itstrajectory recorded every 0.18 s. (Color figure online)

Fig. 5. The data from robot video and H1 neural recording. (1st trace) Robot orientationcalculated from each frame of the video recording. The red curve is the robot orientation. Theblue curve is the filtered robot orientation. (2nd trace) Robot angular velocity after filtering. (3rd

trace) The distance between robot and wall calculated from each video frame. The blue curveshows the distance after filtering. (4th trace) H1-cell spike rate after smoothing with a filter kernel(Kaiser, window length = 10001, alpha = 128). (5th trace) The raw H1-cell extracellularrecording data, with magenta band which indicates the threshold voltage for spike detection.(Color figure online)

Wall Following in a Semi-closed-loop Fly-Robotic Interface 91

The H1-cell activities were sampled using a data acquisition card. They are plottedtogether with the robot’s orientation, angular velocity, and distance to the wall,extracted from the video recording. Neuronal signals and movements of the robot werealigned in time by a synchronisation signal (Fig. 5).

This control algorithm was successful in 5/12 trials under the bang-bang control,i.e. the robot reaching the end of the experimental corridor without colliding with thewall. Some of the trials were showing distance adjustment, but the robot touched thewall before it reached the end of the corridor.

4 Discussion

4.1 Wall Following as a First Approximation of Collision Avoidance

The control algorithm shown in Fig. 2 enabled the robot in several experiments tosuccessfully follow the wall during oscillatory forward movement at an average dis-tance of about 20 cm.

Three essential parameters were crucial for the wall-following algorithm to work.Firstly, a turning radius had to be chosen so that the H1-cell response just exceededabove 100 spikes/second. This response level was sufficiently different from theH1-cell’s spontaneous activities and could be increased or decreased depending onchanges of the robot trajectory. Secondly, the time interval during which the robotturned towards the wall was pre-programmed. The time interval of the microprocessortimer overflow interrupt was chosen based on the time of robot turning a quarter of acircle with selected turning radius at full speed. Thirdly, the length of the time intervalof turning away from the wall was dependent on the H1-cell generating a pre-definednumber of action potentials. As expected, higher spike rates during the turn resulted ina shorter time interval and vice versa, thus establishing negative feedback.

The success rate can be improved in the future by optimising both hardware andsoftware configurations, regarding the signal fidelity and control efficiency.

To advance the hardware, a faster ADC may be used to replace the current on-boardADC, which has a comparatively low sampling rate of 5 kHz. Undersampling of theneuronal signals during digitisation is likely to reduce the detection of action potentialpeak amplitudes, which affects the robustness of spike sorting using a simple thresholdalgorithm. This could easily be improved by increasing the sampling rate to 20 kHz.Furthermore, spike sorting was performed on the same microprocessor used forclosed-loop control. A separate spike sorting microprocessor with adaptive thresholdalgorithm would further increase speed and accuracy of the process.

In the software implementation of an improved control algorithm, more charac-terisations would be helpful to make the control algorithm more efficient. For example,multiple thresholds can be used to replace single threshold in bang-bang control, whichbrings finer adjustment during driving.

92 J.V. Huang et al.

4.2 H1-cell Wall Distance Dependence and Its Model

The H1-cell does not directly encode any distance information. However, considering athird parameter, i.e. the turning radius, it is possible to estimate the distance of obstacleindirectly, based on H1-cell spike rate. We developed a phenomenological model tostudy the relationship between turning radius, distance and H1-cell spike.

Similar to retrograde and prograde movement of planets in Ptolemy’s geocentricmodel of solar system, during self-motion the direction of optic flow generated by anoptical system of an observer depends on the position of visual structures relative to thecentre of the turning radius. If described in egocentric coordinates, visual structurespositioned within a sphere that connects the observer with the centre of the turning radiusmove progressively (front-to-back; blue sphere in Fig. 6 left), while structures outside thesphere move regressively (back-to-front; red area in Fig. 6 left). We may transfer Ptol-emy’s model to the directional optic flow the H1-cell is confronted with while the flyengages on a trajectory that consists of a combination of translational and rotationalself-motion described by a given turning radius, Rt. Relative motion of structures withinthe blue sphere (Fig. 6 left) would be in the anti-preferred direction of the cell while themotion of structures within the red area (Fig. 6 left) would be in the cell’s preferreddirection. Note that the receptive field of the H1-cell, i.e. the area within which motionaffect the activity of the cell, comprises an entire visual hemisphere. In the followingsection, wewill refer to the blue sphere in Fig. 6 (left) as the “inhibitory zone”, and the redarea as the “excitatory zone”. The spike rate of H1-cell depends on the ratio between theinhibitory and excitatory zones and therefore on the turning radius of the robot.

The underlying principle becomes clear when considering the two extreme cases: Aturning radius of zero is equivalent to the fly rotating on the spot, in which case the blue

Fig. 6. (Left) Directional map of optic flow during turns of the robot at a finite turning radius,Rt. The red area indicates back-to-front optic flow, while the blue sphere indicates front-to-backoptic flow. A cross indicates the turning centre. For further explanation, see text. (Right) Opticflow sensitivity field of H1-cell based on relative motion. The yellow arrow indicates theorientation of the robot. The yellow line describes the trajectory of the robot. The red zone is theexcitatory region; the blue zone is the inhibitory region. The black zone is the region where thereis no optic flow caused by relative rotation. The gradients indicate the scalar of the optic flowprojected towards the observer. (Color figure online)

Wall Following in a Semi-closed-loop Fly-Robotic Interface 93

sphere completely vanishes and the cell is confronted only with motion in its preferreddirection (for clockwise rotations – and only with motion in its anti-preferred directionfor counter-clockwise rotations). An infinite turning radius, on the other hand, isequivalent to the robot moving on a straight line in the forward direction in which casethe red area will vanish and the cell experiences only motion in its anti-preferreddirection. This normally results in a perfect inhibition of H1-cell spiking. From thisconsideration, it follows that H1-cell response only contains information about distancewhen the robot is travelling on a finite turning radius.

After analysis of the open-loop experiment data, the H1-cell responses could beplotted as a function of distance to the visual surroundings (Fig. 7, red curve). However,when a wall partially reaches the inhibitory zone, H1-cell experiences both excitatoryand inhibitory optic flow. Further explanation of signal integration at the level of opticflow processing interneurons can be found in publications from the Borst lab [21].

We developed a simple phenomenological model to estimate the H1-cell responsesduring robot movements along a given turning radius based on the directional motionof the surroundings as Eq. 1.

S x; yð Þ ¼ sinp2� cos�1 x� xoð Þ2 þ y� yoð Þ2 þ x� xf

� �2 þ y� yf� �2� xo � xf

� �2� xo � yf� �2

2ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffix� xoð Þ2 þ y� yoð Þ2

q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffix� xf� �2 þ y� yf

� �2q

0

B@

1

CA

�������

�������

0

B@

1

CA

ð1Þ

Where Sðx; yÞ is the sensitivity to an object at a given location during relative rotation;xo; yoð Þ is the turning centre; xf ; yf

� �is the blowfly location.

An optic flow sensitivity map is plotted in Fig. 6 (right).

Fig. 7. The sensitivity to wall based optic flow during turns with various turning radius. The redcurve shows experimental data obtained from H1-cell recordings and the blue curve gives thenormalized output from the model simulation. In the experiments, the robot distance to the wallwas 20 cm and spatial wavelength of the vertical bar pattern on the wall was 30 mm (as seen bythe lateral eye of the fly). (N = 1, n = 2) (Color figure online)

94 J.V. Huang et al.

In addition, a series of points along a line was specified in the model to simulate thewall in the experimental arena with a pattern of a certain spatial wavelength. Thesimulated wall was rotated from an azimuth angle of 135° to 45° (according to blowflyanterior direction) around a vertical axis that coincides with the ventral-dorsal axis ofthe fly, simulating rotation of a blowfly on a certain turning radius. When integratingthe local motion (optic flow), we took into account the spatial resolution on theinter-ommatidial angle of the H1-cell receptive field [22].

The simulation result (Fig. 7) shows that the estimated sensitivity of the cell to awall in relative motion was modulated by the turning radius; the smaller the turningradius, the higher the predicted H1-cell responses. The result (Fig. 7, blue trace) is ingood agreement with neural recording data obtained under open-loop conditions(Fig. 7, red trace).

5 Summary

We were able to demonstrate that spike rate of the H1-cell provides spatial informationabout the visual surroundings under specific circumstances. At a fixed distance to thewall, the spike rates are modulated by the turning radius of the robot.

The current model estimates the spike rate as a function of two parameters: theturning radius and the distance to the wall. In the future, by fixing the turning radius inthe model, the spike rate could be used to estimate the distance to the wall –informationthe robot needs to avoid a collision in a more generic environment.

In future experiments, we will analyse the integration of simultaneous excitatory andinhibitory local optic flow more systematically to derive ideal closed-loop controlarchitectures for a collision-free operation of our fly-robot-interface. Further improve-ments may be achieved by including a forward model to calculate turning radius, inwhich the parameters such as the wheel speeds and the distance between wheels, can belocally measured, to predict the wall distance for collision avoidance control.

Acknowledgments. The authors would like to thank Caroline Golden for improving theproofreading the manuscript. This work was partially supported by US AFOSR/EOARD grantFA8655-09-1-3083 to HGK.

References

1. Fox, J.L., Frye, M.: Animal behavior: fly flight moves forward. Curr. Biol. 23, R278–R279(2013)

2. Reiser, M.B., Dickinson, M.H.: Visual motion speed determines a behavioral switch fromforward flight to expansion avoidance in Drosophila. J. Exp. Biol. 216, 719–732 (2013)

3. Krapp, H.G., Gabbiani, F.: Spatial distribution of inputs and local receptive field propertiesof a wide-field, looming sensitive neuron. J. Neurophysiol. 93, 2240–2253 (2005)

4. Wicklein, M., Strausfeld, N.J.: Organization and significance of neurons that detect changeof visual depth in the hawk moth Manduca sexta. J. Comp. Neurol. 424, 356–376 (2000)

Wall Following in a Semi-closed-loop Fly-Robotic Interface 95

5. Blanchard, M., Rind, F.C., Verschure, P.F.M.J.: Collision avoidance using a model of thelocust LGMD neuron. Robot. Auton. Syst. 30, 17–38 (2000)

6. Bertrand, O.J.N., Lindemann, J.P., Egelhaaf, M.: A bio-inspired collision avoidance modelbased on spatial information derived from motion detectors leads to common routes. PLoSComput. Biol. 11, e1004339 (2015)

7. Kern, R., Boeddeker, N., Dittmar, L., Egelhaaf, M.: Blowfly flight characteristics are shapedby environmental features and controlled by optic flow information. J. Exp. Biol. 215, 2501–2514 (2012)

8. Hausen, K.: Functional characterization and anatomical identification of motion sensitiveneurons in the lobula plate of the blowfly Calliphora erythrocephala. Z. Naturforsch. 31c,629–633 (1976)

9. Krapp, H.G., Hengstenberg, R., Egelhaaf, M.: Binocular contributions to optic flowprocessing in the fly visual system. J. Neurophysiol. 85, 724–734 (2001)

10. Longden, K.D., Krapp, H.G.: Octopaminergic modulation of temporal frequency coding inan identified optic flow-processing interneuron. Front. Syst. Neurosci. 4, 153 (2010)

11. Maddess, T., Laughlin, S.B.: Adaptation of the motion-sensitive neuron H1 is generatedlocally and governed by contrast frequency. Proc. R. Soc. Lond. B Biol. Sci. 225, 251–275(1985)

12. Lewen, G.D., Bialek, W., de Ruyter van Steveninck, R.R.: Neural coding of naturalisticmotion stimuli. Netw. Bristol Engl. 12, 317–329 (2001)

13. Lindemann, J.P., Egelhaaf, M.: Texture dependence of motion sensing and free flightbehavior in blowflies. Front. Behav. Neurosci. 6, 92 (2013)

14. Huang, J.V., Krapp, H.G.: Miniaturized electrophysiology platform for fly-robot interface tostudy multisensory integration. In: Lepora, N.F., Mura, A., Krapp, H.G., Verschure, P.F.M.J.,Prescott, T.J. (eds.) Living Machines 2013. LNCS, vol. 8064, pp. 119–130. Springer,Heidelberg (2013)

15. Huang, J.V.,Krapp,H.G.: A predictivemodel for closed-loop collision avoidance in afly-roboticinterface. In: Duff, A., Lepora, N.F., Mura, A., Prescott, T.J., Verschure, P.F.M.J. (eds.) LivingMachines 2014. LNCS, vol. 8608, pp. 130–141. Springer, Heidelberg (2014)

16. Huang, J.V., Krapp, H.G.: Closed-loop control in an autonomous bio-hybrid robot systembased on binocular neuronal input. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A., Prescott,T.J. (eds.) Living Machines 2015. LNCS, vol. 9222, pp. 164–174. Springer, Heidelberg(2015)

17. Koenderink, J.J., van Doorn, A.J.: Facts on optic flow. Biol. Cybern. 56, 247–254 (1987)18. Lindemann, J.P., Kern, R., van Hateren, J.H., Ritter, H., Egelhaaf, M.: On the computations

analyzing natural optic flow: quantitative model analysis of the blowfly motion visionpathway. J. Neurosci. 25, 6435–6448 (2005)

19. Franceschini, N.: Pupil and pseudopupil in the compound eye of Drosophila. In: Wehner, R.(ed.) Information Processing in the Visual Systems of Anthropods, pp. 75–82. Springer,Heidelberg (1972)

20. Karmeier, K., Tabor, R., Egelhaaf, M., Krapp, H.G.: Early visual experience and thereceptive-field organization of optic flow processing interneurons in the fly motion pathway.Vis. Neurosci. 18, 1–8 (2001)

21. Mauss, A.S., Pankova, K., Arenz, A., Nern, A., Rubin, G.M., Borst, A.: Neural circuit tointegrate opposing motions in the visual field. Cell 162, 351–362 (2015)

22. Petrowitz, R., Dahmen, H., Egelhaaf, M., Krapp, H.G.: Arrangement of optical axes andspatial resolution in the compound eye of the female blowfly Calliphora. J. Comp. Physiol.186, 737–746 (2000)

96 J.V. Huang et al.

Sensing Contact Constraints in a Worm-likeRobot by Detecting Load Anomalies

Akhil Kandhari1(&), Andrew D. Horchler1, George S. Zucker1,Kathryn A. Daltorio1, Hillel J. Chiel2, and Roger D. Quinn1

1 Department of Mechanical and Aerospace Engineering,Case Western Reserve University, Cleveland, OH 44106-7222, USA

{axk751,rdq}@case.edu2 Departments of Biology, Neurosciences, and Biomedical Engineering,Case Western Reserve University, Cleveland, OH 44106-7080, USA

hjc@case.edu

Abstract. In earthworms, traveling waves of body contraction and elongationresult in soft body locomotion. This simple strategy is called peristaltic loco-motion. To mimic this kind of locomotion, we developed a compliant modularworm-like robot. This robot uses a cable actuation system where the actuatingcable acts like the circumferential muscle. When actuated, this circumferentialcable contracts the segment diameter causing a similar effect to the contractiondue to the circumferential muscles in earthworms. When the cable length isincreased, the segment diameter increases due to restoring forces from structuralcompliance. When the robot comes in contact with an external constraint (e.g.,inner walls of a pipe) continued cable extension results in both slack in the cableand inefficiency of locomotion. In this paper we discuss a probabilistic approachto detect slack in a cable. Using sample distributions over multiple trials andnaïve Bayes classifier, we can detect anomalies in sampled data which indicatethe presence of slack in the cable. Our training set included data samples frompipes of different diameters and flat surfaces. This algorithm detected slackwithin ±0.15 ms of slack being introduced in the cable with a success rate of75 %. We further our research in understanding reasons for failure of thealgorithm and working towards improvements on our robot.

Keywords: Robot � Worm � Soft � Probability � Naïve Bayes � Cable �Actuation

1 Introduction

During the crawling movements of the earthworms, sensory feedback provides theanimal with an ability to adapt to different types of environmental perturbations thatmay occur [16]. The importance of sensory feedback for maintaining rhythmiccrawling motion has been established [7]. Mechanosensory organs and stretch, touch,and pressure receptors are the feedback sources in earthworms [15]. Due to the flex-ibility in the earthworm’s body, it is unable to sense its posture from only stretchreceptors [16]. However, the sensory input activities from the setae allows it to adapt toits environment and crawl smoothly on rough surfaces.

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 97–106, 2016.DOI: 10.1007/978-3-319-42417-0_10

Many robots use cable actuation methods, like the Softworm [1–3], exoskeleton-typerobots [19], cable-actuated parallel manipulators [8] and cable-suspended robots [18]. Inour cable actuated robot, slack occurs in the cable when the robot comes in contact with anexternal constraint. It is undesirable as it reduces the efficiency during peristaltic loco-motion. Detection of slack allows us to sense external constraints, which allows the robotto locomote efficiently. In this paper, our question is: can we recognize cable slack bysearching for anomalies in feedback measurements from the actuators?

Anomaly detection techniques traditionally detect irregularities in new data aftertraining on clean data [6]. Probabilistic approaches in predicting these anomalies havebeen studied widely [6, 13]. We use a probabilistic model to detect if slack has beenintroduced in our actuation cable to produce more efficient peristaltic locomotion [1] inour worm-like robot.

2 Slack Detection in a Cable-Actuated Robot

A significant issue faced when using cables is that they allow actuation in only onedirection, i.e., a cable cannot be pushed. Initially, our Compliant Modular Mesh Worm(CMMWorm) robot had a bi-directional actuation system [9]. There were two sets ofcables, one for circumferential actuation, (elongates the segment while making thediameter smaller) and one for longitudinal actuation (reduces segment length whileincreasing segment diameter). Due to nonlinear kinematics, the rate at which thelongitudinal cables are spooled in is not equal to the rate at which the circumferentialcables are spooled out. This difference in rates results in excess circumferential cablebeing spooled out. To prevent this excess cable from tangling or causing uneven wear,circumferential cable tensioning springs were required to keep the cable taut. Thus,cable slack was avoided, but as a result of the antagonistic cables, the design had highbody stiffness.

For CMMWorm, Dynamixel MX-64T actuators are used. These actuators use PID(proportional-integral-derivative) control. Load feedback measures the torque appliedby the actuator internally. We developed an open source library, DynamixelQ [12], forthe OpenCM9.04 microcontroller used by CMMWorm for communication with AXand MX series Dynamixel actuators. It has been shown that the ability for a soft robotto respond to sensed load of its surroundings can permit efficient navigation, e.g., innarrowing pipes [4]. In the previous bi-directionally actuated iteration of CMMWorm[9], we used load measurements from the actuators to detect the walls of the pipe. Loadmeasurements were sampled and then smoothed to filter noise using exponentialsmoothing. The data was then compared to a threshold value based on load datasampled. When a segment came in contact with the wall of a pipe the filtered loadincreased and exceeded the threshold. This allowed the algorithm to stop any furtherexpansion of a segment.

In the current version of the CMMWorm (Fig. 1), only circumferential cables areused for actuation. The longitudinal cables have been replaced by six springs along thelength of each segment such that, as the circumferential cable spools out, the springspassively return the segment to the maximum diameter allowed by the circumferentialcable. In this simpler design, the spring stiffness need only be sufficient to restore the

98 A. Kandhari et al.

segments to their initial maximum diameter. The design is more robust, however, withoutthe longitudinal cable, obstacle detection relies on detecting when the circumferentialcable goes slack, rather than detecting an increase in longitudinal cable tension.

By reconfiguring the robot to use only one cable (circumferential), we can improvethe mechanics of the motion (reducing cable friction losses, plastic deformation of themesh, and limitations on range of motion), but detecting contact becomes a challenge. Asthe circumferential cable is pulled onto the spool, the diameter of a segment is constricteddue to the length of cable available. In the other direction, when the cable is unspooled,the longitudinal springs expand the segment diameter as much as the increased cablelength allows. However, when a segment comes in contact with an external constraint,such as that from the interior wall of a pipe, the segments stop expanding in proportion tothe cable length that is unspooled and the cable can become slack. This causes reducedefficiency in responsive peristalsis that depends on accurate detection of contact [4].Thus, a new way to control the segments of the robot was required.

It was observed that when the cable did experience slack, there was a small spike inthe load values being recorded by the actuator. These load values are noisy and changebased on the PID responses to cable tension. The small peaks due to slack can bethought of as anomalies in the load data being obtained. Hence, we approached thisproblem using a statistical probabilistic model. The problem then becomes to find theprobability of slack occurring in the system given the load value that has been mea-sured. If sampled values are not within the measured distribution, there is a higherprobability that the segment has come in contact with an external force and the cablehas gone slack.

3 Methods

When working with the robot, the actuators are configured to use a position controlscheme. Position limits are specified and the actuator will rotate between these presetlimits for a specified number of rotations (three rotations allow each segment to move

Fig. 1. The six-segment Compliant Modular Mesh Worm (CMMWorm) robot crawling througha 20.3 cm inner diameter pipe. The linear springs that passively return the robot to its maximumdiameter are visible along the length of each segment.

Sensing Contact Constraints in a Worm-like Robot by Detecting LA 99

from maximum to minimum diameter). The actuators are run at their maximum speedand with no torque limits. Load measurements gathered from an actuator, althoughnoisy, tend to spike when slack is introduced in the cable. This is analogous to liftingan object using a cable; if the cable is cut, a short jerk is felt due to the sudden changein tension. If it is possible to detect this spike within the noisy data, then it is possible todetermine that the segment has come in contact against an external obstacle and shouldstop further expansion in order to avoid introducing slack.

To do so, a probabilistic approach is used to capture the occurrence of slack in eachsegment of CMMWorm by measuring the distribution of load values in the cases ofslack and no slack. Load data is gathered from the actuators by expanding each segmentin different diameter pipes and visually observing when the cable becomes slack. Slackis visually determined when the circumferential cable tensioning springs return to theirequilibrium state. We then ensured that the segment was anchored against the walls ofthe pipe. Measurement of load values were continued after slack was introduced in orderto collect data for the case of slack. The noisiness of the data and structural variability[10] leads to inconsistent training sets between segments. Due to nonlinear frictionalproperties, fraying of cables over time, and minor repairs being made over various runs,an individual segment also has variability in the load measurements recorded. Thismeans that the distributions of load values (given slack or no slack) will be different foreach segment and will change every time an adjustment is made to a segment.

To overcome this problem, we used amovingmedian filter for each actuator. Then thefiltered median of the previous ten load measurements is subtracted from the current loadmeasurement. This is referred to as the “relative median load” (RML). The absolute valueof this difference is used such that the relativemedian load is always positive (Fig. 2). Thishelps filter variability in the range of load values among different segments, as well aswithin a segment, yielding a constant range of data for the entire robot.

Time (sec)

RM

LL

oad

Time (sec)

A

B

Fig. 2. (A) Load measurements from the actuator during expansion of a segment in a 20.3 cminner diameter pipe. (B) The median filter measures the median of the previous ten loadmeasurements. The output from this filter is then subtracted from the current load measurement.The absolute value of this is the relative median load (RML). This helps normalize the load datafrom different segments and use a single probability distribution for the entire robot. The spike inthe above graphs at 0.95 s indicates that the cable has gone slack.

100 A. Kandhari et al.

Based on the RML values, we build our distributions (Fig. 3). Data was gatheredthrough ten runs for each of the six segments in 17.4 cm and 20.3 cm inner diameterpipes and on a flat surface. On flat surfaces the segments could expand to their max-imum diameter without obstruction. Load values from these data sets were divided intotwo parts: load given slack in the cable and load given no slack in the cable. From thesedistributions it is observed that higher load values are more prominent when there isslack in the cable as compared to when there is no slack in the cable. Using naïve Bayesclassification the possibility of slack occurring given a particular load value can bepredicted based on probabilities, P, obtained from the training data.

The probability of a load measurement Vt at time t given slack in the cable, S is

P Vt j Sð Þ ¼ P Vtð ÞP S jVtð ÞP Sð Þ ! P S jVtð Þ ¼ P Sð ÞP Vt j Sð Þ

P Vtð Þ ð1Þ

Similarly, the probability of a load measurement Vt at time t given no slack in thecable, �S is

P Vt j S� � ¼ P Vtð ÞP S jVt

� �

P S� � ! P S jVt

� � ¼ P S� �

P Vt j S� �

P Vtð Þ ð2Þ

Dividing (1) by (2), the ratio of likelihood of slack versus no slack given a par-ticular load is

P S jVtð ÞP S jVt� � ¼ P Sð ÞP Vt j Sð Þ

P S� �

P Vt j S� � ð3Þ

To prevent the algorithm from identifying noise as slack, the previous five loadmeasurements are used to find the product of probabilities of these measurements. Thisproduct is a ratio of probability of slack to no slack given a load value and is called theslack confidence (SC), where

SC ¼Y4

i¼0

P Sð ÞP Vt�i j Sð ÞP S� �

P Vt�ij S� � ð4Þ

This helps not classify a noisy measurement for slack. Since the SC can be a largevalue, its natural logarithm, ln(SC), is used for plotting purposes throughout this paper.Through testing it was found that it took at least five probability values to localize apeak caused by slack in the cable.

A peristaltic controller needs to coordinate the simultaneous expansion and con-traction of segments in order to minimize slip. At any given point, there are two activesegments, one contracting and the other expanding in diameter. In between the twoactive segments there is a contracted inactive segment referred to as the “spacer seg-ment” [10]. The remaining three inactive segments are expanded to the permissiblelimit and help in anchoring.

Sensing Contact Constraints in a Worm-like Robot by Detecting LA 101

Once the wave is initiated, load measurements are recorded from the expandingsegment after 150 ms to ignore transients that occur when the actuator starts from rest.After the actuator stabilizes, load measurements are recorded from the actuator of theexpanding segment at 200 Hz. The slack confidence (SC) is then calculated by themicrocontroller using Eq. (4). When slack occurs the SC has a large value compared towhen there is no slack in the cable. At this point a distinct peak is observed (Fig. 4) andexpansion of the segment is stopped and the control transitions to the next expandingsegment. The wave travels down the length of the robot repeating the algorithm in eachexpanding segment. For the contracting segments a preset minimum position allows itto return to its initial configuration.

4 Results

Twenty runs each were done in 15.2 cm, 17.4 cm, and 20.32 cm inner diameter pipesand on a flat surface. The results shown in Table 1 summarize the success rate of slackbeing detected when the segment came in contact with the wall of the pipe or, in the

Prob

abili

ty

Prob

abili

ty

Fig. 3. Distributions showing the probability of load measurements (A) and relative medianload measurements (RML) (B) given no cable slack (blue) and given cable slack (orange). Thedistributions show that the probability of obtaining higher load values decreases when there is noslack in the cable and the probability of obtaining higher load values increases when there isslack in the cable. The same trend is observed in the RML values. The RML distributions (B) areused to calculate slack confidence (SC) on the microcontroller by assigning probabilities to theload values obtained from the actuators. (Color figure online)

Table 1. N = 20 runs each were done in 15.2 cm, 17.4 cm, and 20.3 cm inner diameter pipesand on a flat surface. The percentage of successful runs, false positive errors, and false negativeerrors for the four different cases is shown.

Successful runs False positive errors False negative errors

15.2 cm Pipe 70 % 25 % 5 %17.4 cm Pipe 70 % 20 % 10 %20.3 cm Pipe 75 % 25 % 0 %Flat surface 85 % 15 % 0 %

102 A. Kandhari et al.

A

B

C

Time (sec)

Time (sec)

Load

ln(SC)

Time (sec)

Time (sec)

Load

Time (sec)

Time (sec)

Load

Observed Slack

ln(SC)

ln(SC)

15.2 cm pipe

20.3 cm pipe

Flat surface

Observed Slack

Observed Slack

Observed Slack

Fig. 4. Load measurements and natural logarithm of slack confidence, ln(SC), from the actuatorin three different cases, (A) 15.2 cm diameter pipe, (B) 20.32 cm diameter pipe, and (C) flatsurface. In (A) and (B) the dashed vertical line depicts when slack is visually observed on therobot. The slack confidence has a significant peak (ln(SC) > 10) close to when slack isintroduced in the cable in both cases. In (C), minor peaks are observed, but are not significantcompared to when slack is introduced in the cable due to the walls of the pipe.

Sensing Contact Constraints in a Worm-like Robot by Detecting LA 103

case of a flat surface, the absence of detection of slack throughout. Successful detectionof slack is defined as when ln(SC) > 10 is detected within ±0.15 s of coming incontact with the pipe (Fig. 4). Using this criterion, slack was detected successfully in75 % of the tests. In pipes, 3.75 % of the tests showed a false negative, i.e., nosignificant peak was observed. At this point the cable would unspool even when thesegment came in contact with an external barrier. The majority of the errors, 21.25 %of all the tests, were due to false positives, i.e., a significant peak occurred before thecable went slack. This would cause the robot to stop expansion before it came incontact with an external barrier. Thus a segment that should have been anchoring couldslip relative to the pipe wall.

A detailed comparison between the load measurements and slack confidence forpipes of different diameters and on a flat surface is shown in Fig. 4. There is a lead timebetween when slack confidence peaks and when slack is visually observed in thecables. The lead time is less than 0.1 s. This difference might be due to the complianceof the mesh and the difficulty of precisely detecting slack visually (Fig. 5).

False positive and false negative errors were observed in 25 % of runs. Falsepositive errors cause inefficiency as it might cause the segment to slip. These errorsmight occur because of fraying of cables, high frictional loads on the cable, and

Time (sec) (s)

Time (sec) (s)

Time (sec)

Load

RML

ln(SC)

Observed Slack

A

B

C

Fig. 5. Results from 15.2 cm diameter pipe indicating a false negative, i.e., the segmentcontinued to expand even on coming in contact with the walls of the pipe. (A) Load measurementfrom the actuator, where no significant peak is observed. (B) The relative median load(RML) shows no significant peak throughout the test to quantify a point where slack isintroduced. (C) The logarithm of SC shows a small spike when slack is introduced in the cablebut it is not large enough to indicate that slack has been introduced.

104 A. Kandhari et al.

damaged components. False negative errors also cause inefficiency in the locomotion,as the robot is not able to adapt to its environment. False negative errors also causeexcess cable being spooled out. This cable can get tangled with other components andmight damage or break itself. This error might be due to the high amount of noisecausing interference with the sampled data when slack occurs. False negative errorsmight also be an indication of plastic deformation of the mesh, where the loads are notuniformly transferred along the mesh of the segment.

5 Conclusions

The predictive model of detecting slack using probability distributions has allowed usto redesign our robot using a single cable for actuation. We can now detect slack indifferent diameter pipes along different segments using this algorithm. Using thesensory capabilities of our current actuators, we might be able to eliminate the need foradditional sensors on the robot. Our future work will include achieving higher accuracywithin a smaller time interval using a more dynamic method of predicting load mea-surements, such as Kalman filters [14]. There have also been studies on detectinganomalies using Dynamic Bayesian Networks [5, 13, 17]. We also want to apply asimilar feedback algorithm in conjunction with dynamic oscillator networks, such asstable heteroclinic channels [4, 11] that could help achieve a more adaptive gait pattern.

Currently our robot can react to static environments, by controlling the expansionof each segment on coming in contact with an external constraint, but what if theenvironment was to change suddenly? If the robot slipped while locomoting, would itbe able to adjust to such a change? In the future, the worm-like robot could continu-ously check for an external load and not just during expansion. In this way any changein the environment will trigger it to expand or contract in order to comply to itssurroundings.

Acknowledgements. This work was supported by NSF research grant No. IIS-1065489. Theauthors would like to thank Dr. Soumya Ray and Kenneth Moses for their help during the courseof this project.

References

1. Boxerbaum, A.S., Chiel, H.J., Quinn, R.D.: A new theory and methods for creatingperistaltic motion in a robotic platform. In: Proceedings of IEEE International Conference onRobotics and Automation, Anchorage, pp. 1221–1227 (2010)

2. Boxerbaum, A.S., Shaw, K.M., Chiel, H.J., Quinn, R.D.: Continuous wave peristalticmotion in a robot. Int. J. Robot. Res. 31, 302–318 (2012)

3. Boxerbaum, A.S., Horchler, A.D., Shaw, K.M., Chiel, H.J., Quinn, R.D.: A controller forcontinuous wave peristaltic locomotion. In: Proceedings of IEEE International Conferenceon Intelligent Robots and Systems, San Francisco, pp. 197–202 (2011)

Sensing Contact Constraints in a Worm-like Robot by Detecting LA 105

4. Daltorio, K.A., Boxerbaum, A.S., Horchler, A.D., Shaw, K.M., Chiel, H.J., Quinn, R.D.:Efficient worm-like locomotion: slip and control of soft-bodied peristaltic robots. Bioinspir.Biomim. 8, 035003 (2013)

5. Dean, T., Kanazawa, K.: A model for reasoning about persistence and causation. Comput.Intell. 5(3), 142–150 (1990)

6. Eskin, E.: Anomaly detection over noisy data using learned probability distributions. In:Seventeenth International Conference on Machine Learning, pp. 255–262 (2000)

7. Gray, J., Lissmann, J.W.: Locomotory reflexes in the earthworm. J. Exp. Biol. 15, 5006–5017 (1938)

8. Hassan, M., Khajepour, A.: Analysis of bounded cable tensions in cable-actuated parallelmanipulators. IEEE Trans. Robot. 27(5), 891–900 (2011)

9. Horchler, A.D., Kandhari, A., Daltorio, K.A., Moses, K.C., Andersen, K.B., Bunnelle, H.,Kershaw, J., Tavel, W.H., Bachmann, R.J., Chiel, H.J., Quinn, R.D.: Worm-like roboticlocomotion with a compliant modular mesh. In: Wilson, S.P., Verschure, P.F., Mura, A.,Prescott, T.J. (eds.) Living Machines 2015. LNCS, vol. 9222, pp. 26–37. Springer,Heidelberg (2015)

10. Horchler, A.D., Kandhari, A., Daltorio, K.A., Moses, K.C., Ryan, J.C., Stultz, K.A., Kanu,E.N., Andersen, K.B., Kershaw, J., Bachmann, R.J., Chiel, H.J., Quinn, R.D.: Peristalticlocomotion of a modular mesh-based worm robot: precision, compliance, and friction. SoftRobotics 2(4), 135–145 (2015)

11. Horchler, A.D., Daltorio, K.A., Chiel, H.J., Quinn, R.D.: Designing responsive patterngenerators: stable heteroclinic channel cycles for modeling and control. Bioinpir. Biomim.10(2), 026001 (2015)

12. Horchler, A.D.: DynamixelQ Library, version 1.2. https://github.com/horchler/DynamixelQ13. Liang, K., Cao, F., Bai, Z., Renfrew, M., Cavusoglu, M.C., Podgurski, A, Ray, S.: Detection

and prediction of adverse and anomalous events in medical robots. In: Proceedings ofTwenty-Fifth IAAI Conference, pp. 1539–1544 (2013)

14. Manfredi, V., Mahadevan, S., Kurose, J.: Switching Kalman filters for prediction andtracking in an adaptive meteorological sensing network. In: Proceedings of SecondAnnual IEEE Communications Society Conference on Sensor and AdHoc Communicationsand Networks (SECON), pp. 197–206 (2005)

15. Mill, P.J.: Recent developments in earthworm neurobiology. Comp. Biochem. Physiol. 12,107–115 (1982)

16. Mizutani, K., Shimoi, T., Ogawa, H., Kitamura, Y., Oka, K.: Modulation of motor patternsby sensory feedback during earthworm locomotion. Neurosci. Res. 48(4), 457–462 (2004)

17. Pavlovic, V., Rehg, J.M., Murphy, K.P.: A dynamic Bayesian network approach to figuretracking using learned dynamic models. In: Proceedings of the Seventh IEEE InternationalConference on Computer Vision, vol. 1, pp. 94–101 (1999)

18. Roberts, R., Graham, T., Lippitt, T.: On the inverse kinematics, statics, and fault tolerance ofcable-suspended robots. J. Robot. Syst. 15(10), 581–597 (1998)

19. Veneman, J.F.: A series elastic- and Bowden-cable-based actuation system for use as torqueactuator in exoskeleton-type robots. Int. J. Robot. Res. 25(3), 261–281 (2006)

106 A. Kandhari et al.

Head-Mounted Sensory Augmentation Device:Comparing Haptic and Audio Modality

Hamideh Kerdegari1(&), Yeongmi Kim2, and Tony J. Prescott1

1 Sheffield Robotics, University of Sheffield, Sheffield, UK{h.kerdegari,t.j.prescott}@sheffield.ac.uk

2 Department of Mechatronics, MCI, Innsbruck, Austriayeongmi.kim@mci.edu

Abstract. This paper investigates and compares the effectiveness of haptic andaudio modality for navigation in low visibility environment using a sensoryaugmentation device. A second generation head-mounted vibrotactile interfaceas a sensory augmentation prototype was developed to help users to navigate insuch environments. In our experiment, a subject navigates along a wall relyingon the haptic or audio feedbacks as navigation commands. Haptic/audio feed-back is presented to the subjects according to the information measured from thewalls to a set of 12 ultrasound sensors placed around a helmet and a classifi-cation algorithm by using multilayer perceptron neural network. Results showedthe haptic modality leads to significantly lower route deviation in navigationcompared to auditory feedback. Furthermore, the NASA TLX questionnaireshowed that subjects reported lower cognitive workload with haptic modalityalthough both modalities were able to navigate the users along the wall.

Keywords: Sensory augmentation � Haptic feedback � Audio feedback �Classification algorithm

1 Introduction

Sensory augmentation is an exciting domain in human-machine biohybridicity thatadds new synthesized information to an existing sensory channel. The additional sensesprovided by sensory augmentation can be used to augment the spatial awareness ofpeople with impaired vision [1–5] or for people operating in environments where visualsensing is compromised such as smoked-filled buildings [6–8].

The sensitive tactile sensing capabilities supported by facial whiskers provide manymammals with detailed information about local environment that is useful for navi-gation and object recognition. Similar information could be provided to humans using asensory augmentation device that combines active distance sensing of nearby surfaceswith a head-mounted tactile display [6, 7]. One of the attempts to design such a devicewas the ‘Haptic Radar’ [7] that linked infrared sensors to head-mounted vibrotactiledisplays allowing users to perceive and respond simultaneously to multiple spatialinformation sources. In this device, several sense-act modules were mounted togetheron a band wrapped around the head. Each module measured distance from the user tonearby surfaces, in the direction of the sensor, and transduced this information into a

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 107–118, 2016.DOI: 10.1007/978-3-319-42417-0_11

vibrotactile signal presented to the skin directly beneath the module. Users intuitivelyresponded to nearby objects, for example, by tilting away from the direction of anobject that was moving close to the head, indicating that the device could be useful fordetecting and avoiding collisions. Marsalia [9] has evaluated the effectiveness of ahead-mounted display in improving hazard recognition for distracted pedestrians usinga driving simulator. Results showed that response hit rates improved and responsetimes were faster when participants had a display present.

The above studies indicate the value of head-mounted haptic display for alertingwearers to possible threats. The ‘Tactile Helmet’ [6] was a prototype sensory aug-mentation device developed by the current authors that aimed to be something morethan a hazard detector—a device for guiding users within unsafe, low-visibility envi-ronments such as burning buildings. We selected a head-mounted tactile display as thisfacilitates rapid reactions, can easily fit inside a modified fire-fighter helmet, and leavesthe hands of the firefighters free for tactile exploration of objects and surfaces. Our firstgeneration device (see Fig. 1) comprised a ring of eight ultrasound sensors on theoutside of a firefighter’s safety helmet with four voice coil-type vibrotactile actuatorsfitted to the inside headband. Ultrasound distance signals from the sensors were con-verted into a pattern of vibrotactile stimulation across all four actuators. One of thegoals of this approach was to have greater control over the information displayed to theuser, and, in particular, to avoid overloading tactile sensory channels by displaying toomuch information at once.

Auditory guidance in the form of non-verbal acoustic sound or synthetic speech isanother means for providing augmented navigation information for people with visu-ally impairments or for rescue workers [3–5].

The effectiveness of haptic and audio modalities have been compared in a numberof augmented navigation tasks with mixed results. For example, in [3], audio and hapticinterfaces were compared for way finding by blind pedestrians and it was found thathaptic guidance resulted in closer path-following compared to audio feedback. Marstonet al. [10] also evaluated nonvisual route-following with guidance from audio andhaptic display. Their results showed that haptic feedback produced slightly faster pathcompletion time and shorter distance, however, there was no significant difference

Fig. 1. The first generation ‘Tactile Helmet’ [6] was composed of a ring of ultrasound sensorsand four actuators inside the helmet and was designed to help firefighter’s navigate insidesmoked-filled buildings.

108 H. Kerdegari et al.

between audio and haptic modality. In [11], multimodal feedback strategies (haptic,audio and combined) were compared. Whilst there were no significant differencesbetween modalities in navigation performance, subjects reported that the audio guid-ance was less comfortable than others. Kaul et al. [12] have evaluated audio and hapticguidance in a 3D virtual object acquisition task using HapticHead (a cap consisting ofvibration motors) as a head-mounted display. User study indicated that haptic feedbackis faster and more precise than auditory feedback for virtual object finding in 3D spacearound the user. Finally, in [13] haptic and audio modalities were compared in terms ofcognitive workload, in a short-range navigation task, finding that workload was lowerin haptic feedback compared to audio for blind participants.

The aim of the current paper is to evaluate and compare audio and haptic guidancefor navigation using a head-mounted sensory augmentation device. We designed asecond-generation vibrotactile helmet as a sensory augmentation device for fire fight-ers’ navigation that sought to overcome some of the limitations of our first prototype(Fig. 1) [6] such as low-resolution tactile display. We previously investigated how todesign our tactile interface worn on the forehead [14] to present useful navigationalinformation as a tactile language [15]. Here, we use this tactile language to generatehaptic guidance signals and compare this to audio guidance in the form of syntheticspeech. In order to simulate a wall-following task similar to that faced by fire-fightersexploring a burning building, we constructed temporary walls made of cardboard in theexperimental room and asked subjects to follow these walls using the two alternativeguidance systems. The vibrotactile helmet uses ultrasound sensors to detect the user’sdistance to the walls and then a neural network algorithm to determine appropriateguidance commands (Go-forward/Turn right/Turn left). We evaluated the effectivenessof haptic and audio guidance according to the objective measures of task completiontime, distance of travel and route deviation, and subjective measure of workloadmeasurement using NASA TLX questionnaires.

2 Method

2.1 Subjects

Ten participants - 4 men and 6 women, average age 25 - voluntarily took part in thisexperiment. All subjects were university students or staff. The study was approved bythe University of Sheffield ethics committee, and participants signed the informedconsent form before the experiment. They did not report any known abnormalities withhaptic perception.

2.2 Vibrotactile Helmet

The second generation vibrotactile helmet (Fig. 2) consists of an array of twelveultrasound sensors (I2CXL-MaxSonar-EZ2 by MaxBotic), a tactile display composedof 7 tactors (Fig. 2 (b)) [14], a sound card, a microcontroller unit and two small lithiumpolymer batteries (7.4 V) to provide the system power. Furthermore, five reflective

Head-Mounted Sensory Augmentation Device: Comparing Haptic and AM 109

passive markers were attached to the vibrotactile helmet surface (Fig. 2 (d)) to enableus to track the user’s position and orientation using Vicon motion capture system.

Twelve ultrasound sensors were mounted with approximately 30 degrees separationto the outside of a skiing helmet (Fig. 2 (d)). The ultrasound sensors are employedsequentially one at a time. A minimum pulse-pause time of 50 ms is maintainedbetween consecutive readings to make measurements more stable against ultrasoundreflections. Using a 50 ms pulse-pause time, a complete environmental scan isaccomplished every 0.6 s. The practical measuring range by this ultrasound sensor isbetween 20 cm and 765 cm with 1 cm resolution. The tactile display consists of seveneccentric rotating mass (ERM) vibration motors (Fig. 2 (a)) with 3 V operating voltageand 220 Hz operating frequency at 3 V. These vibration motors are mounted on aneoprene fabric and attached on a plastic sheet (Fig. 2 (b)) with 2.5 cm inter-tactorspacing which can easily be adjusted inside the helmet. Furthermore, a sound card wasconnected to the microcontroller to produce the synthetic speech for audio modality.The ultrasound sensors data are sent to the microcontroller through I2C BUS. Themicrocontroller in the helmet reads the sensors values and sends them to the PCwirelessly using its built-in WiFi support. The PC receives the sensor values andperforms the required processing then generates commands, sending them back to themicrocontroller wirelessly for onward transmission to the tactile display/sound card.

2.3 Haptic and Audio Guidance

In low visibility environments, firefighters navigate using the existing infrastructuresuch as walls and doors. These reference points help them to stay oriented and make amental model of the environment [16]. To facilitate this form of navigation behavior weused a wall-following approach inspired by algorithms developed in mobile roboticsthat maintain a trajectory close to walls by combining steering-in, steering-out andmoving forward commands [17]. Specifically, to navigate the user along the wall, weutilized three commands: turn-left, turn-right, and go-forward. The turn-left/rightcommands are intended to induce a rotation around the user (left/right rotation) in orderto control the orientation of the user; the go-forward command is intended to induceforward motion. These three commands are presented to users in the form of haptic andaudio feedback.

Fig. 2. (a) Eccentric rotating mass vibration motor (Model 310-113 by Precision Microdrives).(b) Tactile display interface. (c) Tactile display position inside the helmet. (d) Vibrotactilehelmet.

110 H. Kerdegari et al.

Haptic feedback in the form of vibrotactile patterns is used to present the com-mands to the user through the vibrotactile display. Figure 3 illustrates the positions oftactors in the vibrotactile display and vibrotactile patterns for presenting differentcommands. Note that tactor 4 is placed in the center of forehead. The turn-left com-mand starts from tactor 3 and ends with tactor 1 while turn-right starts from tactor 5 andfinishes with tactor 7. Go-forward command starts from tactor 3 and tactor 5 simul-taneously and ends with tactor 4. We already investigated the utility and user experi-ence of these commands as our tactile language using the combination of twocommand presentation modes— continuous and discrete and two command types—recurring and single [18]. Results showed that “recurring continuous (RC)” tactilelanguage improved the performance better than other commands.

The continuous presentation mode takes advantage of the phenomena of tactileapparent movement [19]. Specifically when two or more tactors are activatedsequentially within a certain time interval, subjects experience the illusionary sensationof a stimulus travelling continuously from the first stimulation site to the second. Thetwo main parameters that control the feeling of apparent motion are the duration ofstimulus (DoS) and the stimulus onset asynchrony (SOA) [20]. In the current study, aDoS of 400 ms and a SOA of 100 ms were utilized respectively. This results in a totalrendering time of 600 ms for turn right/left commands and 500 ms for go-forwardcommand. However, in the discrete presentation mode the tactors are activatedsequentially with no stimulus overlap that creates the experience of discrete motionacross the forehead for all three commands. As command type, recurring conditionpresents the tactile command to the user’s forehead repeatedly with interval betweenpatterns of 500 ms until a new command is received; while for the single condition thetactile command is presented just once when there is a change in the command.A schematic representation of continuous command presentation and recurring com-mand type for the turn-left command is presented in Fig. 4.

An alternative modality to haptic is audio modality through spoken direction [3].Similar to the haptic modality, our audio modality also uses three commands to nav-igate the user along the wall. However, rather than using tactile language for presentingthese commands, the following synthetic speech is applied: Go-forward, Turn-right and

Fig. 3. Vibrotactile patterns for presenting turn-left, turn-right and go-forward commands in thetactile display.

Head-Mounted Sensory Augmentation Device: Comparing Haptic and AM 111

Turn-left. The duration of each synthetic speech is equal to its similar haptic one andthe interval between patterns is 500 ms like the recurring condition in hapticcommands.

2.4 Procedure

We made a path consisting of several cardboard walls in the experiment room tonavigate the subjects along it (Fig. 5 (a)). In order to track the subject’s position andorientation during the navigation, we used a Vicon motion capture system in theexperiment room. At the beginning of the experiment, each subject was invited into theexperiment room and asked to wear the tactile helmet and a blindfold. They were not beable to see the experiment set-up and cardboard walls before starting the experiment.Participants were told that haptic/audio feedback would assist them to follow the wallseither by turning to the left or right or by maintaining a forward path. Subjects werealso asked to put on headphone playing white noise to mask any sounds from tactorsduring navigation with haptic feedback. Furthermore, subjects were asked to keep theirhead oriented in the direction of travel and to avoid making unnecessary sideways headmovements. A short training session was then provided to familiarize subjects with thetactile language, audio feedback and with the experimental set-up. Once the participantfelt comfortable, the trial phase was started. We considered two starting points (1 and 2as shown in Fig. 5 (a)) to not let the subjects to memorize the paths. Blind-foldedsubjects (Fig. 5 (b)) started the first trial from position 1 and the second trial fromposition 2 and repeated it for the third and fourth trial. When each trial finished,subjects were stopped by the experimenter. Subjects were allowed to rest after eachtrial and started the next trial whenever they were ready. The maximum duration of theexperiment was approximately 20 min. In total, each subject performed 4 trialsincluding 2 feedback types (haptic and audio), each for two times in a pseudo-random

Fig. 4. Schematic representation of the tactile language employed in this study. (a) Tactileapparent motion (continuous) presentation, (b) recurring cue.

112 H. Kerdegari et al.

order. Task completion time, travel distance and route deviation as objective measuresfor each trial were measured.

After finishing the experiment, subjects were asked to complete a paper and pencilversion of the NASA task load index (TLX) [21] to measure subjective workload. Itconsists of six dimensions including mental demand, physical demand, temporaldemand, performance, effort and frustration with 21 graduations. Additionally, subjectswere asked to rate their preference for completing the task with audio and hapticmodality.

3 Classification Algorithm

The wall-following task as a pattern classification problem is nonlinearly separablewhich is in favor of multilayer perceptron neural network [22]. In this work, multilayerperceptron neural network algorithm was utilized to guide the user along the wall asone of common methods used for robot navigation using ultrasound sensors [23]. As aclassification algorithm, it associates the ultrasound data to the navigation commands(go-forward and turn right/left) in the form of haptic or audio modality. In order tocollect data for training the classification algorithm, the experimenter wore the helmetand kept the laptop in her hands and followed the cardboard walls in the experimentroom without wearing a blindfold (Fig. 5 (a)). The datasets are the collection ofultrasound readings when the experimenter follows the walls in a clockwise andanti-clockwise direction, each for 8 rounds. The data collection was performed at a rateof 1 sample (from 12 ultrasound sensors) per 0.6 s and generated a database with 4051samples. Data were labeled during data collection by pressing the arrow key on thelaptop keyboard when turning or going forward is intended (pressing left/right arrowkey button for turn left/right and up arrow key button for go-forward). Ultrasound datain every scan were saved with a related label in a file. Three classes were considered inall the files: (1) Go-forward (2) Turn-right and (3) Turn-left and were used to train theclassifier. We used Multilayer Perceptron (MLP) neural network to classify ultrasounddata into three navigation commands. Our MLP (as shown in Fig. 6) consists of 12input nodes (distance measurement form 12 ultrasound sensors), 1 hidden layer with 15

Fig. 5. (a) Overhead view of the experimental set-up consisting of cardboard walls and motioncapture cameras, position 1 and 2 show the trial stating points. The length of the walls from thestart point to the end is 20 m. (b) Subject is navigating along the wall.

Head-Mounted Sensory Augmentation Device: Comparing Haptic and AM 113

nodes and 3 nodes in output layer (three navigation commands). Back propagationalgorithm was used to train the data and evaluation was done using 10 times 10-Foldscross-validation. Sensitivity and specificity of the MLP algorithm for recognizinggo-forward (G), turn-right (R) and turn-left (L) commands are defined as:

Sensitivity ¼ TPG;R;L

TPG;R;L þFNG;R;Lð1Þ

Specificity Gð Þ ¼ TPRþ TPL

TPRþ TPLþFPG;

Specificity Rð Þ ¼ TPLþ TPG

TPLþ TPGþFPR;

Specificity Lð Þ ¼ TPRþ TPG

TPRþ TPGþFPL

ð2Þ

where TPG;R;L (True Positive) corresponds to successfully classified Go-forward,Turn-right and Turn-left commands, FPG;R;L (False Positive) corresponds to erro-neously classified Go-forward, Turn-right and Turn-left commands and FNG;R;L (FalseNegative) corresponds to missed Go-forward, Turn-right and Turn-left commands [24].

The overall accuracy of the MLP is 94.9 %. Table 1 presents the results of sen-sitivity and specificity of the MLP algorithm for recognizing the go-forward andturning commands. Finally, the proposed trained MLP algorithm was used to navigatethe subjects along the walls.

Fig. 6. The structure of the proposed MLP. It consists of 12 input nodes, 15 hidden nodes and 3outputs.

114 H. Kerdegari et al.

4 Results

An alpha value of 0.05 was chosen as the threshold for statistical significance, allreported p-values are two-tailed. Shapiro-Wilk test showed that data are normallydistributed. We measured task completion time (minute), travel distance (meter) androute deviation (meter) for audio and haptic modality as our objective measures. Taskcompletion time was recorded as the time that subject took to navigate along the wallfrom start point to the end point. Task completion time for audio and haptic modality inFig. 7 (a) shows that subjects navigated faster with haptic modality than audiomodality. However, paired t-test showed no significant difference between audio andhaptic modality in task completion time (t = −1.287, p = 0.33). Travel distance as adistance that subjects have walked along the wall was measured using motion capturesystem. As shown in Fig. 7 (b), subjects traveled shorter distance with haptic modality.A paired t-test revealed no significant difference between audio and haptic modality intravel distance (t = 2.024, p = 0.074). We further measured route deviation usingmotion capture system when navigating with audio and haptic modality. It showssubjects’ position deviation relative to the walls during the navigation. Subjects hadlower route deviation (Fig. 7 (c)) when navigating with haptic modality. A paired t-testshowed a significant difference in route deviation between audio and haptic modality(t = 2.736, p = 0.023).

After completing the experiment, we subjectively measured workload for eachmodality by asking subjects answer the NASA TLX questionnaire. As shown in Fig. 8,physical and temporal demand did not vary much between two modalities which showsboth of them were able to navigate the subjects. However, subjects rated that mentaldemand and effort are higher when navigating with audio feedback. These highermental workload and effort are because subjects had to concentrate more to processaudio feedback to navigate successfully along the wall. Subjects also rated betterperformance and lower frustration with haptic modality, which shows the capability ofhaptic modality for navigation along the wall consistent with our objective measure.Furthermore, subjects were asked to rate their preference for navigation with audio andhaptic modality. This preference was rated on a scale of 1–21 to keep continuity withour NASA TLX, where (1) represents a strong preference for navigation with hapticfeedback and (21) represents strong preference for navigation with audio feedback. Theaverage preference rate of 3.4 as illustrated in Fig. 8 indicated subjects’ preference fornavigating with haptic modality.

Table 1. Sensitivity and specificity for recognizing go-forward and turning commands.

Go-forward Turn-right Turn-left

Sensitivity (%) 95.9 95.3 90.6Specificity (%) 92.7 98.6 98

Head-Mounted Sensory Augmentation Device: Comparing Haptic and AM 115

Fig. 7. Objective measures. (a) Task completion time, (b) Travel distance, (c) Route deviation.The unit of task completion time is in minute and unit of travel distance and route deviation is inmeter. Error bars show standard error.

Fig. 8. Questionnaire feedback. The first six bar plots represent the NASA TLX score for audioand haptic modality. The rating scale is 1–21, where 1 represents no mental, physical andtemporal demand, best performance, no effort required to complete the task and, no frustration.The last bar plot shows subjects’ preference for navigation with haptic modality. The error barsindicate standard error. (Color figure online)

116 H. Kerdegari et al.

5 Conclusion

This paper compares and investigates haptic and audio modalities as non-visualinterfaces for navigation in low visibility environment using the vibrotactile helmet as asensory augmentation device. The haptic modality utilizes our tactile language in theform of vibrotactile feedback while audio modality applies synthetic speech to presentnavigation commands. The objective measure showed that haptic feedback leads tolower route deviation significantly. We also measured task completion time and traveldistance. Although subjects had faster task completion time and lower travel distancewith haptic feedback, no significant difference was found between these two modali-ties. Unlike [13] which blindfolded users had higher cognitive workload in navigationwith haptic modality than with audio modality, our analysis using NASA TLX ques-tionnaire indicated that haptic modality had lower workload on the subjects. The resultsof this study show the effectiveness of haptic modality for guided navigation withoutvision. Future work will use a local map of the environment estimated with theultrasound sensors to generate the navigation commands in place of the MLP algo-rithm. Furthermore, it would be interesting to conduct this experiment with visuallyimpaired people to investigate the potential of haptic and audio modality as a com-munication channel for assisted navigation devices.

Acknowledgment. We would like to thank the subjects for their help in data collection for thisstudy. This work was supported by the University of Sheffield Cross-Cutting Directors ofResearch and Innovation Network (CCDRI), Search and Rescue 2020 project.

References

1. Kärcher, S.M., Fenzlaff, S., Hartmann, D., Nagel, S.K., König, P.: Sensory augmentation forthe blind. Front. Hum. Neurosci. 6, 37 (2012)

2. Kim, Y., Harders, M., Gassert, R.: Identification of vibrotactile patterns encoding obstacledistance information. IEEE Trans. Haptics 8(3), 298–305 (2015)

3. Flores, G., Kurniawan, S., Manduchi, R., Martinson, E., Morales, L.M., Sisbot, E.A.:Vibrotactile guidance for wayfinding of blind walkers. IEEE Trans. Haptics 8(3), 306–317(2015)

4. Shoval, S., Borenstein, J., Koren, Y.: Auditory guidance with the NavBelt-a computerizedtravel aid for the blind. IEEE Trans. Syst. Man Cybern. C Appl. Rev. 28(3), 459–467 (1998)

5. Holland, S., Morse, D.R., Gedenryd, H.: AudioGPS: spatial audio navigation with a minimalattention interface. Pers. Ubiquitous Comput. 6(4), 253–259 (2002)

6. Bertram, C., Evans, M.H., Javaid, M., Stafford, T., Prescott, T.: Sensory augmentation withdistal touch: the tactile helmet project. In: Lepora, N.F., Mura, A., Krapp, H.G., Verschure,P.F., Prescott, T.J. (eds.) Living Machines 2013. LNCS, vol. 8064, pp. 24–35. Springer,Heidelberg (2013)

7. Cassinelli, A., Reynolds, C., Ishikawa, M.: Augmenting spatial awareness with haptic radar.In: 10th IEEE International Symposium on Wearable Computers, pp. 61–64 (2006)

8. Carton, A., Dunne, L.E.: Tactile distance feedback for fire-fighters: design and preliminaryevaluation of a sensory augmentation glove. In: Proceedings of the 4th Augmented HumanInternational Conference, pp. 58–64 (2013)

Head-Mounted Sensory Augmentation Device: Comparing Haptic and AM 117

9. Marsalia, A.C.: Evaluation of vibrotactile alert systems for supporting hazard awareness andsafety of distracted pedestrians, Master Thesis at Texas A&M University (2013)

10. Marston, J.R., Loomis, J.M., Klatzky, R.L., Golledge, R.G.: Nonvisual route following withguidance from a simple haptic or auditory display. J. Vis. Impair. Blind. 101(4), 203–211(2007)

11. Hara, M., Shokur, S., Yamamoto, A., Higuchi, T., Gassert, R., Bleuler, H.: Virtualenvironment to evaluate multimodal feedback strategies for augmented navigation of thevisually impaired. In: IEEE Engineering in Medicine and Biology Society, pp. 975–978(2010)

12. Kaul, O.B., Rohs, M.: HapticHead: 3D guidance and target acquisition through a vibrotactilegrid. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors inComputing Systems, pp. 2533–2539 (2016)

13. Martinez, M., Constantinescu, A., Schauerte, B., Koester, D., Stiefelhagen, R.: Cognitiveevaluation of haptic and audio feedback in short range navigation tasks. In: Miesenberger,K., Fels, D., Archambault, D., Peňáz, P., Zagler, W. (eds.) ICCHP 2014, Part II. LNCS, vol.8548, pp. 128–135. Springer, Heidelberg (2014)

14. Kerdegari, H., Kim, Y., Stafford, T., Prescott, T.J.: Centralizing bias and the vibrotactilefunneling illusion on the forehead. In: Auvray, M., Duriez, C. (eds.) EuroHaptics 2014,Part II. LNCS, vol. 8619, pp. 55–62. Springer, Heidelberg (2014)

15. Kerdegari, H., Kim, Y., Prescott, T.: Tactile language for a head-mounted sensoryaugmentation device. In: Wilson, S.P., Verschure, P.F., Mura, A., Prescott, T.J. (eds.) LivingMachines 2015. LNCS, vol. 9222, pp. 359–365. Springer, Heidelberg (2015)

16. Denef, S., Ramirez, L., Dyrks, T., Stevens, G.: Handy navigation in ever-changing spaces:an ethnographic study of firefighting practices. In: Proceedings of the 7th ACM Conferenceon Designing Interactive Systems, pp. 184–192 (2008)

17. Ando, Y., Yuta, S.: Following a wall by an autonomous mobile robot with a sonar-ring. In:Proceedings 1995 IEEE International Conference on Robotics and Automation, vol. 3,pp. 2599–2606 (1995)

18. Kerdegari, H., Kim, Y., Prescott, T.: Head-mounted sensory augmentation device: designinga tactile language. IEEE Trans. Haptics PP(99), 1 (2016)

19. Sherrick, C.E., Rogers, R.: Apparent haptic movement. Percept. Psychophys. 1(6), 175–180(1966)

20. Kirman, J.H.: Tactile apparent movement: the effects of interstimulus onset interval andstimulus duration. Percept. Psychophys. 15(1), 1–6 (1974)

21. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): Results ofEmpirical and Theoretical Research. In: Hancock, P.A., Meshkati, N. (eds.) Human MentalWorkload, pp. 239–250. Elsevier, North Holland Press, Amsterdam (1988)

22. Freire, A.L., Barreto, G.A., Veloso, M., Varela, A.T.: Short-term memory mechanisms inneural network learning of robot navigation tasks: a case study. In: 6th Latin AmericanRobotics Symposium (LARS), no. 4 (2009)

23. Zou, A.-M., Hou, Z.-G., Fu, S.-Y., Tan, M.: Neural networks for mobile robot navigation: asurvey. In: Wang, J., Yi, Z., Żurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol.3972, pp. 1218–1226. Springer, Heidelberg (2006)

24. Ando, B., Baglio, S., Marletta, V., Valastro, A.: A haptic solution to assist visually impairedin mobility tasks. IEEE Trans. Hum. Mach. Syst. 45(5), 641–646 (2015)

118 H. Kerdegari et al.

Visual Target Sequence Predictionvia Hierarchical Temporal MemoryImplemented on the iCub Robot

Murat Kirtay(B), Egidio Falotico, Alessandro Ambrosano, Ugo Albanese,Lorenzo Vannucci, and Cecilia Laschi

The BioRobotics Institute, Scuola Superiore Sant’Anna, 56025 Pontedera, Pisa, Italy{m.kirtay,e.falotico,a.ambrosano,u.albanese,l.vannucci,c.laschi}@sssup.it

http://sssa.bioroboticsinstitute.it

Abstract. In this article, we present our initial work on sequence pre-diction of a visual target by implementing a cortically inspired method,namely Hierarchical Temporal Memory (HTM). As a preliminary test, weemploy HTM on periodic functions to quantify prediction performancewith respect to prediction steps. We then perform simulation experimentson the iCub humanoid robot simulated in the Neurorobotics Platform.We use the robot as embodied agent which enables HTM to receivesequences of visual target position from its camera in order to predicttarget positions in different trajectories such as horizontal, vertical andsinusoidal. The obtained results indicate that HTM based method canbe customized for robotics applications that require adaptation of spa-tiotemporal changes in the environment and acting accordingly.

Keywords: Hierarchical Temporal Memory · Sequence prediction ·Neurorobotics Platform

1 Introduction

Understanding mammalian brain functions and mechanisms have been attractiveresearch field for robotics and artificial intelligence (AI). These functions andmechanisms provide proof of concept solutions for known problems in roboticsand AI such as operating massively parallel processes with relatively low powerconsumption, processing noisy sensory information for action execution, decisionmaking, etc. Thus, brain-inspired approaches have been investigated to deriveprinciples of brain functions and mechanisms in order to propose solutions forvarious existing challenges. The several known examples are image recognitionin large databases by means of deep neural networks [1], artificial neural networkbased path planning [2], to mention a few. Although these methods perform wellin a specific task with notable accuracy, they lack in biological plausibility andgeneralization of the solution in different domains.

To build more biologically realistic models with generalization capabilities, amethod based on operating principles of Neocortex has been proposed in [3] andc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 119–130, 2016.DOI: 10.1007/978-3-319-42417-0 12

120 M. Kirtay et al.

named as Hierarchical Temporal Memory (HTM). HTM based solutions1 havebeen successfully implemented in a considerable number of applications and ina wide range of areas where sequential learning and generative prediction aimedat anomaly detection, image classification, rogue behavior detection, geospatialtracking. In robotics HTM can be used to anticipate the world future statesin order to properly control motion and deal with the continuously changingenvironment. Humans appear to solve this problem by predicting the changes intheir sensory system as a consequence of their actions [5]. The predictions areobtained using internal models that represent their own bodies and the exter-nal objects dynamics [6–8]. There are three main types of internal models [7]:the forward models, the environment models, and the inverse models. The for-ward models allow predicting the future data from the past perceptions andthe planned actions; the environment models predict the dynamics of externalobjects or agents, and the inverse models find the actions needed to obtain thedesired state starting from the actual one.

So far in robotics there have been a number of implementations of antic-ipatory sensory-motor systems based on internal models. Examples of controlsystems for mobile robots that predict the visual sensory data using forwardmodels are [9,10], while [11–13] proposed systems that anticipate the dynamicsof moving objects in order to accomplish catching tasks. Other models basedon prediction have been proposed to improve performances of humanoid robotsin different tasks: visual pursuit and prediction of the target motion [14–17],anticipation in reaching [18] or manipulation tasks [19–21].

In this study, we focus on sequence prediction of a visual target using HTM.Despite promising features of HTM, there are only limited applications in therobotics field. In study [22], authors used HTM to create and edit various pos-tures in order to enable an inexperienced user to get familiarity with a humanoidrobot’s motion authoring concepts. The study in [23] proposed perception app-roach based on receiving images of various categories such as doors, plants andbookcases. Then, authors performed HTM as classification method to label theseimages and based on the output of image category, the predefined actions ofa conceptual robot determined which are opening a door and move forward.Instead of presenting continuous prediction and online learning which can beconsidered as a core component of HTM, these robotics related studies con-cerned with image classification and action execution applications. On the otherhand, Nguyen et al. investigate a spatiotemporal sequence learning architecturefor autonomous navigation [24]. Although the proposed architecture containsbasics from short and long-term memory mechanisms (STM and LTM) whichare partially similar to HTM, the authors used a robot vision data set to extractsequence properties of the navigated environment. As described in this literaturereview, there are only limited robotics applications and research involving HTM.In this paper, we perform HTM for visual target position prediction of the iCubhumanoid robot in a simulated environment.

1 http://numenta.com/#applications.

Visual Target Sequence Prediction via HTM 121

The rest of paper organized as follows; in the Sect. 2, we highlight operatingprinciples of Neocortex and biological plausibility of HTM. The experimentalsetup and implementation details of the HTM framework are presented in theSect. 3. The obtained results and discussions are explained in the Sect. 4 whileconcluding remarks and future works are emphasized in the Sect. 5.

2 Biological Inspiration Behind HTM

In this section, the biological functions and structure of mammalian neocortexwill be introduced and the biological plausibility of HTM will be addressed.Hierarchical temporal memory was proposed in study [25]. It is a biologicallyconstrained model inspired by operating principles of the neocortex. Neocor-tex is responsible for visual pattern recognition, understanding speech, objectrecognition, manipulation by touch, etc. Biological organization of the neocortexconsists of columnar structure repeated in all over the mammalian brain [26].This structure of repeating pattern led to an idea of the neocortex performingsame computational mechanism to process different inputs from sensory organs(e.g. eye, ear) and produce motor commands to execute necessary actions [3].These biological insights provide support for HTM in order to have a layeredarchitecture which performs same algorithms for different inputs. The aforemen-tioned sensorimotor features of neocortex attract researchers to build ArtificialNeural Networks (ANNs) which are able to implement computational and learn-ing systems that imitate the behavior of neurons (e.g. pyramidal cells). Despiteconsiderable success of the ANN based methods in specific applications, theseapproaches lack in biological plausibility and show performance degradation indifferent domains [27]. For instance, these methods require considerable amountof data and computational resources for training via batch processing and maynot perform well if data have a streaming characteristic.

Unlike classical ANN based approaches, HTM provides more biologicallyrealistic computation concepts to overcome existing difficulties against invari-ant and streaming data. By doing so, HTM employs an encoding phase for pre-processing data, spatial pooling to generate sparse distributed representation ofthe encoders, and temporal pooling (or memory) to capture time-based changesfrom the outputs of spatial pooling. These concepts will be re-emphasized andimplementation details will be described in the Sect. 3.

3 Experimental Setup and Implementation ofHierarchical Temporal Memory

3.1 Neurorobotics Platform (NRP) Setup

In this study we used the Neurorobotics platform (NRP)2 which is being devel-oped as part of Human Brain Project (HBP)3. The aim of NRP is to provide2 http://neurorobotics.net.3 http://www.humanbrainproject.eu.

122 M. Kirtay et al.

Fig. 1. iCub humanoid robot in Neurorobotics Platform

physically realistic simulation environment which enables neuroscientists to testtheir models on different robots such as iCub, Husky and Lauron. Furthermore,the platform can also be used by roboticists to build and customize biologicallyinspired robot control architectures for specific tasks. The software and func-tional details of the NRP with an integration use case is described in study [4].The NRP is being developed for spiking neuron models, yet it also allows usersto extend and integrate new models. We integrate HTM to derive target posi-tion sequence prediction while processing iCub’s camera data. As can be seen inFig. 1, iCub is used as an embodied agent receiving camera data to feed HTMimplementation pipeline (see Fig. 2) in order to predict target position. A circulartarget moves on the screen along predefined paths which are horizontal, verticaland sinusoidal. After collecting images from a camera, the task is restarted andone step ahead prediction is performed using HTM.

We set two distinct goals to conduct simulation experiments with the iCubrobot. In that, we firstly aimed to investigate the NRP and HTM capabilitiesto assess target sequence prediction in online mode. For the latter studies, weare in the process of extending HTM model to integrate sensorimotor transitionmodules for eye movements and grasping while continuously predicting targetsequence.

3.2 Implementation Procedure

In order to exploit HTM for sequence prediction of a visual target, we cus-tomized an existing open-source software platform, Numenta Platform for Intel-ligent Computing (NuPIC)4, that has already been used in various applicationswhere data had streaming charateristics. As a first step of this study, we used

4 http://www.numenta.org.

Visual Target Sequence Prediction via HTM 123

NuPIC, with its default parameters, to predict scalar values of different periodicfunctions. Afterward, we tuned NuPIC parameters in order to predict targetsequences streamed from the simulated environment.

The sequence prediction mechanism works as follows: in a first phase, datareceived from the robot camera is collected until the target has reached the endof the screen (see Figs. 1 and 4). During this phase, the images are processedby extracting the region of interest, using prior knowledge of the target motion,and then by applying a binary threshold to obtain a black and white image. Thethresholded image is converted into a vector where ones correspond to whitepixels and zeroes to black ones. It should be noted that extracting the regionof interest from the image reduces computation times for preprocessing, but thesame results could be achieved by using the whole camera image data. Moreover,in most HTM applications, the encoding phase is usually employed for gettingthe binary representation of the input data before passing it to the HTM imple-mentation pipeline, as depicted in Fig. 2.

Fig. 2. HTM implementation pipeline: inputs and outputs of each phase

The generated binary representation is used as input of spatial pooling phasein order to obtain a sparse distributed representation (SDR) of the receivedinput in the available dataset. The SDR is a cortically inspired representation ofinput data where only a small percentage of the bits are active [29]. Similarly,temporal pooling takes a series of SDR inputs from the spatial pool and generatespredictions by learning sequences. This learning phase enables HTM to formstable sequences of SDRs while capturing temporal relations for prediction.

After the data collecting phase is over, i.e. when the target motion restarts,prediction are started to being produced by the HTM, while the new imagesare still being processed and fed into the spatial and temporal pools. Such pre-dictions can be from one to several steps ahead, depending on the applicationrequirements. In this work, the model is used to perform single and multi-steppredictions for periodic functions and single step predictions of the target imagesequences. A more comprehensive neuroscientific and mathematical background

124 M. Kirtay et al.

about the aforementioned HTM implementation phases can be found in thefollowing studies: [27–29].

4 Results and Discussions

The obtained results will be presented in twofold which are HTM error analysiswith periodic functions and HTM implementation in NRP. We consider thatthe error interpretation will give rise to an idea that whether HTM can beimplemented in visual sequence prediction.

4.1 HTM-Prediction Analysis Using Periodic Functions

In this step, we aim to derive prediction errors while performing HTM on twoperiodic functions with different prediction steps. To obtain prediction values byperforming HTM pipeline, 3000 sequential iteration steps are generated and con-verted to binary representation in encoding phase. As a next step, the encodedvalues are used as inputs of the spatial pooling phase and these values areformed into sparse distributed representations (SDR). Then, the sequence relatedchanges in SDRs are captured by temporal pooling phase in order to generatea prediction. It should be noted that generated predictions share similar repre-sentation. That is why it is necessary to decode this representation to get scalarvalues. The actual values are obtained from below equations and the predictedvalues for 1, 2, and 3 steps in advance are used to assess HTM prediction per-formance. In order to evaluate prediction error, we used the Mean Square Error(MSE) as an evaluation metric for each function.

y = sin(x) (1)

y = sin(2 × x) + cos(4 × x) (2)

Figure 3(a) and (b) show that the obtained one and three steps prediction whichare generated by using the Eq. 1 with iteration range between 2440 and 2600.Similarly, Fig. 3(c) and (d) illustrate single and multi-step prediction by perform-ing Eq. 2 with iteration range between 2440 and 2550. As can be seen from thesefigures HTM can make a single and multi-step prediction for a given function. Itshould be noted that the predefined iteration ranges represent prediction trendsfor both equations.

Table 1. Mean square error results of the performed functions

Prediction step MSE (Eq. 1) MSE (Eq. 2)

1 2.7514e− 05 0.0233

2 5.3602e− 05 0.0263

3 1.2051e− 04 0.0254

Visual Target Sequence Prediction via HTM 125

Moreover, the calculated MSE values in Table 1 are derived for the iterationranges and this values also indicate that HTM can adapt to changes in streamingdata to generate either single or multi-step prediction. We also observed that anincreasing the prediction steps will lead to an increment in the MSE values. Tosum up, we interpret HTM predictions using periodic functions and we observethat obtained results can lead to implementing the same method for visual targetsequence prediction in a simulated environment.

(a) One step ahead sinusoidaltrajectory prediction

(b) Three steps ahead sinusoidaltrajectory prediction

(c) One step ahead periodic function,Eq. 2, trajectory prediction

(d) Three steps ahead periodizfunction, Eq. 2, trajectory prediction

Fig. 3. One step and three steps ahead periodic functions’ trajectory predictions

4.2 HTM-NRP Implementation for Visual Target SequencePrediction

After error analysis step with periodic functions, we consider performing HTMfor visual target sequence prediction in a simulated environment. To implementthis task, we customized HTM to process perceived image data from iCub’scamera in the NRP. Instead of generating input data via predefined functions,we use a visual target which continuously moves along a horizontal, a verticaland a sinusoidal path. A subset of obtained image sequence for the target areshown in Fig. 4. Images in the first row are shown in Fig. 4 and recorded duringhorizontal movement, the second-row images consist of vertical movement andlast row images belong to the sinusoidal movement.

126 M. Kirtay et al.

(a) Sequence 1 (b) Sequence 2 (c) Sequence 3 (d) Sequence 4 (e) Sequence 5

(f) Sequence 6 (g) Sequence 7 (h) Sequence 8 (i) Sequence 9 (j) Sequence 10

(k) Sequence 11 (l) Sequence 12 (m) Sequence 13 (n) Sequence 14 (o) Sequence 15

Fig. 4. iCub camera images

The considered experiment in NRP starts with target movement along oneof the above mentioned trajectories (e.g. sinusoidal) till reach end of the virtualscreen. Throughout this movement, each received camera image is stored in orderto be used as an input for HTM implementation. The number of stored imagesduring learning step for the horizontal sequence is 101, for the vertical sequenceis 100 and for the sinusoidal sequence is 64. As outlined in the Sect. 3, rawpixel values of stored images preprocessed for pipeline and their SDR forms areused for prediction. In order to make a prediction, the same pipeline procedurewill be applied to received images. The SDR form of input will be generatedfor next sequence prediction. To obtain similarity value between input SDRand stored SDRs, it is needed to decode prediction from available similarityscores. The decoding step for scalar values can be easily constructed by NuPICframework. HTM has currently no probabilistic model to reconstruct predictedimage from SDR. To overcome this difficulty, we implement a method to getoverlapping scores based on active bit positions in among learned SDRs andpredicted SDR values. If same overlapping score exists in learned SDRs, werefer the last pattern in the sequence as predicted pattern since we assume thatthe target will continuously move in the forward direction. Instead of performingclassification and regression methods (e.g. K-nearest neighbors), we noticed thatthis simple method will lead to understanding HTM dynamics in detail.

However, applying this method brings about predicting forward jumps in thesequence or getting the same prediction more than once as a next step prediction.That behavior illustrated with the data obtained from Neurorobotics Platformin Fig. 5 where colored ∗ refers to movement trajectory of the target center.For instance, red ∗ characters are obtained prediction values while the targetmoves in a vertical path. Based on overlapping method 21 different predictionsare generated during vertical movement. As can be seen from Fig. 5, predicted

Visual Target Sequence Prediction via HTM 127

Fig. 5. Predicted path trajectories in NRP experiment (Blue - Horizontal, Red- Verti-cal, Black - Sinusoidal) (Color figure online)

sequence are consecutive with gaps and HTM can still capture spatiotemporalchanges in these experiments.

To test overlapping score strategy in a trajectory where one point is far toanother, we construct a sequence which is a combination of horizontal, verticaland sinusoidal, respectively. Such sequence is composed of a number of capturedimages which is lower compared to the previous experiment. In that, we randomlyselect eight patterns from horizontal, nine patterns from vertical and ten patternsfrom sinusoidal paths and convert them to a single sequence (see Fig. 6(a)). Thenwe employ the overlapping method and extracting a prediction as one of thepatterns in the selected sequence which are shown in Fig. 6(b). It is observedthat HTM can successfully predict sequences without having above mentionedproblems.

(a) Sequential trajectory (b) Predicted Sequential trajectory

Fig. 6. Prediction path trajectories (Color figure online)

128 M. Kirtay et al.

5 Conclusion

In this study we implement a cortically inspired computational method, Hier-archical Temporal Memory (HTM), to predict target position sequence (e.g.traversed path) via extracting target’s spatiotemporal features. To do this, wefirstly analyze HTM performance while predicting sequence in advance on differ-ent periodic functions. After obtaining promising results with these predefinedfunctions, we employed HTM in Neurorobotics Platform to predict target posi-tion sequences. The presented results show that HTM paves the way for sequenceprediction by learning spatiotemporal features of the environment and adopt-ing these statistical changes accordingly. This indicates that HTM has promisingdynamics for robotics studies where exploring the dynamics of unknown environ-ment required to take necessary actions. Our ongoing research directions focusedon developing HTM based algorithms on simulated and real humanoid robot inorder to achieve a task which requires sensorimotor coordination and controlsuch as predictions of the eye movements while tracking a visual object andcontinuously learning grasped object dynamics.

Acknowledgements. The research leading to these results has received funding fromthe European Union Seventh Framework Programme (FP7/2007-2013) under grantagreement no. 604102 (Human Brain Project). The authors would like to thank theItalian Ministry of Foreign Affairs, General Directorate for the Promotion of the “Coun-try System”, Bilateral and Multilateral Scientific and Technological Cooperation Unit,for the support through the Joint Laboratory on Biorobotics Engineering project.

References

1. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep con-volutional neural networks. In: Neural Information Processing Systems (NIPS), pp.1106–1114 (2012)

2. Glasius, R., Komoda, A., Gielen, S.: Neural network dynamics for path planningand obstacle avoidance. Neural Netw. 8, 125–133 (1995)

3. Hawkins, J., Blakeslee, S.: On Intelligence. Henry Holt and Company, New York(2004)

4. Vannucci, L., Ambrosano, A., Cauli, N., Albanese, U., Falotico, E., Ulbrich, S.,Pfotzer, L., Hinkel, G., Denninger, O., Peppicelli, D., Guyot, L., Von Arnim, A.,Deser, S., Maier, P., Dillman, R., Klinker, G., Levi, P., Knoll, A., Gewaltig, M.-O.,Laschi, C.: A visual tracking model implemented on the iCub robot as a use casefor a novel neurorobotic toolkit integrating brain and physics simulation. In: IEEE-RAS International Conference on Humanoid Robots, pp. 1179–1184 (2015)

5. Berthoz, A.: The Brain’s Sense of Movement. Harvard University Press, Cambridge(2002)

6. Johansson, R.S.: Sensory input and control of grip. Sensory guidance of movement(1998)

7. Miall, R.C., Wolpert, D.M.: Forward models for physiological motor control. NeuralNetw. 9, 1265–1279 (1996)

Visual Target Sequence Prediction via HTM 129

8. Nguyen-Tuong, D., Peters, J.: Model learning for robot control: a survey. Cogn.Process. 12, 319–340 (2011)

9. Gross, H.-M., Heinze, A., Seiler, T., Stephan, V.: Generative character of percep-tion: a neural architecture for sensorimotor anticipation. Neural Netw. 12, 1101–1129 (1999)

10. Hoffmann, H.: Perception through visuomotor anticipation in a mobile robot.Neural Netw. 20, 22–33 (2007)

11. Bauml, B., Birbach, O., Wimbock, T., Frese, U., Dietrich, A., Hirzinger, G.: Catch-ing flying balls with a mobile humanoid: system overview and design considera-tions. In: 2011 11th IEEE-RAS International Conference on Humanoid Robots(Humanoids), pp. 513–520 (2011)

12. Kim, S., Billard, A.: Estimating the non-linear dynamics of free-flying objects.Robot. Auton. Syst. 60, 1108–1122 (2012)

13. Kober, J., Glisson, M., Mistry, M.: Playing catch and juggling with a humanoidrobot. In: 2012 12th IEEE-RAS International Conference on Humanoid Robots(Humanoids), pp. 875–881 (2012)

14. Falotico, E., Zambrano, D., Muscolo, G.G., Marazzato, L., Dario, P., Laschi, C.:Implementation of a bio-inspired visual tracking model on the iCub robot. In:Proceedings of the 19th IEEE International Symposium on Robot and HumanInteractive Communication (ROMAN 2010), pp. 564–569. IEEE (2010)

15. Vannucci, L., Falotico, E., Di Lecce, N., Dario, P., Laschi, C.: Integrating feedbackand predictive control in a bio-inspired model of visual pursuit implemented on ahumanoid robot. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A., Prescott, T.J.(eds.) Living Machines 2015. LNCS, vol. 9222, pp. 256–267. Springer, Heidelberg(2015)

16. Falotico, E., Taiana, M., Zambrano, D., Bernardino, A., Santos-Victor, J., Dario,P., Laschi, C.: Predictive tracking across occlusions in the iCub robot. In: Pro-ceedings of the 9th IEEE-RAS International Conference on Humanoid Robots(Humanoids 2009), pp. 486–491 (2009)

17. Zambrano, D., Falotico, E., Manfredi, L., Laschi, C.: A model of the smooth pursuiteye movement with prediction and learning. Appl. Bionics Biomech. 7(2), 109–118(2010)

18. Cauli, N., Falotico, E., Bernardino, A., Santos-Victor, J., Laschi, C.: Correctingfor changes: expected perception-based control for reaching a moving target. IEEERobot. Autom. Mag. 23, 63–70 (2016)

19. Datteri, E., Teti, G., Laschi, C., Tamburrini, G., Dario, P., Guglielmelli, E.:Expected perception: an anticipation-based perception-action scheme in robots.In: Proceedings of the 2003 IEEE/RSJ International Conference on IntelligentRobots and Systems (IROS 2003), vol. 1, pp. 934–939 (2003)

20. Laschi, C., Asuni, G., Teti, G., Carrozza, M.C., Dario, P., Guglielmelli, E.,Johansson, R.: A bio-inspired neural sensory-motor coordination scheme for robotreaching and preshaping. In: The First IEEE/RAS-EMBS International Conferenceon Biomedical Robotics and Biomechatronics, BioRob 2006, pp. 531–536 (2006)

21. Laschi, C., Asuni, G., Guglielmelli, E., Teti, G., Johansson, R., Konosu, H., Wasik,Z., Carrozza, M.C., Dario, P.: A bio-inspired predictive sensory-motor coordinationscheme for robot reaching and preshaping. Auton. Robots 25, 85–101 (2008)

22. Seok, K.H., Kim, Y.S.: A new robot motion authoring method using HTM. In:International Conference on Control, Automation and Systems, pp. 2058–2061(2008)

130 M. Kirtay et al.

23. Mai, X., Zhang, X., Jin, Y., Yang, Y., Zhang, J.: Simple perception-action strategybased on hierarchical temporal memory. In: 2013 IEEE International Conferenceon Robotics and Biomimetics (ROBIO), pp. 1759–1764, 12–14 December 2013

24. Nguyen, V.A., Starzyk, J.A., Tay, A.L.P., Goh, W.: Spatio-temporal sequencelearning of visual place cells for robotic navigation. In: The 2010 InternationalJoint Conference on Neural Networks (IJCNN), pp. 1–8, 18–23 July 2010

25. Hawkins, J., George, D.: Hierarchical temporal memory: Concepts, theory andterminology. Technical report, Numenta (2006)

26. Mountcastle, V.B.: The columnar organization of the neocortex. Brain 120(4),701–722 (1997)

27. Hawkins, J., Ahmad, S.: Why Neurons Have Thousands of Synapses. A Theory ofSequence Memory in Neocortex. Frontiers in Neural Circuits (2015)

28. Cui, Y., Surpur, C., Ahmad, S., Hawkins, J.: Continuous online sequencelearning with an unsupervised neural network model (2015). arXiv preprint:arXiv:1512.05463

29. Hawkins, J., Ahmad, S.: Properties of Sparse Distributed Representations andtheir Application to Hierarchical Temporal Memory (2015). arXiv preprint:arXiv:1503.07469v1

Computer-Aided Biomimetics

Ruben Kruiper(&), Jessica Chen-Burger, and Marc P.Y. Desmulliez

Heriot-Watt University, Edinburgh EH14 4AS, Scotland, UKrk22@hw.ac.uk

Abstract. The interdisciplinary character of Bio-Inspired Design (BID) hasresulted in a plethora of approaches and methods that propose different types ofdesign processes. Although sustainable, creative and complex system designprocesses are not mutually incompatible they do focus on different aspects ofdesign. This research defines areas of focus for the development of computa-tional tools to support biomimetics, technical problem solving throughabstraction, transfer and application of knowledge from biological models. Anoverview of analysed literature is provided as well as a qualitative analysis of themain themes found in BID literature. The result is a set of recommendations forfurther research on Computer-Aided Biomimetics (CAB).

Keywords: Bio-Inspired Design (BID) � Biomimicry � Biomimetics � Bionics �Design theory � Innovation � Invention � Computer Aided Design (CAD)

1 Introduction

Bio-Inspired Design (BID) is associated with the application of “nature’s designprinciples” to “create solutions that help support a healthy planet” [1]. Vandevenne(2011) added that the premise of bio-inspired design allows the finding and use ofexisting, optimal solutions. Additional factors include the sustainable image, associa-tion to an organism and ‘high’ probability of leapfrog innovations [2]. Although thetechnology that evolved in nature is not always ahead of man-made technology, theassumption that organisms have ways of implementing functions more efficiently andeffectively than we do is assumed to be true in many cases [3, 4].

However, the search for biological systems and transfer of knowledge is non-trivial.Most Bio-Inspired Design (BID) methods use function, many in terms of the ‘func-tional basis’ to model biological functions and flows, as the analogical connectionbetween biology and engineering [5]. This paper identifies the insufficient definition offunction throughout BID approaches as one of the main obstacles for knowledgetransfer. The Biomimicry 3.8 Institute for example refers to a function as “the roleplayed by an organism’s adaptations or behaviours that enable it to survive. Impor-tantly, function can also refer to something you need your design solution to do” [6].

This paper identifies the main areas of focus for research on Computer-AidedBiomimetics (CAB). Firstly, a definition of biomimetics is given that reflects its focuson technical problem-solving. Secondly, important notions from existing literature onBID are outlined based on themes for qualitative analysis. Finally, the results of thethematic analysis provide recommendations for further research on computationaldesign tools for biomimetics.

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 131–143, 2016.DOI: 10.1007/978-3-319-42417-0_13

2 Definitions

Pahl and Beitz (2007) noted that the analysis of biological systems can lead to usefuland novel technical solutions, referring to bionics and biomechanics as fields thatinvestigate the connection between biology and technology [7]. Biomimetics is oftenregarded as a synonym for BID, bionics and biomimicry, and refers to the transfer ofbiological knowledge from nature to technical applications [8–10]. As BID is anumbrella term we adopt the following definitions, based on Fayemi et al. (2014) [11]:

Biomimetics: Interdisciplinary creative process between biology and technology,aiming to solve technospheric problems through abstraction, transfer and applica-tion of knowledge from biological models.Biomimicry/Biomimesis: Philosophy that takes up challenges related to resilience(social, environmental and economic ones), by being inspired from living organ-isms, particularly on an organizational level.

Bio-inspiration can be useful in early design stages, e.g. the fuzzy front end, whenthe design process has no clear direction. In later stages the search for functional,biological analogies becomes less urgent and the focus lies on transferring quantitativeknowledge from biological systems to technical problems. Nachtigall (2002) introducedthe term technical biology as a field in biology that describes and analyses structures,procedures and principles of evolution found in nature using methodological approachesfrom physics and engineering sciences [12]. According to Speck et al. (2008) technicalbiology is the basis of many biomimetic projects. It “allows one to understand thefunctioning of the biological templates in a quantitative and technologically basedmanner” [13]. However, according to Julian Vincent (personal communication, March31, 2016) technical biology has been known as biomechanics for decades.

Martone et al. (2010) stated that, although biological models provide multifunc-tional properties with high potential for biomimetic applications, they have hardly beenstudied quantitatively in terms of their form-structure-function to transfer knowledge[14]. Figure 1 gives an overview of how methods approach BID. The next section is aliterature review that aims to summarise important notions in a structure that is looselybased on seven themes for qualitative analysis. The themes represent areas that require

Fig. 1. Approaches of methods to ‘enhance’ BID (based on [5])

132 R. Kruiper et al.

attention in research on CAB: the transfer guidelines, the notion of function in literatureand our definition of biomimetics – abstraction, transfer and application of knowledgefrom biological models.

3 Bio-Inspired Design Literature Review

3.1 Direction

Similar to Biomimicry 3.8, Helms et al. (2009) and Speck et al. (2008) indicated twopossible directions of a biomimetics process [13, 15, 16]. The first is a problem driven,top-down process. If the knowledge of identified biological solutions is too little, Specket al. propose an extension that involves further research on the biological system. Thesecond is a solution driven, bottom-up process that searches for technical applicationsof a specific biological solution. Main characteristics are [13]:

Bottom-up (biological solution to technical problem)

• Abstraction often proves to be one of the most important as well as difficult steps• Often several iterative loops• 3–7 years for bottom-up to final product• Possibly multiple implications of technology• Potentially highly inventive

Top-down (technical problem to biomimetics)

• Existing product: Initially define problem and constraints, then search for possiblebiological solutions

• One or two most appropriate selected for further analysis• 6–18 months top-down to functional prototype• Usually not very innovative

Extended top-down

• Existing knowledge of biological model is too low, further research is required• Can be as highly inventive as bottom-up• Smaller range of implications of technology compared to bottom-up• 1–5 years typically, in between top-down and bottom-up

Helms et al. (2009) noted that the steps in these processes in practice do notnecessarily occur in the prescribed sequence. Once a biological solution is selected inthe problem-driven process, the design process tends to be fixated around this onesolution.

Furthermore, they identified several common errors and practice patterns thatemerged during classroom projects on bio-inspired design [15]. Both are listed inTable 1.

Computer-Aided Biomimetics 133

3.2 Formulate Objectives

Lindemann et al. (2004), Stricker (2006), Inkermann et al. (2011) proposed a procedurethat starts with a goal definition, based on the the Müncher Vorgehensmodell(MVM) by Lindemann (2003) [17–20]. The goal, as well as the later search foralternative solutions, are at a relatively abstract level. Stricker (2006) identified severalproblems during BID processes listed in Table 1 [18].

By our definition, the goal in biomimetics is solving technical problems. Stricker(2006) noted that knowing the type of problem you are dealing with, helps planning asolution route (based on Dorner 1987, Badke-Schaub et al. 2004, Erhlenspiel 2003) [18]:

Synthesis barrier: goal is known, but the means to achieve it are not knownDialectical barrier: goal is not known, ambiguous, multi-faceted, interrelated ortoo genericSynthesis and dialectical barrier: combination of above, but knowledge notsufficientInterpolation problem: means and goal known, but not clear how to achieve thegoal

Table 1. Mistakes and trends that often occur during a bio-inspired design process (based on[15, 18]).

Stricker (2006) errors Helms et al. (2009) errors Helms et al. (2009) trends

• Over-reduction of context

• Neglecting form and

processes by focusing mainly

on structure

• Generalising where different

functions originate and simply

copying existing

foreknowledge

• Structure is expected to be

directly transferable (same

elements, same relations)

• Neglecting of constraints

• Vaguely defined problems

• Poor problem-solution

pairing

• Oversimplification of

complex functions

• Using ‘off-the-shelf’

biological solutions

• Simplification of

optimisation problems

• Solution fixation

• Misapplied analogy

• Improper analogical transfer

• Mixing problem-driven versus

solution-driven approach

• Usually focus on structure

instead of function

• The focus on function,

usually problem-driven

• Solution-driven generated

multi-functional designs

• Partial problem definition leads

to compound solutions for new-

found sub-problems.

• Not many problems are framed

as optimisation problems

• Tendency to choose known

biological solutions

• Choice of biological solutions

hard, too many or not enough

134 R. Kruiper et al.

In BID both problem and solution decompositions are transferred [21, 22]. “BIDoften involves compound analogies, entailing intricate interaction between problemdecomposition and analogical reasoning” [21].

3.3 Search for Analogies

Analogies in BID can be useful for solution generation, design analysis and explanation[5, 10]. To validate applicability of analogies through similarity, Inkermann et al.(2011) distinguish four types of similarities [19]:

Formal similarity: same rules and physical principlesFunctional similarity: similar functionStructural similarity: similar structural designIconic similarity: similar form or shape

Databases are a common approach to store biological analogies, usually indexed byfunction. According to Yen et al. “a designer can compare the functions of what theyare designing and also compare the structures and behaviours of their design tobiological systems” [23]. Hill (1997) classified 191 biological systems into 15descriptive technical and biological abstractions on basis of 5 general functions and 3types of transactions: energy, information and matter [18, 20]. DANE, SAPPhIRE andthe more recent Biologue system are databases indexed using theStructure-Behaviour-Function (SBF) model [23–25]. Wilson et al. (2008, 2009) andLiu et al. (2010) used an ontology based knowledge modelling approach to reusestrategies for design [26–28].

Natural language approaches are another way to search for relevant biologicalanalogies. Shu et al. (2014) proposed a semi-automatic search method using functionalkeywords [2, 29]. According to Vandevenne (2011) these studies indicate that repre-sentation of analogues in natural language format should be considered as input forfiltering, analysis and transfer. “Automated characterization of biological strategies,and of the involved organisms, enables a scalable search over large databases” [2].

Yen et al. (2014) noted that the search strategy for biological systems is a problemarea. To make the search process more efficient, Vandevenne (2011) proposedsearching on basis of function and further specification of behaviour and structurebefore commencing knowledge transfer. Other areas that require attention are methodsfor teaching analogical mapping, evaluating good analogies, good designs and gooddesign problems [23].

An example of research on better understanding and supporting the use of bio-logical analogies is the work by Linsey et al. (2014). According to them design byanalogy is powerful, but a difficult cognitive process. To overcome cognitive bias andchallenges they propose several principles and design heuristics. An example is pro-viding uncommon examples to overcome design fixation [30]. The use of analogies andheuristics may be useful for biological inspiration and even supporting transfer ofknowledge. However, “databases can only record history and cannot deduce newrelationships” [31]. Therefore, Vincent (2002, 2006, 2009) proposes the use of TRIZ tofacilitate the comprehension of biological systems [8, 9, 32]. Lindemann et al.

Computer-Aided Biomimetics 135

Gramann (2004) provide a structured, associative checklist to support deduction oftechnical analogies from biological systems that is loosely based on TRIZ [17]. Vincentet al. developed BioTRIZ, a reduced form of TRIZ that inventive principles to bio-logical dialectical problems [8, 33]. “Studying 5,000 examples, the conflict matrix wasreduced from 39 conflict elements to 6 elements that appear in both biology andengineering, and a 6 by 6 contradiction matrix that contains all 40 of the inventiveprinciples was created. These 6 conflict elements are substance, structure, time, space,energy/field, and information/regulation” [5].

3.4 Function

Fratzl (2007) noted biomimetics studies start with the study of structure-functionrelationships in biology; the mere observation of nature is not sufficient [34]. Vincent(2014) noted defining problems in functions is key to knowledge transfer from biologyto technology [4]. According to Stone et al. (2014) natural systems have to be modelledusing normal function modelling techniques to use function as an analogical connec-tion [5]. An example is using the functional basis that defines function as “a descriptionof a device or artefact, expressed as the active verb of the sub-function” [35]. However,Deng (2002) identified two types of functions in literature [36]:

Purpose function: is a description of the designer’s intention or the purpose of adesign (not operation oriented).Action function: is an abstraction of intended and useful behaviour that an artefactexhibits (operation oriented).

Furthermore, Deng (2002) stated that action functions can be described semanti-cally or syntactically. Chakrabarti et al. (2005) adopted this view, an overview ofdefinitions is given in Table 2. For SAPPhIRE Chakrabarti et al. (2005) view functionas “the intended effect of a system (Chakrabarti & Bligh, 1993) and behaviour as thelink between function and structure defined at a given level. Thus, what is behaviour isspecific to the levels at which the function and structure of a device are defined” [25].

Nagel (2014) used a semantic view and noted that a function represents an oper-ation performed on a flow of material, signal or energy [37]. According to Nagel (2014)the use of functional design methods for BID offers several advantages [38]:

Table 2. Action functions from the perspective of semantic and syntactic formulation - based on[25, 36].

Semantic views Syntactic views

Functions as input/output of energy,material and information

Functions using informal representation (e.g.verb-noun transformation)

Functions as a change of state of anobject or system.

Functions using formal representation (e.g.mathematical transformation).

136 R. Kruiper et al.

• archival and transmittal of design information• reduces fixation on aesthetic features or a particular physical solution• allows one to define the scope or boundary of the design problem as broad or

narrow as necessary• encourages one to draw upon experience and knowledge stored in a• database or through creative methods during concept generation

Vattam et al. (2010) adopted the formal definition for function used in Structure-Behaviour-Function (SBF) models, which was developed in AI research on design tosupport automated, analogical design [24]. This is a semantic view on functions forSBF: “A function is represented as a schema that specifies its pre-conditions and itspost-conditions” [39].

Goel (2015) noted that BID presents a challenge for Artificial Intelligence(AI) fields such as knowledge representation, knowledge acquisition, memory, learn-ing, problem solving, design, analogy and creativity [40]. Vandevenne (2011) notedthat the instantiation of functional models is a labour-intensive task that requiresbiology and engineering knowledge [2].

3.5 Abstraction

Abstraction, the reduction of context, is an important aspect in all BID methodologies.According to Vattam et al. (2010) successful BID requires rich and multimodal rep-resentations representations of systems during design. Such representations areorganised at different levels of abstraction. They “explicitly capture functions andmechanisms that achieve those functions on the one hand, and the affordances andconstraints posed by the physical structures for enabling the said mechanisms on theother hand” [41]. Table 3 is an adaption of the lists by Fayemi et al. (2014) onrequirements for abstraction from both a theoretical and a practical perspective.

Abstraction eases the implementation of biological solutions [4, 20]. Chakrabarti(2014) found that exploration at higher levels of abstraction has a greater impact onnovelty of solutions generated [22]. Lindemann et al. (2004) noted that, if one finds no

Table 3. Requirements for abstraction - based on [11]

Computer-Aided Biomimetics 137

technical analogies, the level of abstraction as well as the feasibility of solving theproblem should be reconsidered [17].

Diagrammatic representations of biological systems lead to generation of more andbetter design ideas than textual representations [42]. Descriptive accounts of designlead to more effective educational techniques and computational tools for supportingdesign, advantages include: realism, accuracy of predictions and accuracy of designbehaviour [15].

3.6 Transfer

In order to deduce new relationships, Vincent (2014) suggested using a descriptiveapproach of technology to ease knowledge transfer. “There is very little indication ofhow one can take a concept from its biological context and transfer it to an engineeringor technical context” [31]. Differences in context, high complexity, high amount ofinterrelated and integrated multifunctional elements make transfer the most difficultpart of BID. Lack of biological training increases the difficulty of transferringknowledge. Chakrabarti (2014) defined transfer as “the reproduction of informationfrom a model of a biological system in a model or prototype for a technical system”[22]. There are four kinds of exchanges in bio-inspiration [22, 42]:

1. Use of analogy as idea stimulus2. Exchange of structures, forms, materials3. Exchange of functions, and processes4. Exchange of knowledge about processes, information, chaos

With the aim of initiating work in systematic biomimetics Chakrabarti et al.(2005) developed a generic model for representing causality of natural and artificialsystems [25, 42]. Using this model they analysed existing entries of biological systemsand corresponding technical implementations based on similarities and identifiedseveral mechanisms of transfer [42]. Only the transfer of physical effects and thetransfer of state changes of processes were however considered.

Vandevenne (2011) notes that DANE forces designers to understand the systems ofcandidate biological strategies in detail, as well as the obtained abstraction. Thisfacilitates knowledge transfer and communication [2].

3.7 Application

Helms et al. (2010) sketched a macro-cognitive information processing model of BIDto support the development of computational tools that support this form of design.They noted the complex interplay between problem definition and analogy mapping forknowledge transfer. Salgueiredo (2013) applied C-K theory to BID in early designphases and showed that biological knowledge is usually implemented once the tradi-tional path is blocked. In this case the acquisition of biological knowledge is required tofind ‘unexpected properties’. She noted that the systematic implementation of BID incompanies requires a different form of knowledge management. In essence, C-K theoryis a theory of design knowledge management [10, 43]:

138 R. Kruiper et al.

• The theory captures the iterative expansion of knowledge and enrichment of designconcepts.

• Concept-space can only be partitioned into restrictive and/or expanding.• partitions, relating to constraints to existing knowledge and addition of new

knowledge.• The theory is domain-independent, which is useful in supporting/analysing

multi-disciplinary design.

These findings are in line with the co-evolution of problems and solutions men-tioned by Chakrabarti (2014) and the generation of compound solutions through iter-ative analogy and problem decomposition mentioned by Vattam et al. (2010) [22, 24].Salgueiredo (2013) concludes that “biological knowledge does not offer solutions, itstimulates the reorganization of knowledge bases, creating bridges between differentdomains inside the traditional knowledge. This conclusion is important for reducingthe risks of idolizing nature processes and systems” [10].

4 Results Thematic Analysis

Based on the seven themes 40 features were identified that are of interest for CAB.These features repeatedly occurred in the literature described in the previous sec-tion. 190 relationships between them were documented. Relationships were weightedwith factor 1, 3 or 5, based on their relative importance. Relations weighted with factor 3or 5 were annotated. Figure 2 shows a selection of visualisations, features are repre-sented as nodes and relationships between them as edges. In the software package Gephia radial axis layout was used to create these visualisations, which supports qualitativeanalysis of similarity for the determination of main themes [44, 45]. As one mightexpect, nodes like Transfer and Analogy are highly inter-connected with the network.Less expected are Holistic approach, Validation and Confusion of terms. Other factorsof interest are Transfer-impediment, For computational purposes and the differences inthe sub-groups of abstraction. The annotated edges were used to determine featureinfluence and overlap. Main areas of focus we found and specific points of attention are:

• Holistic approach– Iteratively improve problem/goal and expand knowledge supports validation– Alternating problem-driven and solution-driven approach– Attention to heuristics and principles, e.g. [30]– Attention to macro-cognitive model, e.g. [21, 24]

• Abstraction– Abstraction should maintain relevant constraints and affordances– Use various levels of abstraction for complexity and multi-functional problems– Attention to problem definition and problem types– Attention to validation of abstracted entries in knowledge base

• Analogies– Representation of relevant knowledge for transfer– Attention to analogy mapping and analogical reasoning– Attention to analogy representation: natural language (descriptive), diagrams

Computer-Aided Biomimetics 139

• Confusion of terms and computational approach– Function is key and needs accurate definition for:

• Problem decomposition• Wrong interpretation of interrelated terms (SBF)• Improve search for analogies and transfer

– Automated characterisation improves scalability of knowledge base• Transfer (impediments) and validation

– Descriptive biological knowledge support realism and accuracy– Attention to supporting possible lack of biological knowledge– Attention to validation of analogies, e.g. through similarity

5 Discussion and Outlook

Some attention points for Computer Aided Biomimetics (CAB) were briefly introducedin the previous section. These are derived from a simplified qualitative analysis of theliterature addressed in Sect. 3. The focus during analysis was on finding genericguidelines for the development of computational tools that support a BID process. Theusefulness of these findings is left to the reader to decide.

Fig. 2. Visualisations of features and relationships found during thematic analysis. The groupsdisplay a selected node and their most influential neighbours. The highlighted nodes in A-D arein alphabetical order: Confusion of terms, Holistic approach, Abstraction of problem, Abstractionof function.

140 R. Kruiper et al.

In general, more attention should be paid to definitions, constraints and affordances,optimisation problems, the holistic approach and the validation mechanism. Validationfor example is only possible when the required, relevant knowledge is accessible.However, the extended process described by Speck et al. (2008) implies that knowl-edge expansion drastically affects development time [13]. Investing additional timemay not be desirable and Salgueiredo (2013) is right in concluding that BID should notbe idolised [10].

BioTRIZ has, based on a non-systematic review of the literature on biologicalprocesses, changed the usual classification into ‘matter, energy and information’ to‘substance, structure, time, space, energy/field, and information/regulation’ to allowbetter abstraction of biological phenomena. Further research is to aim at using TRIZtools for CAB and Inventive Design [46] in cooperation with Julian Vincent and DenisCavallucci.

Acknowledgements. The authors thank Julian Vincent and Denis Cavallucci for their advice.

References

1. The Biomimicry Institute Toolbox. http://toolbox.biomimicry.org/. Accessed 14 Mar 20162. Vandevenne, D., Verhaegen, P.-A., Dewulf, S., Duflou, J.R.: A scalable approach for the

integration of large knowledge repositories in the biologically-inspired design process. In:Proceedings 18th International Conference on Engineering Design (ICED 2011), vol. 6,Lyngby/Copenhagen, Denmark (2011)

3. Fish, F.E., Beneski, J.T.: Evolution and bio-inspired design: natural limitations. In: Goel, A.K., McAdams, D.A., Stone, R.B. (eds.) Biologically Inspired Design, Chap. 12,pp. 287-312. Springer, London (2014)

4. Vincent, J.F.V.: Biomimetics in architectural design. In: Intelligent Buildings International,pp. 1–12 (2014)

5. Stone, R.B., Goel, A.K., McAdams, D.A.: Chartering a course for computer-aidedbio-inspired design. In: Goel, A.K., McAdams, D.A., Stone, R.B. (eds.) BiologicallyInspired Design, Chap. 1, pp. 1–16. Springer, London (2014)

6. The Biomimicry Institute Toolbox – Learn more. http://toolbox.biomimicry.org/core-concepts/function-and-strategy/. Accessed 14 Mar 2016

7. Pahl, G., Beitz, W., Feldhusen, J., Grote, K.-H.: Engineering Design: A SystematicApproach, 3rd edn. Springer, Berlin (2007) ISBN 978-1-84628-319-2

8. Vincent, J.F.V., Bogatyreva, O.A., Bogatyreva, N.R., Bowyer, A., Pahl, A.-K.:Biomimetics: its practice and theory. J. Roy. Soc. Interface 3(9), 471–482 (2006)

9. Vincent, J.F.V.: Biomimetics–a review. Proc. Inst. Mech. Eng., Part H: J. Eng. Med. 223(8),919–939 (2009)

10. Salgueiredo, C.F: Modeling biological inspiration for innovative design. In: 20thInternational Product Development Management Conference, Paris, France, June 2013

11. Fayemi, P.-E., Maranzana, N., Aoussat, A., Bersano, G.: Bio-inspired designcharacterisation and its links with problem solving tools. In: Design Conference,Dubrovnik, Croatia, May 2014

12. Nachtigall, W.: Bionik: Grundlagen und Beispiele für Naturwissenschaftler und Ingenieure.Springer, Heidelberg (2002). ISBN 978-3-642-18996-8

Computer-Aided Biomimetics 141

13. Speck, T., Speck, O.: Process sequences in biomimetic research. Des. Nat. 4, 3–11 (2008)14. Martone, P.T., Boiler, M., Burgert, I., Dumais, J., Edwards, J., Mach, K., Rowe, N.,

Rueggeberg, M., Seidel, R., Speck, T.: Mechanics without muscle: biomechanicalinspiration from the plant world. Integr. Comp. Biol. 50, 888–907 (2010)

15. Helms, M., Vattam, S.S., Goel, A.K.: Biologically inspired design: process and products.Des. Stud. 30, 606–622 (2009)

16. The Biomimicry Institute. http://toolbox.biomimicry.org/methods/integrating-biology-design/. Accessed 21 Mar 2016

17. Lindemann, U., Gramann, G.: Engineering design using biological principles. In:International Design Conference, Dubrovnik, Croatia, May 2004

18. Stricker, H.M.: Bionik in der Produktentwicklung unter der Berücksichtigung menschlichenVerhaltens. Ph.D. thesis, Technical University München (2006)

19. Inkermann, D., Stechert, C., Löffler, S., Victor, T.: A new bionic development approachused to improve machine elements for robotics applications. In: Proceedings of IASTED(2011)

20. Gramann, J.: Problemmodelle und Bionik als Methode. Ph.D. thesis, TU München (2004)21. Goel, A.K., Vattam, S.S., Wiltgen, B., Helms M.: Information-processing theories of

biologically inspired design. In: Goel, A.K., McAdams, D.A., Stone, R.B. (eds.)Biologically Inspired Design, pp. 127–152. Springer, London (2014)

22. Chakrabarti, A.: Supporting analogical transfer in biologically inspired design. In: Goel, A.K., McAdams, D.A., Stone, R.B. (eds.) Biologically Inspired Design, pp. 201–220.Springer, London (2014)

23. Yen, J., Helms, M., Goel, A.K., Tovey, C., Weissburg, M.: Adaptive evolution of teachingpractices in biologically inspired design. In: Goel, A.K., McAdams, D.A., Stone, R.B. (eds.)Biologically Inspired Design, pp. 153–200. Springer, London (2014)

24. Vattam, S., Wiltgen, B., Helms, M., Goel, A.K., Yen, J.: DANE: fostering creativity in andthrough biologically inspired design. In: Taura, T., Nagai, Y. (eds.) International Conferenceon Design Creativity, Japan, pp. 115–122. Springer, London (2010)

25. Chakrabarti, A., Sarkar, P., Leelavathamma, B., Nataraju, B.S.: A functional representationfor aiding biomimetic and artificial inspiration of new ideas. AIE EDAM 19, 113–132(2005)

26. Yim, S., Wilson, J.O., Rosen, D.W.: Development of an ontology for bio-inspired designusing description logics. In: International Conference on PLM (2008)

27. Wilson, J.O., Chang, P., Yim, S., Rosen, D.W.: Developing a bio-inspired design repositoryusing ontologies. In: Proceedings of IDETC/CIE (2009)

28. Liu, X., Rosen, D.W., Yu, Z.: Ontology based knowledge modeling and reuse approach inproduct redesign. In: IEEE IRI, Las Vegas, Nevada, USA, August 2010

29. Shu, L.H., Cheong, H.: A natural language approach to biomimetic design. In: Goel, A.K.,McAdams, D.A., Stone, R.B. (eds.) Biologically Inspired Design, pp. 29–62. Springer,London (2014)

30. Linsey, J.S., Viswanathan, V.K.: Overcoming cognitive challenges in bioinspired design andanalogy. In: Goel, A.K., McAdams, D.A., Stone, R.B. (eds.) Biologically Inspired Design,pp. 221–245. Springer, London (2014)

31. Vincent, J.F.V.: An ontology of biomimetics. In: Goel, A.K., McAdams, D.A., Stone, R.B.(eds.) Biologically Inspired Design, pp. 269–286. Springer, London (2014)

32. Vincent, J.F.V., Mann, D.L.: Systematic technology transfer from biology to engineering.Philos. Trans. Roy. Soc. Lond. A 360, 159–173 (2002)

33. Bogatyrev, N., Bogatyrev, O.A.: TRIZ-based algorithm for Biomimetic design. ProcediaEng. 131, 377–387 (2015)

142 R. Kruiper et al.

34. Fratzl, P.: Biomimetic materials research: what can we really learn from nature’s structuralmaterials? J. Roy. Soc. Interface 4(15), 637–642 (2007)

35. Stone, R.B., Wood, K.L.: Development of a functional basis for design. J. Mech. Design 122(4), 359–370 (2000)

36. Deng, Y.-M.: Function and behavior representation in conceptual mechanical design. Artif.Intell. Eng. Des. Anal. Manuf. 16, 343–362 (2002)

37. Nagel, J.K.S.: A thesaurus for bioinspired engineering design. In: Goel, A.K., McAdams, D.A., Stone, R.B. (eds.) Biologically Inspired Design, pp. 63–94. Springer, London (2014)

38. Nagel, J.K.S., Stone, R.B., McAdams, D.A.: Function-based biologically inspired design. In:Goel, A.K., McAdams, D.A., Stone, R.B. (eds.) Biologically Inspired Design, pp. 95–126.Springer, London (2014)

39. Goel, A.K., Rugaber, S., Vattam, S.S.: Structure, Behavior and Function of ComplexSystems: The SBF Modeling Language. https://home.cc.gatech.edu/dil/uploads/SBF2.pd

40. Goel, A.K.: Biologically inspired design: a new paradigm for AI research on computationalsustainability? In: Computational Sustainability, Workshop Papers (2015)

41. Vattam, S.S., Helms, M., Goel, A.K.: Biologically inspired design: a macrocognitiveaccount. In Proceedings of the ASME IDETC/CIE (2010)

42. Sartori, J., Pal, U., Chakrabarti, A.: A methodology for supporting “transfer” in biomimeticdesign. AI Eng. Des. Anal. Manuf. 24, 483–506 (2010)

43. Hatchuel, A., Weil, B.: C-K design theory: an advanced formulation. Res. Eng. Des. 19,181–192 (2009)

44. Gephi, Open Graph Viz Platform. https://gephi.org/45. Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2),

77–101 (2006)46. Cavallucci, D., Rousselot, F., Zanni, C.: An ontology for TRIZ. Procedia Eng. 9, 251–260

(2009)

Computer-Aided Biomimetics 143

A Neural Network with Central Pattern GeneratorsEntrained by Sensory Feedback Controls

Walking of a Bipedal Model

Wei Li(✉), Nicholas S. Szczecinski, Alexander J. Hunt, and Roger D. Quinn

Department of Mechanical and Aerospace Engineering, Case Western Reserve University,Cleveland, OH 44106-7222, USA

wxl155@case.edu

Abstract. A neuromechanical simulation of a planar, bipedal walking robot hasbeen developed. It is constructed as a simplified musculoskeletal system to mimicthe biomechanics of the human lower body. The controller consists of a dynamicneural network with central pattern generators (CPGs) entrained by force andmovement sensory feedback to generate appropriate muscle forces for walking.The CPG model is a two-level architecture, which consists of separate rhythmgenerator (RG) and pattern formation (PF) networks. The presented planar bipedmodel walks stably in the sagittal plane without inertial sensors or a centralizedposture controller or a “baby walker” to help overcome gravity. Its gait is similarto humans’ with a walking speed of 1.2 m/s. The model walks over small obstacles(5 % of the leg length) and up and down 5° slopes without any additional higherlevel control actions.

Keywords: Biologically inspired · Central pattern generator · Sensoryfeedback · Bipedal walking

1 Introduction

It is generally accepted that basic rhythmic motor signals driving walking and otherforms of locomotion in various animals are generated centrally by the neural networksin the spinal cord (reviewed by [1–3]). These neural networks, capable of producingcoordinated and rhythmic locomotor activity without any input from sensory afferentsor from higher control centers are referred to as central pattern generators (CPGs). Catswith transection of spinal cord and removal of major afferents can still produce rhythmiclocomotor patterns [4, 5]. Despite lacking clear evidence of CPGs in humans, observa‐tions in patients with spinal cord injury provide some support [6]. Sensory input alsoplays an important role in locomotion. Locomotion is the result of dynamic interactionsbetween the central nervous system, body biomechanics, and the external environment[7]. As a closely coupled neural control system, sensory feedback provides the infor‐mation about the status of the biomechanics and the relationship between the body andthe external environment to make locomotion adaptive to the real environment. Sensoryinput could reinforce CPG activities, provide signals to ensure correct motor output forall body parts, and entrain the rhythmic pattern to facilitate phase transition when a

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 144–154, 2016.DOI: 10.1007/978-3-319-42417-0_14

proper body posture is achieved [8, 9]. There is strong evidence [10, 11] to suggest thatsensory feedback affects motoneuron activity through common networks, rather thandirectly through reflex pathways acting on specific motoneurons. It implies sensoryfeedback and CPGs are highly integrated networks in the spinal cord.

Several 2D bipedal walking models have been previously developed with controllerscontaining CPGs and reflexes. Some simulation models [12, 13] had over-simplifiedmulti-link structures driven by motors at joints, which lack the viscoelasticity of muscu‐loskeletal structures. This lack provides poor mechanical properties that may reducestability. Some models [14, 15] adopt more realistic musculoskeletal structures, but theirCPG activities are not affected by sensory feedback. Instead, sensory afferents areconnected directly with motoneurons. This is contradictory to findings that sensoryfeedback could strongly affect CPG activities [11, 12]. Geyer et al. [16] produced real‐istic human walking and muscle activities in a 2D model. They claim that their modelonly relies on muscle reflexes without CPGs or any other neural networks. It is realizedby two sets (swing phase and stance phase) of coupled equations for every muscle toproduce human walking and muscle activities. However, these equations could beviewed as abstract mathematical expressions of neural networks controlling muscleactivities and the transition between two phases works like a finite-state machine. Morerecently, CPGs were added to a similar muscle-reflex model to control muscles actuatingthe hip joint [17]. Both of these works rely on precisely tracking muscle and otherbiomechanical parameters and provide limited insight into how the central nervoussystem is structured in humans. CPGs have also been applied to physical bipedal robots[18, 19], but usually these CPGs are abstract mathematical oscillators tuned to generatedesirable trajectories of joints angles or torques, and the phases of the oscillators couldsimply be reset by reflexes. Klein and Lewis [20] claimed that their robot is the first thatfully models human walking in a biologically accurate manner. But it has to rely on ababy walker-like frame to prevent it from falling down in the sagittal plane. Its kneesremain bent distinctly before the foot touches the ground and its heels are always off theground. It looks like it is pushing a heavy box, fundamentally different from a human’snormal gait.

In this paper we present a biologically rooted neuromechanical simulation model ofa planar, bipedal walker. It has a musculoskeletal structure and is controlled by a two-level CPG neural network incorporating biological reflexes via sensory afferents. Themodel is developed in Animatlab, which can model biomechanical structures andbiologically realistic neural networks, and simulate their coupled dynamics [21]. Wedemonstrate that natural-looking, stable walking can be achieved by this relativelysimple but biologically plausible controller combined with a human like morphologicalstructure.

2 Structure of the Model

We based the biomechanical structure of our model on the human lower body, whichhas been shaped and selected by millions of years’ evolution, and is energetically closeto optimal [22]. The planar skeleton is made up of rigid bodies, and is actuated by

A Neural Network with CPGs Entrained by Sensory Feedback Controls 145

antagonistic pairs of linear Hill muscles. The morphology structure employed signifi‐cantly reduces the control effort with its inherent self-stability and provides a basis forcomparison to the real human gait. The simulated biped (Fig. 1) is modeled as a pelvissupported by two legs. Each leg has 3 segments (thigh, shank and foot), which areconnected by 3 hinge joints (hip, knee and ankle) and actuated by 6 muscles. The footis composed of two parts connected by one hinge joint, spanned by a passive spring withdamping (spring constant = 16 kN/m, damping coefficient = 20 N·m/s). This makes thefoot flexible, which is the basis of the toe-off motion. The foot has a compliance of10 μm/N to mimic the soft tissue of a human foot. The mass and length ratio of differentsegments (except the foot length) are based on human data [23]. The total mass of themodel is 50 kg and its height is approximately 1 m (leg length is 0.84 m). The mass ofthe pelvis, thigh, shank and foot are 25 kg, 8 kg, 4 kg and 0.5 kg. The lengths of thethigh, shank and foot are 0.42 m, 0.42 m and 0.24 m.

Fig. 1. Left: biomechancal model in Animatlab. Right: the muscle and skeleton system of thesimulated biped.

3 Neural Control System

3.1 Neural Model

The neurons are leaky “integrate-and-fire” models. Each neuron is modeled as a “leakyintegrator” of currents it receives, including synaptic current and injected current:

(1)

where is the membrane potential, is the membrane capacitance, is negativecurrent due to the passive leak of the membrane, is the current from total synaptic

146 W. Li et al.

input to the neuron, is the current externally injected into the neuron. The leakcurrent is:

(2)

where is the leak conductance, is the resting potential. When the membranepotential reaches the threshold value the neuron is said to fire a spike, and is reset to

.Transmitter-activated ion channels are modeled with a time-dependent postsynaptic

conductance . The current that passes channels depends on the reversalpotential of the synapse and the postsynaptic neuron membrane potential :

(3)

If , the synapse is inhibitory and it induces negative inhibitory postsy‐naptic current. If , the synapse is excitatory and it induces positive excitatorypostsynaptic current. The postsynaptic conductance increases to a value with atime delay to the arrival of the presynaptic spike and then decays exponentially to zero:

(4)

where is the initial amplitude of the postsynaptic conductance increase, denotesthe arrival time of a presynaptic spike, is the time delay between the presynapticspike and the postsynaptic response, and is the time constant of the decay rate.

3.2 Half-Center Oscillator

A basic neural pattern generator is a half-center neural oscillator composed of tworeciprocally inhibited neurons. When one neuron spikes, it inhibits the other. Due to thefacilitation of the synapse [24] the repeated spiking of the presynaptic neuron graduallyhampers the synaptic transmission and reduces the postsynaptic conductance so theamplitude of the postsynaptic current gradually declines. The postsynaptic neuron wouldspike again as soon as the total current input recovers to the threshold value. When itstarts to spike it would inhibit the other neuron in the same way.

3.3 CPG

Rybak et al. [25] proposed a two-level CPG architecture composed of a central rhythmgenerator (RG) network controlling several coupled unit pattern formation (UPF)modules in the pattern formation (PF) level. Sensory afferents could access and affectRG and PF networks separately. In this model the CPG network adopts a similar archi‐tecture as shown in Fig. 2. The RG is a half-center oscillator that generates the basicperiodic stance signal for both legs. The PF network contains half-center oscillators as

A Neural Network with CPGs Entrained by Sensory Feedback Controls 147

UPF modules and each joint of each leg (hip, knee, and ankle) has one. Each half-centerin the UPF module is connected to the corresponding extensor and flexor motoneuronsto drive the extensor and flexor muscles for each joint. All half-center oscillators consistof two reciprocally inhibited “integrate-and-fire” neurons as described previously.

Fig. 2. The architecture of the CPG network. Black lines are synaptic connections between CPGneurons. Blue lines are synaptic connections from sensory neurons. (Color figure online)

Each neuron in the RG has an excitatory connection with a hip extension neuron andthe other leg’s hip flexion neuron. It encourages the legs to step in antiphase. At the PFlevel the hip flexion neuron has an excitatory connection with the ankle dorsiflexionneuron and an inhibitory connection with knee extension neuron. The hip extensionneuron has an excitatory connection with the knee extension neuron. This controlleruses a layer of interneurons to combine and filter afferent inputs from different receptors.These interneurons have inhibitory connections to the interneurons in the RG and PFnetworks.

3.4 Sensory Afferents

It is generally agreed that sensory input affects CPG timing directly since stimulationof corresponding afferents entrains or resets the locomotor rhythm [26]. In leggedanimals, there are three main categories of afferents that satisfy this requirement; groupI muscle afferents in extensor muscles, cutaneous afferents in feet, and muscle afferentsaround the hip joint signaling its position. The first two largely depend on leg loadinginformation, and the last is based on movement.

148 W. Li et al.

1. Ground Contact Afferents

Studies show that increasing the load on an infants’ feet while they walk with supportsignificantly increases the duration of the stance phase. It suggests that load receptorreflex activity can strongly modify and regulate gait timing and resides within the spinalcircuits [27]. In our model, separate force sensors located on the plantar surface of theheels and toes separately detect and measure the ground contact force. As shown inFig. 2, heel ground contact sensors excite afferent neurons which make inhibitoryconnections to the hip flexion, knee flexion, and ankle plantarflexion neurons at the PFlevel. They also apply inhibition to the other leg’s stance neuron in the RG. Toe groundcontact sensors inhibit the knee flexion and ankle dorsiflexion neurons at the PF level.

2. Hip Middle Position Afferents

During normal walking the knee extends in swing before the foot touches the ground.The knee reaches its peak flexion between 25 % and 40 % of the swing phase beforeextending. As shown in Fig. 2 in our model the hip middle position afferent is connectedto the knee flexion neuron at the PF level and inhibits it when hip movement passes thepredetermined hip angle during hip flexion.

4 Experiment and Results

In software simulation experiments, we confined the movement of the body to thesagittal plane, but it has no gravitational or pitching support, so it can fall forward orbackward.

4.1 Normal Walking

The model walks at a moderate speed (1.2 m/s) on a rigid surface. Figure 3(a) shows aframe by frame comparison to that of a human. It is interesting to observe that theyappear similar at corresponding phases of the cycle. Figure 3(b) compares hip, knee,and ankle angles between the model and human data during one gait cycle. The model’smotion matches a human’s in several aspects. The range of motion and general shapesof hip, knee and ankle angle curves are similar to a human’s, although the model’s kneemotion lacks initial flexion at early stance phase, and the ankle dorsiflexion is insuffi‐cient. The knee reaches maximum flexion at about 75 % of the cycle and begins to extendbefore the hip reaches the anterior extreme position to prepare for ground contact. Thetransition from stance to swing is at about 60 % of the cycle, when the hip and kneebegins to flex, and ankle reaches maximum plantarflexion. Figure 3(a) also comparesthe motion of the model’s foot with a human’s foot at early, middle and late stancephases.

A Neural Network with CPGs Entrained by Sensory Feedback Controls 149

4.2 Interaction Between Sensory Feedback and CPGs

When the right leg enters stance (frame A in Fig. 3(a)) the right heel load sensor causesthe heel ground contact sensor neuron to spike. This triggers the right leg stance RGneuron, causing right leg hip and knee UPF extension neurons to spike and initiate stance(time A in Fig. 4(c)). At time B in Fig. 3(a) the right heel just leaves ground and rightheel ground contact sensor neuron stops spiking (time B in Fig. 4(a)). The right ankleUPF plantarflexion neuron is released from inhibition and begins to spike, causing rightankle plantarflexion (time B in Fig. 4(c)). The double support phase can be seen justfollowing time C in Fig. 3(a). At this point, both the right toe and the left heel are in

Fig. 3. Comparison of gait between the model and human. (a) Series of frames of the model andhuman at the same points in the gait cycle. (b) Model’s hip, knee, and ankle angles compared withhuman’s in the sagittal plane [28].

150 W. Li et al.

contact with the ground. The left heel ground contact sensor neuron begins to spike (timeC in Fig. 4(b)) and inhibit the right leg stance RG neuron (time C in Fig. 4(c)). The leftleg enters stance and begins to take load from the right leg. Loading of the left leg triggersright leg swing and further encourages stance in the left leg. At time D in Fig. 3(a) theright foot toe part leaves the ground. Right toe sensor neuron stops spiking (time D inFig. 4(a)), releasing right knee UPF flexion neuron and right ankle UPF dorsiflexionneuron from inhibition (time D in Fig. 4(c)). Right knee further flexes and right anklegoes into dorsiflexion. Toward the end of the right leg’s swing phase (time E in Fig. 3(a))the right hip middle position sensor neuron spikes (time E in Fig. 4(a)), inhibiting theright knee UPF flexion neuron from spiking (time E in Fig. 4(c)). This causes the kneeto extend and prepare for ground contact.

Fig. 4. Neuron membrane potentials in the gait cycle. (a) Right leg sensor neuron membranepotentials. GC: ground contact, Mid: middle position. (b) Left leg sensor neuron membranepotentials. (c) Right leg RG and PF neuron membrane potentials. EXT: extension, FLX: flexion,PF: plantarflexion, DF: dorsiflexion.

4.3 Walking on Irregular Terrain

The model can walk on some irregular terrains with the CPG network alone. No highercontrol actions are needed. It can walk over a 45 mm high obstacle, approximately 5 %of the leg length (Fig. 5(a)). It can also walk up a 5° incline (Fig. 5(b)) and walk downa 5° decline (Fig. 5(c)).

A Neural Network with CPGs Entrained by Sensory Feedback Controls 151

Fig. 5. Walking on irregular terrain (a) Walking over a 45 mm high obstacle. (b) Walking up a5° incline. (c) Walking down a 5° decline.

5 Conclusions

In this paper we present a bipedal simulation model that can walk naturally and stably inthe sagittal plane controlled by a biologically inspired neural network. It notably doesnot have inertial sensors or a central posture controller, nor does it use a “baby walker”to oppose gravity. Its structure and actuators are modeled after a human’s lower body.Our model uses Hill’s muscle model as actuators instead of motors at joints, whichmodels the force profile of muscles and stores and dissipates energy during walkingsimilar to humans. A two-component foot structure not only enhances the leg’s groundcontact late in the stance phase but also makes it possible to provide accurate dynamicsensory information of the relation between the foot and ground during stance by usingindividual sensors on toe and heel.

Using a neural network as a controller provides a more tangible and consistentimitation of the human central nervous system related to walking than switching betweentwo sets of abstract equations to coordinate muscle reflexes. Compared to other bipedalmodels or robots adopting CPGs, our neural network has several more biologicallyplausible features. The rhythmic patterns are generated by oscillators composed ofbiological neuron models, not by mathematical oscillators tuned to reproduce the trajec‐tory of joint movement. In our model sensory inputs are all biologically realistic andthey access the CPG network and entrain its output instead of directly modifying activ‐ities of motoneurons. In the two-level architecture the movement of hip joints are

152 W. Li et al.

coordinated under a higher level rhythm generator instead of directly coupling hippattern generators of two legs.

Our developed controller and simulation not only demonstrates that stable bipedalwalking in the sagittal plane can be achieved through a biologically based neuralcontroller, but can also be used as a platform to help researchers test hypotheses relatedto the neural control of human locomotion.

References

1. Guertin, P.A.: Central pattern generator for locomotion: anatomical, physiological, andpathophysiological considerations. Front. Neurol. 3, 4–15 (2013)

2. Mackay-Lyons, M.: Central pattern generation of locomotion: a review of the evidence. Phys.Ther. 82, 69–83 (2002)

3. Ijspeert, A.J.: Central pattern generators for locomotion control in animals and robots: areview. Neural Netw. 21, 642–653 (2008)

4. Grillner, S., Zangger, P.: On the central generation of locomotion in the low spinal cat. Exp.Brain Res. 34, 241–261 (1979)

5. Pearson, K.G., Rossignol, S.: Fictive motor patterns in chronic spinal cats. J. Neurophysiol.66, 1874–1887 (1991)

6. Dimitrijevic, M.R., Gerasimenko, Y., Pinter, M.M.: Evidence for a spinal central patterngenerator in humans. Ann. N. Y. Acad. Sci. 860, 360–376 (1998)

7. Dickinson, M.H., et al.: How animals move: an integrative view. Science 288, 100–106 (2000)8. Pearson, K.G.: Common principles of motor control in vertebrates and invertebrates. Ann.

Rev. Neurosci. 16, 265–297 (1993)9. Rossignol, S., Dubuc, R., Gossard, J.P.: Dynamic sensorimotor interactions in locomotion.

Physiol. Rev. 86, 89–154 (2006)10. McCrea, D.A.: Spinal circuitry of sensorimotor control of locomotion. J. Physiol. 533,

41–50 (2001)11. Pearson, K.G.: Generating the walking gait: role of sensory feedback. Prog. Brain Res. 143,

123–129 (2004)12. Taga, G., Yamaguchi, Y., Shimizu, H.: Self-organized control of bipedal locomotion by

neural oscillators in unpredictable environment. Biol. Cybern. 65, 147–159 (1991)13. Aoi, S., Tsuchiya, K.: Locomotion control of a biped robot using nonlinear oscillators. Auton.

Robots 19, 219–232 (2005)14. Ogihara, N., Yamazaki, N.: Generation of human bipedal locomotion by a bio-mimetic neuro-

musculo-skeletal model. Biol. Cybern. 84, 1–11 (2001)15. Paul, C., Bellotti, M., Jezernik, S., Curt, A.: Development of a human neuro-musculo-skeletal

model for investigation of spinal cord injury. Biol. Cybern. 93, 153–170 (2005)16. Geyer, H., Herr, H.: A muscle-reflex model that encodes principles of legged mechanics

produces human walking dynamics and muscle activities. IEEE Trans. Neural Syst. Rehabil.Eng. 18(3), 263–273 (2010)

17. Van der Noot, N., Ijspeert, A.J., Ronsse, R.: Biped gait controller for large speed variations,combining reflexes and a central pattern generator in a neuromuscular model. In: IEEEInternational Conference on ICRA 2015, pp. 6267–6274 (2015)

18. Morimoto, J., Endo, G., Nakanishi, J., Cheng, G.: A biologically inspired bipded locomotionstrategy for humanoid robots: modulation of sinusoidal patterns by a coupled oscillator model.IEEE Trans. Robot. 24(1), 185–191 (2008)

A Neural Network with CPGs Entrained by Sensory Feedback Controls 153

19. Righetti, L., Ijspeert, A.J.: Programmable central pattern generators: an application to bipedlocomotion control. In: Proceedings of 2006 IEEE International Conference on ICRA 2006,pp. 1585–1590 (2006)

20. Klein, T.J., Lewis, M.A.: A physical model of sensorimotor interactions during locomotion.J. Neural Eng. 9(4), 1–14 (2012)

21. Cofer, D.W., Cymbalyuk, G., Reid, J., Zhu, Y., Heitler, W.J., Edwards, D.H.: AnimatLab: a3D graphics environment for neuromechanical simulation. J. Neurosci. Methods 187(2),280–288 (2010)

22. Fischer, M., Witte, H.: Legs evolved only at the end. Philos. Trans. R. Soc. A 365, 185–198(2006)

23. Plagenhoef, S., Evans, F.G., Abdelnour, T.: Anatomical data for analyzing human motion.Res. Q. Exerc. Sport 54, 169–178 (1983)

24. Zucker, R.S., Regehr, W.G.: Short-term synaptic plasticity. Annu. Rev. Physiol. 64(1),355–405 (2002)

25. Rybak, I.A., Stecina, K., Shevtsova, N.A., Lafreniere-Roula, M., McCrea, D.A.: Modelingspinal circuitry involved in locomotor pattern generation: insights from deletions duringfictive locomotion. J. Physiol. 577, 617–639 (2006)

26. Dietz, V., Duysens, J.: Significance of load receptor input during locomotion: a review. Gaitand Posture 11, 102–110 (2000)

27. Pang, M.Y.C., Yang, J.F.: The initiation of the swing phase in human infant stepping:importance of hip position and leg load. J. Physiol. 528(2), 389–404 (2010)

28. Kadaba, M.P., Ramakrishnan, H.K., Wootten, M.E.: Measurement of lower extremitykinematics during level walking. J. Orthop. Res. 8, 383–392 (1990)

154 W. Li et al.

Towards Unsupervised Canine PostureClassification via Depth Shadow Detectionand Infrared Reconstruction for Improved

Image Segmentation Accuracy

Sean Mealin(B), Steven Howell, and David L. Roberts

North Carolina State University, Raleigh, USA{spmealin,robertsd}@csc.ncsu.edu, schowel2@ncsu.edu

Abstract. Hardware capable of 3D sensing, such as the MicrosoftKinect, has opened up new possibilities for low-cost computer visionapplications. In this paper, we take the first steps towards unsuper-vised canine posture classification by presenting an algorithm to performcanine-background segmentation, using depth shadows and infrared datafor increased accuracy. We report on two experiments to show that thealgorithm can operate at various distances and heights, and examine howthat effects its accuracy. We also perform a third experiment to show thatthe output of the algorithm can be used for k-means clustering, result-ing in accurate clusters 83% of the time without any preprocessing andwhen the segmentation algorithm is at least 90% accurate.

1 Introduction

Recently there has been a growing interest in the use of technology to interpretdogs’ postures and behaviors. Existing work in this space has fallen into two maincategories: on-body systems that rely primarily on inertial measurement units(c.f., [1]) and off-body systems that use computer vision techniques (c.f., [7]).On-body techniques have proven effective for classifying postures, but aren’tuseful for identifying dogs’ position or orientation in space.

Previous work has used a Microsoft Kinect and supervised learning algo-rithms [7] for canine posture detection; however, they identified several challenges.Specifically, changes in illumination negatively impacted the segmentation of thedog from the background, and the accuracy of the classification algorithm. Fur-ther, identifying the dog’s paws proved difficult due to noise in the data producedby the Kinect and the proximity of the paws to the floor. Another practical chal-lenge with their system is that supervised algorithms require the collection of siz-able corpora of labeled data to allow a model to be built and successfully validated.Depending on the algorithm and dog, these hand-labeled data may be required forevery dog, which does not scale to larger applications and may prove difficult fornon-technical experts (i.e., pet owners) to provide.

We propose that an unsupervised approach for canine posture detectionwould avoid several of those problems including the collection and labeling of ac© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 155–166, 2016.DOI: 10.1007/978-3-319-42417-0 15

156 S. Mealin et al.

training dataset. Our approach uses both depth and infrared data to performcanine-background segmentation more accurately than just relying on depthdata alone. As a heuristic for locating the dog in the scene, we use depth shad-ows, while infrared data helps to differentiate the dog’s paws from the ground.We also present empirical data from two experiments: the first showed that theKinect must be positioned within 2 m of the canine for best performance, whilethe second experiment indicated that the algorithm is capable of detecting thecanine at most angles. We then conclude with some preliminary results showingthat the output of the algorithm is suitable for k-means clustering for body partdifferentiation, as a step towards unsupervised classification. Since the focus ofthis paper is on the performance of the algorithm in different environments,and to ensure that our results are as reproducible and rigorous as possible, weexclusively used two life-sized anatomically correct canine models for the exper-iments. In future work, we will expand our experiments to include a wide varietyof canine breeds.

2 Related Work

For more than three decades the problem of image segmentation has been studiedin the field of computer vision (c.f., [4,6,11]). With the prevalence of hardwarelike the Microsoft Kinect that is capable of detecting depth as well as color, tech-niques for segmentation, object tracking, and gesture recognition have changedto take advantage of the additional data [3]. One method for segmentation usesdepth shadows as a feature for increased accuracy. A depth shadow is a regionin the image that is set to an invalid value, due to a number of possible reasons;identifying that reason can assist with segmentation of objects in the scene [2].

In addition to allowing 3D interaction for humans, the Kinect has provento be a useful tool for a variety of animal related tasks. Previous studies haveexamined the suitability of the Kinect for monitoring cows [9], and estimatingthe activity and weight of pigs [5,12]. The use of the infrared data returnedby the Kinect has also enabled the automatic recognition of rat posture andbehavior [10] using supervised machine learning algorithms. For entertainmentpurposes, the Kinect has tracked cats for human-feline games [8].

To our knowledge, the only study focusing specifically on the use of theKinect with dogs resulted in a system for canine wellness [7]. The study useddepth data provided by the Kinect as input to a Structural Support VectorMachine (SSVM), which then classified seven body parts of the dog. The SSVMis a supervised machine-learning algorithm, which required the manual labelingof a training dataset based on data collected during their trials. Our approachmay aid this approach by improving SSVM accuracy through the use of infrareddata to more accurately capture the location and contours of the canine’s bodyparts; however, our main goal is to lay the foundation for an unsupervised postureclassification system, which removes the burden of manually annotating data,and training a classifier for each dog.

Towards Unsupervised Canine Posture Classification 157

3 Approach

We used a Microsoft Kinect 2.0 to capture depth and infrared data, and a laptopwith an Intel Core i7-3720QM processor and 16 gigabytes of memory for theanalysis. The Kinect is capable of capturing color frames at 1080p resolution at30 frames per second, and depth and infrared frames at a resolution of 512× 424pixels at 30 frames per second. According to Microsoft, for optimal results, theKinect should be positioned between 0.5 m and 4 m from the subject, and shouldnot be in direct sunlight.

For the purposes of this work, we separated the data capture and analysisprocesses. We used the Kinect SDK 2.0 to save each frame to a file withoutany preprocessing. We implemented the segmentation algorithm in Python 3.5,relying on the Python bindings of OpenCV 3.1, Numpy 1.9.3, and Scipy 0.16.1.

Our approach has two steps: first, we isolate a region of interest (ROI), which(ideally) contains the canine in the scene; and second, we refine the depth dataassociated with the canine. For this discussion, an image consists of both depthframe and the corresponding infrared frame. It is necessary to take an image ofthe background without the canine present to do background subtraction. LetIMG denote an image with the canine present, while BG denotes an image ofthe background without the canine. A subscript D or I refers to the depth orinfrared portion of an image respectively; for example, IMGD refers to the depthdata of an image which contains a canine.

To find the ROI, we convert a copy of IMGD and BGD to binary images usingan inverse binary threshold filter. This effectively singles out the depth shadowsin the image. As explained by Deng et al. [2], a depth shadow results whenthe infrared light used by the Kinect for distance measurements is occludedby an object in the scene. We then apply the morphological operation dilateto the binary image of BGD; dilate is a standard image manipulation techniqueprovided by OpenCV which causes thin lines in an image to become thicker. Thatwill cause the depth shadows in the background of the scene to become moreprominent. We then use background subtraction to remove the depth shadowsthat naturally occur in the scene due to fixed objects, while also reducing thenoise that sometimes leads to false shadows when an object does not reflectinfrared light back to the Kinect. To close any holes in the depth shadow, whichcould interfere with the next step, we then re-run the morphological operationon the subtracted image. At this point, the most prominent depth shadow is theoutline of the dog, so contour detection returns the bounding box of the canine.

We next convert the portion of IMGI that corresponds to the ROI identifiedin the IMGI to a threshold image. The value for the threshold function can betaken from a part of the image that contains mainly background, such as theedge of the bounding box. On the generated threshold image, we run contourdetection, giving preference to the contour closest to the center of the imageif there are multiple detected. That contour serves as a bit mask, removing alldepth readings not covered by the contour. This process results in all depthreadings being set to 0, with the exception of the canine in the scene.

158 S. Mealin et al.

(a) Depth Data (b) Infrared Data

Fig. 1. Image showing the depth data of a dog(a), where the paws are indistinct, and imageof the infrared data (b), where the paws arewell-defined.

Noise in the Kinect depth read-ings causes details of objects toget lost when near flat surfaces,such as the floor. This problemis exacerbated by uneven services,such as carpeting. Figure 1(a) is anexample where the dog’s paws arealmost completely lost due to noise.Our approach uses infrared data torecover the information from thenoise. Where the depth data isnoisy, the infrared data maintains

a clean contour in most circumstances. Figure 1(b) shows the same view, only ininfrared, where the paws are clearly defined against the floor.

(a) Depth Data (b) Infrared Data (c) Binary Image

Fig. 2. Image showing the effects of infrared reflections on depth data (a), infrareddata (b), and a binary image to highlight the dog and reflection (c).

Vertical flat objects, such as walls, cause the infrared data from the Kinectto be especially noisy. As an example, Fig. 2(a) is the depth data, Fig. 2(b) is theinfrared data of the same scene, and Fig. 2(c) is the result of applying a binarythreshold filter. The Figure shows that the infrared reflection causes an imaginaryobject at the top of the image. To reduce this noise look for a nearby depth shadow;if there is not one, the algorithm ignores the object. If there are multiple objectswith depth shadows, the algorithm selects the one that is most central.

4 Experiment One

To understand the physical constraints of the Kinect’s positioning on thealgorithm, we captured a total of 14 images of a canine in two locations atmultiple distances and heights. A human then assigned a binary score based onwhether the canine was visible in the output of the algorithm, which determinedwhether the algorithm was successful at that distance-height combination. Weused this data to determine the best position of the Kinect for future experi-ments.

Towards Unsupervised Canine Posture Classification 159

4.1 Experiment One Methodology

Fig. 3. A color image of thedog in location 1, side facingthe camera.

In order to get clean and repeatable data, our exper-iment used a life-sized anatomically correct stuffedyellow Labrador Retriever, measuring 95 cm longfrom nose to tail, 29 cm wide, and 55 cm tall. Thestuffed dog was in a standing posture, with headturned slightly to the left of center. We positionedthe canine sideways relative to the Kinect in everyimage for this experiment to provide an ideal anglefor segmentation. Figure 3 shows a sample of thestuffed dog’s positioning.

The Kinect does not work best in direct sunlight, so both locations wereindoors. Location 1 was in a room that had vinyl flooring, with the dog located inthe corner of the room. We selected this to examine if the algorithm had troublewith the slant of the two walls creating the corner. Due to space restrictions, wewere only able to capture images of the dog at distances of 1 m, 1.5 m, and 2 mfrom the dog, and at heights of 1 m and 1.5 m above ground level, for a totalof 6 positions. Location 2 contained carpeted flooring, doorways to open spaces,and various pieces of furniture within the image. We chose this location to seehow carpet and clutter effected the performance of the algorithm. The Kinectwas positioned at distances of 1 m, 2 m, 3 m, and 4 m away from the dog, and atheights of 1 m and 1.5 m above ground level, resulting in 8 positions.

4.2 Experiment One Results

To be successful, we determined that the output of the algorithm must con-tain the canine either completely, or partially, no matter how small. We wereas lenient on the criteria as possible, to only filter out the positions that thealgorithm completely failed to find the canine in the scene. In location 1, thealgorithm was able to identify the canine in 4 out of the 6 images. At distances1 m and 1.5 m, and at heights 1 m and 1.5 m, the canine was visible in the outputof the algorithm. In the two remaining locations, at distance 2 m and heights 1 mand 1.5 m, the infrared reflections on the two walls proved to be too much for thealgorithm, which resulted in output containing the wall instead of the canine.

In location 2, the Kinect was able to locate the dog in 6 of the 8 images, atdistances 1 m, 2 m, and 3 m, at both heights 1 m and 1.5 m. When positioned 4 maway from the canine at either height, the area taken up by the canine was toosmall for the algorithm, causing it to focus on other objects in the scene.

We found that when infrared light reflects off a surface directly at the Kinect,such as the wall directly behind the canine in location 2, the algorithm is moresuccessful at filtering it out, compared to the angled surfaces that create thecorner in location 1. We conclude that when possible, the Kinect should not bedirectly facing a corner created by two walls. We also note that the algorithmwas able to find the dog when it was either small or large compared to the totalarea of the image, indicating the algorithm is resilient to dog size.

160 S. Mealin et al.

5 Experiment Two

To evaluate the effect of the canine’s orientation on the algorithm, we capturedimages of three canines at different angles in the same two locations as before,at the best distances and heights identified by the first experiment.

5.1 Experiment Two Methodology

The two canines used in this experiment consisted of two life-sized anatomicallycorrect stuffed dogs. The first was the stuffed yellow Labrador Retriever previouslydescribed, and the second an identical black one. We posed the two dogs in fivepositions: directly facing the Kinect, directly away from the Kinect, sideways rel-ative to the Kinect, and at 45◦ angles between those three positions; those anglesare designated as front, back, left, front-left, and back-left respectively below.

The two locations were the same as the first experiment. Since the Kinectfailed to isolate the canine at a distance of 2 m in location 1, we limited thedistances to 1 m and 1.5 m. Likewise, 4 m failed to result in useful output inlocation 2, so we limited the Kinect to distances 1 m, 2 m, and 3 m, at bothheights as before.

5.2 Experiment Two Results

A human scored the output of the algorithm based on three criteria:1. Located canine: at least some part of the canine must be visible in the output

for this criteria to be considered successful.2. Isolated canine: The canine must be the only object in the output for this

criteria to be considered successful.3. Present body parts: A point is awarded for each of the head, torso, tail, and

all four paws if they are visible in the output of the algorithm. We did notpenalize the algorithm if a body part was visually occluded, as determinedby examining the color image. The final score is reported as the ratio of thenumber of detected body parts to the number of visible body parts.

Results for Criteria 1 and 2: For the first two criteria, in every image, thealgorithm either exclusively identified the canine, or completely failed. Therewere no images where the canine was not completely isolated in the output, sowe simultaneously report on the first two criteria below.

Table 1 contains the results for location 1 with the stuffed dogs. The algorithmwas able to locate both stuffed dogs at distance 1 m at both heights, with theexception of the black lab in the front-left position at h = 1.5 m. At D = 1.5 m,the algorithm had trouble finding the dogs. At the lower height, it missed theblack lab in every position, while at the upper height it was able to find it inthree out of five positions.

Table 2 contains the results for location 2 with the stuffed dogs. Encourag-ingly, both the yellow and black labs were identified in every image throughdistance 2 m. At 3 m neither dog was detected reliably by the algorithm, sug-gesting that a distance of 2 m or less is ideal.

Towards Unsupervised Canine Posture Classification 161

Table 1. Whether the stuffed yellow lab or stuffed black lab was detected in location1 at a given distance and height, and at a given position. Each row is a position, whileeach column is a distance-height measured in meters. If a Y appears, it means that thestuffed yellow lab was detected, while a B means the stuffed black lab was detected.

Posture 1D1H 1D1.5H 1.5D1H 1.5D1.5H

Front Y,B Y,B Y Y,B

Front-Left Y,B Y Y

Left Y,B Y,B Y Y

Back-Left Y,B Y,B Y,B

Back Y,B Y,B Y Y,B

Table 2. Whether the stuffed yellow lab or stuffed black lab was detected in location2 at a given distance and height, and at a given position. Each row is a position, whileeach column is a distance-height measured in meters. If a Y appears, it means that thestuffed yellow lab was detected, while a B means the stuffed black lab was detected.

Posture 1D1H 1D1.5H 2D1H 2D1.5H 3D1H 3D1.5H

Front Y,B Y,B Y,B Y,B B

Front-Left Y,B Y,B Y,B Y,B B

Left Y,B Y,B Y,B Y,B Y,B Y,B

Back-Left Y,B Y,B Y,B Y,B Y,B Y,B

Back Y,B Y,B Y,B Y,B

Results for Criteria 3: The complete results for the third criteria are inTables 3 and 4 for location 1 and location 2 respectively. The score is for thealgorithms performance for both the yellow and black labs combined.

In location 1, the algorithm had trouble finding all parts of the canines.At distance 1 m, it consistently found both dogs, however it only scored 100 %when the canine was in the front and back positions, depending on the height.

Table 3. The score of the stuffed yellow lab and stuffed black lab in location 1. Eachrow is a position, while each column is a distance-height measured in meters. N/Ameans that neither dog was detected.

Posture 1D1H 1D1.5H 1.5D1H 1.5D1.5H

Front 100.00 % 75.00 % 100.00% 87.50 %

Front-Left 83.33 % 60.00 % 60.00% N/A

Left 90.00 % 70.00 % 80.00% 90.91 %

Back-Left 60.00 % 75.00 % N/A 100.00 %

Back 87.50 % 100.00 % 100.00% 100.00 %

162 S. Mealin et al.

Table 4. The score of the stuffed yellow lab and stuffed black lab in location 2. Eachrow is a position, while each column is a distance-height measured in meters. N/Ameans that neither dog was detected.

Posture 1D1H 1D1.5H 2D1H 2D1.5H 3D1H 3D1.5H

Front 100.00 % 100.00 % 100.00% 87.50 % 100.00 % N/A

Front-Left 90.00 % 100.00 % 80.00% 70.00 % N/A 100.00%

Left 100.00 % 100.00 % 100.00% 100.00 % 90.00 % 100.00%

Back-Left: 100.00 % 100.00 % 100.00% 100.00 % 91.67 % 91.67 %

Back 100.00 % 100.00 % 100.00% 100.00 % N/A N/A

For distance 1.5 m, the algorithm sometimes failed to find a canine at all, butinterestingly, scored the highest at h = 1.5 m with the lowest score being 87.5 %.

Table 4 shows that the algorithm did very well through the 2 m distancemark, where the lowest score was 70 %. Even at distance 3 m, when the algorithmstarted to struggle to find the dogs, the lowest score was 90 %, indicating thatit only missed at most one body part.

6 Experiment Three

As a step towards the unsupervised recognition of canine posture, we presentsome preliminary results of k-means clustering on the output of the algorithm.Whereas the third criteria for the second experiment indicated which body partswere visible after segmentation as judged by a human, this experiment examineshow successful an unsupervised learning technique, k-means clustering, is atidentifying those body parts.

6.1 Experiment Three Methodology

We converted the depth data provided by our algorithm to a point cloud, usingspatial transforms provided by the Kinect. For the below examples, we did nofurther processing of the data before running k-means clustering.

For this experiment, we ran clustering on all of the images from the secondexperiment, a total of 100 images. For each image, we ran k-means with k = 7to account for the background and up to six distinct body areas we hoped toidentify. A human then scored each image based on whether the clusters covered(depending on visibility in the raw image) up to six distinct body parts: thehead, torso, rump, tail, and two sets of paws. We counted the algorithm correctif it grouped the two front and two back paws together.

Towards Unsupervised Canine Posture Classification 163

6.2 Experiment Three Results

(a) Left Orientation (b) Front-left Orienta-tion

Fig. 4. An image showing the dog in the left orienta-tion (a). There are different clusters for its head, torso,rump, front legs, back legs, and tail. In the front-leftorientation (b) there are different clusters on its head,torso, rump, front legs, and back legs (the tail wasn’tvisible in the raw image).

Table 5 contains the resultsof the clustering algorithm,scored by human graders.Note, for these results, wedo not report a percentageof clusters that were cor-rect, but an all-or-nothingclassification. Since cluster-ing is an important stepfor unsupervised classifica-tion, we were as critical ofthe criteria for success aspossible. The first columnrepresents a threshold forthe criteria from the secondstudy. Out of the 50 images

that scored a 100 %, k-means resulted in sensible clusters for 41 images. Inter-estingly, as the accuracy of visible body parts decreased to 90 %, the clusteringalgorithm was still able to maintain a high success rate.

Figure 4(a) is an example of very good results, where the background iscleanly distinguished from the dog and all six body parts are clustered neatly.Figure 4(b) illustrates another example of a successful result where the head,torso, rump, front paws, and back paws are all clustered as well. Because thetail wasn’t visible and k-means was set to find seven clusters, an extra clusterappears on the chest.

Table 5. Results of the clustering algorithm for images of varying quality as determinedby criteria three of the second experiment. The left column indicates a range of accuracyas defined in the second study. The next columns are the number of images for thataccuracy range, the number of images that were successfully clustered, and a successrate given by dividing those two columns.

Percent of un-occluded Number of images Number of successfully Success rate

body parts visible clustered images

100% 50 41 82.00 %

90% - 99.99 % 12 10 83.33 %

80% - 89.99 % 12 6 50.00 %

70% - 79.99 % 8 5 62.50 %

60% - 69.99 % 6 2 33.33 %

0% (Dog Not Found) 12 0 0.00 %

Total 100 64 64.00 %

164 S. Mealin et al.

7 Discussion

As expected, the algorithm started to lose effectiveness as the Kinect’s distanceand height increased in the two locations. Surprising to us, the algorithm wasable to find the stuffed yellow lab more of the time in location 1, while in location2, the stuffed black lab was found more often. One possible explanation is that inlocation 1, The infrared reflections from the walls were more effective at drowningout the black lab. The data supports this as the black lab was lost as the Kinectcould see more of the walls. For location 2, where the opposite was true, werequire additional data to develop a satisfactory explanation.

Another observation is that in location 1, the algorithm performed slightlybetter at the taller height when D = 1.5 m. An explanation is that the increasedheight, which forced the Kinect’s angle to be more sharply pointed towards theground actually helped reduce the impact of the infrared reflections. This couldindicate that when working in environments with unavoidable reflections, theKinect should be positioned higher for optimal results.

(a) Depth 2m (b) Depth 3m

Fig. 5. Images showing the dog in the back-leftposition at 2 m (a) and 3m (b). Its head, torso,tail, and three legs are clear at 2 m, while itshead, torso, tail, and two legs are clear at 3m.

In location 2, the algorithmgives satisfactory results at dis-tances 1 m and 2 m. In all of thecaptured images, at both heights,the output was either perfect ormissed at most three body partsfor both dogs. After eliminat-ing D = 2 m and H = 1.5 m, thealgorithm only missed two bodyparts in the worst case, a tail anda paw. Based on these data, wesuggest that the Kinect be posi-tioned a maximum of 2 m fromthe dog for optimal results, whichFig. 5(a) demonstrates. Interest-

ingly, even when that is exceeded, the algorithm can still produce acceptableresults, as demonstrated by Fig. 5(b).

8 Future Work

In the future, we need to collect more data to better understand how the colorof the dog and coat length effects the algorithm’s accuracy. We were unable toexplain why the algorithm was able to find the stuffed yellow lab and not thestuffed black lab in location 1, while being able to find the opposite in location2 at extreme distances. We also will expand testing to real dogs, with a focuson dogs of various sizes, colors, and coat lengths. In conjunction with this, weare going to introduce additional postures such as when the canine is sitting orlaying down.

As another way to increase the total accuracy of the algorithm, we plan tointroduce multiple Kinect units. We believe that it would help when much of the

Towards Unsupervised Canine Posture Classification 165

canine’s body parts are visually occluded, such as in the front or back positions.Having a second Kinect at another angle would also enable us to create largerpoint clouds, and capture the dog in more dimensions, which could potentiallyhelp with the k-means clustering algorithm.

9 Conclusion

In this paper, we presented an algorithm capable of canine-background segmen-tation in images captured by the Microsoft Kinect. The algorithm used depthshadows to identify the location of the canine, and infrared data to accuratelyfind parts of the canine located near other surfaces, such as the floor.

We reported on two experiments that evaluate the positioning of the Kinectand the algorithm’s performance as the position of the dog changes. The firstexperiment showed that the algorithm’s performance is degraded by infraredreflections, however, the negative effects can be somewhat minimized by rais-ing the height of the Kinect to angle it down towards the ground. The latterindicated that the algorithm can identify the canine in a variety of angles atvarious distances. We plan to run additional studies with a variety of dog breedsto more-fully evaluate the algorithm.

We finally presented the preliminary results of k-means clustering on theoutput of the algorithm. The clusters were sensibly formed in 83 % of imageswhen the segmentation algorithm is at least 90 % accurate. It is our belief thatadding preprocessing steps such as smoothing, and using multiple Kinect unitscould possibly increase the accuracy of the clusters. In the future, we will furtherrefine the algorithm sensor placement with data gathered by using real, ratherthan stuffed, dogs.

Successfully identifying the position and orientation of a dog in an environ-ment with little or no expensive-to-obtain hand-labeled data is a critical first steptowards enabling novel computer-mediated interactions for dogs. The potentialapplications in this space are enormously-broad, ranging from computer assistedtraining, to welfare monitoring, and live evaluation of working dogs. The workpresented in this paper represents an important first step towards realizing unsu-pervised localization and posture identification using commodity sensing hard-ware.

Acknowledgments. This material is based upon work supported by the National Sci-ence Foundation, under both Graduate Research Fellowship Grant No. DGE-1252376and NSF grant 1329738. Any opinion, findings, and conclusions or recommendationsexpressed in this material are those of the authors and do not necessarily reflect theviews of the National Science Foundation.

166 S. Mealin et al.

References

1. Brugarolas, R., Roberts, D., Sherman, B., Bozkurt, A.: Machine learning basedposture estimation for a wireless canine machine interface. In: 2013 IEEE TopicalConference on Biomedical Wireless Technologies, Networks, and Sensing Systems(BioWireleSS), pp. 10–12. IEEE (2013)

2. Deng, T., Li, H., Cai, J., Cham, T.J., Fuchs, H.: Kinect shadow detection andclassification. In: Proceedings of the IEEE International Conference on ComputerVision Workshops, pp. 708–713 (2013)

3. Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with microsoftkinect sensor: a review. IEEE Trans. Cybern. 43(5), 1318–1334 (2013)

4. Haralick, R.M., Shapiro, L.G.: Image segmentation techniques. Comput. Vis.Graph. Image Process. 29(1), 100–132 (1985)

5. Kongsro, J.: Estimation of pig weight using a microsoft kinect prototype imagingsystem. Comput. Electron. Agric. 109, 32–35 (2014)

6. Peng, B., Zhang, L., Zhang, D.: A survey of graph theoretical approaches to imagesegmentation. Pattern Recogn. 46(3), 1020–1038 (2013)

7. Pistocchi, S., Calderara, S., Barnard, S., Ferri, N., Cucchiara, R.: Kernelized struc-tural classification for 3d dogs body parts detection. In: 2014 22nd InternationalConference on Pattern Recognition (ICPR), pp. 1993–1998. IEEE (2014)

8. Pons, P., Jan, J., Catal, A.: Developing a depth-based tracking systems for inter-active playful environments with animals. In: 12th International Conference onAdvances in Computer Entertainment Technologies, Second International Congresson Animal Human Computer Interaction (2015)

9. Salau, J., Haas, J., Thaller, G., Leisen, M., Junge, W.: Developing a multi-kinect-system for monitoring in dairy cows: object recognition and surface analysis usingwavelets. Anim. Int. J. Anim. Biosci., 1–12 (2016)

10. Wang, Z., Mirbozorgi, S.A., Ghovanloo, M.: Towards a kinect-based behavior recog-nition and analysis system for small animals. In: 2015 IEEE Biomedical Circuitsand Systems Conference (BioCAS), pp. 1–4. IEEE (2015)

11. Zhang, Y.J.: A survey on evaluation methods for image segmentation. PatternRecogn. 29(8), 1335–1346 (1996)

12. Zhu, Q., Ren, J., Barclay, D., McCormack, S., Thomson, W.: Automatic animaldetection from kinect sensed images for livestock monitoring and assessment. In:2015 IEEE International Conference on Computer and Information Technology;Ubiquitous Computing and Communications; Dependable, Autonomic and SecureComputing; Pervasive Intelligence and Computing (CIT/IUCC/DASC/PICOM),pp. 1154–1157. IEEE (2015)

A Bio-Inspired Model for Visual CollisionAvoidance on a Hexapod Walking Robot

Hanno Gerd Meyer1(B), Olivier J.N. Bertrand2, Jan Paskarbeit1,Jens Peter Lindemann2, Axel Schneider1,3, and Martin Egelhaaf2

1 Biomechatronics, Center of Excellence ‘Cognitive InteractionTechnology’ (CITEC), University of Bielefeld, Bielefeld, Germany

hanno.meyer@uni-bielefeld.de2 Department of Neurobiology and Center of Excellence ‘Cognitive Interaction

Technology’ (CITEC), University of Bielefeld, Bielefeld, Germany3 Embedded Systems and Biomechatronics Group, Faculty of Engineering

and Mathematics, University of Applied Sciences, Bielefeld, Germany

Abstract. While navigating their environments it is essential forautonomous mobile robots to actively avoid collisions with obstacles.Flying insects perform this behavioural task with ease relying mainlyon information the visual system provides. Here we implement a bio-inspired collision avoidance algorithm based on the extraction of nearnessinformation from visual motion on the hexapod walking robot platformHECTOR. The algorithm allows HECTOR to navigate cluttered envi-ronments while actively avoiding obstacles.

Keywords: Biorobotics · Bio-inspired vision · Collision avoidance ·Optic flow · Elementary motion detector

1 Introduction

Compared to man-made machines, insects show in many respects a remarkablebehavioural performance despite having only relatively small nervous systems.Such behaviours include complex flight or walking manoeuvres, avoiding colli-sions, approaching targets or navigating in cluttered environments [11]. Sensingand processing of environmental information is a prerequisite for behaviouralcontrol in biological as well as in technical systems. An important source ofinformation is visual motion, because it provides information about self-motion,moving objects, and also about the 3D-layout of the environment [7].

When an agent moves through a static environment, the resulting visualimage displacements (optic flow) depend on the speed and direction of ego-motion, but may also be affected by the nearness to objects in the environment.During translational movements, the optic flow amplitude is high if the agentmoves fast and/or if objects in the environment are close. However, during rota-tional movements, the optic flow amplitude depends solely on the velocity of ego-motion and, thus, is independent of the nearness to objects. Hence, informationc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 167–178, 2016.DOI: 10.1007/978-3-319-42417-0 16

168 H.G. Meyer et al.

about the depth structure of an environment can be extracted parsimoniouslyduring translational self-motion as distance information is immediately reflectedin the optic flow [7]. To solve behavioural tasks, such as avoiding collisions orapproaching targets, information about the depth structure of an environment isnecessary. Behavioural studies suggest, that flying insects employ flight strategieswhich facilitate the neuronal extraction of depth information from optic flow bysegregating flight trajectories into translational and usually much shorter rota-tional phases (active-gaze-strategy, [4]). Furthermore, the neuronal processing ofoptic flow has been shown to play a crucial role in the control of flight stabilisa-tion, object detection, visual odometry and spatial navigation [3].

In flying insects, optic flow is estimated by a mechanism that can be modelledby correlation-type elementary motion detectors (EMDs, [2]). A characteristicproperty of EMDs is that the output does not exclusively depend on velocity,but also on the pattern properties of a moving stimulus, such as its contrast andspatial frequency content. Hence, nearness information can not be extractedunambiguously from EMD responses [5]. Rather, the responses of EMDs to puretranslational optic flow have been concluded to resemble a representation of therelative contrast-weighted nearness to objects in the environment, or, in otherwords, of the contours of nearby objects [18].

Recently, a simple model for collision avoidance based on EMDs was pro-posed [1]. The model is based on three successive processing steps: (a) theextraction of (contrast-weighted) nearness information from optic flow by EMDs,(b) the determination of a collision avoidance direction from the map of near-ness estimates and (c) the determination of a collision avoidance necessity, i.e.whether to follow (i.e. potential obstacles are close) or not to follow the collisionavoidance direction (i.e. potential obstacles are still rather distant). When cou-pled with a goal direction, the algorithm is able to successfully guide an agentto a goal in cluttered environments without collisions.

In this study, this collision avoidance model was implemented on the insect-inspired hexapod walking robot HECTOR [15]. In contrast to flight, walkingimposes specific constraints on the processing of optic flow information. Due tothe mechanical coupling of the agent to the ground the perceived image flow issuperimposed by continuous rotational components about all axes correlated tothe stride-cycle [9]. Therefore, nearness estimation from optic flow during trans-lational walking might be obfuscated, potentially reducing the reliability of thecollision avoidance algorithm. Further, in contrast to the 360◦ panoramic visionused in [1], a fisheye lens was mounted on the front segment of the robot withits main optical axis pointing forward, limiting the field of view for retrievingoptic flow information.

In the following, the implementation of the collision avoidance and vision-based direction control on the robotic platform will be described and the perfor-mance assessed in artificial and natural cluttered environments in a simulationframework of HECTOR. After optimisation of parameters, HECTOR will beable to successfully navigate to predefined goals in cluttered environments whileavoiding collisions.

Bio-Inspired Visual Collision Avoidance on a Walking Robot 169

2 The Simulation Framework

The performance of the model of collision avoidance and direction control wasassessed in a dynamics simulation of HECTOR which is coupled with a render-ing module. The simulation allows parameter optimisation and tests in differentvirtual environments. The simulation framework is depicted in Fig. 1 and can beseparated into four processing modules: (a) walking controller, (b) robot simula-tion, (c) renderer and (d) vision-based directional controller.

Fig. 1. The simulation framework used for testing the implementation of the visual col-lision avoidance model on HECTOR. The dashed box (Vision-Based Direction Con-troller) indicates the algorithm used for controlling the robot’s behaviour based onnearness estimation from optic flow.

2.1 Robot Simulation and Walking Controller

The hexapod robot HECTOR is inspired by the stick insect Carausius morosus.For its design, the relative positions of the legs as well as the orientation of thelegs’ joint axes have been adopted. The size of the robot was scaled up by a factorof 20 as compared to the biological example which results in an overall length ofroughly 0.9 m. This size allows the robot to be used as an integration platformfor several hard- and software modules. All 18 drives for the joints of the six legsare serial elastic actuators. The mechanical compliance of the drives is achievedby an integrated, sensorised elastomer coupling [14] and is the foundation forthe robot’s ability to passively adapt to the structure of the substrate duringwalking. The bio-inspired walking controller is a conversion of the WALKNET

170 H.G. Meyer et al.

approach [17] and allows the robot to negotiate rough terrain [15]. Abstractingthe complex task of leg coordination, only the heading vector must be providedexternally, e.g. by a vision-based direction controller as proposed here.

To simulate a multitude of controller parameters, a dynamics simulationhas been set up based on ODE (Open Dynamics Engine) which also simulatesthe elastic joint actuation. The HECTOR simulator is controlled by the samecontroller framework as the physical robot.

2.2 Renderer

To obtain optic flow information resulting from ego-motion in different virtualenvironments the images of a camera attached to the robot’s main body arerendered using the graphics engine Panda3D [6]. The robot’s orientation andposition are obtained from the robot simulation module. To emulate the widefield of view of insect eyes [21], virtual camera images are rendered simulating anequisolid fisheye lens [12]. The lens is parametrised to a horizontal and verticalfield of view of 192◦. The images obtained have a resolution of 400× 400 pixelswith a resolution of 10 bit per RGB color channel and a sampling frequencyof 20 Hz. Although blowflies possess color vision, evidence suggests that thepathways involved in motion detection are monochromatic [20]. Therefore, onlythe green color channel is used (Fig. 2A).

Head Stabilisation. During walking, the extraction of distance informationon the basis of optic flow processing may be impaired by stride-coupled imageshifts. For example, walking blowflies hardly ever show purely translational loco-motion phases. Rather, they perform relatively large periodic rotations of theirbody around all axes due to walking [9]. While stride-induced body rotationsaround the roll and pitch axes are compensated by counter-rotations of the head,body rotations around the yaw axis are not [8]. To minimise stride-coupled imagedisplacements, movements of the camera around the roll and pitch axis are com-pensated in simulation. This is achieved by setting the roll and pitch angles ofthe camera to fixed values independent of the movement of the robot’s mainbody, effectively keeping the center of the optical axis of the camera parallel tothe ground plane.

2.3 Vision-Based Direction Controller

The sequences of camera images obtained from the renderer are processed by thevision-based direction controller, which can be subdivided into four processingsteps:

(a) preprocessing of images, in order to emulate the characteristics of the visualinput of flying insects,

(b) estimation of a relative nearness map by processing of optic flow via EMDs,(c) computation of a collision avoidance direction based on the relative nearness

of objects and a goal direction, and(d) controlling the walking direction of the robot.

Bio-Inspired Visual Collision Avoidance on a Walking Robot 171

Fig. 2. (A) Camera image of a virtual environment rendered with an equisolid fish-eye lens. Image has been resized and reduced to 8-bit dynamic range for reproduc-tion. (B) Camera image remapped to a rectilinear representation, spatially filtered andscaled to an array of photoreceptors. Each pixel position represents a luminance valueas perceived by the according photoreceptor. Image has been resized and reduced to8-bit dynamic range for reproduction. (C) Relative contrast-weighted nearness map μr

obtained from optic flow estimation via horizontally and vertically aligned EMDs. Thecolor-code depicts near (red) and far (blue) estimated relative nearnesses. (D) Polarrepresentation of the relative nearness averaged over the azimuth (blue). The arrow(black) depicts the sum of nearness vectors (COMANV ) used for determining the col-lision avoidance direction CADfov (red). Based on the weighted sum of the directionto a goal α (yellow) and the CADfov, a heading direction γ (green) is computed. Thegreyed out area indicates the surrounding of the robot not covered by the field of view.(Color figure online)

172 H.G. Meyer et al.

(a) Image Preprocessing. The non-linear fisheye lens, used here, possessesdistortion characteristics which are such that objects along the optical axis ofthe lens occupy disproportionately large areas of the image. Objects near theperiphery occupy a smaller area of the image. Since the lens enlarges objects inthe vicinity of the optical axis, those objects are transmitted with much greaterdetail than objects in the peripheral viewing region, thus, obfuscating nearnessestimation from optic flow. Hence, images obtained from the fisheye lens areremapped to a rectilinear representation [12].

The compound eye of insects consists of a two-dimensional array of hexago-nally aligned ommatidia comprising the retina. Each ommatidium contains a lensand a set of photoreceptor cells. The lattice of ommatidia has characteristics of aspatial low-pass filter and blurs the retinal image. To mimic the spatial filtering ofthe eye, the remapped images are filtered by a two-dimensional Gaussian-shapedspatial low-pass filter according to Shoemaker et al. [19]. After spatial filtering,each input image is scaled down to a rectangular grid of photoreceptors, with aninterommatidial angle of 1.5◦. The acceptance angle of an ommatidium is set to1.64◦ in order to approximate the characteristics of the eyes of the blowfly [16].The grid covers a field of view of 174◦ horizontally and vertically, resulting inan array of 116× 116 photoreceptors (i.e. luminance values) (Fig. 2B).

(b)Optic FlowProcessing. In the model used here (see [18]), optic flow estima-tion is based on two retinotopic arrays of either horizontally or vertically alignedEMDs. Individual EMDs are implemented by a multiplication of the delayed sig-nal of a receptive input unit with the undelayed signal of a neighbouring unit. Onlyinteractions between direct neighbours are taken into account, for both horizon-tally and vertically aligned EMDs. The luminance values from the photoreceptorsare filtered with a first-order temporal high-pass filter (τhp = 20 ms) to remove themean from the overall luminance of the input. The filtered outputs are fed into thehorizontally and vertically aligned EMD arrays. The delay operator in each half-detector is modelled by a temporal first-order low-pass filter (τlp = 35 ms). EachEMD consists of two mirror-symmetric subunits with opposite preferred direc-tions. Their outputs are subtracted from each other. For each retinotopic unit themotion energy is computed by taking the length of the motion vector given bythe combination of the responses of a pair of the horizontal hEMD and the verticalvEMD at a given location (x,y) of the visual field:

μr(x,y) =√

v2EMD(x, y) + h2

EMD(x, y) (1)

The array of the absolute values of these local motion vectors μr resembles amap of contrast-weighted relative nearness to objects in the environment [18],providing information about the contours of nearby objects (Fig. 2C).

(c) Navigation and Collision Avoidance. Once the relative nearness map μr

is known, collision avoidance is achieved by moving away from the maximum near-ness value (e.g. objects that are close) (see [1]). However, the contrast-weighted

Bio-Inspired Visual Collision Avoidance on a Walking Robot 173

nearness map also depends on the textural properties of the environment. Toreduce the texture dependence, the nearness map is averaged along the elevation ε,giving the average nearness for a given azimuth φ. Each of these averaged nearnessvalues can be represented by a vector in polar coordinates, where the norm of thevector is the averaged nearness, and its angle corresponds to the azimuth. The sumof these vectors points towards the average direction of close objects (Fig. 2D).This vector is denoted center-of-mass-average-nearness-vector (COMANV ; [1])

COMANV =∑ ⎛

⎝⎛⎝ cos(φ)

sin(φ)

⎞⎠ 1

n

∑μr (ε, φ)

⎞⎠ , (2)

where n is the number of elements in the azimuth. The inverse of the COMANVvector, scaled to the horizontal field of view θ of the photoreceptor array, pointsaway from the closest object and, thus, can be used as the direction of the robotto avoid collisions (collision avoidance direction, CADfov; Fig. 2D; [1]):

CADfov =− arctan (COMANVy, COMANVx)

2πθ

(3)

The length of the COMANV vector increases with nearness and apparent size ofobjects. Its length is a measure of the collision avoidance necessity (CAN ; [1]):

CAN =‖ COMANV ‖ . (4)

The CAN measure is used to control the heading direction γ of the robot betweenavoiding collisions and following the direction to a goal (α; Fig. 2D) [1]:

γ = W (CAN) · CADfov + (1 − W (CAN)) · α (5)

W is a sigmoid weighting function based on the CAN :

W (CAN) =1

1 +(

CANn0

)−g , (6)

and driven by a gain g and a threshold n0 [1].

(d) Behavioural Control of the Walking Robot. The walking direction ofthe robot is controlled based on the heading direction γ obtained from estimatingrelative nearness values to objects from optic flow and the goal direction. Infor-mation about the spatial structure of an environment can only be extracted fromoptic flow during translational movements, as rotational flow components do notprovide distance information [7]. Inspired by the active-gaze-strategy employedby flying insects [4], the control of walking direction is implemented by segre-gating the motion trajectory into translational and rotational phases. Duringtranslation, the robot moves forward with a constant velocity of 0.2 m/s for 4 s(corresponding to 80 camera images), while averaging the heading direction γ.

174 H.G. Meyer et al.

After that, the robot switches to a rotational state and turns towards the aver-aged heading direction γ until the vector is centred in the field of view. Whenthe optic flow field is estimated by EMDs, the nearness estimations also dependon the motion history due to the temporal filters. Hence, the optic flow obtainedduring the rotational phase interferes with the optic flow measurements duringthe translational phase. This effect decreases over time. Therefore, the headingdirection γ is only averaged for the last 3 s of the translational phase.

Due to the restricted horizontal field of view, no information about the near-ness of objects outside of the camera’s field of view can be obtained (grey areain Fig. 2D). However, as the camera is pointing forward along the direction ofwalking during translation, information about the nearness of objects sidewardsor behind the robot is not essential. In situations where the goal direction doesnot reside within the field of view, the CAN is set to zero, effectively inducinga turn of the robot in the rotational phase until the goal direction is centered inthe field of view.

3 Visual Collision Avoidance in Cluttered Environments

Parameter Optimisation. The implementation of the collision avoidancemodel in the dynamics simulation of HECTOR was tested in several clutteredenvironments. In a first step, the threshold n0 and gain g of the weighting func-tion W [see Eq. (6)] were optimised in an artificial environment. The environ-ment consisted of a cubic box with a cylindrical object placed in the center(see Fig. 3B–D). Both, the box and the object were covered with a Perlin noisetexture. The robot was placed at a starting position (S) in front of the object,facing a goal position (G) behind the object. The distance between starting posi-tion and goal was set to 10 m. For each of the possible parameter combinationsof the gain g = [1.0, 2.0, ..., 10.0] and threshold n0 = [0.0, 1.0, ..., 20.0] the tra-jectory length for reaching the goal (G) without colliding with the object wastaken as a benchmark of the performance of the collision avoidance model (seeFig. 3A). A collision was assumed if the position of the camera crossed a radiusof 1.5 m around the center of the object (black dashed circle, Fig. 3B–D) and therespective combination of parameters was discarded. For each combination ofparameters 3 trials were performed.

If the threshold n0 is set to low values, the computation of the heading direc-tion γ [see Eq. (5)] mainly depends on the collision avoidance direction CADfov,whereas the goal direction α is only taken into account to a small extent. Hence,the robot will more likely avoid collisions than navigate to the goal (G). Further,a steeper slope of the sigmoid weighting function W , set by the gain g, leads tohigher temporal fluctuation of the heading direction γ. As a consequence, whensetting the threshold to n0 = 0.0 and the gain to g = 10.0, the resulting tra-jectories were relatively long (Fig. 3A) and showed erratic movement patterns.However, all trajectories reached the goal position for the given parameter com-bination (Fig. 3B). Due to the robot following the collision avoidance directionCADfov, in several cases the goal direction did not reside within the field of view,

Bio-Inspired Visual Collision Avoidance on a Walking Robot 175

Fig. 3. (A) Length of simulated trajectories (color-coded) in a cubic box with a singleobject (see B–D) for different combinations of the weighting function parameters gaing = [1.0, 2.0, ..., 10.0] and threshold n0 = [0.0, 1.0, ..., 20.0] [see Eq. (6)]. The size ofthe box was 14 m× 14m× 10 m (length×width× height) and the radius of the objectr = 1 m (height h =10 m). The walls of the box and the object were uniformly coveredwith a Perlin noise texture (scale= 0.05). When the trajectory crossed a circle of aradius of 1.5 m around the center of the object (dashed line in B–D) a collision wasassumed (white areas). (B–D) Simulated trajectories (n=10) in a cubic box with asingle object (filled circle). Starting positions are given as S and goal positions as G.Weighting function parameters were set to (B) g = 10.0 and n0 = 0.0, (C) g = 10.0and n0 = 20.0 and (D) g = 1.0 and n0 = 12.0. The grey dotted lines in B indicatethe main optical axis before and after recentering the goal direction in the visual field.(E) Simulated trajectories in a cubic box with randomly placed objects (filled circles)for different starting positions (S1–S3). The size of the box was 25m× 25 m× 10 m(length×width× height). The radius of each object (n = 30; height: 10 m) was setrandomly in a range from 0.25 m to 1.0 m. The walls of the box and the objects wereuniformly covered with a Perlin noise texture (scale = 0.05). Weighting function para-meters were set to g = 1.0 and n0 = 12.0 (see A and D). For each starting position 3trajectories are shown. It is notable, that the variability for trajectories with the samestarting positions arises due to the initialization of the robot with differing body pos-tures, effectively influencing the initial perceived image flow. (F) Simulated trajectories(n = 5) in a reconstructed natural environment. Weighting function parameters wereset to g = 1.0 and n0 = 12.0 (see A and D). The distance between starting position(S) and goal position (G) was 48.83 m.

176 H.G. Meyer et al.

resulting in a recentering of the goal vector along the main optical axis (as indi-cated by the grey dashed lines in Fig. 3B). This strategy led to reaching the goalposition for all trajectories.

In contrast, when setting the threshold n0 to high values, the computationof the heading vector γ mainly takes the goal direction α into account, whereasthe influence of the collision avoidance direction (CADfov) is reduced. As aconsequence, the robot will more likely follow the direction to the goal withoutavoiding obstacles. Therefore, when setting the threshold to n0 = 20.0 and thegain to g = 10.0, the robot directly approached the goal position, consequently,colliding with the object (Fig. 3A and C).

Figure 3D shows the trajectories for a combination of the parameters gain andthreshold which resulted in short trajectory lengths without collisions (n0 = 12.0,g = 1.0). Here, the robot almost directly approached the goal, while effectivelyavoiding the object. This combination of the threshold and gain parameters wasused in subsequent simulations in more complex cluttered environments.

Artificial Cluttered Environment. After optimisation of the threshold andgain parameters the performance of the collision avoidance model was tested inan Artificial cluttered environment (Fig. 3E) which was set up in a cubic box.Several cylindrical objects (n = 30) were placed at random positions in thex,y-plane. The radius of the objects was set individually to a random numberof r = [0.25, 1.0] m. Both, the box and the objects were covered with a texturegenerated from Perlin noise. The robot was placed at different starting positions(S1–S3), with the main optical axis oriented in parallel with the x-axis. Thedistance to the goal position was d1,3 = 21.36 m for the starting positions S1

and S3 and d2 = 20 m for the starting position S2. The parameters of thesigmoid weighting function W were set to n0 = 12.0 and g = 1.0 accordingto Fig. 3A. For each starting position 3 trajectories were simulated. In all casesthe robot successfully reached the goal position without collisions and withoutencountering local minima (see however [1] for a more detailed analysis).

Natural Cluttered Environment. We further tested the performance of thecollision avoidance model in a reconstructed natural environment (Fig. 3F). Theenvironment consisted of a dataset obtained from several laser scans [22]. Thestarting (S) and goal position (G) were set so that the robot had to avoidcollisions with trees to reach the location of the goal. Also in the natural envi-ronment – which substantially differs in the textural pattern properties from thetested artificial environment – for all trials (n = 5) the combination of weightingfunction parameters n0 = 12.0 and g = 1.0 resulted in trajectories successfullyleading to the goal, without colliding with objects.

4 Conclusion

A prerequisite for autonomous mobile robots is to navigate their environmentswhile actively avoiding collisions with obstacles. Whereas, to perform collision

Bio-Inspired Visual Collision Avoidance on a Walking Robot 177

avoidance, autonomous robots nowadays normally rely on active sensors (e.g.laser range finders [13]) or extensive computations (e.g. Lucas-Kanade opticflow computation [10]), insects are able to do so with minimal energetic andcomputational expenditure by relying mainly on visual information. We imple-mented a bio-inspired model of collision avoidance in simulations of the hexa-pod walking robot HECTOR solely based on the processing of optic flow bycorrelation-type elementary motion detectors (EMDs). EMDs have previouslybeen accounted for playing a key role in the processing of visual motion infor-mation in insects [3]. As could be shown, although the responses of EMDs tovisual motion are entangled with the textural properties of the environment [5],the relative nearness information obtained from optic flow estimation via EMDsis sufficient to direct HECTOR to a goal location in cluttered environments with-out colliding with obstacles. This holds true either for artificially generated envi-ronments as well as for a reconstructed natural environment, which substantiallydiffer in their textural pattern properties. Moreover, by employing behaviouralstrategies such as (a) an active-gaze strategy and (b) active head stabilisation –both also found in insects – the influence of rotational optic flow componentswhich potentially obfuscate the estimation of relative nearness information fromoptic flow is reduced. Hence, on the physical robot a prototype for mechanicalgaze-stabilisation has been implemented and is currently compared to a softwareimplementation.

The simulation results shown here will serve as a basis for the implementationof more complex bio-inspired models for visually-guided navigation in hardwarewhich is currently under development. These models will comprise strategies fornavigation and search behaviour based on the insect-inspired processing of opticflow.

Acknowledgments. This work has been supported by the DFG Center of ExcellenceCognitive Interaction TEChnology (CITEC, EXC 277) within the EICCI-project. Wethank Dr. Wolfgang Sturzl for kindly providing us with a dataset of a laser scannedoutdoor environment.

References

1. Bertrand, O.J., Lindemann, J.P., Egelhaaf, M.: A bio-inspired collision avoidancemodel based on spatial information derived from motion detectors leads to commonroutes. PLoS Comput. Biol. 11(11), e1004339 (2015)

2. Borst, A.: Modelling fly motion vision. In: Feng, J. (ed.) Computational Neuro-science: A Comprehensive Approach, pp. 397–429. Chapman and Hall/CTC, BocaRaton, London, New York (2004)

3. Borst, A.: Fly visual course control: behaviour, algorithms and circuits. Nat. Rev.Neurosci. 15(9), 590–599 (2014)

4. Egelhaaf, M., Boeddeker, N., Kern, R., Kurtz, R., Lindemann, J.P.: Spatial visionin insects is facilitated by shaping the dynamics of visual input through behavioralaction. Front. Neural Circuits 6(108), 1–23 (2012)

178 H.G. Meyer et al.

5. Egelhaaf, M., Kern, R., Lindemann, J.P.: Motion as a source of environmentalinformation: a fresh view on biological motion computation by insect brains. Front.Neural Circuits 8(127), 1–15 (2014)

6. Goslin, M., Mine, M.R.: The panda3d graphics engine. Computer 37(10), 112–114(2004)

7. Koenderink, J.J.: Optic flow. Vis. Res. 26(1), 161–179 (1986)8. Kress, D., Egelhaaf, M.: Head and body stabilization in blowflies walking on dif-

ferently structured substrates. J. Exp. Biol. 215(9), 1523–1532 (2012)9. Kress, D., Egelhaaf, M.: Impact of stride-coupled gaze shifts of walking blowflies

on the neuronal representation of visual targets. Front. Behav. Neurosci. 8(307),1–13 (2014)

10. Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with anapplication to stereo vision. In: IJCAI, vol. 81, pp. 674–679 (1981)

11. Matthews, R.W., Matthews, J.R.: Insect Behavior. Springer, Netherlands (2009)12. Miyamoto, K.: Fish eye lens. JOSA 54(8), 1060–1061 (1964)13. Montano, L., Asensio, J.R.: Real-time robot navigation in unstructured environ-

ments using a 3d laser rangefinder. In: Proceedings of the 1997 IEEE/RSJ Inter-national Conference on Intelligent Robots and Systems, IROS 1997, vol. 2, pp.526–532. IEEE (1997)

14. Paskarbeit, J., Annunziata, S., Basa, D., Schneider, A.: A self-contained, elasticjoint drive for robotics applications based on a sensorized elastomer coupling -design and identification. Sens. Actuators A Phys. 199, 56–66 (2013)

15. Paskarbeit, J., Schilling, M., Schmitz, J., Schneider, A.: Obstacle crossing of a real,compliant robot based on local evasion movements and averaging of stance heightsusing singular value decomposition. In: 2015 IEEE International Conference onRobotics and Automation (ICRA), pp. 3140–3145. IEEE (2015)

16. Petrowitz, R., Dahmen, H., Egelhaaf, M., Krapp, H.G.: Arrangement of opticalaxes and spatial resolution in the compound eye of the female blowfly calliphora.J. Comp. Physiol. A 186(7–8), 737–746 (2000)

17. Schilling, M., Hoinville, T., Schmitz, J., Cruse, H.: Walknet, a bio-inspired con-troller for hexapod walking. Biol. Cybern. 107(4), 397–419 (2013)

18. Schwegmann, A., Lindemann, J.P., Egelhaaf, M.: Depth information in naturalenvironments derived from optic flow by insect motion detection system: a modelanalysis. Front. Comput. Neurosci. 8(83), 1–15 (2014)

19. Shoemaker, P.A., Ocarroll, D.C., Straw, A.D.: Velocity constancy and models forwide-field visual motion detection in insects. Biol. Cybern. 93(4), 275–287 (2005)

20. Srinivasan, M., Guy, R.: Spectral properties of movement perception in the droneflyeristalis. J. Comp. Physiol. A 166(3), 287–295 (1990)

21. Sturzl, W., Boddeker, N., Dittmar, L., Egelhaaf, M.: Mimicking honeybee eyes witha 280 field of view catadioptric imaging system. Bioinspir. Biomim. 5(3), 036002(2010)

22. Sturzl, W., Grixa, I., Mair, E., Narendra, A., Zeil, J.: Three-dimensional modelsof natural environments and the mapping of navigational information. J. Comp.Physiol. A 201(6), 563–584 (2015)

MIRO: A Robot “Mammal” with a BiomimeticBrain-Based Control System

Ben Mitchinson and Tony J. Prescott(✉)

Department of Psychology and Sheffield Robotics, University of Sheffield,Western Bank, Sheffield, S10 2TN, UK

{b.mitchinson,t.j.prescott}@sheffield.ac.uk

Abstract. We describe the design of a novel commercial biomimetic brain-basedrobot, MIRO, developed as a prototype robot companion. The MIRO robot isanimal-like in several aspects of its appearance, however, it is also biomimetic ina more significant way, in that its control architecture mimics some of the keyprinciples underlying the design of the mammalian brain as revealed by neuro‐science. Specifically, MIRO builds on decades of previous work in developingrobots with brain-based control systems using a layered control architecturealongside centralized mechanisms for integration and action selection. MIRO’scontrol system operates across three core processors, P1-P3, that mimic aspectsof spinal cord, brainstem, and forebrain functionality respectively. Whilstdesigned as a versatile prototype for next generation companion robots, MIROalso provides developers and researchers with a new platform for investigatingthe potential advantages of brain-based control.

1 Introduction

Many robots have been developed that are animal-like in appearance; a much smallernumber have been designed to implement biological principles in their control systems[1, 2]. Of these, even fewer have given rise to commercial platforms that demonstratethe potential for brain-based, or neuromimetic, control in real-world systems. Buildingon more than two decades of research on robots designed to emulate animal behaviorand neural control [3–7] —that has developed key competences such as sensorimotorinteraction, orienting, decision-making, navigation, and tracking—we teamed with anindustrial designer, experts in control electronics, and a manufacturer, to create anaffordable animal-like robot companion. The resulting platform, MIRO, was originallydesigned to be assembled in stages, where each stage constitutes a fully-operationalrobot that demonstrates functionality similar to that seen in animals. These stages looselyrecapitulate brain development, as well as, to some extent, brain/phylogenetic evolution.with the finalized robot emulating some of the core functionality of a generalizedmammal. Previous publications have reported on the potential of the MIRO robot as abiomimetic social companion [8] and as a platform for education and entertainment [9].In this article we (i) describe the principles of brain-based control that have inspiredMIRO, (ii) outline the morphology and hardware design, and (iii) detail the three keylevels of the MIRO control architecture and the functionality to which they give rise.

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 179–191, 2016.DOI: 10.1007/978-3-319-42417-0_17

We end our article by briefly discussing some of the trade-offs we have made in devel‐oping a functioning brain-based robot as a commercial product.

2 Principles of Mammal-Like Brain-Based Control

Living, behaving systems display patterns of behavior that are integrated over space andtime such that the animal controls its effector systems in a coordinated way, generatingsequences of actions that maintain homeostatic equilibrium, satisfy drives, or meet goals.How animals achieve behavioral integration is, in general, an unsolved problem inanything other than some of the simplest invertebrates. This has also been called theproblem of architecture, and it is equally as problematic for robots as it is for animals[10]. Today’s robots are notoriously “brittle” in that their behavior—which may appearintegrated and coordinated with respect to a well-defined task—can rapidly break downand become disintegrated when task parameters go outside those anticipated by therobot’s programmers. Animals can also go into states of indecision and disintegrationwhen challenged by difficult situations [11], but generally show a robustness andcapacity to quickly adapt that is the envy of roboticists [12].

We believe that neuroscience and neuroethology have important lessons for roboticsconcerning the problem of architecture. Specifically, theoretical and computationalanalyses of animal nervous systems point to the presence of “hybrid” control architec‐tures that combine elements of reactive control with integrative mechanisms that operateboth in space, coordinating different parts of the body, and in time, organizing behaviorover multiple time-scales (for discussion, see, [1, 13–15]).

One key principle, whose history dates at least to the 19th century neurologist JohnHughlings Jackson [16], is that of layered architecture. A layered control system is onein which there are multiple levels of control at which the sensing apparatus is interfacedwith the motor system [17]. It is distinguished from hierarchical control by the constraintthat the architecture should exhibit dissociations, such that the lower levels still operate,and exhibit some sort of behavioral competence, in the absence (through damage orremoval) of the higher layers but not vice versa. A substantial body of the neuroscienceliterature can be interpreted as demonstrating layered control systems in the vertebratebrain; layering has also been an important theme in the design of artificial controlsystems, for instance, for autonomous robots [18]. The notion of a layered architecturehas been mapped out in some detail in the context of specific types of behavior. Forexample, in [15], we described how the vertebrate defense system—the control systemthat protects the body from physical harm—can be viewed as being instantiated inmultiple layers from the spinal cord (reflexes), through the hindbrain (potentiatedreflexes), midbrain (coordinated responses to species-specific stimuli), forebrain (coor‐dinated responses to conditioned stimuli), and cortex (modification of responsesaccording to context). In this system the higher layers generally operate by modulating(suppressing, potentiating, or modifying) responses generated by the lower layers.

Whilst the brain shows clear evidence of layered control there are other importantgoverning principles in its organization. Indeed, a system that worked by the principlesof layered control alone would be too rigid to exhibit the intelligent, flexible behavior

180 B. Mitchinson and T.J. Prescott

that mammals are clearly capable of. One proposal, stemming from the research of theneurologist Wilder Penfield, is of a centralized, or centrencephalic, organizing principlewhereby a group of central, sub-cortical brain structures serves to coordinate and inte‐grate the activity of both higher- and lower-level neural systems [19]. Candidate struc‐tures include the midbrain reticular formation—which may be important in integratingbehavior within the brainstem, and in regulating behavior during early development—and the basal ganglia, a group of mid- and forebrain structures that we have argued playa critical role in action selection. We have previously developed several embodiedmodels of these brain systems (see Fig. 1) and have demonstrated their sufficiency togenerate appropriate behavioral sequences for mobile robots engaged in activities suchas simulated foraging [4, 20].

Fig. 1. Neurorobotic models of control architectures. Left: [4] embedded a model of thevertebrate basal ganglia in a table-top robot and showed its ability to control action selection andbehavioural sequencing for a simulated foraging task. Right: Shrewbot [5] is one of series ofwhiskered robots developed to explore the effectiveness of brain-based control architectures ingenerating life-like behaviour.

Our research on biomimetic robot control architectures is predicated on the notionthat the principles of both centrencephalic organization and layered control are at workin mammalian brains and can be co-opted to generate coordinated and robust behaviorfor robots. Over recent years we have developed a number of neurorobotic models tofurther test this proposition [5, 21], of which the MIRO robot is the first commercialinstantiation.

A further question with regard to the problem of control architecture concerns thefundamental units of selection. The neuroethology literature suggests a decompositionof control into behavioral sub-systems that then compete to control the animal (see [15,22], an approach that has been enthusiastically adopted by researchers in behavior-basedrobotics (see, e.g. [23]). An alternative hypothesis emerges from the literature on spatialattention, particularly that on visual attention in primates including humans [24]. Thisapproach suggests that actions, such as eye movements and reaches towards targets, aregenerated by first computing a ‘salience map’ that integrates information about the rele‐vance (salience) to the animal of particular locations in space into a single topographic

MIRO: A Robot “Mammal” 181

representation. Some maximization algorithm is then used to select the most salientposition in space towards which action is then directed. Of course, the approaches ofbehavioral competition and salience map competition are, again, not mutually exclusiveand it is possible to imagine various hierarchical schemes, whereby, for instance, abehavior is selected first and then a point in space to which the behavior will be directed.In the mammalian brain, sensorimotor loops involving the cortex, superior colliculus,basal ganglia, and midbrain areas such as the periaqueductal gray, interact to controlhow the animal orients towards or away from different targets and what actions andbehaviors are then selected with respect to these targets [15]. Other structures providecontextual information based on past experience—the hippocampal system, for instance,contributes to the animal’s sense of time and place—thereby promoting better decisionsin the here-and-now [25].

In the following we briefly describe the physical instantiation of MIRO as a robotplatform and then return to the question of how MIRO has been designed to support abrain-based control architecture.

3 The MIRO Platform

The MIRO platform (see Fig. 2) is built around a core of a differential drive base and athree degree-of-freedom (DOF) neck (lift, pitch, yaw). Additional DOFs include twofor each ear (curl, rotate), two for the tail (droop, wag), and one for the eyelids (open/close). Whilst these latter DOFs target only communication, the movements of the neckand body that serve locomotion and active sensing play a significant role in communi‐cation as well. The platform is also equipped for sound production and with two arraysof colored lights, one on each side, both elements serving communication and/oremotional expression.

All DOFs in MIRO are equipped with proprioceptive sensors (potentiometers forabsolute positions and optical shaft encoders for wheel speed). Four light level sensorsare placed at the corners of the base, two task-specific ‘cliff sensors’ point down fromits front face, and four capacitive sensors are arrayed along the inside of the body shellproviding sensing of direct human contact. In the head, stereo microphones (in the baseof the ears) and stereo cameras (in the eyes) are complemented by a sonar ranger in thenose and an additional four capacitive sensors over the top and back of the head (behindthe ears). Accelerometers are present in both head and body.

MIRO has a three-level processing stack (see below). Peripheral components arereached on an I2C bus from the ‘spinal processor’ (ARM Cortex M0), which commu‐nicates via SPI with the ‘brainstem processor’ (ARM Cortex M0/M4 dual core), whichin turn communicates via USB with the ‘forebrain processor’ (ARM Cortex A8). Allperipherals and some aspects of processing are accessible from off-board through WiFiconnectivity (with MIRO optionally configured as a ROS—Robot Operating System—node), and the forebrain processor can be reprogrammed if lower-level access is required(lower processors can be re-programmed if desired, though with more onerous require‐ments to respect the specifics of the platform).

182 B. Mitchinson and T.J. Prescott

4 Control Architecture of the MIRO Robot

As explained above, a fundamental feature of the MIRO control architecture is its layeredform as further illustrated in Fig. 3 (which is not exhaustive but includes the key archi‐tectural elements). Processing loops are present at many different levels, generatingactuator control signals based on sensory signals and current state. In addition, highersystems are able to modulate the operation of loops lower down (a few examples areshown in the figure) thus implementing a form of subsumption [18]. Each layer buildsupon the function of those below, so that the architecture is best understood from thebottom-up. We place the loops into three groups, each loosely associated with a broadregion of the mammalian central nervous system, as follows.

Spinal CordThe first layer, which we denote “spinal cord”, provides two types of processing. Thefirst is signal conditioning—non-state-related transformations that can be appliedunconditionally to incoming signals. This includes robot-specific operations such asremoving register roll-overs from the shaft encoder signals, but also operations withbiological correlates.. An example of the latter is automatic acquisition of the zero-point of accelerometer signals (accounting for variability of manufacture) which isfunctionally comparable to sensory habituation in spinal cord neurons. The resulting“cleaned” or “normalized” signals provide the input to the second type of processingin this layer—reflex loops. A bilateral “cliff reflex” inhibits forward motion of eachwheel at the lowest level if the corresponding cliff sensor does not detect a floorsurface. A parallel “freeze reflex” watches for signals that might indicate the pres‐ence of another agent (a tilting acceleration-due-to-gravity vector, or touch on anyof the touch sensors) and inhibits all motions, sounds, and lighting effects when

Fig. 2. The MIRO prototype companion robot. Some example MIRO behavior can be seen athttps://youtu.be/x4tya6Oj5sU

MIRO: A Robot “Mammal” 183

triggered. If left alone, MIRO will slowly recover and begin to move and vocalizeonce more.

Fig. 3. Control architecture of MIRO loosely mapped onto brain regions (spinal cord, brainstem,forebrain). Signal pathways are excitatory (open triangles), inhibitory (closed triangles), orcomplex (closed circles). See text for description of components.

184 B. Mitchinson and T.J. Prescott

All of the reflexes can be inhibited by higher systems, allowing them to be “switchedoff” if a higher-level understanding of MIRO’s context demands it. Overall, this layercan be characterized as implementing “reactive control”.

BrainstemWe group some of the most central elements of MIRO’s biological control system intothe second layer, denoted “brainstem”. This layer is concerned with simple action selec‐tion, the computation and maintenance of affective state, simple spatial behaviors andthe generation of motor patterns to drive the actuators.

Affect is represented using a circumplex model derived from affective neuroscience[26], that comprises a two-dimensional state representing valence (unpleasantness,pleasantness) and arousal. Fixed transforms map events arising in MIRO’s sensoriuminto changes in affective state: for example, stroking MIRO drives valence upwards,whilst striking him on the head drives valence down. Baseline arousal is computed froma number of sources including the real-time clock. That is, MIRO has a circadian rhythm,being more active during daylight hours. General sound and light levels also affectbaseline arousal, whilst discrete events cause acute changes of affective state (very loudsound events raise arousal and decrease valence, for example).

MIRO expresses affect in a number of ways. Most directly, a set of “social” patterngenerators (SPG) drive the light displays, as well as movement of the ears, tail, andeyelids, so as to indicate affective state [8]. Meanwhile, MIRO’s vocalization model, acomplete generative mechano-acoustic model of the mammalian vocal system [27], ismodulated by affect, so that MIRO’s voice can range from morose to manic, angry torelaxed. More indirectly, MIRO’s movements are modulated also by affect: low/higharousal slows/speeds movement, and very low arousal leads to a less upright posture ofthe neck.

The other major system in the brainstem layer is a spatial behavior system modeledon the management of spatial attention and behavior in superior colliculus and relatednuclei in mammals [28]. This system comprises a topographic salience map of the spacearound MIRO’s head which is driven by aspects of both visual and aural inputs. Onefilter generates positive salience from changes in brightness in camera images, so thatmovement is typically a key generator of salience. Another, alongside, uses a Jeffressmodel [29] to localize the source of loud sound events so that a representation of theirintensity can be added to the salience map at the appropriate location. Other aspects ofthese sensory streams, as well as signals from other sensory modalities, can be config‐ured to contribute to this global salience map in a straightforward way [30].

Simple hard-coded filters generate behavioral plans from this map: “where” iscomputed as the maximum of the map; “what” is computed by combining MIRO’scurrent affective state with the nature of the stimulus (for example its size, location, ortemporal nature). The system generates behavioral plans including “orient” (turn tovisually “foveate” the stimulus), “avert” (turn away from the stimulus), “approach” and“flee” (related behaviors with locomotion components), and assigns a priority (a scalarvalue) to each plan. A model of the basal ganglia (BG) [4, 22] is then used to select,with persistence and pre-emption, one of these plans for execution by the motor plantat any one time. Overall, this system corresponds closely to similar, hard-wired, behavior

MIRO: A Robot “Mammal” 185

systems that have been identified in several animal species, including rodents [31] andamphibians [14].

This loop is closed through a motor pattern generator (MPG) that takes as inputbehavioral plans and generates time series signals for the actuators. Any behavioral planis encoded as an open-loop trajectory for a point in the frame of reference of one of therobot’s kinematic links. In all current plans, the point chosen corresponds to a “gener‐alized sensory fovea” [30, 31] just in front of the nose; thus, MIRO is “led by the nose”as a behavioral plan executes. The MPG comprises a kinematic self-model that iscomputed by moving the guided point and then identifying the remaining parameters(undriven) of the model through a principle of “least necessary movement”, starting withthe most distal DOFs. This computation is performed using a non-iterated coordinatedescent procedure. The lack of iteration limits the quality of the approximate solution,but is very cheap to compute, biologically plausible, and performs reasonably well. Inprevious work, we have used an adaptive filter model as a pre-processing stage to thisMPG, greatly improving accuracy, and suggested that this may be a role played bymammalian cerebellum [31].

A second, distinct, kinematic self-model is used to estimate MIRO’s configurationfor the interpretation of sensory signals. The model combines motor efferent signals withsensory afferent (proprioceptive) signals through a complementary filter [32] to derivea timely estimate of MIRO’s instantaneous configuration. This configuration is availableas an input to the analysis of data with a spatial component; for example, it determinesthe optical axis of the cameras when a video frame was captured.

The brainstem layer contains several other sub-systems that have biological corre‐lates. For instance, sleep dynamics are implemented as a relaxation oscillator, withwakefulness and exhaustion the two oscillator states. Thus, MIRO spends around fivein every twenty minutes “asleep”, expressed by closed eyes and a lowered head. Motorreafferent noise is present in MIRO’s sensory streams in several forms—particularsources include obstruction of the cameras by blinking of the eyelids, corruption of videoframes through self-motion (blurring), and the presence of audio noise whilst motorsare active. All of these forms of noise are eliminated from the incoming data streams bygating, based on efferent and afferent cues of their presence. Thus, for example, MIROwill not attempt to detect motion when it is, itself, in motion. Selective suppression ofsensory streams during some forms of motion is also a feature of biological vision [33].

ForebrainMIRO’s forebrain control systems are under present development. Figure 2 gives anindication of the character of components that are anticipated for this layer. The natureof the control architecture allows that these “higher” systems can be built on top of theexisting layers, taking advantage of already-implemented functionality. For example, ahigher system intended to perform task-specific orienting would not need to replicatethe orienting system that is already present. Rather, a suitable modulation can be appliedto the existing spatial salience filters, or additional filters added, and the orientingbehavioral plan can be “primed” [30]. The result is a tendency to perform orientingtowards the primed region of signal space (or physical space). In MIRO, all lowersystems are amenable to modulation; highlighted in the diagram are implemented

186 B. Mitchinson and T.J. Prescott

modulation routes allowing affect to be driven by influences from the forebrain layer orreflexes to be inhibited completely allowing the recovery of direct control. Currentresearch is directed at implementing a spatial cognition module modeled on the mamma‐lian hippocampus that will support inhibition-of-return during exploratory behavior andwill allow the robot to learn about, and navigate to, important sites such as a home ‘bed’.

The centrality of the basal ganglia model to any extension of the motor repertoire isnotable. Since there is only one motor plant, only one motor pattern should be selectedat any one time (simultaneous activation of multiple motor plans through the same outputspace constituting a motor error). Therefore, some selection mechanism is required sothat only one plan is disinhibited at any one time. There is substantial support for thehypothesis that the vertebrate basal ganglia is such a centralized selection mechanism,that may implement a form of optimal decision-making between competing actions, thatoperates across the different layers of the neuraxis, and that has contributed to theflexibility and scalability of the vertebrate brain architecture [4, 10, 22, 31].

Processing Stack of MIRO RobotThe layers of MIRO’s biomimetic control architecture are mirrored in their implemen‐tation distributed across three on-board processors as shown in Fig. 4. A fourth level ofprocessing, denoted “P4”, is available by inclusion of off-board systems into the controlstack. The rationale for this arrangement has a pedagogic aspect (ease of understanding)but the key benefits are functional.

One important feature is that the control latency of loops through the lowest reprog‐rammable processor, P1, can be as low as a few milliseconds. This contrasts very favor‐ably with the control latency through an off-board processor, P4, which—even underfavorable conditions—can be hundreds of milliseconds. The inherent unreliability ofwireless communications means that off-board latency can, on occasion, be longer still.Thus, safety critical aspects of the control policy, such as that implemented by the cliffreflex, will display superior performance if implemented in P1 versus, say, P4. There is,unsurprisingly, a continuum of latencies from P1 (~10 ms) through P2 (~30 ms), P3(~50-200 ms), and P4 (100 ms or more).

Conversely, computational power (as well as energy consumption) increases as wemove upwards through the processing stack (see Figure). This means that there is alsoa continuum of “competence”, or control sophistication. P1 can respond fast, but lacksthe power to make sophisticated decisions. P2 is able to perform spatial processing, andrespond quickly to the spatial nature of events, but lacks the power to perform patterndiscriminations or image segmentation, say. P3 is more capable still, but as an on-boardprocessor on a battery-powered mobile robot still has tight computational constraints.The characteristic of increasing latency and control sophistication as we move upthrough the different levels of layered architecture is shared by vertebrate brains [15].Latencies of escape reflexes implemented in spinal cord can be ~10 ms, but reflexresponses are relatively unsophisticated and involve minimal signal processing; mean‐while, midbrain responses to visual events begin after 50 ms [34], whilst classificationof objects by human visual cortex begins to emerge at around 100 ms [35] but allows amuch more sophisticated response.

MIRO: A Robot “Mammal” 187

This distribution of substrates from “fast and simple” through to “slow and sophis‐ticated” may be a potentially useful design element for many robots. One aspect of“simple”, that can easily be forgotten, is “less likely to fail”. Especially during devel‐opment, sophisticated systems such as P3 are highly prone to transient failures; havinglower-level systems that protect the robot from possible damage will be beneficial. Atthe same time, higher processors can be put to sleep when they are not required, savingpower and leaving lower processors to watch for events that may turn out to be behav‐iorally-relevant. The downside to this tiered processing stack is design complexity. Cost,however, may not be a serious concern, since the simpler processors are rather cheapparts.

A concern specific to robotics is increasing accessibility as we move up the stack.Running new control code on P4 can be as simple as pressing a key or clicking a mouse;on P3, at least a network file transfer will be required, and perhaps also a cross-compilestep; changing the control code in P2 requires reprogramming the majority of the sectors

Fig. 4. The biomimetic MIRO control architecture is implemented across three on-boardprocessors, P1-3, each loosely associated with a broad region of the mammalian brain. The designphysically displays dissociation, since P2 (and, to a more limited extent, P1) is able to control therobot independently of higher processing layers. “P4” denotes off-board processing, and “P0”non-reprogrammable peripheral-specific processors.

188 B. Mitchinson and T.J. Prescott

of the on-board FLASH, an operation that takes a few seconds, and requires theharnessing of P3 as a mediator; reprogramming P1 requires that the robot be powereddown and undergo a minor wiring change first, before being updated, reconfigured andpowered back up. Changing the code in any P0 (non-reprogrammable) processorrequires installing a new part. Thus, the development cycle tends to favour placing codethat is changing often (typically sophisticated) higher up, and code that is more stable(typically simple) lower down. Whilst brain evolution is fundamentally different fromthis style of robot design, there is a similar tendency in nature towards conservation ofstructure and function towards the lower end of the neuraxis (spinal cord/brainstem),and increase in flexibility and adaptability at the upper end (cortex) [15].

5 Conclusion

We began with the goal of creating an affordable animal-like robot in which we couldembed a biomimetic control architecture that we had previously developed on expensivebespoke robotic platforms. Important constraints in the design process, that were laterrelaxed but still strongly influenced the outcome, were the need to have a platform thatcould operate in an integrated way at multiple stages of construction, and that no singlecomponent should cost more than $10. We have found that a brain-based design isactually well-suited to these challenges of incremental construction and use of cheap,off-the-shelf parts. During the course of evolution, the mammalian brain has adaptedand scaled to many different body types and ecological niches; the MIRO robot showsthat future living machines, built of non-biological components such as plastic andsilicon, can also make use of layered control architectures inspired by, and abstractedfrom, those we find in animals.

Acknowledgments. The development of the MIRO robot was funded by Eaglemoss Publishingand Consequential Robotics with contributions from Sebastian Conran, Tom Pearce, Victor Chen,Dave Keating, Jim Wyatt, Maggie Calmels, and Emily Collins. Our research on layeredarchitectures was also supported by the FP7 WYSIWYD project (ICT-612139), and the EPSRCBELLA project (EP/I032533/1).

References

1. Prescott, T.J., et al.: Embodied Models and Neurorobotics. In: Arbib, M.A., Bonaiuto, J.J.(eds.) From Neuron to Cognition via Computational Neuroscience. MIT Press, Cambridge(in press)

2. Floreano, D., Auke, J., Ijspeert, J., Schaal, S.: Robotics and neuroscience. Curr. Biol. 24(18),R910–R920 (2014)

3. Prescott, T.J., Ibbotson, C.: A robot trace-maker: modeling the fossil evidence of earlyinvertebrate behavior. Artif Life 3, 289–306 (1997)

4. Prescott, T.J., et al.: A robot model of the basal ganglia: behaviour and intrinsic processing.Neural Netw. 19(1), 31–61 (2006)

5. Pearson, M.J., et al.: Biomimetic vibrissal sensing for robots. Philos. Trans. R. Soc. Lond. BBiol. Sci. 366(1581), 3085–3096 (2011)

MIRO: A Robot “Mammal” 189

6. Lepora, N.F., et al.: Optimal decision-making in mammals: insights from a robot study ofrodent texture discrimination. J. R. Soc. Interface 9(72), 1517–1528 (2012)

7. Mitchinson, B., et al.: Biomimetic tactile target acquisition, tracking and capture. Robot.Auton. Syst. 62(3), 366–375 (2014)

8. Collins, E.C., Prescott, T.J., Mitchinson, B.: Saying it with light: a pilot study of affectivecommunication using the MIRO robot. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A.,Prescott, T.J. (eds.) Living Machines 2015. LNCS, vol. 9222, pp. 243–255. Springer,Heidelberg (2015)

9. Collins, E.C., et al.: MIRO: a versatile biomimetic edutainment robot. In: 12th Conferenceon Advances in Computer Entertainment, Iskandar, Malaysia (2015)

10. Prescott, T.J.: Forced moves or good tricks in design space? landmarks in the evolution ofneural mechanisms for action selection. Adapt. Behav. 15(1), 9–31 (2007)

11. Hinde, R.A.: Animal Behaviour: a Synthesis of Ethology and Comparative Psychology.McGraw-Hill, London (1966)

12. McFarland, D., Bosser, T.: Intelligent Behaviour in Animals and Robots. MIT Press,Cambridge (1993)

13. Verschure, P.F.M.J., Krose, B., Pfeifer, R.: Distributed adaptive control: the self-organizationof structured behavior. Robot. Auton. Syst. 9, 181–196 (1992)

14. Arbib, M.A., Liaw, J.S.: Sensorimotor transformations in the worlds of frogs and robots. Artif.Intell. 72(1–2), 53–79 (1995)

15. Prescott, T.J., Redgrave, P., Gurney, K.N.: Layered control architectures in robots andvertebrates. Adapt. Behav. 7(1), 99–127 (1999)

16. Jackson, J.H.: Evolution and dissolution of the nervous system. In: Taylor, J. (ed.) SelectedWritings of John Hughlings Jackson. Staples Press, London (1884/1958)

17. Prescott, T.J.: Layered control architectures. In: Pashler, H. (ed.) Encyclopedia of Mind, pp.464–467. Sage, London (2013)

18. Brooks, R.A.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom.RA-2, 14–23 (1986)

19. Penfield, W.: Centrencephalic integrating system. Brain 81, 231–234 (1958)20. Humphries, M.D., Gurney, K., Prescott, T.J.: Is there a brainstem substrate for action

selection? Philos. Trans. R. Soc. Lond. B Biol. Sci. 362(1485), 1627–1639 (2007)21. Prescott, T.J., et al.: Whisking with robots: From rat vibrissae to biomimetic technology for

active touch. IEEE Robot. Autom. Mag. 16(3), 42–50 (2009)22. Redgrave, P., Prescott, T., Gurney, K.N.: The basal ganglia: a vertebrate solution to the

selection problem? Neuroscience 89, 1009–1023 (1999)23. Brooks, R.A.: New approaches to robotics. Science 253, 1227–1232 (1991)24. Gandhi, N.J., Katnani, H.A.: Motor functions of the superior colliculus. Annu. Rev. Neurosci.

34, 205–231 (2011)25. Fox, C., et al.: Technical integration of hippocampus, basal ganglia and physical models for

spatial navigation. Front. Neuroinformatics, 3(6) (2009)26. Posner, J., Russell, J.A., Peterson, B.S.: The circumplex model of affect: an integrative

approach to affective neuroscience, cognitive development, and psychopathology. Dev.Psychopathol. 17(3), 715–734 (2005)

27. Hofe, R., Moore, R.K.: Towards an investigation of speech energetics using ‘AnTon’: ananimatronic model of a human tongue and vocal tract. Connection Sci. 20(4), 319–336 (2008)

28. Mitchinson, B., Prescott, T.J.: Whisker movements reveal spatial attention: a unifiedcomputational model of active sensing control in the rat. PLoS Comput. Biol. 9(9), e1003236(2013)

190 B. Mitchinson and T.J. Prescott

29. Jeffress, L.A.: A place theory of sound localization. J Comp Physiol Psychol. 41(1), 35–39(1948)

30. Mitchinson, B.: Attention and orienting. In: Lepora, P.T.J.N., Verschure, P.F.M.J. (eds.)Living Machines: A Handbook of Research in Biomimetic and Biohybrid Systems OUP:Oxford (in press)

31. Prescott, T.J., et al.: The robot vibrissal system: understanding mammalian sensorimotor co-ordination through biomimetics. In: Krieger, P., Groh, A. (eds.) Sensorimotor Integration inthe Whisker System, pp. 213–240. Springer, New York (2015)

32. Higgins, W.T.: A comparison of complementary and Kalman filtering. IEEE Trans. Aerosp.Electron. Syst. AES-11(3), 321–325 (1975)

33. Burr, D.C., Morrone, M.C., Ross, J.: Selective suppression of the magnocellular visualpathway during saccadic eye movements. Nature 371(6497), 511–513 (1994)

34. Redgrave, P., Prescott, T.J., Gurney, K.: Is the short-latency dopamine response too short tosignal reward error?. Trends Neurosci. 22(4), 146–151 (1999)

35. Thorpe, S.J.: The speed of categorization in the human visual system. Neuron 62(2), 168–170 (2009)

MIRO: A Robot “Mammal” 191

A Hydraulic Hybrid Neuroprosthesis for Gait Restorationin People with Spinal Cord Injuries

Mark J. Nandor1,2(✉), Sarah R. Chang1,3, Rudi Kobetic1, Ronald J. Triolo1,4,and Roger Quinn2

1 Louis Stokes Cleveland Department of Medical Affairs Medical Center, Cleveland, USAmnandor@aptcenter.org, rkobetic@fescenter.org

2 Department of Mechanical Engineering, Case Western Reserve University, Cleveland, USArdq@case.edu

3 Department of Biomedical Engineering, Case Western Reserve University, Cleveland, USAsrc74@case.edu

4 Department of Orthopedics, Case Western Reserve University, Cleveland, USArxt24@case.edu

Abstract. The Hybrid Neuroprosthesis (HNP) is a hydraulically actuatedexoskeleton and implanted Functional Electrical Stimulation (FES) system thathas been designed and fabricated to restore gait to people with spinal cord injuries.The exoskeleton itself does not supply any active power, instead relying on animplanted FES system for all active motor torques. The exoskeleton insteadprovides support during quiet standing and stance phases of gait as well as sensoryfeedback to the stimulation system. Three individuals with implanted functionalelectrical stimulation systems have used the system to successfully walk shortdistances, but were limited in the flexion torques the stimulation system couldprovide.

1 Introduction

1.1 Functional Electrical Stimulation Enabled Gait

Restoring gait in people with spinal cord injuries is a long sought after and desired goalin rehabilitation research. This has been accomplished through orthotic bracing/exoskel‐etons (both passive and active), functional electrical stimulation (FES), and a combi‐nation or bracing and FES.

A spinal cord injury is a breakdown of communication – muscle activation signalscannot be sent through the damaged spinal column, however in many cases the subject’smuscles are still receptive to commands. FES utilizes an external command unit to issuecommands to the user’s own muscles.

Electrodes placed on the skin have been utilized for their relative ease of setup, focusingon eliciting hip and knee extensor muscles for standing, and eliciting a withdrawal reflexfor stepping. A typical surface stimulation consists of 4–6 channels [10, 11]. Implantedsystems have been able to provide a greater number of channels (upwards of 48) [9], withthat additional bandwidth providing a greater degree of control [12]. FES gait is typicallyrun open loop as a timed pattern of muscle triggerings.

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 192–202, 2016.DOI: 10.1007/978-3-319-42417-0_18

The primary disadvantage of FES enabled is the quick onset of muscle fatigue.Combined with the open loop nature of the system, walking distances are typically short.

1.2 Orthotics for People with Spinal Injuries

Many different orthoses have been designed for use by people with spinal cord injuries.Typically used is a Reciprocating Gait Orthosis (RGO), a passive device that whenstanding will lock the user’s knees and ankles in a neutral position, and mechanicallycouple contralateral hip motion – hip flexion on one side causes hip extension in theother. By prevention bilateral hip flexion, this arrangement is statically stable as well asfacilitates a reciprocating gait. However, the locked knees force the user to exert a largeamount of upper extremity forces to achieve sufficient foot/floor clearance during swing.

1.3 Hybrid FES/Orthotic Devices

Combining electrical stimulation with an external orthotic has been accomplished previ‐ously in a variety of forms. Early efforts [2–4] focused on combining stimulation withpassive orthotics, such as a reciprocating gait orthosis. However, joint kinematics wasstill restricted by the orthoses used.

Later efforts sought to create a more natural looking gait by enabling the joints ofthe orthotic to lock and unlock based on sensor feedback [5, 6]. These devices relied onstimulation to provide all active motor power. More recent developments include effortsto combine stimulation with a fully active, robotic exoskeleton [7, 8].

Discussed here is an exoskeleton that utilizes passive electrohydraulic joints and FESthat combines the strengths while negating the weaknesses of each individual approach.It utilizes implanted stimulation (capable of providing relatively high torques - 60 Nmin hip flexion, 63 Nm in hip extension, 15 Nm in knee flexion, 80 Nm in knee extension[9]) for all of the propulsive power. Like an RGO, the exoskeleton is capable of lockingthe knee joints and reciprocally coupling the hip joints, providing stability during quietstanding and stance phases of gait. Unlike an RGO, the exoskeleton is able to automat‐ically change its joint constraints as needed, such as unlocking the knee of the swingleg, allowing for knee flexion and toe clearance. This reduces the need for compensatorymechanisms.

2 Design Overview

2.1 System Architecture Overview

The complete HNP exoskeleton (as shown in Fig. 1) consists a mechanical frame(a torso and two leg uprights) carrying 3 hydraulic subsystems – each knee contains aDual State Knee Mechanism (DSKM), and the hip joints are controlled by a hydraulicVariable Constraint Hip Mechanism (VCHM). Figure 2 shows the hydraulic circuitrycontained within each device. Additionally, the exoskeleton carries a variety of sensorsand electronics capable of providing wireless, independent operation outside of a lab

A Hydraulic HNP for Gait Restoration in People with Spinal Cord Injuries 193

environment, including joint angle encoders, force sensitive resistors in the soles of theshoe, and an inertial measurement unit carried on the torso of the exoskeleton.

Fig. 1. The Hybrid Neuroprosthesis exoskeleton

Fig. 2. The HNP hydraulic circuitry diagram

2.2 Dual State Knee Mechanism

The purpose of the DSKM is to lock the knee and provide support during quiet standingand the stance phases of gait, while unlocking and providing free, unimpeded motionduring the swing phase of gait, with knee flexion and extension torques being provided

194 M.J. Nandor et al.

by the FES system. Determination between stance and swing phases of gait is performedby the onboard sensors and microcontrollers.

To accomplish this, each DSKM consists of a 7/8 in. bore, 3 in. stroke hydrauliccylinder (Clippard Minimatic), with the rod and blind side of the cylinder connected toeach other. To control the flow of fluid, a two-way/two-position solenoid valve (AllenairCorporation) is placed in between. In its default unpowered state, the solenoid valve isclosed, preventing fluid flow from occurring and consequently the cylinder fromextending and retracting. A small accumulator (Bimba Corporation) is also placed inthe circuit to accommodate the fluid volume differential between blind side and rod side.Maximum operation pressure is limited to 600 psi, the maximum rating of the hydraulichose utilized.

To translate the linear motion of the cylinder to rotary motion of the joint, a threebar linkage is utilized. Other transmission concepts such as a rack and pinion gear setwere considered but dismissed as being too large and heavy. The linkage was designedand optimized according to a few specifications:

– Minimum range of motion: 10° of hyperextension to 110° of flexion.– Locking torque at neutral: minimum 70 Nm.– Minimum posterior protrusion of the hydraulic cylinder.

Figure 3 shows the final locking torque capability of the DSKM as a function of kneeangle.

Fig. 3. DKSM locking torque capability vs. joint angle

A Hydraulic HNP for Gait Restoration in People with Spinal Cord Injuries 195

2.3 Variable Constraint Hip Mechanism

Unlike the knee joint hydraulics, which is two independent units, both hip joints arecontrolled by a single mechanism. The hydraulics of the VCHM consist of two 7/8 in.bore, 3 in. stroke hydraulic cylinders (one for each hip joint), three four way/two positionhydraulic valves (Hydraforce), and a small accumulator. Like the DSKM, the VCHMis designed to let each hip joint operate independently locked an unlocked. Additionally,it is capable of reciprocally coupling hip motion. By directly connecting the rod andblind sides of the two cylinders, hip flexion on one side with cause hip extension on thecontralateral side and vice versa.

In its unpowered state, the VCHM defaults to the reciprocally coupled hip state.Combined with the locked knee default/unpowered state of the DSKM, in the case ofcomplete battery discharge or unintended power failure, the HNP exoskeleton willsimply act as a conventional reciprocating gait orthosis (RGO), a traditional orthoticdevice prescribed to people with spinal cord injuries. This is an important safety featureof the HNP exoskeleton – in the case of malfunction or power loss, the user is neverstranded, but can still utilize the exoskeleton as a familiar unpowered orthotic (Fig. 4).

Fig. 4. The VCHM locking torque capability vs. joint angle

The VCHM also utilizes 3 bar linkages to translate the linear cylinder motion intorotary hip joint motion. Similar to the knee joint transmission selection process, othertransmission options such as rack and pinion gear transmissions were considered anddismissed as too heavy and complex. The linkage was again constrained and optimizedby the following conditions;

– Range of motion – 20° of extension to 100° of flexion.– Locking Torque at neutral – minimum 70 Nm.– Minimum posterior protrusion of the hydraulic cylinder.

196 M.J. Nandor et al.

2.4 Mechanical Elements of the Exoskeleton

The mechanical frame of the exoskeleton is designed to resist maximum force from thecylinders. At 2482 kPa (600 psi), each cylinder is capable of exerting over 1596 N (360lbs.) of force. Finite element analysis was performed to ensure structural integrity undermaximum load. By placing all cylinder mounts in double shear, the bending momentdue to cylinder force on the exoskeleton frame is minimized, and the exoskeletal frameis only moderately stressed. Peak Von Neumann stresses from the simulation are under29 MPa (4.2 ksi.). The frame is constructed of 6061-T6 aluminum, with a yield strengthof 276 Mpa (40 ksi).

The shank portions of the exoskeleton uprights are designed to accept standard ther‐moformed ankle foot orthoses (AFOs). These are prescribed and fabricated by a certifiedorthotist and are custom fit to each individual user for best possible fit and preventionof pressure sores during use.

The exoskeleton is designed to accommodate users from 1.52 m (60 in.) to 1.93 m(76 in.) with a 100 kg. (220 lb.) maximum weight limit. To facilitate this, each legsegment can adjust its length, and the torso unit is able to adjust both width and anterior/posterior position of the hip centers. Total device weight is 17 kg (38 lbs.). VCHMvalves, accumulators, and electronic microprocessor, stimulation boards, battery, andvoltage regulator are all carried on the posterior portion of the torso, as shown in Fig. 5.

Fig. 5. Posterior mounted VCHM hydraulics and exoskeleton electronics

2.5 Electronics

Part of the exoskeleton’s purpose is to provide enough sensory feedback to determinewhat phase of gait the user is in, closing the loop with the actions of the stimulationsystem. To accomplish that, the following sensors are placed onto the exoskeleton:

A Hydraulic HNP for Gait Restoration in People with Spinal Cord Injuries 197

– Encoders at the hip and knee joints, reporting joint angle.– Force sensitive resistors within the soles of the shoes, capable to detecting heel strike

and toe off events– A nine axis inertial measurement unit (IMU) placed on the torso component of the

exoskeleton. This device reports absolute orientation of the torso in space.

These sensor readings are all read by a custom designed PCB with the appropriatesignal conditioning circuitry and an 8-bit microprocessor (Atmega Corp.). The micro‐controller is responsible for running the gait event detector (GED), and issuingcommands to the hydraulic valves and stimulation system. The user issues commandsvia a wireless finger switch held in the hand with four buttons.

The microcontroller communicates via Bluetooth to a nearby computer, runninga graphical user interface. This interface provides real time sensor data, displays thecurrent phase of gait determined by the GED, and records the sensor data for lateranalysis.

3 Device Evaluation

3.1 Locking Compliance

The exoskeleton must be able to lock and support the user’s body weight when necessary,such as during single/double stance phases of gait. To test this functionality, the exoskel‐eton joints were individually connected to a robotic dynamometer (Biodex MedicalSystems, Shirley, NY), capable of applying a measured torque while the joint is hydraul‐ically locked. The resultant angular displacement is shown in Figs. 6 and 7.

Fig. 6. DSKM locking compliance

3.2 Passive Resistance

Because all of the joint torque is provided by the stimulation, it is important to minimizethe passive resistance of system. To measure this passive resistance, the exoskeletonjoints were again attached to the dynamometer. While the joints were commanded to be

198 M.J. Nandor et al.

free, the dynamometer was commanded to rotate at constant angular velocity; recordingthe torque it was exerting to maintain that speed. Figures 8 and 9 display the datacollected for the DSKM and VCHM.

Fig. 8. DSKM passive resistance torque vs. speed

Fig. 9. VCHM passive resistance vs. joint speed

At the knee joint, approximately 20 % of the torque generated by stimulation in kneeflexion was needed to overcome the hydraulic passive resistance at 120°/s. 4 % of the

Fig. 7. VCHM locking compliance

A Hydraulic HNP for Gait Restoration in People with Spinal Cord Injuries 199

maximum generated extension torque is necessary to overcome the resistance in exten‐sion. At the hip joint, approximately 23 % of the generated stimulation torque wasnecessary to overcome the passive resistance in both flexion and extension.

3.3 Experimental Use

To date, three different subjects with implanted stimulation have walked with the HNP,with levels of injury ranging from T11 to C7. Two of the three had previous experiencewalking with stimulation only. A walker was used during all trials to provide stabilityand support. Early walking trials were meant to be a proof of concept and allow fordebugging and fine tuning of the controller/stimulation patterns.

All subjects transferred from their wheelchair onto a chair containing the exoskeletonfor donning. Standing was initiated by button press, at which time the exoskeleton jointswould unlock and after a short delay; stimulation of the extensor muscles was rampedup for standing. Once standing, the exoskeleton joints were locked and the encoderszeroed. Initiation of walking was triggered with another button press, with each subse‐quent step triggered by another button press. During walking, the hip joints remainedcoupled at all times, and the knee joints would unlock for the swing phase leg whileremaining locked for the stance phase leg. All data was collected wirelessly with acomputer.

While all subjects (shown in Fig. 10) were able to complete multiple steps, but wereultimately limited in walk length and duration by fatigue of flexor muscles, limiting kneeflexion (causing insufficient foot floor clearance) and hip flexion (limiting step lengthand making step initiation difficult).

Fig. 10. The HNP in use, on 3 separate volunteers with implanted stimulation

Figure 11 shows data collected from 4 steps from a single trial. The hip angle plotshows the effect of the hip-hip reciprocal coupling – driving the swing hip into flexioncauses the stance hip to extend. The data also shows the lack of knee flexion – stimulationwas unable to provide more than 30° of knee flexion.

200 M.J. Nandor et al.

Fig. 11. Recorded data from an experimental trial

3.4 Conclusion and Future Work

Described here is an electromechanical exoskeleton that is meant to expand on previousFES + RGO gait research. Like previous hybrid FES + RGO experiments, this HNPderives all of its active torques from stimulation. Unlike a purely passive RGO, theexoskeleton described here utilizes an onboard sensors, microcontroller and hydraulicactuators to detect the user’s phase of gait, and apply joint constraints accordingly.

Ultimately, the limitation of the HNP system is muscle fatigue. While this exoskel‐eton described here has the capability to unlock the knee joints for the swing phase ofgait, fatigue of the knee flexors means the user is unable to take full advantage of thatdesign feature. Similarly, fatigue of hip flexor muscles limits overall step length andwalk duration.

Future controller work will take advantage of the onboard telemetry and use that tomodulate the stimulation patterns. Typical FES enabled gait (including the preliminarywork shown here) is simply a timed pattern run open loop, without feedback. With thisexoskeleton, the sensors should be able to detect the onset of fatigue and increase themuscle activation levels to compensate.

Additionally, hardware revisions that aim to reduce the passive resistance of thejoints, as well as provide a small amount of assistive power can further delay the onsetof muscle fatigue, and increase walking speed and distance. Concepts that place smallknee flexion assist springs in the DKSM, or small assistive motors (sized to allow stim‐ulation to still provide a majority of the necessary walking torques) at each hip in theVCHM have been discussed.

Acknowledgements. The author would like to thank all APT Center and FES Centerinvestigators and engineers that contributed to this work, including Dr. Musa Audu, KevinFoglyano, and John Schnellenberger. This work was supported by grants from the Department of

A Hydraulic HNP for Gait Restoration in People with Spinal Cord Injuries 201

Defense (W81XWH-13-1-0099), and the Rehabilitation Research and Development Service ofVeteran Affairs (B0608-R).

References

1. To, C.S., Kobetic, R., Schnellenberger, J.R., Audu, M.L., Triolo, R.J.: Design of a variableconstraint hip mechanism for a hybrid neuroprosthesis to restore gait after spinal cord injury.IEEE/ASME Trans. Mechatron. 13(2), 197–205 (2008)

2. Hirokawa, S., Grimm, M., Le, T., Solomonow, M., Baratta, R.V., Shoji, H., D’Ambrosia,R.D.: Energy consumption in paraplegic ambulation using the reciprocating gait orthosis andelectrical stimulation of the thigh muscles. Arch. Phys. Med. Rehabil. 71, 687–694 (1990)

3. Solomonow, M., Baratta, R., Hirokawa, S., Rightor, N., Walker, W., Beaudette, P., Shoji, H.,D’Ambrosia, R.: The RGO generation II: muscle stimulation powered orthosis as a practicalwalking system for thoracic paraplegics. Orthopedics 12, 1309–1315 (1989)

4. Isakov, E., Douglas, R., Berns, P.: Ambulation using the reciprocating gait orthosis andfunctional electrical stimulation. Paraplegia 30, 239–245 (1992)

5. Goldfarb, M., Durfee, W.: Design of a controlled-brake orthosis for FES-added gait. IEEETrans. Rehabil. Eng. 4(1), 13–24 (1996)

6. Goldfarb, M., Korkowski, K., Harrold, B., Durfee, W.: Preliminary evaluation of a controlled-brake orthosis for FES-aided gait. IEEE Trans. Neur. Syst. Rehabil. Eng. 11(3), 241–248(2003)

7. Ha, K.H., Murray, S.A., Goldfarb, M.: An approach for the cooperative control of FES witha powered exoskeleton during level walking for persons with paraplegia. IEEE Trans. Neur.Syst. Rehabil. Eng. (99), 1

8. del-Ama, A., Gil-Agudo, Á., Pons, J., Moreno, J.: Hybrid FES-robot cooperative control ofambulatory gait rehabilitation exoskeleton. J. NeuroEng. Rehabil. 11(1), 27 (2014)

9. Kobetic, R., Marsolais, E.B.: Synthesis of paraplegic gait with multichannel functionalneuromuscular stimulation. IEEE Trans. Rehabil. Eng. 2(2), 66–79 (1994)

10. Marsolais, E.B., Kobetic, R.: Development of a practical electrical stimulation system forrestoring gait in the paralyzed patient. Clin. Orthop. (233), 64–74 (1988)

11. Alojz, K., et al.: Gait restoration in paraplegic patients: a feasibility demonstration usingmultichannel surface electrode FES. J. Rehabil. R&D/Veterans Adm. Dept. Med. Surg.Rehabil. R&D Serv. 20(1), 3–20 (1983)

12. Kobetic, R., Triolo, R.J., Marsolais, E.B.: Muscle selection and walking performance ofmultichannel FES systems for ambulation in paraplegia. IEEE Trans. Rehabil. Eng. 5(1),23–29 (1997)

202 M.J. Nandor et al.

Principal Component Analysisof Two-Dimensional Flow Vector Fields

on Human Facial Skin for EfficientRobot Face Design

Nobuyuki Ota, Hisashi Ishihara(B), and Minoru Asada

Graduate School of Engineering, Osaka University, Suita, Japanishihara@ams.eng.osaka-u.ac.jp

Abstract. In this study, deformation patterns of an adult male lowerface are measured and analyzed for efficient face design for androidrobots. We measured flow vectors for 96 points on the right half of thelower face for 16 deformation patterns, which are selected from Ekman’saction units. Namely, we measured 16 flow vector fields of facial skin flow.The flow vectors were created by placing ink markers on the front of theface and then video filming various facial motions. A superimposed imageof vector fields shows that each point moves in various directions. Princi-ple component analysis was conducted on the superimposed vectors andthe contribution ratio of the first principal component was found to be86 %. This result suggests that each facial point moves almost only inone direction and different deformation patterns are created by differentcombinations of moving lengths. Based on this observation, replicatingvarious kinds of facial expressions on a robot face might be easy becausean actuation mechanism that moves a single facial surface point in onedirection can be simple and compact.

1 Introduction

Facial expression is one of the important communication channels for humans;therefore, designers of communication robots have tried to replicate human facialexpressions on robot faces [1,2,4–14]. When humans change facial expressions,entire skin surfaces deform in complicated patterns, which is difficult for robotdesigners to replicate. To replicate several facial expressions on a single robotface, the designers should know how each point on a human face moves for severalfacial deformation patterns.

Traditionally, robot face designers [2,4,6,7,10,11,13,14] have decided thelocations of facial moving points and their directions based on Ekman’s actionunits (AUs), which are summarized and introduced in the Facial Action Cod-ing System (FACS) [3]. AUs are facial deformation patterns created when oneor more facial muscles are activated, and FACS explains that different facialexpressions are realized by different combinations of one or several action units.

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 203–213, 2016.DOI: 10.1007/978-3-319-42417-0 19

204 N. Ota et al.

However, AUs are not enough to design effective actuation mechanisms fora robot face because only verbal qualitative descriptions of deformations areintroduced in FACS. Instead, quantitative information such as flow vector fieldsmust be used to effectively design robot faces. Cheng et al. [2] have pointed outthis issue and measured facial deformations with a 3D scanner. However, theirobservation was limited to only four typical facial expressions, which were smile,anger, sadness, and shock.

Therefore, in this study, we measured every conceivable deformation patternespecially in the lower face of a Japanese adult male. Namely, we measuredflow vector fields of facial skin. Then, two kinds of compensation processing forthe measured vector fields, such as head movement compensation and neutralface matching, are executed. Finally, Principal Component Analysis (PCA) wasconducted to determine variations in movements and flow field trends of eachfacial point.

2 Method

2.1 Motion Patterns

FACS introduces sixteen deformation patterns in the lower face and they areselected for this experiment. Table 1 summarizes the selected deformation pat-terns. Each AU has its own AU number and name defined in FACS. Basically,

Table 1. Deformation patterns (Ekman’s action units) measured in this experiment

AU number Name

9 Nose Wrinkler

10 Upper Lip Raiser

11 Nasolabial Furrow Deepener

12 Lip Corner Puller

13 Sharp Lip Puller

14 Dimpler

15 Lip Corner Depressor

16 Lower Lip Depressor

17 Chin Raiser

18 Lip Pucker

20 Lip Stretcher

22 Lip Funneler

23 Lip Tightener

24 Lip Presser

25 Lips Part

28 Lips Suck

PCA of Two-Dimensional Flow Vector Fields on Human Facial Skin 205

the name contains information about the responsible part of the face (e.g., lipcorner) and its deformation pattern (e.g., puller).

FACS describes how to activate each AU and an example of its associatedfacial image. A Japanese adult male (hereafter demonstrator) activated each AUand his facial deformations were measured in this experiment.

2.2 Measurement

Ninety six measurement points were identified on the demonstrator’s skin, andink markers were placed on them, as shown in Fig. 1. Each marker was approxi-mately 2–5 mm in size, and their locations were decided so that they were placednot only on anatomically distinctive points, such as corners of the mouth andthe top of the nose, but also on other plane surfaces such as the cheek.

The movements (or flows) of the markers for each AU were recorded by videofilming (SONY HANDYCAM HDR-CX420) at the front of the demonstrator.While being filmed, the demonstrator began with a neutral face, activated oneof the selected AUs, and kept its deformation for several seconds. Sixteen videorecordings were created, each of which contains both the demonstrator’s neutralface and his deformed one.

Fig. 1. Locations of ink markers on the right half of a Japanese adult male’s lower face

2.3 Data Processing

To measure flow vectors of the markers, two video frames were selected from eachvideo recording of each AU. The first one contains the demonstrator’s neutralface and the second one contains his deformed one.

By comparing pixel coordinates of each marker in these two images, we canroughly calculate each marker’s flow vector. However, two kinds of image cor-rections are necessary to calculate more precise flow vectors and to superimpose

206 N. Ota et al.

several flow vectors of the same marker. The first one is the Head MovementCompensation. While activating each AU, the demonstrator’s head can move;therefore, the markers’ pixel coordinates can move even if the marker does notmove on face. The second one is Neutral Face Matching between different AUs tosuperimpose several vectors for the same marker. Even though the demonstratortried to show the same neutral face with the same head position in each videorecording session, these neutral faces can not be exactly the same. Therefore,starting points for flow vectors of the same marker are not in the same positionbetween different AUs.

Head Movement Compensation. For head movement compensation, fivereference points on the facial images shown in Fig. 2 were used. Reference pointswere placed on the top of the nose, on the mid-point between the eyes, and onthe mid-point between the corner of the eye and the ear. These points are knownto be relatively static on humans faces [2]. In addition to these three points, twoadditional reference points were placed on the top of the head and the chin.

The sizes and positions of the two images for each AU were adjusted so thateach reference point matched. This adjustment was manual because matchingfive points was relatively simple.

Fig. 2. Reference points on a facial image for head movement compensation

Neutral Face Matching. For neutral face matching between different AUs,first, the neutral face image for AU9 was randomly chosen as a reference image.Affine transformations were applied to 15 sets of 97 marker coordinates (96 inkmarkers and the reference point on the top of the chin) in neutral face imagesso that these marker sets would match with a marker set of the reference image.

PCA of Two-Dimensional Flow Vector Fields on Human Facial Skin 207

Two-dimensional coordinate vector Xi,j of a marker i in an image j is con-verted to a coordinate vector X ′

i,j by the equation

X ′i,j = AjXi,j + bj ,

where Aj is a linear transformation matrix and bj is a translation vector foran image j. The purpose is to find the parameters of Aj and bj so that thecoordinate vectors X ′

i,j(i = 1, . . . , 97) match with vectors Xi,9 for an neutralface image of AU9.

To find the appropriate parameters, Steepest Descent Method was imple-mented with an error function defined as

F (Aj) =√

Σ97i=1‖Xi,9 − X ′

i,j‖2 .

3 Result

3.1 Flow Vector Fields

Figures 3 and 4 show the measured flow vector fields for AU12 and AU16, respec-tively. Green arrows represent flow vectors, and the facial images depict deforma-tion for each AU. These flow vectors were obtained after the Head Movement Com-pensation. After the compensation, the maximum average error of five referencepoints between two images was 1.9 pixels. Considering the actual head size was275 mm and the head image size was 600 pixels, the estimated error was 0.9 mm.

These figures show how facial deformations are extensive and complicatedand how the flow fields are different between AUs. For example, in AU12, whichis Lip Corner Puller, we can see entire surfaces around the cheek, lips, andjaw move. The flow vectors near the lip corner have longer lengths, and their

Fig. 3. Measured skin flow vector field for AU12 (Lip Corner Puller) (Color figureonline)

208 N. Ota et al.

Fig. 4. Measured skin flow vector field for AU16 (Lower Lip Depressor) (Color figureonline)

directions are aligned toward the top side of the face. On the other hand, inAU16, which is Lower Lip Depressor, we can see the entire surface below themouth movement, and these directions are not aligned toward the side bottom.

3.2 Superimposed Field

Figure 5 represents the superimposed vector field before Natural Face Match-ing, which is the compensation processing to match vector starting points forthe same marker. Vector colors indicate the vector angle. We can find that thestarting points are not matched.

On the other hand, the starting points are matched well after Neutral FaceMatching, as shown in Fig. 6. From this figure, we can determine how far and inwhich directions each point on the face moved. The point near the face midlinemoved only vertically. However, other points moved in several directions. Thepoints near the mouth moved farther than points near the nose or the eye. Themaximum vector length was 22 mm and it appeared at the corner of the lowerlip for AU13.

Thus, the flow vectors seem to be too complex to replicate on a robot face.However, there seems to be a trend in the flow lines that connect the chin, thecorner of the eye, and the corner of the lips. We will analyze this trend in thenext section.

3.3 Principal Component Analysis

Figure 7 shows the first and second principal components of each marker’s flowvectors whose starting points are matched by Neutral Face Matching. The direc-tions of two double-headed arrows for each marker represent the directions of

PCA of Two-Dimensional Flow Vector Fields on Human Facial Skin 209

Fig. 5. Superimposed vector field before Neutral Face Matching. Starting points arenot matched. The positions of the eye, nose, and mouth are indicated by gray areas asa reference. (Color figure online)

Fig. 6. Superimposed vector field after Neutral Face Matching. Starting points arematched. The positions of the eye, nose, and mouth are indicated by gray areas as areference. (Color figure online)

the first and second principal components, respectively, while the lengths of thetwo arrows represent the variances of their scores. Namely, the longer the arrows,the farther the marker moved for different AUs. We found that the arrows closerto the mouth are longer.

210 N. Ota et al.

Fig. 7. The first and second principal components of flow vectors for each marker.The directions of two double-headed arrows for each marker represent the directionsof the first and second principal components, respectively, while the lengths of the twoarrows represent variances of their scores. The positions of the eye, nose, and mouth areindicated by gray areas as a reference. The s-shape trend of the flow lines is indicatedby a green line. (Color figure online)

Figure 8 shows the histogram for the contribution ratios of the first principalcomponents for every marker. Their average was 86 %, which means that almostall marker movement occur only in one direction at each point.

0

5

10

15

20

25

Freq

uenc

y

Contribution ratio

Fig. 8. Histogram for the contribution ratios of the first principal components for everymarker

Figure 9 shows the histogram for the minimum reproduction errors of everyflow vectors when every markers move only in the directions of each first principalcomponent. The reproduction error of each marker in each AU is its second

PCA of Two-Dimensional Flow Vector Fields on Human Facial Skin 211

Fig. 9. Histogram for the reproduction errors of flow vectors when every markers moveonly in the directions of each first principal component

principal component score. Over eighty percent of the reproduction errors wasunder 2 [mm] and over 95 percent of them was under 4 [mm].

Furthermore, we can find an s-shape trend of flow lines that connect the sideof the chin, the corner of the lips, the side of the cheek, and the corner of theeye. This trend is represented in Fig. 7 as a green line.

4 Conclusion

The main findings of the measurement and analysis of 16 patterns of facial flowvector fields are as follows:

– Although facial surfaces seem to move in a complicated manner when severaldeformation patterns are expressed, almost all movements occur only in onedirection in each measured point on the face.

– Such principal directions for all of the points are continuous through theentire facial surface; such continuous flows form an s-shape trend.

– Moving lengths for each point for several deformation patterns vary moresignificantly around the mouth.

These findings suggest that various kinds of facial expressions are created bycombinations of different movement lengths of facial surface points, each of whichmoves almost only in one direction.

These observations can facilitate replicating various kinds of facial expres-sions on a robot. This is because an actuation mechanism that moves a facialsurface point in one direction can be simple and compact while a mechanismthat moves the point in several directions would be complex.

However, several problems remain to be solved to obtain further effectivedesign policies for face robots. First, the flow vectors lack depth informationsince we obtained these vectors from two-dimensional images. Therefore, three-dimensional flow vectors should be obtained for more precise analysis. Second,

212 N. Ota et al.

the number of points required to replicate the flow vector fields on a robot shouldbe less than 96 because that is too many to prepare actuation mechanisms foreach point. Therefore, neighbor points that move similarly should be treatedas a single point representing its peripheral area. Third, the second principalcomponent should not be ignored. Although its contribution ratio is low, skinflows of the second principal component exist and they can be dominant in someof deformation patterns. Further flow field analyses are necessary on each AU tounderstand when flows of the second principal component occur.

Acknowledgment. This work was supported by a Grant-in-Aid for Specially Pro-moted Research No. 24000012 and for Young Scientist (B) No. 15K18006.

References

1. Bickel, B., Kaufmann, P., Skouras, M., Thomaszewski, B., Bradley, D., Beeler, T.,Jackson, P., Marschner, S., Matusik, W., Gross, M.: Physical face cloning. ACMTrans. Graph. 31(4), 118:1–118:10 (2012)

2. Cheng, L.C., Lin, C.Y., Huang, C.C.: Visualization of facial expression deformationapplied to the mechanism improvement of face robot. Int. J. Soc. Robot. 5(4), 423–439 (2013)

3. Ekman, P., Friesen, W.V., Hager, J.C.: Facial action coding system (FACS): man-ual. In: A Human Face (2002)

4. Hanson, D., Andrew, O., Pereira, I.A., Zielke, M.: Upending the uncannyvalley. In:Proceedings of the International Conference on Artificial Intelligence (2005)

5. Hashimoto, M., Yokogawa, C.: Development and control of a face robot imitatinghuman muscular structures. In: Proceedings of the International Conference onIntelligent Robots and Systems, pp. 1855–1860 (2006)

6. Hashimoto, T., Hiramatsu, S., Kobayashi, H.: Development of face robot for emo-tional communication between human and robot. In: Proceedings of the Interna-tional Conference on Mechatronics and Automaton, pp. 25–30 (2006)

7. Hirth, J., Schmitz, N., Berns, K.: Emotional architecture for the humanoid robothead ROMAN. In: Proceedings of the International Conference on Robotics andAutomation, pp. 2150–2155 (2007)

8. Ishiguro, H.: Android science: conscious and subconscious recognition. ConnectionSci. 18(4), 319–332 (2006)

9. Ishihara, H., Yoshikawa, Y., Asada, M.: Realistic child robot Affetto for under-standing the caregiver-child attachment relationship that guides the child develop-ment. In: IEEE Proceedings of the International Conference on Development andLearning, pp. 1–5 (2011)

10. Kobayashi, K., Akasawa, H., Hara, F.: Study on new face robot platform for robot-human communication. In: 8th International Workshop on Robot and HumanInteraction, pp. 242–247 (1999)

11. Lin, C.Y., Cheng, L.C., Tseng, C.K., Gu, H.Y., Chung, K.L., Fahn, C.S., Lu, K.J.,Chang, C.C.: A face robot for autonomous simplified musical notation reading andsinging. Robot. Auton. Syst. 59(11), 943–953 (2011)

PCA of Two-Dimensional Flow Vector Fields on Human Facial Skin 213

12. Minato, T., Yoshikawa, Y., Noda, T., Ikemoto, S., Ishiguro, H., Asada, M.:CB2: a child robot with biomimetic body for cognitive developmental robotics. In:Proceedings of the 7th IEEE-RAS International Conference on Humanoid Robots,pp. 557–562 (2007)

13. Tadesse, Y., Priya, S.: Graphical facial expression analysis and design method: anapproach to determine humanoid skin deformation. J. Mech. Robot. 4(2), 1–16(2012)

14. Yu, Z., Ma, G., Huang, Q.: Modeling and design of a humanoid robotic face basedon an active drive points model. Adv. Robot. 28(6), 379–388 (2014)

Learning to Balance While Reaching:A Cerebellar-Based Control Architecture

for a Self-balancing Robot

Maximilian Ruck1,2, Ivan Herreros1(&), Giovanni Maffei1,Martí Sánchez-Fibla1, and Paul Verschure1,3

1 SPECS, Technology Department, Universitat Pompeu Fabra, Carrer de RocBoronat 138, 08018 Barcelona, Spain

maximilianruck1@gmail.com, {ivan.herreros,

giovanni.maffei,marti.sanchez,paul.verschure}@upf.edu2 Department of Mechanical Engineering, Westfälische Hochschule, University

of Applied Sciences, Bocholt, Gelsenkirchen, Germany3 ICREA, Institució Catalana de Recerca i Estudis Avançats, Passeig Lluís

Companys 23, 08010 Barcelona, Spain

Abstract. In nature, Anticipatory Postural Adjustments (APAs) are actions thatprecede predictable disturbances with the goal of maintaining a stable bodyposture. Neither the structure of the computations that enable APAs are knownnor adaptive APAs have been exploited in robot control. Here we propose acomputational architecture for the acquisition of adaptive APAs based on cur-rent theories about the involvement of the cerebellum in predictive motorcontrol. The architecture is applied to a simulated self-balancing robot(SBR) mounting a moveable arm, whose actuation induces a perturbation of therobot balance that can be counteracted by an APA. The architecture comprisesboth reactive (feedback) and anticipatory-adaptive (feed-forward) layers. Thereactive layer consists of a cascade-PID controller and the adaptive one includescerebellar-based modules that supply the feedback layer with predictive signals.We show that such architecture succeeds in acquiring functional APAs, thusdemonstrating in a simulated robot an adaptive control strategy for the cancel-lation of a self-induced disturbance grounded in animal motor control. Theseresults also provide a hypothesis for the implementation of APAs in nature thatcould inform further experimental research.

Keywords: Cerebellar control � Anticipatory postural adjustments �Self-balancing robot � Adaptive control

1 Introduction

Nowadays self-balancing robots are becoming pervasive in everyday situations andthere exist numerous prototypes that explore new configurations and applicationdomains. However, most of the commercial self-balancing robots (SBRs) use solely

Research supported by socSMC-641321—H2020-FETPROACT-2014.

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 214–226, 2016.DOI: 10.1007/978-3-319-42417-0_20

pre-programmed feedback control [1]. In this paper we explore the use of adaptivecontrol techniques in order to deal with a problem that will arise the moment SBRs areto include additional actuators. That is, how to adapt and prevent the momentary loss ofbalance that occurs once a SBR changes its body configuration by extending amanipulator to reach for objects, thereby modifying the position of its center of mass?

Our approach to solve this problem is grounded in nature, namely in experimentalpsychology and neuroscience. On the one hand, humans and other bipedal primates arefaced with the same difficulty when they reach for objects. They have to adapt theirstance in order to preserve balance. Indeed, bipedal animals and SBRs are both dif-ferent instances of inverted pendulum systems. From a control theory perspective, onecan interpret the extension of one’s arm as self-induced disturbances. In humans,changes in body posture or in muscular activity that precede these types of disturbancesare known as anticipatory postural adjustments (APAs) [2]. Experimental manipula-tions have shown that healthy subjects rapidly acquire APAs, adjusting them both intiming and amplitude to cancel the effect of predictable disturbances. Such APAsusually comprise the predictive activation of the same muscles that before learningwere recruited reactively [3]. Therefore, learning can be characterized as a (partial)transference of the responses from feedback to feed-forward control [4]. Additionally,patients with damages in the cerebellum show impaired and non-adaptive APAs. Thisevidence, together with neural recording in monkeys, has pointed to the cerebellum asthe critical substrate for the acquisition of APAs [3].

To simulate the acquisition of APAs by the cerebellum we apply a model ofcerebellar learning. The classical theory of cerebellar learning originated in the late 60 s[5, 6]. According to that theory, the cerebellum works as a supervised learning devicewhere inputs from climbing fiber pathway, one of its two input pathways, instruct theprocessing of the information that comes via the mossy fiber pathway, the second inputpathway. As a result, the cerebellum adaptively adjusts a mapping of mossy fiber inputsinto Purkinje cell output activity. From an information processing perspective, theinferior olive, via its climbing fiber afferents, informs the cerebellar cortex about anerror while plasticity at the cerebellar cortex aims at changing the cerebellar output toreduce that error. Cerebellar anatomy and physiology will not be treated in detail in thispaper, but the reader can find a computationally oriented review in [7]. The basicintuition is that the cerebellum receives simultaneously error signals (through theclimbing fiber pathway) and context information through the mossy fiber pathway.That context information can relate both to states of the world or the agent (as in ourcase, where it codes the agent’s state of being next to move the arm). Finally, thecoincidence between context and error information allows for the acquisition ofadaptive outputs that can be issued once contexts that preceded errors arere-encountered.

Cerebellar-based controllers are commonly used in the robotic community, espe-cially following the cerebellar model for articulatory control (CMAC) [8]. Indeed,CMAC and other cerebellar-based models have already been applied to the task ofcontrolling a SBR [9–12]. However, the research described here is novel in the fol-lowing ways:

Firstly, it addresses the problem of adjusting the reactions of the two-wheeled robotto changes in its body configuration. This problem will be relevant for SBR with

Learning to Balance While Reaching 215

additional actuators, such as wheeled humanoids. Secondly, the mixed feed-forward/feedback architecture is built according to the hypothesis that the cerebellum facilitatesfine motor control by acquiring sensory predictions. This is in contrast with the stillprevailing view that interprets the cerebellum as an inverse model issuing motor

Fig. 1. Scheme of SBR and task. A. Display of the SBR from the lateral side balancing on asurface. The state variables θ and v are indicated in relation to their reference states rh and rvrespectively. B. The feedback motor control system consists of two overlapping feedbackcontrollers (fb). The outer feedback controller reacts to the mismatch ev of reference velocity rvand velocity (v). Such controller is set with low gains, thus it is slowly reacting and changing thereference angle rh. Whereas the inner feedback controller is highly reactive to sudden changesand issues motor commands (u) according to the error eh. The motor command computed fromthe feedback control system (grey) is conveyed to the plant (blue), that models the robot’sdynamical behavior. C. Trajectory of SBR with feedback control. The plant (blue) starts the trialat t0 = 0 s in an equated position (timeline grey). The extension of the gripper (black) is initiatedat t1 = 1 s and ends at t2 = 2 s. At t3 = 3 s extra weight is attached to the gripper and released att4 = 5 s. The flexion of the gripper is initiated at t5 = 6 s and finishes at t6 = 7 s. Afterwards therobot returns steadily into a balanced state. The arrow at the wheel indicates the current velocityand sets the angular reference state rh. The black line displays the current angular orientation ofthe robot, whereas the dashed black line as well as the shape represents the desired angularorientation of the robot (all values are approximated simply to indicate the relations between thedifferent variables at different stages of the task). (Color figure online)

216 M. Ruck et al.

commands [3, 13, 14]. Whereas Kawato and collaborators used sensory errors toacquire motor commands [15], our model acquires sensory error-predictions based insensory errors.

Operationally, we have defined the task of balancing the SBR as that of maintainingthe linear velocity of its wheel axis at zero. Maintaining balance while reaching forwardbecomes a disturbance-rejection task with a self-generated, hence predictable, distur-bance. At the basic and non-adaptive level, this task is managed by cascade PIDcontroller that uses both the linear velocity and the angular position of the robot body tocompute the control signal. The reliance on two plant outputs implies that two separateerror signals (velocity and angle errors) drive each layer of the PID controller (Fig. 1A−B). The outer layer of the PID, based on the velocity error computes the target anglethat would eventually keep the robot still and balanced. The inner layer aims atreducing the error between the current and the previous target angle layer. Note that interms of performance, only the error in velocity matters. As long as the error in velocityis minimized, the error in angular position is irrelevant.

That duplicity in the feedback controller allows interfacing an adaptivefeed-forward module with each level of the reactive layer. In one case, supplying theouter PID controller with predicted errors in the velocity (pv), or the second casedriving the inner PID with predicted errors in the angle (ph). That made also possible totest the effect of providing both PID controllers with predictive inputs simultaneously.Thus, a third novelty of our adaptive approach is that even if there is only one degree offreedom to control, we propose to use a hierarchy of adaptive modules that mirrors thehierarchy of the feedback controller.

To summarize, we propose (1) that cerebellar-based control strategies can beapplied to endow SBRs with anticipatory behaviors that resemble those observed inanimals, especially as they have been measured in the APA paradigm; (2) that theadaptive control strategy can be successful even if, instead of directly accessing theplant, the adaptive component or components only affect behavior by recruiting thefeedback control with sensory predictions; and (3) that even if we faced a singledegree-of-freedom problem, the use of two feed-forward adaptive components that mapdirectly onto different stages of the feedback control can entail an increase in perfor-mance compared to strategies that use a single predictive component.

2 Methods

A. Robot Model and Feedback Controller. A two-wheeled SBR is a vehicle able tokeep its center of mass above a pivot (the wheels’ axis) (Fig. 1-A). In the standardconfiguration the robot makes use of a feedback control loop to balance and withstanddisturbances.

We model the robot with a standard non-linear state-space equations of a balancingsystem [16]. The rigid body equations model two masses attached by rigid link, wherethe mass at the base is actuated by a force (u) parallel to the ground. The model hasthree state variables: angular position h, linear velocity v and angular acceleration _h.The model equations are given below.

Learning to Balance While Reaching 217

ddt

hv_h

2

4

3

5 ¼

_h�ml sin hð Þ _h2 þmg ml2

Jt

� �sin hð Þ cos hð Þ�cv�clm cos hð Þ _hþ u

Mt�m ml2Jtð Þ cos hð Þ2

�ml2 sin hð Þcos hð Þ _h2 þMtgl sin hð Þ�cl cos hð Þv�c Mtmð Þ _hþ lcosðhÞu

JtMtmð Þ�m lcos hð Þð Þ2

2

6664

3

7775ð1Þ

The parameters of the model are set to match the approximate values of a physicalprototype currently under development (see Table 1).

Using this model of the plant, reaching the gripper and loading a weight aresimulated as changes in the relative position (pgr) and total weight of a mass situated atthe end of the gripper (mgrÞ that affect the moment of inertia (Jt), the total mass (Mt) andthe distance from the wheel axis to the center of mass (1). Specifically, during thereaching we simulated a forward movement of the mass, perpendicular to the robotbody. After extending it, the gripper is loaded (e.g. by another agent) with an extraweight. To complete the task, the robot must maintain balance with the additional loadfor a few seconds, release the load and retract the gripper. At each time the new centerof masses (c.o.m.) and moments of intertia are computed, and the relative change in thec.o.m. angle with respect to the previous time step is added to the state variable θ.

The robot stabilizes by using a cascade PID controller (Fig. 1B) that is defined asfollows:

rh tð Þ ¼ �kvpv tð Þ � kvi

Z t

0v sð Þds ð2Þ

u tð Þ ¼ khp rh tð Þ � h tð Þð Þþ khd _rh tð Þ � _h tð Þ� �

ð3Þ

A first (outer) proportional-integral (PI) controller receives as input the linear errorin velocity (ev), which by definition of the problem is equal to minus the velocity (v),and outputs a target angle (rh). Such target angle becomes the reference for a second(inner) proportional-derivative (PD) controller. The inner controller then receives asignal coding the current error in the angular position (eh), computed comparing theactual angle (h) with rh. Indeed, this cascade PID controller can be interpreted as a twostep strategy to solve the balancing problem: first computing a desired stance (rh), inthis case, an inclination of the robot’s body, and secondly, computing the motorcommand (u) that brings the robot to that target stance. We set the parameters of thefeedback controller such robot is stable near the equilibirum point. Note that reactivecontrol maintained stability of the SBR during all stages of the reaching task, at theexpenses of introducing large errors in the linear velocity.

B. Basic Cerebellar Model. Our adaptive feed-forward modules desing is based on thecerebellar microcircuit. In the cerebellum, each microcircuit integrates diverse inputscoming from the mossy/parallel fiber into a single output relayed by Purkinjecells/cerebellar deep-nuclei. According to the cerebellar learning theory, the integrationof the mossy-fiber information is determined (or instructed) by a highly specific inputcoming from the inferior olive via the climbing fibers. In the same line of the adaptive

218 M. Ruck et al.

filter theory of the cerebellum, we have chosen to model a cerebellar microcircuit as anadaptive linear element, implementing a decorrelation learning or Widrow-Hofflearning rule. More precisely, the model uses an adaptive linear element that combinesa set of basis to produce an output signal.

Each basis signals (pðtÞ) results from a double exponential convolution of an inputsignal. In our case the inputs to each set of bases are triggers that, through differentlabeled lines, anticipate each of the six stages of the task (e.g., reaching onset, reachingoffset, load weight, unload weight and onset and offset of the retraction). In ourimplementation we generated 30 bases for each input, with gives a total of approxi-mately 180 bases. To have a diverse set of signals to combine, each basis implements afilter with different time constants (the details of the basis generation have already beenintroduced in [17]). The bases are mixed according to a set of weights wðtÞ. Using theconvention that both wðtÞ and p tð Þ are column vectors, the output of the cerebellum(y tð Þ) can be generated as follows:

y tð Þ ¼ w tð ÞTp tð Þ ð4Þ

The weights are updated according to the decorrelation learning rule [18] (alsoknown as Widrow-Hoff rule [19]). In this particular experiment the following versionof the update-rule is used:

ddtwc ¼ �ge tð Þp t � dð Þ ð5Þ

where wc, intialized at 0 at each trial onset, stores the change in weights to be applied atthe end of the trial; g is a learning rate; eðtÞ, the error signal; and d, the anticipatorydelay. The anticipatory delay is a crucial parameter, distinctive of our modellingapproach to cerebellar function (see [4, 17, 20]). It operationalizes the idea that to avoidan error occuring at time t the cerebellum should have acted at time t � d. The ruleupdates the weights by associating errors at a given time with the information encodedin the filter bases (p) d seconds earlier. Setting d requires approximate knowledge of theplant (or closed-loop) dynamics. That is, it requires knowing the lag in the responsedynamics of the system, in order to anticipate responses earlier enough. Additionally,this parameter also implies that cerebellar responses in this model are anticipatory,which is not the standard in computational models of the cerebellum (e.g., see [21]).

C. Structure of the Adaptive Layer. In our modelling of a cerebellar microcircuit,each adaptive element receives layer through its climbing fiber pathway an error signalcomputed in the reactive control layer and sends output to the associated error feedbackcontroller. The mossy/parallel fiber information that is provided to each elementconsists of a series of time varying signals (alpha-like functions) that are triggered by amotor intention. The wiring of the cerebellar components to implement an adaptivelayer is displayed in Fig. 2. The two cerebellar controllers (V_CRB and A_CRB, forvelocity and angle, respectively) receive the error signals that drive each individual PIDcontroller, namely ev and eh. Based on these teaching signals, they acquire the pre-dictive outputs pv and ph, which are added to the actual measured errors, driving the

Learning to Balance While Reaching 219

feedback controllers. Hence, the feed-forward predictive commands issued by bothcerebellar components affect behavior only after passing through the feedbackcontrollers.

Regarding the inputs that enter the cerebellar models through the mossy fiberpathway, the different basis signals generated in each cerebellar model represent theresponse to an input signal encoding motor intentions (MI in Fig. 2). This signals canalso be thought of as efference copies arriving from the system controlling the robot’sarm which allow the generation of a feed-forward sensory error signal that, driving thefeedback controllers, cancels the error introduced by the self-induced disturbance.

D. Simulations. The two-wheeled SBR and the motor control architectures areimplemented inMATLAB 2015a Simulink and run in a laptop computer. The parametersare detailed in Table 1. The learning experiments were conducted in 15 s duration trials.The structure of each trial is detailed in Fig. 1C. The simulations, deterministic andnoiseless, were performed with a fixed size ODE solver, with a fixed time-step of 5 ms.

In the simulations we used three different versions of the architecture, dependingupon whether both or just one of the adaptive cerebellar-like components wereinstantiated. That way we could first asses the individual contribution of eachfeed-forward module to performance and the effect of having both adaptive modulesworking in synergy. We refer to the architecture that only includes V_CRB as thevelocity-only adaptive (VOA) architecture, to the one including only A_CRB as theangle-only adaptive (AOA) architecture and the one instantiating both modules as thedual adaptive (DA) architecture.

Fig. 2. Two-layered control architecture of the SBR. The reactive motor control system isextended with a second anticipatory layer. The anticipatory layer learns to associate the motorintention of the plant with the error signal ex and outputs the counterfactual sensory errorprediction px. The prediction is incorporated into the control loop and processed from thefeedback controller (fb). The cerebellar controllers V-CRB and A-CRB (refer to veleocity andangle respectively) are implemented in cooperation.

220 M. Ruck et al.

3 Results

We examine first the performance of the full control architecture (DA) that comprisestwo adaptive feed-forward controllers. The DA architecture succeeds in reducing theerrors significantly. Indeed, the primary measure of performance, the error in thevelocity ev signal, decreases from a range spanning from −0.7 to 1.1 m/s to a rangespanning from −0.4 to 0.2 m/s (Fig. 3A). Hence, after learning, the disturbance causedby the actuation of the robot arm was greatly reduced in amplitude. The errors in theangular position (Fig. 3B) were also reduced after learning, albeit in a lesser extent.

Regarding the output of the predictive controllers (Fig. 3C and D), they resemblesmoothed (and in the case of the pv, advanced in time) versions of the errors measuredin the first trial.

In a second set of trials we separately assess the impact of the delay parameters oneach module’s performance after. We use only one adaptive the VOA and AOAarchitectures (see Methods). In each case only one of the feedback controllers receivespredictive signals. The most relevant results are obtained with the VOA architecture.First, this architecture, which only includes one adaptive component, yields a reductionof the relative root mean square error (rRMSE) for the velocity of approximately 95 %(Fig. 4A). This means that, this architecture succeeds almost completely in decreasingdisplacement while executing the reaching actions. Moreover, a large range of values

Fig. 3. A, B. First error of naive (gray) and last (color) error of trained agent for velocity(A) and angle (B). C, D. Complete sensory error signal of a trained robot for velocity (C) andangle (D). Dashed line indicates the adaptive signal (pv and ph, respectively) and the colored solidlines, the sum of adaptive and reactive error signals.

Learning to Balance While Reaching 221

of the anticipatory delay parameter dv, from 150 ms to 600 ms achieves an almostequal performance.

The results for the AOAarchitecture are markedly different. Performance, measuredin term of does not improve for any of the values tested for the anticipatory delayparameter (dh) (Fig. 4A). Indeed, in terms of avoiding displacements induced by thereaching actions, the AOA architecture always performs worse than an architecturerelying only in the PID cascade (without anticipation). Secondly, the error in theangular position is actually reduced, even beyond the level attained by the VOAarchitecture (Fig. 4B). The final difference between the AOA and the VOA architectureaccording to the different values of their delay parameters is, that in the VOA archi-tecture there is clear and wide range of values of preferred values for dv, whereasperformance in the AOA architecture performance increases as the dh decreases.

These results can be summarized as follows: First, in both cases the adaptivecomponents succeed in predicting their corresponding errors signals, as evidenced bythe marked decreases in their measured errors (Figs. 4A and B). Secondly, predictingerrors associated with the reference of the closed-loop (ev) is relevant increasing per-formance in the disturbance-rejection task, whereas acquiring predicting the errorsignal of the inner feedback controller (eh) does not decrease the performance error(eh). Finally, performance of the VOA architecture was very tolerant to changes in thekey parameter ðdv).

Finally, we compare the results of the three different architectures; the twoincluding only one adaptive component (VOA and AOA), and the one with two (DA).Indeed, after the previous results it is unclear to what extent the adaptive component forthe error in the angle has any positive impact in performance. However, the outcome ofthis last set of simulations suggests that working in synergy with the adaptive com-ponent for ev (V_CRB) the adaptive component for eh (A_CRB) ameliorates perfor-mance as well (Fig. 5A and C). First, the DA architecture reduces slightly the error invelocity (Fig. 5A). Secondly, the DA architecture shows an improvement relative to theVOA in terms of control effort. Indeed, taking pure feedback control as the baseline and

Fig. 4. Relative root mean square errors (rRMSE) as function of the anticipatory delay. A, B.Effect of the anticipatory delay of A-CRB (blue) and V-CRB (red) in the velocity error (A) andthe angular error (B). For both figures normalization is performed using the cumulative sum ofthe errors in the first trial compared to the errors in the last five trials. The black circle indicatesthe delay with the least error in the respective space. (Color figure online)

222 M. Ruck et al.

computing the proportional reduction in the integral of the absolute control, the VOAarchitecture minimizes the control effort by 33 % and the DA architecture by 41 %(Fig. 5C). Hence, the DA architecture not only improves the balance of the robot whilereaching, it also decreases the control effort.

4 Conclusions

We have introduced a cerebellar-based control architecture that allows a simulatedwheeled-SBR to avoid displacements and maintain balance while actuating an arm,loading and unloading weight.

Even if the simulated robot was already able to maintain balance by means of thefeedback control provided by a cascade of PID controller, the introduction of sup-plementary feed-forward predictive signals improved the performance reducing theRMSE by a 95 % over the pure feedback strategy (Fig. 4A).

Notably, even if the feedback controller is two-staged, almost all of the perfor-mance gain in terms of error reduction comes from providing the outer feedbackcontroller with predictive signals. Indeed, even though the cascade PID includes twoerror-correction steps, in control theory terms, there is just one external reference signal– i.e., behavior has only one goal that is to keep the robot still. Controlling the position(inclination) is merely an instrumental step that enables the robot to remain still.Therefore, our results seem to imply that acquiring a predictive signal to complement

Fig. 5. Performance over session trials. A, B. rRMSE of the velocity (A) and the angle (B) forthree different architectures, VOA (red), AOA (blue), DA (purple) during the trials. C. Motoreffort reduction over trials for three different architectures. Motor effort reduction is calculated asthe ratio between the accumulated squared motor output at each trial over the one on the first trial.(Color figure online)

Learning to Balance While Reaching 223

the external reference signal is enough for rejecting the disturbance. We believe thatsuch result indicates that the basic approach of connecting an adaptive-anticipatorymodule to an error-feedback controller might be sufficient for improving performance awide range of problems dealing with a single degree of freedom.

However, the addition of the second adaptive controller further improves the per-formance of the SBR in the disturbance-rejection task in terms of reduction of thecontrol effort. In this case, the reduction in effort must be related to the smoothness ofthe predicted error signal (ph) compared with the actual error signal (eh). Even if botherror signals, the one experienced at the beginning of training and the one entering theinner feedback controller have similar amplitudes, the slower transients present in theinput signal after learning ensure that the PD-controller issues a smaller control signal.

Wheeled SBRs, are non-minimum phase systems. This implies that any error invelocity must be first made bigger before it can be decreased. For instance, a still robotthat has to move forward must first move backwards in order to induce a forward-tiltthat, causing a transient acceleration of the robot’s center of mass, makes the forwardtranslation stable. In practice, that means that error-feedback controllers fornon-minimum phase systems introduce unavoidable lags in reference tracking. Hence,the benefit for anticipation in those system stems from being able to bypass suchlatency by predicting the change in the reference by a sufficiently large interval. Thisproperty explains the results for the anticipatory delay parameter (dv) of the adaptivevelocity signal controller. Indeed, in order to achieve a significant gain of performance,this parameter must be set to at least 150 ms.

From a pure neuroscience or neuro-robotic perspective these results illustrate howthe cerebellum, which is considered to be crucial for accurate motor control, can enableprecise predictive motor control operating as a purely sensory-prediction device.Additionally, the results show that the same cerebellar model can be wired toerror-feedback controllers with different properties: a memory-less PD controller and acontroller with an internal state. In nature, this result may illustrate how the samecerebellar algorithm could contribute to feedback computations located in brainstructures with very different dynamics, such as the motor cortex or the spinal cord.

In the context of APAs, our implementation of a control architecture embodies atheory of how anticipatory postural adjustments could occur in nature. Of course, thereare substantial differences between controlling a self-balancing robot and maintaining abipedal stance, e.g., in terms of body structure, means of actuation, etc. However, oursolution does suggest, that as a general principle, APAs could be acquired usingsensory-error predictions that are sent to a reactive controller.

Due to the non-linearity and intrinsic difficulty of controlling SBRs, acquiring ananticipatory motor command that precisely controls the robot stance while it changesits body configuration and loads objects is not a trivial task. With our results we show,in simulation, that a very general-purpose architecture can succeed in this task usingmotor learning principles from neuroscience literature. As further work, the robustnessof the control approach could be assessed either by adding noise in the sensors and/oractuators in simulations or by mounting it in a real robot. And finally, the contextinformation could be enriched, coding also for the intended velocity of the armmovement and/or expected weight, thereby allowing the generalization of the APA to aset of disturbances.

224 M. Ruck et al.

Acknowledgements. The research leading to these results has received funding from theEuropean Commission’s Horizon 2020 socSMC project (under agreement number: socSMC-641321H2020-FETPROACT-2014) and by the European Research Council’s CDAC project:(ERC-2013-ADG 341196).

Appendix

See Table 1.

Table 1. Simulation parameters.

feedback control P I D

feedforward control V-CRB A-CRB

Outer fb -0.070 -0.020 — number of basis 150 210

Inner fb 35 — 0.01 learning rate 0.00258 0.00258

anticipatory delayδ

0.380 0.050

time constantsfor bases (see [18])

slow 1.000

fast 3.000

finh 3.000

physical properties of the robot

Mt 1.580 kg mass of SBR

m 0.040 kg mass of wheels

mGr 0.030 kg mass of gripper

mextra 0.005 kg extra weight

g 9.800 m/s2 gravity

l 0.0331 m distance of center of mass to wheels

Jt 1.9828 * 10-3 m2kg moment of inertia

0.010 Nms rotational friction

c 0.100 Ns/m rolling friction

of the arm movement

h 0.150 m height of gripper

pgr 0 to 0.100 m extension of gripper

taccel 0.100 s duration of gripper acceleration

Jgr 6.7500 * 10-4 m2 kg moment of inertia of gripper

T 0.0045 Nm torque of recoil

recoil 6.670 rad/s2 applied angular acceleration (T/Jgr)

Learning to Balance While Reaching 225

References

1. Chan, R.P.M., Stol, K.A., Halkyard, C.R.: Review of modelling and control of two-wheeledrobots. Annu. Rev. Control 37(1), 89–103 (2013)

2. Massion, J.: Movement, posture and equilibrium: Interaction and coordination. Prog.Neurobiol. 38(1), 35–56 (1992)

3. Horak, F.B., Diener, H.C.: Cerebellar control of postural scaling and central set in stance.J. Neurophysiol. 72(2), 479–493 (1994)

4. Maffei, G., Herreros, I., Sánchez-Fibla, M., Verschure, P.F.: Acquisition of anticipatorypostural adjustment through cerebellar learning in a mobile robot. In: Lepora, N.F., Mura,A., Krapp, H.G., Verschure, P.F., Prescott, T.J. (eds.) Living Machines 2013. LNCS, vol.8064, pp. 399–401. Springer, Heidelberg (2013)

5. Marr, D.: A theory of cerebellar cortex. J. Physiol. 202(2), 437–470 (1969)6. Albus, J.S.: A theory of cerebellar function. Math. Biosci. 10(1–2), 25–61 (1971)7. Dean, P., Porrill, J., Ekerot, C.-F., Jörntell, H.: The cerebellar microcircuit as an adaptive

filter: experimental and computational evidence. Nat. Rev. Neurosci. 11(1), 30–43 (2010)8. Albus, J.: A new approach to manipulator control: The cerebellar model articulation

controller (CMAC). J. Dyn. Syst. Meas. Control 97(3), 220–227 (1975)9. Li, C., Li, F., Wang, S., Dai, F., Bai, Y., Gao, X., Kejie, L.: Dynamic adaptive equilibrium

control for a self-stabilizing robot. In: 2010 IEEE International Conference on Robotics andBiomimetics, ROBIO 2010, pp. 609–614 (2010)

10. Chiu, C.H., Peng, Y.F.: Design and implement of the self-dynamic controller for two-wheeltransporter. In: IEEE International Conference on Fuzzy Systems, pp. 480–483 (2006)

11. Ruan, X., Chen, J.: On-line NNAC for two-wheeled self-balancing robot based onfeedback-error-learning. In: Proceedings - 2010 2nd International Workshop on IntelligentSystems and Applications, ISA 2010 (2010)

12. Tanaka, Y., Ohata, Y., Kawamoto, T., Hirata, Y.: Adaptive control of 2-wheeled balancingrobot by cerebellar neuronal network model. In: 2010 Annual International Conference ofthe IEEE Engineering in Medicine and Biology Society, EMBC 2010, pp. 1589–1592(2010)

13. Gao, J.H., Parsons, L.M., Bower, J.M., Xiong, J., Li, J., Fox, P.T.: Cerebellum implicated insensory acquisition and discrimination rather than motor control. Science 272, 545–547(1996)

14. Miall, R.C., Wolpert, D.M.: Forward models for physiological motor control. Neural Netw.9(8), 1265–1279 (1996)

15. Kawato, M., Furukawa, K., Suzuki, R.: A hierarchical neural-network model for control andlearning of voluntary movement. Biol. Cybern. 57(3), 169–185 (1987)

16. Astrom, K.J., Murray, R.M.: Feedback Systems: An Introduction for Scientists andEngineers (2012)

17. Herreros, I., Maffei, G., Brandi, S., Sanchez-Fibla, M., Verschure, P.F.M.J.: Speedgeneralization capabilities of a cerebellar model on a rapid navigation task. In: 2013IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 363–368 (2013)

18. Fujita, M.: Adaptive filter model of the cerebellum. Biol. Cybern. 45(3), 195–206 (1982)19. Widrow, B., Lehr, M.A., Beaufays, F., Wan, E., Bilello, M.: Adaptive signal processing. In:

Proceedings of the World Conference on Neural Networks, p. 11 (1993)20. Herreros, I., Verschure, P.F.M.J.: Nucleo-olivary inhibition balances the interaction between

the reactive and adaptive layers in motor control. Neural Netw. 47, 64–71 (2013)21. Wolpert, D.M., Kawato, M.: Multiple paired forward and inverse models for motor control.

Neural Netw. 11, 1317–1329 (1998)

226 M. Ruck et al.

Optimizing Morphology and Locomotionon a Corpus of Parametric Legged Robots

Gregoire Passault(B), Quentin Rouxel, Remi Fabre, Steve N’Guyen,and Olivier Ly

LaBRI, University of Bordeaux, Bordeaux, France{gregoire.passault,quentin.rouxel,remi.fabre,

steve.nguyen,olivier.ly}@labri.fr

Abstract. In this paper, we describe an optimization approach to thelegged locomotion problem. We designed a software environment tomanipulate parametrized robot models. This environment is a platformdeveloped for future experiments and for educational robotics purpose.It allows to generate dynamic models and simulate them using a physicsengine. Experiments can then be made with both morphological and con-troller optimization. Here we describe the environment, propose a simpleopen loop generic controller for legged robots and discuss experimentsthat were made on a robot corpus using a black-box optimization.

1 Introduction

Eadweard Muybridge did an early work in studying animals locomotion, ana-lyzing photographs of animals during different phases of strides [1]. What isnoticeable is that there is a lot of similarities in how different animals move atcorresponding paces.

These similarities are explained by studies on Central Pattern Generators(CPGs), that pointed out that the nervous system is able to produce rhythmicmotor patterns even in vitro [2]. This is called fictive motor patterns, and thispoints out that animals locomotion may be based on hardware encoded a prioris.

Simple physical models such as the SLIP (spring-loaded inverted pendulum),can also describe running animals. It is interesting to mention that animals withtwo to eight legs can fit this model when running, where groups of legs acts inconcert so that the runner is an effective biped [3].

One of our goals is to explore how some classical gaits like the trot, and inlesser extent the walk, could appear naturally by optimizing trajectory-basedlocomotion controllers. We explore how the locomotion solutions of animals,provided by hundreds of millions years of evolution, corresponds to solutionsissued from statistical optimization on pure physics based criteria, and in thisway appear to be canonical.

What we propose here is to optimize the locomotion trajectories on a para-metric robot corpus as well as morphology/anatomy. In this context, we proposea controller with a priori knowledge (such as symmetry) allowing to simplifythe search space of the optimization process.c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 227–238, 2016.DOI: 10.1007/978-3-319-42417-0 21

228 G. Passault et al.

This work is based on a technology developed by authors allowing to simulatea parametric family of legged robots. It is part of the Metabot [4] project, whichconsist in developing an open-source low cost robot for education and research.

We will first describe the architecture of the system that was developed, andthen the generic controller that can make all the robots of the corpus walk. Then,we will discuss the optimization experiments that were made and the obtainedresults.

2 Related Works

The GOLEM project [5] proposed to evolve both the robot’s morphology andits controller, using neural networks and quasi-static simulation. They generatedmanufacturable walking robots without any a priori.

More recently, Disney Research [6] proposed a system allowing casual users todesign and create 3D-printable creatures. They proposed a method to generatemotion using statically stable moves on a robot model given by the user. Theyalso proposed a way to generate printable geometry from the custom parts.

The presented work is not restricted to statically stable motions. Instead,dynamic locomotion patterns are explored - which mean that it is not assumedthat the robot have to be statically stable at each time step. The ability to printthe robots is also proposed.

[7] co-evolved pure-virtual robots in dynamics simulator and proposed ascript language for robot structures.

[8] produced real-world robots whose morphology and controller were opti-mized by simulation. The study underlines an important gap between simulationthe physical world. Our simulation tries to reduce this gap by including an esti-mation of the shocks and the backlash.

[9] studied the legged locomotion space, trying to adapt the locomotion whenthere is damaged parts using pre-computed behaviour-performance maps.

There are also several works on fixed architecture robot’s locomotion tuning,like the teams that worked on the Aibo robots [10] or Bostom Dynamic’s LittleDog [11], or the locomotion learning from scratch [12].

In our system, not only the robot locomotion but also its morphology isoptimized.

3 System Architecture

3.1 Robots Modelling

To model the robots, we developed a custom tool that uses the OpenSCAD [13]language as backend. This language can be used to describe 2D or 3D models usingcode, which makes extensive parametrization possible. This code is actually basedon constructive solid geometry (CSG), where boolean operations are applied tobasic objects to combine them. OpenSCAD can produce meshes and CSG trees,which contains the basic objects, boolean operations and transform matrices.

Optimizing Morphology and Locomotion 229

We customized OpenSCAD to add metadata in the code. We extended theparts description language to add markers in the CSG tree so that we can automat-ically retrieve information such as reference frames such as anchor or tips. Thus,we are able to specify in the generated part some reference frames that we canretrieve once the part is compiled and to tag specific parts of the CSG tree.

Fig. 1. A component contains models,parts and anchor points.

Fig. 2. An overview of the robot editor,using parametric components library.

Moreover, we also added the notion of component, which contains models of“real-life” existing objects (such as motors), parts that should be manufacturedto make the robot and anchors that are points where other compatible anchorscan be attached (see Fig. 1). Components can have parameters, that can modifysome features (for instance mechanical lengths) of the parts that it contains.

We then created a components library and an editor that can be used toinstantiate and attach components together. The edited robot is actually rep-resented as a tree with components as nodes and anchors as child relation (seeFig. 2)1. We also added global robot parameters that can impact multiple com-ponents. This results in parametric robots that can be re-generated with newparameters to change some morphological features.

All the parts of a robot can be exported, generating meshes that can be usedfor example for 3D printing.

3.2 Kinematic and Dynamics

Since we know the transformation matrices for each part of the robot tree, wecan compute the kinematic model for a given robot topology and its parameters.Leg tips are also tagged in the components.

It is also possible to compute the dynamics model. To achieve this, parts andmodels are turned into small voxels (cubes) of typically 1mm3. This can be doneusing even-odd rule [14] on each point of the bounding box. With these cubes,we can deduce the volume of the part and the mass distribution (see Fig. 3). Thecenter of mass can then be known using a weighted average and the inertia withan integration of the cubes.

1 Videos showing the editor is available at https://www.youtube.com/watch?v=smHctwi05Ic.

230 G. Passault et al.

Fig. 3. Voxels are computed from theactual part meshes to compute the com-plete dynamic properties of the model.

Fig. 4. Robot parts and models and itspure shape collisions approximation forsimplified collision computations.

Parts have a pre-determined homogeneous density, and models (for examplea well-known motor) can have a fixed overall mass, in this case, the density isadjusted to fit the mass.

All the parts have a corresponding (parametric) pure shape approximation.This is made easy thanks to the constructive geometry, as it is already basedon basic objects. Most of the part collision approximations simply consist inremoving items from the tree such as screw holes. Then, we parse the CSG treeand directly retrieve the pure shapes (see Fig. 4). To avoid auto-collision glitches,these shapes can be automatically “retracted” (each pure shape is slightly down-sized, so that near shapes such as a motor horn don’t collide because of numericalapproximations when rotating).

We can then inject all these information into a dynamics engine.

3.3 Corpus

We designed a corpus containing 22 robots, with 4 to 8 legs and 8 to 24 motors.The parts are designed to be manufacturable (3D printable). The modeled motoris similar to an on-shelf low-cost motor (Robotis XL-320). Robots vary in shapeand thus have different morphological parameters (see Fig. 5).

Fig. 5. An overview of six robots of the corpus.

Optimizing Morphology and Locomotion 231

All the robots are facing the X-axis, which is the arbitrary “front” of therobot. Some degrees of freedom range were limited artificially to disambiguatethe kinematic (see below).

4 Generic Controller

Here we introduce a generic controller for locomotion that takes a robot modeland generates motor trajectories. This controller allows to steer the robot accord-ing to a given speed vector.

4.1 Principle

The locomotion task to be solved consists in following a given speed and rota-tional velocity described by the dynamic controls (x, y, θ). Since the robots havea significant number of degrees of freedoms, it is clear that there are many differ-ent ways to achieve this. For more convenience, the legs are ordered using theiranti-clockwise position around the z-axis. Thus, the front left leg is always thefirst one and the front right the last.

In order to reduce the parameter’s space, we consider that the robot’s heightduring the walk remains constant and that the body yaw and roll are null.

If the robot is indeed following the dynamic controls, the leg trajectory onthe ground during the support phase will then be determined. We then useiterative inverse kinematics using simple stochastic method (we apply randomvariations on the angles and check if one of these variation lead to a result closestto the target and iterate again until it doesn’t). Since we are dealing with smallnumber of degrees of freedom per leg, this is efficient enough (This consumes farless computational power than the dynamics engine itself). Thus, we are able tocontrol the tip position in the Cartesian (x, y, z) coordinates.

Fig. 6. Model of the leg trajectory that uses cubic splines and three locus, the “0”point is the position of the tip when the dynamic controls are null.

The leg trajectory is described using splines. In order to ensure smoothmotions (and finite speeds) we have chosen cubic splines position for trajec-tories representation. We define three couples point/velocity (or locus) for this

232 G. Passault et al.

spline. The first two are defining the segment followed by the leg on the floorand the last one being the point reached when the leg rises (C).

Here, the two loci on the floor have both a fixed position and speed: theposition is determined by the steps length and the speed is null during supportphase. The third locus height is a parameter that defines how high the robotwill rise its legs. Its position and speed belong to the optimized parameters ofthe controller that change the leg trajectory when raised (see Fig. 6).

We assume that the support duration is the same for each leg and is a para-meter of the controller. Leg phases (i.e. synchronization) are also parameters.The first leg (front left) is the reference phase that can be changed along withall the other leg phases accordingly. The frequency of the whole stride is anotherparameter.

Finally, the robot posture (i.e. the position of the legs when dynamic controlsare null and the robot is static) can be parametrized with x, y and z, where z isthe height of the robot and x, y are multiple of the leg position on the originalrobot model (if x = 1 and y = 1 the position of the legs are exactly the sameas in the model, if x = 0.5 and y = 2, the legs are near the center of the robotalong the sagittal plane and farther along the frontal plane, the symmetry ispreserved).

4.2 Parameters Summary

– locus x, speed and height: are locus parameters (see above and Fig. 6)– support: the duration of the support of each leg– p2, p3, ... pn (where n is the number of legs): the leg phases (p1 is the “refer-

ence” leg, so there is only a set of parameters for a given gait)– frequency: the number of stride per second– x, y and z: are posture parameters that define the robot posture when the

dynamic controls are null

Thus, the total number of optimized parameters for the walk controller is(n − 1) + 8 where n is the number of legs.

5 Experiments

5.1 Dynamics Engine

We use the Bullet physics library, an efficient open-source dynamics engine [15].In this case, it can simulate rigid bodies, torque controllable hinges and collisions.

To simulate the motors, we applied torques directly on the components.Joints control is made by applying a target velocity from the position error(proportional), and then applying a target torque from the velocity error (pro-portional). To do so we use gains manually tuned. The maximum motor speedand torque curve is considered, the maximum torque that can be applied dependslinearly on the current speed, reducing to zero when the max speed is reached.

Optimizing Morphology and Locomotion 233

In order to get more realistic results, we added a backlash simulation on thedegrees of freedom, using a cone/twist constraint.

The collisions information can be retrieved from the dynamic simulation.They were used to detect both auto-collisions between the robot’s parts andcollisions between other parts than legs with the floor to add a penalty on thescore (to avoid crawling behaviors for instance).

Simulations were made using a time accuracy (Δt) of 1 ms, and all the partshad a friction coefficient of 0.5.

In order to make experiments possible, morphological parameters wererounded to millimeters and a cache system was added to avoid compiling andvoxelizing parts that were already met before (see Fig. 7).

5.2 Optimization

We use the black-box evolutionary algorithm CMA-ES [16,17] as optimizingalgorithm, with the following meta-parameters (all the problem parameters werenormalized between 0 and 1): algorithm: BIPOP CMA-ES, restarts: 3, elitism:2, f-tolerance 10−6, x-tolerance 10−3.

Notice that the f tolerance is maybe the more critical meta-parameter,because it determines how accurately the score should be tuned. The optimiza-tion indeed stops when the score is not optimized more than this tolerance duringa certain number of iterations. In our case, since each fitness evaluation typicallytakes a few seconds, a bad value for this can lead to hours of useless computationtrying to tune meaningless small values.

The dimension of experiments vary with morphological parameters and num-ber of legs (because of the phases) from 15 to 20.

5.3 Score

The goal of the first experiment consists in walking forward, i.e. going as far aspossible on the X axis. Two scores were tested, the first is simply the inverse ofthe distance walked (minimization), and the second is the inverse of the distancewalked multiplied by the energy cost, which is the sum of all the impulses (N.m.s)sent to the motors.

A second experiment consists in reaching checkpoints. Four near points haveto be attained successively by walking forward and turning to get the bodyaligned with the target. This introduces the trajectory control which adds a fewmore parameters to be optimized. Here, giant steps were used on the score witheach checkpoint passed. In order to help the convergence of the optimization, ifthe last checkpoint is not reached, the score is related to the distance to the nextmissed checkpoint. If the last checkpoint is reached, two scores were tested, thefirst is the duration of the experience (trying to minimize it), and the second isthe duration multiplied by the energy cost.

Finally, during this two experiments, we optimize the walk using a small fixedfrequency to compare the resulting gait.

234 G. Passault et al.

5.4 Workflow

The overall architecture is shown on Fig. 7. CMA-ES runs parallel simulationswith parameters set, each simulation first compile the dynamics of the robot,trying to hit the (filesystem) cache, speeding up the process. Simulations areheadless, but can also stream their state and are viewed using a custom 3Dclient GUI.

Fig. 7. Software architecture.

6 Results

While there is no guarantee that CMA-ES finds the global optimum, the opti-mization still provides very satisfactory sets of parameters for each robot.

6.1 Leg Phases

A first result can be viewed on the leg phases of the optimized robots. The(p2, p3, p4) plot of all optimized quadruped robots (all experiments mixed) isshown on Fig. 8, where all the points appear to be along a long quasi ellipsoid.

The trot gait is on the middle of the ellipsoid, the walk on one side andthe “cross” walk on the other side (see Fig. 10). Doing a Principal ComponentAnalysis (PCA) reveals that the first axis explains 65 % of the variance and thesecond one 32 % (see Fig. 9).

6.2 Locus

We plotted the histogram of locus for all experiments, resulting in Fig. 11.We can notice that the locus X values are mostly in the [0, 0.5] interval, which

means that the leg trajectory will go slightly forward when rising.

Optimizing Morphology and Locomotion 235

0.00.2

0.40.6

0.81.00.0

0.20.4

0.60.8

1.0

0.00.20.40.60.81.0

trotcross-walkwalk

Fig. 8. p2, p3 and p4 for all optimizedquadruped are along an ellipsoid (p3 wasshifted by 0.5 to center it) (Color figureonline)

32

10

12

332

10

12

3

3210123

65.7%32.7%1.6%

Fig. 9. PCA analysis of the points fromFig. 8, data are standardized and the leg-end corresponds to the proportion of vari-ance explained by each of the principalcomponents. (Color figure online)

Fig. 10. Different quadruped gaits mentioned in this paper. The diagram on the rightis the order of the rising legs (which correspond to leg phases).

Fig. 11. Histogram of locus X for all experiments (see Fig. 6).

6.3 Walking with Low Frequency

We also tried optimizations with fixed frequencies that were not tuned.We observe that this results in higher robot postures (see Fig. 12). Which

indeed reduce energy cost but lose stability. More robots were able to exhibit awalk (or cross walk) during this experiment.

236 G. Passault et al.

Fig. 12. The posture height (z) ofthe optimized robots from the corpusfor optimized frequency and constrainedfrequency. (Color figure online)

Fig. 13. The speed of the robots that thecontroller tries to reach (frequency ∗ dx)for the walking and checkpoints exper-iments (with energy cost in the score).(Color figure online)

6.4 Checkpoints

The “walk forward” experiment often leads to over-tuned running robots, thatsometime seem very unrealistic and likely not controllable. Moreover, walkingforward doesn’t prove that the robot can be reliably steered. This is why intro-duced the checkpoints experiment (see above). All the experiments converged torobots that successfully reach all the checkpoints.

The resulting robots walk slower (see Fig. 13) that those from the walkexperiment.

6.5 Fast Walk

The experiment that consists in a fast walk disregarding the energy cost unsur-prisingly converges to bigger robots in every cases (see Fig. 14). This is likelybecause the reachable space of legs is simply bigger, allowing bigger steps.

Fig. 14. The size of robots (which is the sum of morphological parameters) are com-pared in the walking experiment based on energy cost score and walking experimentbased on pure distance score. (Color figure online)

Optimizing Morphology and Locomotion 237

7 Conclusion and Future Work

We presented a framework allowing to generate and simulate parametric multi-legged robots. We also proposed a simple generic controller and used it throughsimulations experiments on a corpus of parametric robots. This controller wasable to produce locomotion for all the robots of the corpus, but also to controlthem to reach arbitrary checkpoints2.

Analysis of the simulations revealed that the search space could be reduced,like the leg phases for quadruped robots and the locus position that is almostalways on the front. We also noticed that reducing the robot’s energy cost imply asmaller morphology in every simulations, while making it steerable (with check-points experiments) reduces the speed. Moreover, the obtained leg phases foroptimized quadruped robots seems very similar to animals trot and walk.

In a future work, the phases ellipsoid result will be checked on real robotswith an appropriate experimental setup.

Some other optimizable parameters could also be added to the controller,like the orientation of the body.

The controller could also be improved in order to be able to handle robotswith spine and legs kinematic chains that share degrees of freedom, thus greatlyextending the capabilities of the robots.

It would also be interesting to compare CMA-ES with other optimizing algo-rithm for this specific experiment. For example a multi-objective optimizationmethod could allow to find not only “optimal” candidates but also optimal pop-ulations of candidates.

Finally we could also produce automatically robot morphologies from scratchwithout a pre-defined corpus. This raises some problems like kinematic singu-larities, upstream filtering of bad candidates (for example all the motors on thesame plane) and particularly adapting the optimization algorithm.

References

1. Muybridge, E.: Animals in Motion. Courier Corporation (2012)2. Marder, E., Bucher, D.: Central pattern generators and the control of rhythmic

movements. Curr. Biol. 11(23), R986–R996 (2001)3. Holmes, P., Full, R.J., Koditschek, D., Guckenheimer, J.: The dynamics of legged

locomotion: models, analyses, and challenges. SIAM Rev. 48(2), 207–304 (2006)4. Gregoire Passault, F.P., Rouxel, Q., Ly, O.: Metabot: a low-cost legged robotics

platform for education (Submitted)5. Pollack, J.B., Lipson, H.: The GOLEM project: evolving hardware bodies and

brains. In: 2000 Proceedings of the Second NASA/DoD Workshop on EvolvableHardware, pp. 37–42. IEEE (2000)

6. Megaro, V., Thomaszewski, B., Nitti, M., Hilliges, O., Gross, M., Coros, S.: Inter-active design of 3D-printable robotic creatures. ACM Trans. Graph. (TOG) 34(6),216 (2015)

2 A video of the obtained behaviors is available: https://www.youtube.com/watch?v=GF1KM7JrmC0.

238 G. Passault et al.

7. Marbach, D., Ijspeert, A.J.: Co-evolution of configuration and control for homoge-nous modular robots. In: Proceedings of the Eighth Conference on IntelligentAutonomous Systems (IAS8), BIOROB-CONF-2004-004, pp. 712–719. IOS Press(2004)

8. Samuelsen, E., Glette, K.: Real-world reproduction of evolved robot morpholo-gies: automated categorization and evaluation. In: Mora, A.M., Squillero, G.(eds.) Applications of Evolutionary Computation. LNCS, vol. 9028, pp. 771–782.Springer, Heidelberg (2015)

9. Cully, A., Clune, J., Tarapore, D., Mouret, J.-B.: Robots that can adapt like ani-mals. Nature 521(7553), 503–507 (2015)

10. Hengst, B., Ibbotson, D., Pham, S.B., Sammut, C.: Omnidirectional locomotionfor quadruped robots. In: Birk, A., Coradeschi, S., Tadokoro, S. (eds.) RoboCup2001. LNCS (LNAI), vol. 2377, pp. 368–373. Springer, Heidelberg (2002)

11. Neuhaus, P.D., Pratt, J.E., Johnson, M.J.: Comprehensive summary of the insti-tute for human and machine cognition’s experience with little dog. Int. J. Robot.Res. 30(2), 216–235 (2011)

12. Maes, P., Brooks, R.A.: Learning to coordinate behaviors. In: AAAI, pp. 796–802(1990)

13. The programmers solid 3D CAD modeller. http://www.openscad.org/14. Hormann, K., Agathos, A.: The point in polygon problem for arbitrary polygons.

Comput. Geom. 20(3), 131–144 (2001)15. Boeing, A., Braunl, T.: Evaluation of real-time physics simulation systems. In:

Proceedings of the 5th International Conference on Computer Graphics and Inter-active Techniques in Australia and Southeast Asia, pp. 281–288. ACM (2007)

16. Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolutionstrategies. Evol. Comput. 9(2), 159–195 (2001)

17. Multithreaded C++11 implementation of CMA-ES family for optimization of non-linear non-convex blackbox functions. https://github.com/beniz/libcmaes/

Stick(y) Insects — Evaluation of Static Stabilityfor Bio-inspired Leg Coordination in Robotics

Jan Paskarbeit1(B), Marc Otto2, Malte Schilling3, and Axel Schneider1,4

1 Biomechatronics Group, Center of Excellence ‘Cognitive InteractionTechnology’ (CITEC), University of Bielefeld, Bielefeld, Germany

jpaskarbeit@uni-bielefeld.de2 Robotics Research Group, Faculty of Mathematics and Computer Science,

University of Bremen, Bremen, Germany3 Neuroinformatics Group, Center of Excellence ‘Cognitive Interaction

Technology’ (CITEC), University of Bielefeld, Bielefeld, Germany4 Embedded Systems and Biomechatronics Group, Faculty of Engineering

and Mathematics, University of Applied Sciences, Bielefeld, Germany

Abstract. As opposed to insects, todays walking robots are typically notconstructed to withstand crashes. Whereas insects use a multitude of sen-sor information and have self-healing abilities in addition, robots usuallyrely on few specialized sensors that are essential for operation. If one ofthe sensors fails due to a crash, the robot is unusable. Therefore, mosttechnical systems require static stability at all times to avoid damagesand to guarantee utilizability, whereas insects can afford occasional fail-ures. Despite the failure tolerance, insects also possess adhesive, “sticky”pads and claws at their feet that allow them to cling to the substrate, thusreducing the need for static stability. Nevertheless, insects, in particularstick insects, have been studied intensively to understand the underlyingmechanisms of their leg coordination in order to adapt it for the controlof robots. This work exemplarily evaluates the static stability of a singlestick insect during walking and the stability of a technical system that iscontrolled by stick insect - inspired coordination rules.

1 Introduction

Man-made, technological systems designed after biological examples are usuallybuilt of different materials as compared to the natural systems. This leads toadditional constraints for the technical design which, in some cases, contradictthe transfer of the original, bio-inspired idea. In contrast, biological systemsoften have to conform to side conditions that are of no importance to the techni-cal system. To resolve this antagonism, a careful abstraction is important whenonly subsystems or substructures are transferred from the biological exampleto the technological system, in particular if mass, inertia or dimensions differ.This work highlights this perspective for the concept of static stability in thesix-legged walking robot Hector, depicted in Fig. 1(b), modeled after the stickinsect Carausius morosus as shown in Fig. 1(a). Unlike most robots, insects can

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 239–250, 2016.DOI: 10.1007/978-3-319-42417-0 22

240 J. Paskarbeit et al.

afford to disregard stability. If they lose static stability, they either fall withnegligible negative consequences or they cling to the ground with their tarsi.Their inviolability is e.g. due to their small size and mass combined with thecompound materials they are made from. Hector is neither designed to fallover nor are there any special ground attachment (anchoring) mechanisms at itsleg tips. This work starts with an exemplary evaluation of a walking stick insectand then focuses on the question under which conditions the leg coordinationframework in bio-inspired Walknet-type controllers [12] provides stable walk-ing. It finishes with conclusions on how to add stability as a side-condition tothe walking controller without negative interferences with the bio-inspired legcoordination.

Fig. 1. (a) Biological model, Carausius morosus, and rendering of Hector in top view.The CoMs are marked in both cases. (b) Image of Hector while climbing over a smallobstacle.

2 Evaluation of Static Stability for Walking Stick Insects

The static stability of stick insects has been discussed in literature with differentoutcomes. According to Jander, stick insects maintain static stability during tri-pod walking [6], whereas Kindermann found that stick insects topple backwardsif they stand on loose ground and one of the hind legs is lifted [7]. This viewis supported when looking at the detailed kinematics data of walking sticks aspublished recently by Theunissen et al. [15,16]. As an example, in one trial (“Ani-mal12 110415 00 22”, visualized in Fig. 2) the insect would be deemed multipletimes statically unstable (shown in red in Fig. 2(c) and (d)). Even though threelegs are on the ground the centre of mass (CoM) leaves the support polygon(CoM was assumed to be located between the hind coxae, see [2]). The footfallpattern for this run is shown in Fig. 2(b). Although at any point in time at leastthree feet are on the ground (black bars indicate stance phases), the stabilitymargin varies. The distance between the projection of the Center of Mass (CoM)onto the walking surface and the closest line of the support polygon is shown

Evaluation of Static Stability for Bio-inspired Leg Coordination 241

in Fig. 2(a). Negative distances stand for stable, positive distances for unsta-ble walking phases. As can be seen in conjunction with the footfall pattern inFig. 2(b), the distance increases every time one of the hind legs (L3, R3) is lifted.Even though the exact contact positions might vary depending on the unknownorientation of the tarsi, this would not change the result as the CoM leaves thesupport polygon by a considerable amount.

As the accompanying video of the trial shows, the stick insect does to nottopple even if the CoM is outside the support polygon since the insect is ableto cling to the ground with its tarsi. This points in the same direction as thefindings of Kindermann.

(a)

(b) (d)

(c)

(c) (d)

} }

L1L2L3R1R2R3

0 2 4 6 8 10 12-20-10

010

0 2 4 6 8 10 12

dist

. [m

m]

time [s]

time [s]

-40 -30 -20 -10 0 10 20 30 40-40

-30

-20

-10

0 L3

R2

L1

R1

L2L3

R3

40 60 80 100 120 140-40-30-20-10

R3

L2L1

R1R2

L3

x-pos [mm]

x-pos [mm]

y-po

s [m

m]

y-po

s [m

m]

Fig. 2. Experimental data from walking stick insect (based on data from [16]). (a)and (b) show the distance of the CoM to the support polygon and the correspondingfootfall pattern over time. A negative distance in (a) corresponds to a statically stableposture, a positive distance denotes an unstable posture. In (c) and (d), exemplary footpositions are plotted for three points in time that correspond to the colored, verticallines in (a) and (b). The connecting lines between the footpoints represent the supportpolygons for the respective postures. The positions of the CoM that correspond to thetimes for which the support polygons are shown, are marked by colored circles. Themovement of the CoM is plotted as a black line.

3 Bioinspired Leg Coordination

For bioinspired control of walking in hexapod and quadrupedal robots there aretwo distinct general approaches which both do not necessarily inflict stability onthe robot. On the one hand, open-loop control applies precomputed or plannedcontrol signals to the actuators [5]. As an advantage, such control approaches arenot affected by sensory noise and the communication does not deal with latenciesfrom sensory signals. On the other hand, in many cases animals have to deal withhighly unpredictable environments, e.g. for a stick insect when walking in uneven

242 J. Paskarbeit et al.

terrain or when climbing on twigs. Such approaches require close considerationof feedback from the environment which is mediated through sensory input.Closed-loop controllers allow for adaptive behavior by taking sensory feedbackinto account.

As mentioned, in locomotion those different approaches might fulfill differentroles. This work focuses on slow walking as an adaptive behavior which takes thesensed environment into account. An example for such a controller is Walknetwhich is inspired by experiments on the walking behavior of stick insects [12].Walking in a hexapod robot poses a difficult problem as it requires the coor-dinated movement of six legs. In walking, mainly two different behaviors aredistinguished on the leg level: On the one hand, as depicted in Fig. 3(b), theswing movement [14] during which the leg is lifted and protracted towards theAnterior Extreme Position (AEP). On the other hand, during the stance move-ment a leg supports the body as it is retracted towards its Posterior ExtremePosition (PEP) [11].

In Walknet, the leg coordination is distributed in six individual controllers,one for each leg. This is inspired by biological findings on the organization of thecontrol system in stick insects [12]. Each of these leg controllers has to decidewhich of the two behaviors (swing or stance) is executed. The decision is drivenby sensory signals and in particular the transitions between the two behaviorsare initiated by sensory input: A leg continues with a swing movement until ittouches the ground. During the stance phase, the leg is pushed backwards inorder to propel the body forwards. The stance movement continues until the legreaches its PEP. As each leg cycles through those two behaviors individually,a coordination mechanisms between the leg controllers is required. A simplesolution is given through a set of local coordination rules inspired by biologicalinsights on stick insect behavior [12]. Section 4 will deal with the question of staticstability in the light of this coordination scheme. The first three coordinationrules act directly on the transition from stance to swing to prolong or shortenthe stance phase (for effective directions for all rules see Fig. 3(a)):

Rule 1 prolongs the stance phase of the receiver leg if the sender leg is inswing phase. After the sender leg starts its stance phase, the influence persistsfor a short time at a reduced strength. The duration is speed-dependent. Rule 2facilitates the start of the receiver leg’s swing phase after the sender leg estab-lished ground contact at the end of its own swing phase. Between the detectionof ground contact and the activation of the influence, a short delay is provoked.Rule 3 facilitates the start of the receiver leg’s swing phase whenever the senderleg crosses a threshold position on its way towards the PEP. The influence lastsonly for a short time. The location of the threshold position is speed-dependentand differs for ipsilaterally and contralaterally neighbouring legs.

Rules 4–6 act in different ways on the coordination. These rules have sel-domly been implemented and are missing in this implementation as well as in thereference implementation of [12]. Early, theoretical analyses of some of the rules,though not in combination, are given in [1]. For a detailed description of thecoordination rules, see [12]. Walknet has successfully been tested in dynamics

Evaluation of Static Stability for Bio-inspired Leg Coordination 243

simulations and on robots [4], most recently on the robot Hector [10]. It hasproven to produce stable walking patterns at different velocities and leading todifferent types of insect gaits. In addition, the Walknet approach has beenused for a simple strategy on curve walking [11]. However, in tight curves theworking ranges of the individual legs are unequal. Legs situated on the innerside of the curve are barely moving while the outer legs’ working ranges increase(a detailed analysis of curve walking in stick insects is provided in [3]). Theseunequal stance distances represent a challenging situation for Walknet. Thiswork analyzes systematically how this affects stability and how the parametersof the coordination rules have to be adapted to produce stable walking.

L1

L2

L3

R2

R3

1,2,5

3,4,5,6

1,2,5

3,4,5,6

1,2,5

3,4,5,6

1,2,5

3,4,5,6

2,5

2,3

3?

5?

R12,3

iPEP

iAEPstance PEP

AEPx y

z

5

(a) (b)

Fig. 3. (a) Directions of influence for the coordination rules within Walknet.(b) Schematic depiction of a stance-swing-cycle for the left middle leg. The leg stancesfrom AEP to PEP. Both positions are shifted relative to the constant intrinsic positions(iAEP, iPEP) by the coordination rules.

4 Static Stability in Forward and Curve Walking for aTechnical System

To test the suitability of the described bio-inspired leg coordination withoutmodifications for the control of a hexapod robot, a physics simulation of Hectorwas used. To assess stability, the virtual robot was commanded to walk on flatterrain for a given time. At the end of each trial, the fraction of iterations wasdetermined, during which the robot was statically stable. For this evaluation,the first 5 s of each trial were excluded as this work is not concerned with thetransient behaviour at the beginning of walking. Therefore, a fraction of 1 sig-nifies that the robot has been statically stable after the first seconds. A reducedstability fraction indicates unstable situations during the experiment.

The bio-inspired leg coordination is based mostly on the shifting of the PEPof each leg (based on the constant, intrinsic PEP (iPEP), triggered by its neigh-bouring legs). Therefore, the unit of rule strengths will be given in meters.

244 J. Paskarbeit et al.

For the first experiments, the walking controller as reported by Schilling et al.was used for slow and fast walking [12]. To evaluate the influence of the individualcoordination rule strengths, the intensities of the coordination rules were variedwithin intervals of [−0.5 m, 0.0 m] for rule 1 and [0 m, 0.25 m] for rules 2 and 3.The ipsilateral and contralateral influences of rule 3 were adjusted individually,as their start thresholds are computed differently (see [12]). As the PEP-shiftsof rule 3 in the front legs are found to be stronger than in the hind legs [12], afixed ratio of 3:1 is assumed and only the values for the front legs will be given inthe results. Since the leg coordination is influenced by the initial posture, eachrun was conducted from three different, predefined postures to rule out peaksin stability due to a incidentally chosen perfect starting posture. All data thatis shown represents the worst result for these three trials. In stick insects, theload of the legs is sensed by campaniform sensilla that are located close to thejoints [17]. For the data shown in this work, force sensors were simulated at theleg tips. To obtain an immediate feedback for the detection of ground contacts,a threshold of 0.001 N is used if not otherwise specified.

In the first experiment, the robot walked with two different speeds (slow:0.15 m/s and fast: 0.25 m/s) using the default controller settings that are givenin [12]. Only the strengths of the coordination influences (the rule-specific PEP-shifts) were varied systematically during the experiment. For slow walking, mul-tiple parameter sets were found that create permanently stable coordination (seeFig. 4(a)). For the fast walking experiment, for which the results are shown inFig. 4(b), no such parameter set could be found. The maximum stability fractionwas 0.94. Therefore, in 6 % of the iterations the robot was considered unstable.In practice, a short duration of instability will not necessarily result in a crash.Due to the inertia of the main body it takes some time for the robot to tilt.Also, if a leg is in swing phase during the tilting movement, it may strut therobot as soon as it touches ground and switches to stance phase. Therefore, thevirtual as well as the real robot have successfully performed multiple walks –even on rough terrain [10]. Nevertheless, for the general application of the bio-inspired controller on real hexapod robots, the stability should be maintained inall situations and independent of the initial posture.

To achieve this, the complete set of controller parameters has been optimizedusing Simulated Annealing [8], a global optimization technique which involvesa probabilistic search function. In addition to the already mentioned variationof the PEP-shifts, the deactivation delay of rule 1 and the activation delay ofrule 2 were optimized. Also, the contact force threshold between the leg and theground, at which the leg switches to stance at the end of a swing phase, wasoptimized.

With the parameter set listed in Table 1, a stability fraction of 0.998 couldbe achieved for fast walking. Although the optimization focused only on fastwalking, with these optimized parameters, slow walking was still found to be sta-tically stable. Although this result misses the goal of permanent, static stability,the controller will likely preserve the robot from falling in almost all situationsduring forward walking.

Evaluation of Static Stability for Bio-inspired Leg Coordination 245

rule 2 [m]

0.5 0.6 0.7 0.8 0.9 1fraction of stable iterations

rule 2 [m]

(a)

(b)

0.000

0.125

0.250

0.0000.1250.2500.0000.1250.2500.0000.1250.2500.0000.1250.2500.0000.125

rule

3 i

[m]

psila

tera

l

rule 3 contralateral [m]

0.000

0.125

0.250

0.000

0.125

0.250

0.000

0.125

0.250

0.000

0.125

0.250

0.000

-0.125

-0.250

-0.375

-0.500

0.000 0.063 0.125 0.188 0.25

rule

1 [m

]

0.250

0.000

0.125

0.250

0.0000.1250.2500.0000.1250.2500.0000.1250.2500.0000.1250.2500.0000.125

rule

3 i

[m]

psila

tera

l

rule 3 contralateral [m]

0.000

0.125

0.250

0.000

0.125

0.250

0.000

0.125

0.250

0.000

0.125

0.250

0.000

-0.125

-0.250

-0.375

-0.500

0.000 0.063 0.125 0.188 0.25

rule

1 [m

]

0.250

Fig. 4. Fraction of stable iterations per total iterations for different strengths of thecoordination rules. (a) shows the results for slow walking (0.15 m/s), (b) for fast walking(0.25 m/s). The fields marked by red frames represent the parameter sets that achievemaximum stability fractions. (Color figure online)

As has been shown in [12], after some initial steps, the Walknet-basedcontroller usually generates a regular gait. In the data collected in this work,this is observable as well. However, during straight forward movement, the stancedistances of the legs are equal in average. Therefore, the question remains howthe controller handles situations, in which the stance distances of the legs vary,e.g. in curve walking. To examine this question, the robot was commanded towalk curves with a constant radius as shown in Fig. 6(a). The deviations from aperfect circular trajectory are due to the inherent compliance of Hector that

246 J. Paskarbeit et al.

Table 1. Parameter set obtained by simulated annealing

Parameter Value

Rule 1 −0.0765 m

Rule 2 0.2464m

Rule 3 contralateral 0.2499m

Rule 3 ipsilateral 0.0364m

Rule 1 deactivation delay 0.3547 s

Rule 2 start delay 0.0591 s

Ground contact force threshold 3.7385N

was also modelled in the simulation. During these trials, no course correctionwas applied, as this could have influenced the results.

The overall performance regarding the stability fraction decreases consider-ably for curve walking. Therefore, in order to find an optimal set of coordinationrule strengths, the parameter variation was performed as already shown for for-ward walking. As Fig. 5 shows, no parameter set could be found that createscomparable stability fractions as obtained during forward walking. The beststability fraction that was achieved for curve walking is 0.85. Thus, it seemslikely that substantial adaptions of the co-ordination mechanisms are requiredto achieve statically stable curve walking.

0.5 0.6 0.7 0.8 0.9 1fraction of stable iterations

rule 2 [m](b)

0.000

0.125

0.250

0.0000.1250.2500.0000.1250.2500.0000.1250.2500.0000.1250.2500.0000.125

rule

3 i

[m]

psila

tera

l

rule 3 contralateral [m]

0.000

0.125

0.250

0.000

0.125

0.250

0.000

0.125

0.250

0.000

0.125

0.250

0.000

-0.125

-0.250

-0.375

-0.500

0.000 0.063 0.125 0.188 0.25

rule

1 [m

]

0.250

Fig. 5. Fraction of stable iterations per total iterations for different strengths of thecoordination rules during fast curve walking (0.25 m/s). The fields marked by red framesrepresent the maximum stability fractions. (Color figure online)

Evaluation of Static Stability for Bio-inspired Leg Coordination 247

-0.2 0.0 0.2 0.4 0.60.00.10.20.30.40.50.60.70.8

assumed x-position of center of mass [m]

fract

ion

of s

tabl

e ite

ratio

ns

cent

er o

f mas

sof

bio

logi

cal m

odel

cent

er o

f mas

sof

robo

t

opt. fast walking (0.25 m/s)curve walking (0.25 m/s)

(b)

fast walking (0.25 m/s)(0.25 m/s)

0.8

0.91.0 slow walking

(0.15 m/s)

-2 -1 0 1 2-4

-3

-2

-1

0

1

x-position [m]

y-po

sitio

n [m

]

Fig. 6. (a) Exemplary trajectories for curve walking. (b) Maximum fraction of stableiterations over all iterations for different virtually shifted CoMs. The position of theCoM is given relative to the onset of the hind legs. (Color figure online)

Regarding stability, one of the most relevant differences between the insectand the robot is the shifted CoM. Whereas the CoM lies roughly between thehind leg onsets in the insect, in the robot, it is located slightly in front of themiddle legs (see Fig. 1(a)). To analyze the influence of this aspect, the stabilityfraction was re-evaluated by virtually shifting the location of the CoM alongthe longitudinal axis of the robot. In practice, a real shift of the CoM wouldprobably result in different outcomes, especially for the extreme positions thatwould make coordinated locomotion impossible.

Figure 6 shows the highest stability fractions that could be achieved for any ofthe coordination rule parameter sets for slow and fast walking (first experiment),optimized fast and curve walking for different CoMs (relative to the position ofthe hind legs). For forward walking, the peak stability can be obtained for a CoMclose to that of the robot. For curve walking, the optimal CoM lies closer to theback of the robot. Although these results do not reflect the actual outcome ifthe CoM would have been actually shifted, it is both notable and plausible thatthe stability fraction would be reduced for an insect-like positioned CoM.

5 Conclusion

Unlike most robots, insects typically have the ability to cling to the ground.Depending on their habitat they might still require static stability to keep fromtoppling over. An example are grass-cutting ants that usually walk on looseground [9]. On the other hand, stick insects like Carausius morosus live onbranches, often hanging upside-down. For them, the substrate is rigid enough tohold on to it. Thus, static stability is not a key requirement for them in theirnatural habitat. However, for walking on plain ground, different results have beenreported regarding their ability to maintain static stability. As Jander reports,the lift-off of hind legs is triggered only when the CoM can be supported by theremaining legs on the ground [6]. Kindermann observed backward toppling if theinsects could not cling to the ground and one of the hind legs was lifted [7].

248 J. Paskarbeit et al.

Despite the stability issue, stick insects seem to be an optimal model forwalking machines due to their adaptive gait patterns. With increasing movementspeed, they change from wave gait to tetrapod and tripod continuously. Thus,for low speeds, many legs are on the ground which in principle increases stabilityand resilience. For high speeds, the fastest potentially stable gait (tripod gait)is adopted.

While these adaptive gaits are desirable for walking machines, due to theirrequirement to maintain static stability to prohibit falling, the direct adaptabil-ity of the leg coordination concepts as found in stick insects for technical systemswas unclear due to the contradictory reports. In this paper, kinematic data of awalking stick insect is analyzed regarding the position of its CoM relative to thesupport polygon. It was found that the stability margin decreases whenever oneof the hind legs switches from stance to swing phase. For two exemplary situa-tions, it was shown that the CoM left the support polygon, thus indicating anunstable posture. As a consequence, the suitability of the coordination conceptsfor walking machines, in the light of their need for unconditioned stability, isunclear.

Based on the setup of the bio-inspired robot Hector, the coordination rulesas reported by [12] have been evaluated for their ability to maintain static sta-bility in different scenarios. The most relevant differences between the robot andits biological model, Carausius morosus, are the inability of the robot to clingto the ground and a shift of its CoM towards the onsets of the middle legs asopposed to the hind legs in the insect. The inability to cling to the ground makesthe robot prone to falling if the stability is not maintained by safe arrangementof the support polygon at any time. In a physics simulation, the robot, con-trolled by the bio-inspired coordination concept, was tested in three differentscenarios and a multitude of control parameters. For slow forward walking atspeeds of 0.15 m/s, multiple sets of coordination parameters were found thatmaintained static stability throughout the test runs. For fast forward walkingat 0.25 m/s, none of the parameter sets could stabilize the robot at all times.With an optimized parameter set, however, a stability fraction of 0.998 couldbe achieved. Since the coordination rules have been derived from observation offorward walking stick insects, the concept of Walknet and the implementedcoordination rules are not explicitly designed for curve walking. During tests, atwhich the robot was commanded to walk curves, the stability was significantlydecreased to stability fractions of about 0.85.

As the evaluation of the kinematic data showed, stick insects do not relyon static stability for locomotion. In comparison, technical systems such as thehexapod robot Hector must maintain stability at all times. Hector has beenmodeled explicitly after the model of the stick insect. Thus, for example, therelative positions of the legs have been preserved (scaled up by a factor of 20).The results suggest that the existing, bio-inspired coordination rules alone do notguarantee unconditioned stability in a robot like Hector or other robot setupswithout ground attachment mechanisms or specifically adapted leg distances.

Evaluation of Static Stability for Bio-inspired Leg Coordination 249

However, to benefit from the adaptability of bio-inspired leg coordination,provided by Walknet, in arbitrary multi-legged robot setups, additional con-trol layers must be introduced. One way to guarantee stability would be theobservation of the distance between the center of gravity and the closest line ofthe support polygon. If the body movement would lead the CoM too close tothe border of the support polygon, the movement could be modified to counter-act the danger of toppling. Also, if a leg is supposed to switch to swing phase,it could be checked prior to its lift-off whether this stance-swing-switch wouldendanger the stability and, if required, the stance phase could be extended. Thiscan easily be formulated as an additional, technological rule that amends theotherwise purely bio-inspired rule set of Walknet.

Another approach would be a cognitive layer that detects whenever a situa-tion occurs that does not comply with the requirements of the technical system[13]. In these cases, an internal model of the robot could be used to search foradequate solutions of the problem. In the cases depicted in Fig. 2, in which thelift-off of a hind leg compromised the stability of the insect, a solution could bethe repositioning of the opposite hind leg such that it can support the center ofgravity.

For robust locomotion on complex terrain, combinations of the mentionedapproaches could combine the elegance of insect locomotion with the require-ments of technical systems.

Acknowledgments. This work has been supported by the DFG Center of Excellence‘Cognitive Interaction TEChnology’ (CITEC, EXC 277) within the EICCI-project.

References

1. Calvitti, A., Beer, R.D.: Analysis of a distributed model of leg coordination. I.Individual coordination mechanisms. Biol. Cybern. 82(3), 197–206 (2000)

2. Cruse, H.: The function of the legs in the free walking stick insect, Carausiusmorosus. J. Comp. Physiol. B 112(2), 235–262 (1976)

3. Durr, V., Ebeling, W.: The behavioural transition from straight to curve walking:kinetics of leg movement parameters and the initiation of turning. J. Exp. Biol.208(12), 2237–2252 (2005)

4. Espenschied, K.S., Quinn, R.D., Beer, R.D., Chiel, H.J.: Biologically based dis-tributed control and local reflexes improve rough terrain locomotion in a hexapodrobot. Robot. Auton. Syst. 18(1–2), 59–64 (1996)

5. Ijspeert, A.J.: Central pattern generators for locomotion control in animals androbots: a review. Neural Netw. 21(4), 642–653 (2008)

6. Jander, J.: Mechanical stability in stick insects when walking straight and aroundcurves. In: Gewecke, M., Wendler, G. (eds.) Insect Locomotion, pp. 33–42. PaulParey, Berlin, Hamburg (1985)

7. Kindermann, T.: Positive Ruckkopplung zur Kontrolle komplexer Kinematikenam Beispiel des hexapoden Laufens: Experimente und Simulationen. Ph.D. thesis,Universitat Bielefeld (2003)

8. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing.Science 220, 671–680 (1983)

250 J. Paskarbeit et al.

9. Moll, K., Roces, F., Federle, W.: How load-carrying ants avoid falling over: mechan-ical stability during foraging in Atta vollenweideri grass-cutting ants. PLoS ONE8(1), e52816 (2013)

10. Paskarbeit, J., Schilling, M., Schmitz, J., Schneider, A.: Obstacle crossing of a real,compliant robot based on local evasion movements and averaging of stance heightsusing singular value decomposition. In: IEEE International Conference on Roboticsand Automation, ICRA 2015, Seattle, WA, USA, 26–30 May 2015, pp. 3140–3145(2015)

11. Schilling, M., Paskarbeit, J., Schmitz, J., Schneider, A., Cruse, H.: Grounding aninternal body model of a hexapod walker - control of curve walking in a biologicalinspired robot–control of curve walking in a biological inspired robot. In: Proceed-ings of IEEE/RSJ International Conference on Intelligent Robots and Systems,IROS 2012, pp. 2762–2768 (2012)

12. Schilling, M., Hoinville, T., Schmitz, J., Cruse, H.: Walknet, a bio-inspired con-troller for hexapod walking. Biol. Cybern. 107(4), 397–419 (2013)

13. Schilling, M., Paskarbeit, J., Hoinville, T., Huffmeier, A., Schneider, A., Schmitz,J., Cruse, H.: A hexapod walker using a heterarchical architecture for action selec-tion. Front. Comput. Neurosci. 7 (2013)

14. Schumm, M., Cruse, H.: Control of swing movement: influences of differentlyshaped substrate. J. Comp. Physiol. A 192(10), 1147–1164 (2006)

15. Theunissen, L., Bekemeier, H., Durr, V.: Stick insect locomotion (2014).toolkit.cit-ec.uni-bielefeld.de/datasets/stick-insect-locomotion-data

16. Theunissen, L.M., et al.: A natural movement database for management, documen-tation, visualization, mining and modeling of locomotion experiments. In: Duff, A.,Lepora, N.F., Mura, A., Prescott, T.J., Verschure, P.F.M.J. (eds.) Living Machines2014. LNCS, vol. 8608, pp. 308–319. Springer, Heidelberg (2014)

17. Zill, S.N., Schmitz, J., Buschges, A.: Load sensing and control of posture andlocomotion. Arthropod Struct. Dev. 33(3), 273–286 (2004)

Navigate the Unknown: Implicationsof Grid-Cells “Mental Travel” in Vicarious

Trial and Error

Diogo Santos-Pata1(B), Riccardo Zucca1, and Paul F.M.J. Verschure1,2

1 Universitat Pompeu Fabra, SPECS group, N-RAS,Roc Boronat, 138, 08018 Barcelona, Spain

{diogo.pata,riccardo.zucca,paul.verschure}@upf.edu2 ICREA, Barcelona, Spainhttp://www.specs.upf.edu

Abstract. Rodents are able to navigate within dynamic environments byconstantly adapting to their surroundings.Hippocampalplace-cells encodethe animals current location and fire in sequences during path planningevents. Place-cells receive excitatory inputs from grid-cells whose metricsystem constitute a powerful mechanism for vector based navigation forboth known and unexplored locations. However, neither the purpose or thebehavioral consequences of such mechanism are fully understood. Duringearly exploration of a maze with multiple discrimination points, rodentstypically manifest a conflict-like behavior consisting of alternating headmovements from one arm of the maze to the other be- fore making a choice,a behavior which is called vicarious trial and error (VTE). Here, we sug-gest that VTE is modulated by the learning process between spatial- andreward-tuned neuronal populations. We present a hippocampal model ofplace- and grid-cells for both space representation and mental travel thatwe used to control a robot solving a foraging task. We show that place-cellsare able to represent the agents current location, whereas grid-cells encodethe robots movement in space and project their activity over unexploredpaths. Our results suggest a tight interaction between spatial and rewardrelated neuronal activity in defining VTE behavior.

Keywords: Biomimetics · Navigation · Grid-cells · Mental travel · VTE

1 Introduction

Representing ones’ location in space and decode future trajectories are essentialfeatures for surviving in the world. Rodents are exceptional foragers capable ofextensively explore large and complex environments, find their way back homeand adapt to dynamic environments. Their robotic counterparts, however, stillfind difficulties in integrating their path over space, decide routes to take andadapt to novel environmental configurations. Studies on the rodent hippocampushave reported multiple cell types encoding for spatial properties, such as animal

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 251–262, 2016.DOI: 10.1007/978-3-319-42417-0 23

252 D. Santos-Pata et al.

position and orientation. Place-cells, a type of neuron encoding for specific loca-tions of the explored environment have been found in the dentate gyrus, CA3and CA1 regions of the rodent hippocampus [15]. Since the discovery of place-cells, the hippocampus has been considered to play a fundamental role in spatialnavigation and representation.

Place-cells receive their excitatory projections from the entorhinal cortexwhere grid-cells are found [1]. The firing activity of grid-cells in the medialentorhinal cortex is typically arranged in an hexagonal pattern tessellation cov-ering the explored environment. Grid-cells encode the environment through mul-tiple scales that progressively increase along the dorsal-ventral axis of the medialentorhinal cortex, allowing to represent space at multiple levels of resolution.Grid-cells have been suggested to serve as a metric component necessary forplace-cells in the proper hippocampus to tune their activity to specific locationsof the explored environment. Indeed, computational models of place-cell forma-tion have relied on the input-output transformation of grid-cells signals throughinhibitory network competition processes [14].

An universal metric system combined with place-specific firing activity allow-ing to decode spatial positions is a crucial feature for optimal spatial navigation.However, in order to reach goal-locations, one needs to be able to plan future tra-jectories and make sense of previously encoded spatial memories. Hippocampalplace-cells are not only active at their specific locations, but they have been alsofound to fire in spatial sequences during both locomotion, a phenomenon calledtheta-sequences [5], and stationary periods, the so-called sharp-wave ripples [17].Place-cells fire in spatial sequences when the animal is at decision points [5] of amultiple t-maze configuration. Such mechanism of spatial representation encod-ing future trajectories, allowing the animal to decide which action to take, is amajor advantage of foraging animals when compared to the current capabilitiesof their robotic counterparts.

Despite a myriad of studies regarding grid-cells formation and functionality,the grid-cells metric system is still far from being exploited in robot navigation,path-planning and the so called “mental traveling”. Because grid-cells activatebased on attractor dynamics, show periodicity, and are modulated by the animalmovements, their primary role in path integration is widely accepted and suchtype of signaling is a powerful candidate to build spatial representations of theexplored environments. However, it has been recently proposed that a secondfunction of grid-cells is to perform “mental traveling” processes [10,11]. Becausetheir activity is independent of environmental sensory cues, they constitute apotentially robust context-independent metric system for dynamic environments.Furthermore, a mechanism has been proposed in which grid-cells potentiallydrive vector-based navigation to both known and unexplored locations [11].

However, the mechanisms generating such sequences of future locations arestill unclear. “Mental travel” implies a neural signal of spatial navigation with-out actual motor movement [10,11]. Thus, “mental travel” refers to an imageryprocess of translating ones’ body in space while remaining stationary. Becausegrid-cells activity is independent of environmental sensory cues or the spatial

Grid-Cells “Mental Travel” and VTE Behavior 253

context, they constitute a potentially robust context independent metric sys-tem to ‘virtually’ explore dynamic environments. Given their underlying low-dimensional continuous attractor dynamics in synaptic connectivity [16], as wellas their ramp-driven spiking behavior [4], grid-cells do not need intrinsic learningin order to project their activity into imaginary locations. Thus, at the computa-tional level, motion related signals such as orientation and direction are sufficientto generate grid-like activity [6]. Hippocampal theta oscillations have been sug-gested to drive the hippocampal representational mode [10]. That is, during thefirst half of a theta cycle, the animal computes its current location, while dur-ing the second half, the animal computes future trajectories, through a ‘virtualspeed’ input arriving to this representational system. However, the origin of thisvirtual speed signal the role of hippocampal theta oscillations in driving thehippocampal mode remains unclear.

Here, we argue that septal/hippocampal theta generated signals function asa representation mode switching mechanism and that the virtual speed signal forperforming “mental travel” is encoded in the actual behavior of the animal. Thus,there is no need of extra anatomical projections in order to perform simulatedvector based navigation. Previously, we have presented an hippocampal basedrobotic system for spatial representation where populations of head-direction-cells encoding for robot’s orientation and grid-cells performing path-integrationwere sufficient to drive place-cells encoding the robot’s current location [2,3]. Inthis study, we present an improved implementation of the previously presentedhippocampal system. Here, grid-cells not only perform path-integration used forthe place-cells location encoding, but also allow the robot to anticipate futurelocations through “mental travel” processes. Being able to simulate a trajec-tory without physically navigating within the environment allows to constantlyupdate planned trajectories as well as predict where do novel routes will lead to.Thus, it seems reasonable to develop such mechanisms to be employed on realworld foraging robots.

2 Methods

2.1 Neural Populations

In order to simulate future trajectories, a foraging robot has to be able to under-stand the properties of an environment and being capable of situating itself inspace. To do so, we first implemented rate-based grid-cell neurons equally dis-tributed among 6 modules (N = 6000), each with different grid-scale properties(see [2,3,6] for a detailed explanation of the model’s implementation). Grid-cellsactivity was initialized with random activity between 0 and 1/N (number ofneurons in each grid-cell module) and were modulated by the speed-vector ofthe robot movements at each time step. As in [6], the activity of each cell attime t was defined using a linear transfer function given by:

Bi(t + 1) = Ai(t) +N∑j=1

Aj(t)Wij (1)

254 D. Santos-Pata et al.

where, Ai is an average normalization mechanism to maintain stability ofthe network (see [6] for details), i and j are indexes of cells in the network andWij is the synaptic weight between cell i and j. Because grid-cells are periodicand modulated by the robot’s velocity, their synaptic weights were set basedon low-continuous attractor mechanisms. The attractor dynamics were definedthrough the synaptic distribution within each grid-cell modules as:

Wij = Iexp

(−‖ci − cj‖2tri

σ2

)− T (2)

where, I(= 0.3) is the synaptic strength intensity parameter, σ(= 0.24) mod-ulates the synaptic distribution Gaussian size, and T (= 0.05) is a parametersdefining inhibitory and excitatory connections in the synaptic distribution, triregulates the bump of activity to be moved along a toroidal network accordinglywith the robot’s movements (see [6] for more details).

A second neural population was set to reproduce place-cells like activity overthe environment. That is, each cell should increase its firing rate at specificlocations of the explored environment. If so, the robot will be able to develop aspatial representation and awareness of its current position at each time.

Similarly to [14], we implemented 10000 rate-based place-cells and their activ-ity was modulated by the excitatory inputs arriving from multiple grid-cells alongthe dorsal-ventral axis with progressive scales:

Fplace = Igrid · H(Igrid − (1 − K) · gridmax(r)) (3)

where, H is a Heaviside function, gridmax(r) is the maximum activity pro-jected from a grid- to a place-cell at position r, and K(= 0.9) determines thethreshold for every place-cell to compete through the winner-take-some process:

F iplace =

{F iplace, if F i

place ≥ placemax · (1 − K)0, otherwise

(4)

where F iplace is the activity of place-cell i and placemax is the activity of

the place-cell with higher firing rate at each time. Because we did not impose aspecific synaptic weights distribution from grid- to place-cells, place-cells weresusceptible of having multiple and spread firing fields. In order to refine place-cells receptive fields, during the exploration phase we set a potentiation ruledetermining that when the amount of active place-cells would exceed 25 % oftotal number of place-cells, a Hebbian learning mechanism [13] would be appliedto the 5 % strongest activated cells, while the grid- to place-cells synaptic weightsof the remaining 95 % of neurons would be suppressed.

We have described the emergence of VTE events during deliberative stagesof task contingencies learning [8,9]. Because such behavior is involved in goal-directed navigation, usually towards a reward location, the hippocampal for-mation alone would not be sufficient to trigger VTE. We thus implemented anapproximation of striatal cells encoding for reward locations [12]. Reward-cellswere modeled through rate-cells activity with bidirectional connectivity between

Grid-Cells “Mental Travel” and VTE Behavior 255

hippocampal place-cells. Learning association between the place-cells populationvector and reward-cells was performed through a single layer perceptron. Thus,two sources of input arrive to reward-cells. On one hand, every time the simu-lated agent crosses a reward location, reward-cells are set to high firing rate. Onthe other hand, place-cells activity is also projected into reward-cells, allowingto decode reward locations during “mental traveling” towards the rewarded armof the maze (see Fig. 1C). A single hippocampal-reward perceptron circuit wasimplemented and output activity of its reward-cell was considered as a mean-field approximation of striatal output. Synaptic weights between reward- andplace-cells were initially set to random values ranging from 0 to 1, with an addi-tional bias unit. The learning rate (γ) of hippocampal-reward associations wasset to 0.7. During learning, reward cells activity was set as:

Rewardr =N∑j=1

PCj · Wrj (5)

where PCj corresponds to the activity of every place-cell connected toreward-cell (r), and Wrj is the synaptic weight between these two cells. Atevery update of the reward-cell activity, the error between the expected activityprovided from the reward signaling when the agent crossed a reward locationand the actual reward-cell activity was given by:

RewardError = RewardExpected − f(Rewardr) (6)

where, f(Rewardr) is a step-function given by:

f(Rewardr) =

{1, if Rewardr ≥ 0,

0, otherwise.(7)

The synaptic weights between place- and reward-cells was finally updatedgiven by:

Wrj = γ · RewardError · PCj (8)

where γ is the learning rate for the synaptic weights update.

2.2 Robotic Setup

Our setup consisted of a virtual agent performing a navigation task towards pre-viously explored locations. First, in order to test whether our implementationwas properly set, we allowed the agent to randomly explore an open field arena.Thus, we could assess grid- and place-cells rate maps (Fig. 2). To validate oursystem we have configured a virtual environment resembling a T-maze alternatetask. At every time the virtual agent arrived to the T-maze decision point, itcould orient towards either left or right sides of the maze until eventually decideto move on to the orientation correspondent alley. Trajectories from decision-point until arriving back to that same location were programmatically defined.

256 D. Santos-Pata et al.

Given the structure of our neural network implementation resembling the hip-pocampal formation and striatal reward mechanisms, the simulated agent couldmodulate its decisions based on the reward-cells activity every time it would“mental travel” towards a specific alley of the maze. Thus, after learning place-reward contingencies, the agent’s go signal was triggered every time reward-cellswould exceed an activity rate of 0.75. Because at the decision point the agentwas set to randomly orient to the maze alleys and perform grid-cells “mentaltravel” simulation, it would affect both place- and reward-cells activity, by meansof virtually traveling towards a reward-location. Thus, the agent could learn thebest action to take in order to get its reward. A maximum of 8 “mental travel”simulations (4 towards the right arm) was set so that the agent would not getstuck at the decision-point when no go signal was triggered. In those trials, theagent randomly picked a direction to move on. Moreover, in this study, we con-sidered as VTE events, those trials at which the agent could not take a decisionbased on the reward-cell activity.

3 Results

In order to quantify the effects of grid-cells “mental travel” in VTE behavior,we implemented the described hippocampal model in a simulated agent. Sixpopulations of grid-cells with distinct scales were generated (Fig. 2-bottom). Atotal of 6000 cells were made available to project excitatory input to the place-cells population where individual cells place-fields showed to be specific to uniquelocations of the explored environment (Fig. 2-top).

Open Field Arena. In the first quantification session, we set the agent to ran-domly explore the open field arena where no target was assigned and contactwith environmental borders was avoided. Figure 2 illustrates rate-maps examplesof both grid- and place-cells obtained from this exploratory session. Thus, wecould assess that a grid-to-place spatial representation from our hippocampalimplementation was properly functioning. Rate-maps of four grid-cells belong-ing to different populations depict changes in grid-scale (Fig. 2), mimicking thedorsal-ventral axis of medial entorhinal cortex layer 2 [6,7].

T-maze Test. We then tested our simulations in an alternate task mimickinga T-maze configuration (Fig. 3). Rate-maps of grid-cells during exploration ofthe T-maze were obtained. As expected, firing-fields from single cells coveredmultiple regions of the maze, implicating that position could not be assessedfrom single cells activity.

We have described our hippocampal implementation for a simulated agentperforming navigational tasks. Nonetheless, hippocampal specific activity per seis not sufficient to trigger VTE. Additionally to our previous implementation[2,3], we added a population of rate-based neurons mimicking the role of striatalcells. Thus, we expected that by learning associations between place-cells popu-lation vector and reward-cells activity when the agent is at the reward-location,

Grid-Cells “Mental Travel” and VTE Behavior 257

Fig. 1. Spatial representation and ‘virtual’ exploration. (A) Grid-cells receive eitherreal velocity inputs from the robot movements or ‘virtual’ velocity inputs for decodingfuture positions. (B) Situations of ’mental’ traveling. At decision points, the robotsimulates exploration towards left (1) and right (2) possible trajectories. In an openfield arena (3), the agent probes optimal orientations to achieve a target location. (C)Artificial neural network implementation. Speed signal input grid-cells. Each grid-cellis synaptically connected to every cell in its network forming an all-to-all connectivitymatrix per population. The synaptic weights are modulated by the Cartesian distanceof each cell pair in the population matrix, as illustrated in the cell at the top (colorcode ranges from inhibitory to excitatory weights). Grid-cells activity is projected tothe population of place-cells where competition through inhibitory interneurons tuneindividual cells to specific spatial locations. Place-cells population vector is than pro-jected into reward-cells such that a reward signal is obtained when place-cells activitymatch with activity at reward location. (D) Illustration of hippocampal/septal thetageneration and its correspondent real/virtual speed alternation mode. (Color figureonline)

258 D. Santos-Pata et al.

Fig. 2. Spatial rate maps of place- (top) and grid-cells (bottom) obtained from a sim-ulated agent randomly navigating withing a squared arena.

we would have been able to activate reward-cells while the animal rested at thedecision-point, but virtually navigated towards the reward-location. We havehypothesized that the process of ‘mental travel’ would intrinsically modulateVTE events. Furthermore, we suggested that the learning phase of place-rewardassociations directly predicts the strength of VTE behavior (i.e., the amount ofturns towards the possible choices until the agent makes a decision). An algorith-mic description of such process is illustrated in Algorithm1: VTE from “mentaltravel”. When the agent is situated at the decision-point of the T-maze, it isallowed to perform a set of “mental travels” towards each arm of the maze.Within each simulation, a maximum of 100 network cycles are allowed. Simula-tions were performed by inputting a constant speed scalar of one unit, but itsorientation component was defined the agent’s head orientation. At each cycle,the activity of grid-cells resembled their activity when the agent was activelynavigating within the maze. Similarly, place-cells consistently activated as theagent would be crossing their respective place-field. Figure 4 shows the corre-lation of place-cells population activity during ‘mental travel’ events for therewarded location, suggesting a proper decoding of future outcomes. Becausereward-cells activity has been associated with specific population vector activ-ity of hippocampal place-cells, by virtually crossing the rewarded location, alsoreward-cells would become active, and thus triggering the go signal of the deci-sion process. In case the amount of simulation events reached the maximumnumber of allowed choices, the agent’s orientation was set to towards the correctarm and navigation started.

Changed Task Contingencies. We have hypothesized that the learning processbetween place- and reward-cells would modulate VTE events when an animal isat the decision-point of a T-maze task where the reward is placed in one arm ofthe maze. Moreover, as observed in Tolman’s studies [8], VTE events increaseafter changing of task contingencies, that is, varying the starting position or thelocation of the reward. In order to assess that our implementation of hippocam-pal and striatal networks in combination with mechanisms of “mental travel”

Grid-Cells “Mental Travel” and VTE Behavior 259

Fig. 3. Rate maps of grid-cells obtained from the agent performing an alternate taskwithin a t-maze configuration. (Color figure online)

would reproduce similar effects in a simulated navigational task, we have testeda virtual agent in situations where task contingency would be manipulated. Atotal of 14 sessions were performed, VTE events quantified and averaged acrosssessions. Because our agent began navigation with randomly assigned weightsbetween place- and reward-cells, we would expect that VTE events would behigh at the initial stage of exploration. Indeed, that was confirmed at the veryfirst trial where ‘mental traveling’ was performed towards both possible deci-sion orientations and no activation of reward-cells above 0.7 of its maximumrate (1.0) was obtained (Fig. 5). However, because the learning rate (γ) of ourlearning mechanism between place- and reward-cells was set to 0.75, the simu-lated agent was able to perform a decision toward the correct arm at the thirdtrial. Again, studies on task contingency changes have shown that after alteringthe reward location to the opposite site of the maze makes the animal engagingVTE behavior, a period where rodents look back and forth towards possibleroutes to take. We have tested increases in look-ahead events after removing thereward from the learned location and place it at the opposite arm of the maze.As expected, the amount of VTE events increased after reconfiguration of thetask, an effect from dissociating the population vector activity of place-cells andthe absence of reward-related activity (Fig. 5).

260 D. Santos-Pata et al.

Algorithm 1. VTE from “mental travel”1: while eventnumber < maximumChoices do � Try random orientation2: eventorientation ← RandomLeft||Right

3: gridcellsinput ← speedvirtual + headorientation4: for i 1: maximumSimulationCycles do5: gridcellsactivity ← equation1, 26: placecellsactivity ← equation3, 47: rewardcellsactivity ← equation58: if rewardcell > 0.75 then9: return eventorientation

10: end if11: end for12: end while

Fig. 4. Correlation of place-cells population vector along both left and right arm ofthe maze at decision point. (Color figure online)

Fig. 5. Modulation of VTE events during learning (early trials) and after rewardreplacement (trial 9). (Color figure online)

Grid-Cells “Mental Travel” and VTE Behavior 261

4 Discussion

Situating oneself in space and being able to reach target locations is a majorsurvival feature that every organism must accomplish. The rodent hippocampusis an example of a great deal of resources to be deployed on real world foragingrobots. Electrophysiological studies have revealed a myriad of cell type to beencoding spatial properties. Head-direction cells in both thalamic anterodorsaland hippocampal postsubicular and early medial entorhinal cortex layers, pro-vide the proper hippocampus with a sense of global orientation [18]. From acomputational perspective, orientation signals help to maintain a relationshipbetween encoded locations and build a trajectory scheme to reach target loca-tions [3]. Similarly, grid-cells found in the medial entorhinal cortex allow theanimal to cover explored environments with tessellating patterns and obtain ametric system with global coordinates facilitating the encoding of specific loca-tions [1]. Finally, place-cells are the ultimate positional encoding mechanism,allowing the animal not only to situate itself in space, but also, to decode futuretrajectories to reach a target [5,17]. The implementation of grid- and place-cellsin navigating machines has been accomplished. However, there is more that thehippocampal region can offer to robotics. In this study, we have brought recentlydeveloped theories of the grid-cells functionality into a simulated robotic plat-form. Specifically, we have presented a mechanism in which by ‘mentally trav-eling’ towards future trajectories with minimal computational effort and zeroenergetic cost, a simulated agent is able to: (1) predict future locations throughspecific trajectories; (2) constantly monitor the value of a chosen route and;(3) ‘mentally’ explore never taken paths. We have shown that a simulated agentcould decode what was about to encounter by mentally probing its options on aT-maze configuration scenario. Such mechanism might be of great advantage forplanning routes in highly dynamic environments, where unknown routes must beconsidered. In this study we have combined a previously described implementa-tion of the hippocampal formation for space representation [2] and path-planning[3]. Here, we have extended our previous work by integrating a neural populationmimicking the striatal rewarding circuitry motivating the animal towards spe-cific spatial locations. Our results suggest a mechanism where neural activity inpopulations encoding spatial locations directly modulate animal decision-makingprocess and consequent VTE behavior. Our implementation of grid-cells “men-tal travel” resembles the activation of theta-sequences in hippocampal formationfound in [5]. Further investigation would reveal neural dynamics in depictingfuture trajectories during quiescent and ongoing navigation.

Acknowledgments. The research leading to these results has received funding fromthe European Research Council under the European Union’s Seventh Framework Pro-gramme (FP7/2007-2013)/ERC grant agreement 341196; CDAC.

262 D. Santos-Pata et al.

References

1. Hafting, T., et al.: Microstructure of a spatial map in the entorhinal cortex. Nature436(7052), 801–806 (2005)

2. Pata, D.S., Escuredo, A., Lallee, S., Verschure, P.F.M.J.: Hippocampal based modelreveals the distinct roles of dentate gyrus and CA3 during robotic spatial naviga-tion. In: Duff, A., Lepora, N.F., Mura, A., Prescott, T.J., Verschure, P.F.M.J.(eds.) Living Machines 2014. LNCS, vol. 8608, pp. 273–283. Springer, Heidelberg(2014)

3. Maffei, G., Santos-Pata, D., Marcos, E., Sanchez-Fibla, M., Verschure, P.F.: Anembodied biologically constrained model of foraging: from classical and operantconditioning to adaptive real-world behavior in DAC-X. Neural Netw. 72, 88–108(2015)

4. Domnisoru, C., Kinkhabwala, A.A., Tank, D.W.: Membrane potential dynamics ofgrid cells. Nature 495(7440), 199–204 (2013)

5. Johnson, A., David Redish, A.: Neural ensembles in CA3 transiently encode pathsforward of the animal at a decision point. J. Neurosci. 27(45), 12176–12189 (2007)

6. Guanella, A., Kiper, D., Verschure, P.: A model of grid cells based on a twistedtorus topology. Int. J. Neural Syst. 17(04), 231–240 (2007)

7. Brun, V.H., et al.: Progressive increase in grid scale from dorsal to ventral medialentorhinal cortex. Hippocampus 18(12), 1200–1212 (2008)

8. Tolman, E.C.: Cognitive maps in rats and men. Psychol. Rev. 55(4), 189 (1948)9. Redish, A.D.: Vicarious trial and error. Nat. Rev. Neurosci. 17(3), 147–159 (2016)

10. Sanders, H., et al.: Grid cells and place cells: an integrated view of their navigationaland memory function. Trends Neurosci. 38(12), 763–775 (2015)

11. Bush, D., et al.: Using grid cells for navigation. Neuron 87(3), 507–520 (2015)12. van der Meer, M.A., et al.: Triple dissociation of information processing in dorsal

striatum, ventral striatum, and hippocampus on a learned spatial decision task.Neuron 67(1), 25–32 (2010)

13. Hebb, D.O.: The Organization of Behavior: A Neuropsychological Theory.Psychology Press, New York (2005)

14. de Almeida, L., Idiart, M., Lisman, J.E.: The input output transformation of thehippocampal granule cells: from grid cells to place fields. J. Neurosci. 29(23), 7504–7512 (2009)

15. O’Keefe, J., Dostrovsky, J.: The hippocampus as a spatial map. Preliminary evi-dence from unit activity in the freely-moving rat. Brain Res. 34(1), 171–175 (1971)

16. Yoon, K.J., et al.: Specific evidence of low-dimensional continuous attractor dynam-ics in grid cells. Nat. Neurosci. 16(8), 1077–1084 (2013)

17. Pfeiffer, B.E., Foster, D.J.: Hippocampal place-cell sequences depict future pathsto remembered goals. Nature 497(7447), 74–79 (2013)

18. Taube, J.S., Muller, R.U., Ranck, J.B.: Head-direction cells recorded from thepostsubiculum in freely moving rats. I. Description and quantitative analysis. J.Neurosci. 10(2), 420–435 (1990)

Insect-Inspired Visual Navigation for Flying Robots

Andrew Philippides1(✉), Nathan Steadman1, Alex Dewar2, Christopher Walker1,and Paul Graham2

1 Department of Informatics, Centre for Computational Neuroscience and Robotics,University of Sussex, Brighton, UK

{andrewop,N.Steadman,Chris.Walker}@sussex.ac.uk2 School of Life Sciences, Centre for Computational Neuroscience and Robotics,

University of Sussex, Brighton, UK{A.Dewar,paulgr}@sussex.ac.uk

Abstract. This paper discusses the implementation of insect-inspired visualnavigation strategies in flying robots, in particular focusing on the impact ofchanging height. We start by assessing the information available at differentheights for visual homing in natural environments, comparing results from anopen environment against one where trees and bushes are closer to the camera.We then test a route following algorithm using a gantry robot and show that arobot would be able to successfully navigate a route at a variety of heights usingimages saved at a different height.

Keywords: Visual navigation · Insect-inspired robotics · Visual homing · UAVs ·Flying robots

1 Introduction

Navigation is a vital ability for animals and robots. In the latter, where GPS is unavailableor unreliable, visual homing methods can be used. For flying robots, the limitations onpayload mean that it is important that any algorithms are efficient in terms of computationas this reduces power consumption. It is therefore natural that engineers turn for inspi‐ration to insects such as ants and bees [1], who use vision to navigate long distancesthrough complex natural habitats despite limited neural and sensory resources [2–5]. Inthis spirit, we have previously developed a view-based homing algorithm based on thebehavior of desert ants. While this algorithm has been tested for ground-based robots insimulation, here we test the ability of this algorithm to generalize to a flying robot.

The first generation of biomimetic algorithms for insect-like navigation wereinspired by the fact that an insect’s use of vision for navigation is often a retinotopicmatching process (Ants: [2, 4, 5]; Bees: [3]; Hoverflies: [6]; Waterstriders: [7]; Review:[8]) where remembered views are compared with the currently experienced visual scenein order to set a direction or drive the search for a goal. Insect-inspired robotic modelsof visual navigation have thus been dominated by snapshot-type models where a singleview of the world, as memorized from the goal location, is compared to the current viewin order to drive a search for the goal ([3]; for review see [9]). Importantly, Zeil andcolleagues [10, 11] used simple metrics based on the sum-square differences in pixel

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 263–274, 2016.DOI: 10.1007/978-3-319-42417-0_24

values between images, and thus agnostic of the details of the model used, to analysethe range over which a single image can be used for visual homing.

This work and others showed that, while snapshot models work in a variety of envi‐ronments, these approaches are limited in that they generally allow for navigation onlyin the immediate vicinity of the goal [10–13]. We therefore developed a familiarity-based model of route navigation in which individual views are used as a visual compassto recall the direction the agent was facing at that point, rather than the direction to thelocation as in most snapshot models. This allowed us to develop a holistic route memorywhich allowed a ground-based agent to navigate through simulated natural environmentswith route showing many characteristics of ant routes [14]. Following on from this work,similar models have been shown to work with a biologically plausible neural networkin simulation [15] and on a robot in a natural environment [16].

In all these previous works, however, all images were taken from the ground level.To assess the whether this model can also be used for a flying robot, we thus need totest the model with images gathered from different heights. To do this we follow themethods of [10, 11, 17] to first assess the extent over which single images gathered fromdifferent heights through two natural environments can provide information for visualhoming. We show that in line with Zeil [10], a snapshot stored at one height cansuccessfully be used as either an attractor snapshot or visual compass for an agent trav‐elling at a different height. We then test our route navigation algorithm with similar datagathered using a high precision gantry robot in an indoor environment. The success ofthe algorithm provides proof of principle that an aerial robot could use our route navi‐gation algorithm despite being at a different height to that at which the original routewas travelled.

2 Measuring the Informational Content of Natural Images fromDifferent Heights

To get an understanding of the information that is present in images for homing, wefollow the procedures of [10, 11, 17] and use simple metrics to estimate the range overwhich a single image can be used either as an attractor type snapshot, to recover adirection towards a goal, or as a visual compass, to recall the heading at which the agentwas facing when the goal image was stored.

2.1 Data Collection and Image Processing

The process starts by capturing sets of images using a panoramic imaging device withintwo natural environments at different heights (Fig. 1). Data was collected using a KodakPixPro SP360 camera to take panoramic images at regular intervals at a range of heightsfrom straight-line routes in two locations. The heights investigated were 0 cm, 40 cm,70 cm, 100 cm, 150 cm and 200 cm. With the exception of 0 cm, when the camera wasplaced directly upon the ground, images were taken from a tripod with an in-built spiritgauge, allowing each image to be taken at a level setting. In addition, it allowed us toexamine the impact of pitch by including an image taken with a 45° downward tilt at

264 A. Philippides et al.

each of the heights and locations investigated. The tripod was aligned to a commonheading to ensure all images from each location were taken facing in the same direction.

Fig. 1. Sample images from two locations. A: Location A – open transect through Queen’s Park,Brighton. B: Location B – transect through wooded copse in East Brighton Park. Imagesunwrapped from fish-eye lens via PixPro software.

Location A: The initial dataset was collected from a field in Queen’s Park, Brighton.These transects covered a distance of 36 m taken in 2 m increments through an openarea. This location was selected as it is an open environment without any nearby treesor foliage and in which the trees which constitute the skyline were at least 50 m fromthe camera location with the majority being several hundred metres away (Fig. 1A). Thismeans the visual data changed slowly relative to movement along the route.

Location B: The second dataset was collected from a small copse in East BrightonPark, Brighton. Transects of 12 m were covered in 1 m intervals. The treeline was at nopoint further than 5 m from the camera in any of the images taken. The decreased intervalbetween images was used to compensate for the small distance of the route. Due to theforested nature of this site (Fig. 1B), the route passed noticeably between shade and lightwith distinct lens flare on certain images taken. The images were not manipulated tocompensate for this.

The images were processed in three ways: they were first unwrapped from a fish-eyeimage to a panoramic image using the software provided with the PixPro camera. Imageswere then scaled-down to 408 × 1632 pixels, converted to grey-scale and the sky homo‐genized to a uniform value of 255 (white). The first two stages were achieved withMatlab functions imresize and rgb2gray, respectively. Sky homogenization was alsoperformed in Matlab automatically and quite roughly by thresholding the blue-channelat a value of 170 (determined by trial and error) and setting above 170 values to 255.Occasionally, parts of clouds were below the 170 threshold, and so we also convertingany isolated ‘not-sky’ pixels which were not connected to the ground to 255. Skyhomogenization is necessary as the light gradient can provide a very strong, but spuriouscue, by which homing algorithms can gain information.

Insect-Inspired Visual Navigation for Flying Robots 265

2.2 Assessing the Region in Which an Image Can Be Used for Visual Homing

By measuring the image difference between a reference image and images fromsurrounding points, we can build an image difference function (IDF) that shows howimages change with distance from a goal view [10]. The image difference between twoimages X and Y is defined as:

(1)

where X(i,j) is the pixel in the i’th row and j’th column of image X and P is the numberof pixels. Notice that this value is dependent on the alignment of image X and Y to acommon heading. The IDF for an image from Location A is shown in Fig. 2.

Fig. 2. An example IDF from Location A at a height of 100 cm. IDF values are shown as aproportion of the maximum and plotted against distance from the goal. x’s mark points whereimages were taken from. Points within the catchment area are marked with red squares. Noise inthe IDF limits the catchment area to the left of the goal while to the right it extends 6 m.

To assess the range over which a single image can be used for navigation, we quantifythe region around a goal image in which the difference between images increases withincreasing distance from the reference image. The presence of an increasing IDF issignificant as it shows that the information needed for view-based homing is available[10, 17]. We therefore define the catchment area (CA) of the IDF as the region wherethe IDF is generally increasing. Note that as we are applying this to a straight line transectthis is not strictly an ‘area’ but an indication of the distance over which a single imagecan be used. Nevertheless we use the term ‘catchment area’ to be consistent with thegenerally used terminology. To estimate this region, we take the number of consecutivelocations spreading out from the goal position where the IDF is strictly increasing rela‐tive to the direction of movement from the goal (Fig. 2). This is a lower bound on theregion within which an image could be used for homing as a single uneven/bad imagecan introduce a spurious minimum in this discrete data (e.g. point at distance -2 inFig. 2). While this could be overcome by taking more images and smoothing the datain some way, as we are using the same metric to compare two data sets, we use the raw

266 A. Philippides et al.

data as a lower bound. Likewise, the fact that the distribution of catchment areas isasymmetric (goal positions at the centre have CAs that can extend in both directionsunlike those at the edge) is not so important as we are comparing the same positions fordifferent heights.

Figure 3A-B shows that images gathered at different heights do carry sufficientinformation for visual homing. While there is quite a spread of results at each heightfrom different goal positions, a good proportion of the route can be traversed using asingle image. As expected, a greater distance could be traversed with a single image inLocation A than Location B. Interestingly, in the more cluttered Location B, theperformance increased with increasing height. The cause of this can be seen in Fig. 4.When there are nearby objects, a limited field of view in the elevation access means thatclose objects dominate the skyline more for lower heights, leading to inaccuracy in thematching process. We also tested routes taken with a level camera against goal imageswhere the camera was angled at 45o to the ground plane (Fig. 3 C-D). The idea of thiswas to see whether the effect of pitch changed with changing height. For this data set,the impact of pitch was detrimental to the route coverage as has been seen in [18] anddid not vary with height, but at least in Location A, there remains some information forhoming.

Fig. 3. Catchment areas in locations A (A and C) and B (B and D). X-labels identify the heightof the route. In the top row (A and B), data is shown for goal images taken from the route tested.In contrast, height labels followed by a ‘T’ (bottom row, C-D) indicate that the goal images onlywere captured at a 45° tilt. Red lines indicate the median catchment area, whole boxes cover the25th and 75th percentile and whiskers extend to 1.5 times the inter-quartile range. (Color figureonline)

We next tested whether images that were taken at a particular goal height, could besuccessfully used to home for a robot travelling at a different height (Fig. 5). Results

Insect-Inspired Visual Navigation for Flying Robots 267

were mixed but showed that: firstly, in principle this is possible; secondly, and unsur‐prisingly, that the closer in height the goal and route, the better the match; And finally,and more interestingly, that there was a slight trend of better performance for goals androutes at higher heights. The reasons for the benefits of height are likely to do with thelessening amount of the ground plane in the image as well as a cleaner skyline, thoughthis needs further investigation.

Fig. 5. Catchment areas where the goal (on the x-axis) is at a different height from the route (y-axis). Colours show the mean catchment area across all goal positions in Location A (A) and B(B). (Color figure online)

2.3 Assessing the Region Over Which Views Can Be Used to Recall a Heading

While the analysis of the catchment area of natural images demonstrates that the infor‐mation required for visual navigation is present, to more directly link it to our familiaritybased algorithm we need to repeat the analysis using views as a visual compass. Wetherefore next determine what we termed rotational catchment areas (RCAs) [17], which

Fig. 4. The impact of increased height at Location B. A - Location B taken at a height of 40 cm.B - Location B at a height of 150 cm. The increased elevation in B results in a more distinct outlineof the treeline to the left of the image. This in turn leads to better IDF comparisons as there is nowmore visual information.

268 A. Philippides et al.

indicate the region in which an agent would be able to use a goal image to recall theheading it was facing when the goal images was stored.

To determine the RCA, one first calculates the rotational IDF (RIDF) by evaluatingthe IDF between a reference image and the current image rotated (in silica) in steps of1° of azimuth resulting in a 1 × 360 RIDF (Fig. 6). The minimum value in the RIDFdefines an orientation of the current image which gives the closest match with the refer‐ence image. In the vicinity of the reference image these orientations will be similar tothe reference image orientation [10] and so can be used to recall a heading or a movementdirection for a forward facing agent [17]. We define the rotational catchment area as theregion spreading out from the location of the reference image where the minimum inthe RIDF is less than 45° from the true orientation of the reference image (Fig. 6).

Fig. 6. Sample RIDF using an image from Location A, 100 cm height, as both goal and currentimage. IDF is normalised to the maximum value and plotted against the degree of azimuthalrotation. Dashed lines mark the 45° threshold region. In this case the best match is at 0o meaningthe agent is facing in the same direction as when the image was stored.

Analysis of RCAs for routes and goals at the same height shows that there are strongvisual compass cues across the routes (Fig. 7 A-B) with much increased catchment areascompared to the IDF analysis. In Location A, performance increases with height of theroute. In Location B the highest routes seem to slightly underperform but this is likelya ceiling effect (the maximum possible RCS is 11 m for Location B. Comparing goalsat different heights to routes again there are large catchment areas for the visual compasswith much more matching across different heights than was evident in the IDF data. InLocation A, there seem to be two broad blocks of matching with all routes from 100 cmdown broadly matching while the two highest routes match each other. The picture forLocation B is similar but with a more broad matching across heights. While the rathercrude measure of mean RCA and the ceiling effect for Location B means more investi‐gation is needed, these results bode well for our route following algorithm to functionat different heights as it is based on the visual compass method.

Insect-Inspired Visual Navigation for Flying Robots 269

Fig. 7. Rotational catchment areas as a proportion of route length in locations A (left: A and C)and B (right: B and D). X-labels identify the height of the route. A and B show the distributionof rotational catchment areas for routes and goal images at the same height. Red lines indicate themedians, boxes cover the 25th and 75th percentile and whiskers extend to 1.5 times the inter-quartile range. C and D show rotational catchment areas where the goal (on the x-axis) is at adifferent height from the route (y-axis). Colours show the mean catchment area across all goalpositions in Location A (C) and B (D). (Color figure online)

3 Route Navigation

We next test route navigation for images gathered at different heights using the famili‐arity-based algorithms described in [14]. However, as we need to test the algorithm overa region, we need to use a more controlled robotic platform and move to an indoor gantryrobot.

The first version of the algorithm, dubbed ‘Perfect Memory’ as it stores all trainingviews, proceeds as follows. The agent first traverses a (here pre-defined) route storinggreyscale, panoramic images at set distances (represented by red crosses in Fig. 8).Crucially, the stored views are oriented in the direction in which the agent was “facing”at that point. In order to recapitulate the route, the agent then obtains a best-guess of thecorrect heading by comparing the current and stored views across rotations, with therotation yielding the smallest image difference (from among all stored views) taken asthe agent’s goal bearing. The second method involves the use of an InfoMax familiarity-based neural network [14] and is referred to as Infomax. In this variant, instead ofremembering all the views and comparing the current and stored views on an individualbasis, a single layer neural network is trained with the stored views to learn the familiarityof the training views using an Infomax training rule [19]. The network is then presented

270 A. Philippides et al.

with the current view at a range of rotations and the orientation which yields the smallestactivation (and thus the greatest familiarity) is taken as the best-guess heading.

Fig. 8. Estimated headings for different homing algorithms at a range of heights using the robotgantry. A – Best-guess headings using the Perfect Memory algorithm. Reliable headings areobtained over a range of locations and the algorithm is robust to changes in height. Blue arrowsindicate the headings, red crosses the locations for different stored views and green lines thepositions of the cardboard boxes. The scale, as described in the text, is in reality a tenth of thatshown in the figure. B – Best-guess headings using the InfoMax algorithm. Performance isvirtually equivalent to that obtained with the Perfect Memory algorithm, despite requiring farfewer computational resources. Colour scheme is as for A. (Color figure online)

Insect-Inspired Visual Navigation for Flying Robots 271

To test these two algorithms, we used an indoor gantry robot, which is comprised ofa panoramic camera which outputs a 720 × 60 pixel image covering 360o of azimuthand 50o of elevation (extending roughly equally above and below the camera’s‘horizon’). The camera is mounted onto a robot arm capable of moving along x, y andz dimensions to any arbitrary point within our 2.7 × 1.8 × 1.2 m arena. As this was rathersmall for our purposes, we treated the arena as though it (and the objects within it) were10 times as large and scaled the agent’s movements appropriately. Henceforth distancesin this paper refer to the new artificial scale. For this test we placed a number of cardboardboxes of different heights (min = 0.98 m; max = 6.04 m) within the arena to providevisual stimuli. Additionally, the walls of the arena were removed so more distal visualcues were available in the form of the visual panorama of the office in which the gantryis housed. First we acquired the stored views for an arbitrary route within the arena at aseparation of 25 cm (n = 60; indicated by red crosses in Fig. 8). We then calculated thebest-guess headings for the Perfect Memory and InfoMax algorithms (Fig. 8A and B,respectively) for images collected from the gantry at a range of x, y and z coordinates.

Figure 8 shows that robust performance is given by both the Perfect Memory andInfoMax flavours of the visual homing algorithm. Although performance is understand‐ably poorer for locations where the original training route is obscured by boxes, none‐theless, the algorithms mostly yield headings parallel to the training route, indicatingthat there is sufficient distal visual information (in the office) to drive reliable homing.Moreover, performance does not decay substantially with changes in height, whichsuggests that even quite dramatic amounts of visual noise in a real-world environmentwould be unlikely to lead the agent astray. The quality of performance is particularlyremarkable for the InfoMax case, where the only data required in memory is a 644 × 644matrix of weights, as opposed to the entire cache of stored views (58 × 720 × 60 pixelsin total). Accordingly, the time taken to compute a heading is also considerably fasterfor InfoMax.

4 Conclusions

Here we have shown that parsimonious ant-inspired homing strategies are suitable foraerial robots. Training data collected at one height can be used to robustly recall aheading direction from a range of different heights, indicating that variations in flightheight, especially when not too close to the ground, will be tolerated by the algorithm.However, more work needs to be undertaken to prove this performance in closed loopsystems, and to test the route algorithm outdoors.

The robustness of performance of visual compass based methods across the rangeof perspectives also gives hope that route memories could be transferred between robots.The use of a holistic route memory, which is easy to transfer between robots, will aidthis endeavor and is the subject of on-going work. In particular, we are interested to seeif for instance a UAV could be used to follow a path specified by a ground-based robot,or vice versa. This has implications in, for instance, search and rescue operations wheredifferent robots could be used for exploration and retrieval.

272 A. Philippides et al.

Finally, it would be interesting to observe how the flight height of bees vary duringforaging. However, currently positional data on bees is generally taken from radar whereinformation on height is unavailable, though efforts are on-going to improve these radarsystems which could prove very insightful.

Acknowledgements. Thanks to the anonymous reviewers for helpful suggestions. This workwas funded by the Newton Agri-Tech Program RICE PADDY project (no. STDA00732). AP alsoreceived funding from the European Union’s Seventh Framework Programme for research,technological development and demonstration under grant agreement no. 308943.

References

1. Graham, P., Philippides, A.: Insect-inspired vision and visually guided behavior. In: Bhushan,B., Winbigler, H.D. (eds.) Encyclopedia of Nanotechnology, pp. 1122–1127. Springer,Netherlands (2015)

2. Wehner, R., Räber, F.: Visual spatial memory in desert ants. Cataglyphis Bicolor. Experientia35, 1569–1571 (1979)

3. Cartwright, B.A., Collett, T.S.: Landmark learning in bees - experiments and models. J. Comp.Physiol. 151, 521–543 (1979)

4. Wehner, R.: Desert ant navigation: how miniature brains solve complex tasks. J. Comp.Physiol. A. 189, 579–588 (2003). Karl von Frisch lecture

5. Wehner, R.: The architecture of the desert ant’s navigational toolkit (Hymenoptera:Formicidae). Myrmecol News 12, 85–96 (2009)

6. Collett, T.S., Land, M.F.: Visual spatial memory in a hoverfly. J. Comp. Physiol. A. 100,59–84 (1975)

7. Junger, W.: Waterstriders (Gerris-Paludum F) compensate for drift with a discontinuouslyworking visual position servo. J. Comp. Physiol. A. 169, 633–639 (1991)

8. Collett, T.S., Graham, P., Harris, R.A., Hempel-De-Ibarra, N.: Navigational memories in antsand bees: memory retrieval when selecting and following routes. Adv. Study Behav. 36,123–172 (2006)

9. Möller, R., Vardy, A.: Local visual homing by matched-filter descent in image distances.Biol. Cybern. 95, 413–430 (2006)

10. Zeil, J., Hofmann, M., Chahl, J.: Catchment areas of panoramic snapshots in outdoor scenes.J. Opt. Soc. Am. A: 20, 450–469 (2003)

11. Stürzl, W., Zeil, J.: Depth, contrast and view-based homing in outdoor scenes. Biol. Cybern.96, 519–531 (2007)

12. Smith, L., Philippides, A., Graham, P., Baddeley, B., Husbands, P.: Linked local navigationfor visual route guidance. Adapt. Behav. 15, 257–271 (2007)

13. Smith, L., Philippides, A., Graham, P., Husbands, P.: Linked local visual navigation androbustness to motor noise and route displacement. In: Asada, M., Hallam, J.C.T., Meyer,J.-A., Tani, J. (eds.) SAB 2008. LNCS (LNAI), vol. 5040, pp. 179–188. Springer, Heidelberg(2008)

14. Baddeley, B., Graham, P., Husbands, P., Philippides, A.: A model of ant route navigationdriven by scene familiarity. PLoS Comput. Biol. 8(1), e1002336 (2012)

15. Ardin, P., Peng, F., Mangan, M., Lagogiannis, K., Webb, B.: Using an insect mushroom bodycircuit to encode route memory in complex natural environments. PLoS Comput. Biol. 12(2),e1004683 (2016)

Insect-Inspired Visual Navigation for Flying Robots 273

16. Kodzhabashev, A., Mangan, M.: Route following without scanning. In: Wilson, S.P.,Verschure, P.F.M.J., Mura, A., Prescott, T.J. (eds.) Living Machines 2015. LNCS, vol. 9222,pp. 199–210. Springer, Heidelberg (2015)

17. Philippides, A., Baddeley, B., Cheng, K., Graham, P.: How might ants use panoramic viewsfor route navigation? J. Exp. Biol. 214, 445–451 (2011)

18. Ardin, P., Mangan, M., Wystrach, A., Webb, B.: How variation in head pitch could affectimage matching algorithms for ant navigation. J. Comp. Physiol. A. 201(6), 585–597 (2015)

19. Lulham, A., Bogacz, R., Vogt, S., Brown, M.W.: An infomax algorithm can perform bothfamiliarity discrimination and feature extraction in a single network. Neural Comput. 23,909–926 (2011)

274 A. Philippides et al.

Perceptive Invariance and Associative MemoryBetween Perception and Semantic

Representation USER a Universal SEmanticRepresentation Implemented in a System

on Chip (SoC)

Patrick Pirim(B)

Brain Vision Systems, Paris, Francepatrick.pirim@bvs-tech.com

http://www.bvs-tech.com

Abstract. USER (Universal SEmantic Representation) is a bio-inspiredmodule implemented in a system on a chip (SoC), which builds a linkbetween multichannel perception and semantic representation. The inputdata are projected into a generic bioinspired higher dimensional non-linear semantic space with high sparsity. A pooling of these semantic rep-resentations (global, dynamic and structural) is done automatically by aset of dynamic attractors embedding spatio-temporal histograms, beingdrastically more efficient than back-propagation. A supervised learn-ing is used to build the association between the invariant multimodalsemantic representations (histogram results) and the labels (‘words’).The invariant recognition is achieve thanks to multichannel multiscaledynamic attractors and bilinear representations - imitating brain atten-tional processes. USER modules can be cascaded, allowing to work atdifferent levels of abstraction (or complexity). Due to its low consump-tion, small size and minimal price, USER targets deep learning, robotics,and Internet of Things (IoT) applications.

Keywords: USER · Invariant representation · Dynamic attractor ·Pattern recognition · Semantic representation · Associative memory ·Multi-sensory · Deep learning · Sparsity · Embedded system · SoC · IoT

Introduction. Semantic representations allowed by the invariance of perceptivesituations are of tremendous importance for humans. Here we propose a genericframework ascribing a semantic representation to a combination of pluralities ofsenses, with the following properties: 1. Filtering of the input sensor cues towardsthree feature maps: Global map (Gm), Dynamic map (Dm) and Structural map(Sm), 2. Extraction of semantic elementary representations through dynamicattractors (DA) on each of the feature maps, 3. Dynamical control of the sensorrange in order to obtain perceptive invariance during the first processing, per-ceptive invariance despite the environment’s variability being a prerequisite toc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 275–287, 2016.DOI: 10.1007/978-3-319-42417-0 25

276 P. Pirim

build a universal single sensory semantic representation, 4. Association of ele-mentary semantic representations into a unique global representation (Learningmode). 5. Ability to associate a ‘word’ to a given perception (Request mode) andability to retrieve the word associated to a given situation (Get mode), and 6.Silicon implementation allowing scalability and low power consumption.

1 Electronic Model of Neural Population Activities

Figure 1 shows (from top to bottom):

Biologically-Inspired Implementation. The visual cortex exhibits threeremarkable properties: blobs perception, local motion, and line orientation [5].These three generic functions exist all over the entire cortex [1]. Each of these fea-tures is 2D mapped (cortical column organization), with data in w bits as input,into 2w bits Global map (Gm), Dynamic map (Dm) and Structural map (Sm).In order to account for the topographical organization of the cortical columns,a Position map (Pm) is added.

Features Extractor. The main difference between biological and electronicimplementations lays between the parallel (biology) or serial processing (hard-ware) However, the time to process the information are equivalent since the fastelectronic processing compensates its lack of parallelism. The feature extractorsare hard-wired embedded as a set of four filters, i.e. the four maps (with outputsin 2w bits):

– Global map (Gm): the filter capitalizes the general instantaneous characteris-tics of sensory data.

– Dynamic map (Dm): the filter capitalizes their time-change characteristics.– Structural map (Sm): the filter capitalizes their geometrical characteristics.– Position map (Pm) is the position of the current data, at each elementary step

time, within the actual temporal sample.

By contrast with biological processing that is event-driven (spikes), the elec-tronic processing is time-driven with temporal sequence of frames involvingthe sub-sequences of spatial informations. It is a global process implementinga temporo-spatial transform [11].

Dynamic Attractor. The 2w bits data from the four maps are processed in abihistogram module. Simultaneously, two classifier modules compare these data,each with two bounds. In each of them, a binary output is validated wheneverthe data are inside these bounds. The bounds of the first classifier module areupdated internally at each end of cycle. Those of the second one are updatedexternally. The later module has the function of an elementary request (R). Alogical AND (“spike”) on all eight binary outputs is sent to the four bihistogrammodules, as validation signals for their computations. This loop is essential forautomatic convergence on the optimal information. In vision for example, thetwo attributes (for bihistogram computation) are hue and saturation for Gm,

Perceptive Invariance and Associative Memory 277

Fig. 1. USER presentation. Top: Transcription from biological (cerebral cortex)function to electronic implementation. Middle: Scheme of the dynamic attractor (DA).Bottom: Concept’s global view embedded in SoC.

278 P. Pirim

direction and velocity of motion for Dm, oriented gradient and curvature forSm, and x and y for Pm. This module transforms the feature extractor’s results,by use of the bihistograms’ medians, into a spatio-temporal information (elemen-tary ‘what’ and ‘where’ [4,12]). The geometrical covariation between patterns isan efficient way for recognition, and for that bihistogram (2D histogram) com-putation facilitates linear classification on two different parameters.

The bihistogram is implemented using a memory of 22w counters (thereforethe bihistogram size). This value is updated after each information validation ofthe classification results set. A small number of registers are initialized at thebeginning of cycle and updated hereafter. At the end of a cycle, one of themcounts the number of validation information (Nbpts = global energy value).The maximal energy is represented by the three registers, the bihistogram’smaximal value and its 2D position. An additional group of registers drives theautomatic classification for the first classifier. At the end of the cycle, the mediansof bihistogram are outputted. Additionally, the counter is resetted, and the cyclestarts over again, either at the end of sequence in time-driven mode, or whenNbpts is over the threshold in event-driven mode. In order to compensate thedelay introduced by the use of previous cycle results, the difference between thecurrent median and the previous one is added to the classification bounds as ananticipation indicator.

At the beginning of each cycle, the DA is activated if the previous Nbpts isabove a threshold value (minimal energy), otherwise it is inhibited. When acti-vated, the classification registers are upgraded, and embed the DA parameters.The DA outputs, when activated, are the four groups of median registers. Theycode for the elementary semantic representations that have been founded (F).

As soon as a DA departs from an histogram maximum value (Rmax, a“winner-take-all”), another one starts (for the same process) on the complemen-tary of the previous active activation data, and so on. This process is repeateduntil a threshold value (Nbpts) is reached. This dynamic recruitment forms atree-organization, and is controlled by the “Tree Organization” bus (T).

USER: An Embedded Concept. Each layer of the USER architecture is madeof generic modules (Fig. 6c). The sensory data are sent to a first feature extractormodule which outputs four data, representative of the feature decomposition andits position. These information are sent to several DA modules, which processseveral elementary semantic representations data. These elementary semanticrepresentations are the inputs of an associative memory module that receivesor sends out, depending on the context, a word that labels the global semanticrepresentation.

2 Associative Memory

Memory Types. Memories allow two different addressing modes, RAM forRandom Access Memory and CAM for Content-Addressable Memory [7]. Gen-erally only one function is integrated, such as RAM in conventional PC or CAM

Perceptive Invariance and Associative Memory 279

for Internet use. In our case, the semantic elementary representations are asso-ciated with words, the addresses of those data being transparent. We need twomemory modules, each one with selectable CAM or RAM functions, for thesetwo kinds of data, a first one linked to word representation, and the secondone to elementary semantic representations. Both types of data are mapped byinternal addresses.

Figure 2 shows the three features of functioning with the required memorytypes:

Fig. 2. Associative memory modes. In Learning mode, the multi-modes percep-tion is associated with a labelling word, and memorized in the associative memory.In Request mode, from a given word, the associative memory delivers the elementarysemantic representations for searching the result in the multi-modes perception. In Getmode, from the multi-modes perception, the word is extracted.

Learning Mode in order to memorize the association between the elementarysemantic representation from the perception and a dedicated word (Fig. 2 left).The two data are stored in two memory modules addressed by a common address.The two memory modules are set in RAM.

Request Mode allows to find -for a given word- the corresponding pattern inthe perceptual environment (Fig. 2 middle). In this case, the first memory moduleis set in CAM with word as the content and delivers an address to the secondmemory module, set in RAM function. The output result is the correspondingelementary semantic representations.

Get Mode in order to find the dedicated word from the elementary semanticrepresentations of the perceptual environment (Fig. 2 right). A CAM is usedin the second memory module with semantic elementary representation as thecontent, and delivers an address to the first memory module set in RAM. Theoutput result is the corresponding word.

Electronic implementation of the associative memory with two non-volatilememories (either RAM or CAM). Its conception is a delicate process, difficult

280 P. Pirim

to implement, involving high energy consumption, as also high price. To accountfor all theses constraints, a Magneto-resistive Random Access Memory (MRAM,non-volatile) has been selected. The CAM function is then emulated with twomemory blocs, one (addressed by the data) stores a virtual address that is usedto address the second memory bloc in order to store this data. The completeassociative memory is obtained with four memory blocks, controlled by a smallmicroprocessor. The best compromise is obtained when data of W bits are storedin 2W addresses. With today’s market, W=16 or 32 is a good choice (availableon shelf), and allows an implementation on one chip of 4 Mb with a 64 K wordslearning limit. Since the three mode functions (Learning, Request and Get) areexecuted in the same clock time, the clock frequency can be low.

3 Perceptive Invariance

Vision. Fig. 3a shows a standard acuity curve [17]. Between the fovea axis and20◦ retinal eccentricity, the acuity is divided by a factor of 10; and thus the sizerepresented by a pixel is multiplied by 100. The perceptive invariance resultshere from the adaptation between two scales of sensory representations (Fig. 3b).From sensor data, a first low resolution mapping (MAP1) with DAs gives acontextual pattern representation, that is detected as a region of interest (ROI),and drives a second map with higher resolution in standardized size (MAP2). Itis necessary for MAP1 that its resolution is adjusted to obtain only a single DA.Then, the loop adjusts the resolution of MAP2 so that the two bihistogram’smedians (elementary semantic representations) are inside useful ranges.

Fig. 3. Perceptive invariance in vision. (a) Standard acuity curve [17]. (b) MAPsrepresentations. (c) MAPs example detailed in Fig. 4. (d) Electronic implementation.Note that the ratio between MAP1 and MAP2 definitions (20 Mp and VGA 0.3 Mp) issimilar to the ratio of visual acuities at fovea and 20◦.

Figure 3c illustrates this concept with the invariant representation of the word‘digit 2’. In MAP1 the DA perceives the element as a blob. Then this DA drives

Perceptive Invariance and Associative Memory 281

Fig. 4. Detailed functioning of the dynamic attractors. MAP1 is the contex-tual representation in environment. Top line: a single dynamic attractor (DA) deliversthe elementary semantic representation. MAP2 is the representation in standard size(perceptive invariance). Bottom left: associative memory results. (Color figure online)

the computation towards this ROI and other DAs detect parts of this moredetailed pattern in MAP2. Figure 3d shows how high-definition sensors (20 Mpor more) may be connected to a spatial adaptation function that carries twosignals, one low-resolution of the global view and the second with high-resolutioninside the ROI. The latter signal is driven by the DA of MAP1 and each signalis connected to a USER process.

Audition. In the mammalian cochlea, the basilar membrane’s (BM) mechan-ical responses are amplified, and frequency tuning is sharpened through activefeedback from the electromotile outer hair cells (OHCs). To be effective, OHCfeedback must be delivered to the correct region of the BM and introduced atthe appropriate time in each cycle of BM displacement [6]. Frequency analysis,

282 P. Pirim

Fig. 5. Audition recognition tests (Top) Upper top: Voice recording when vowelsare successively pronounced Middle: Corresponding sonagrams with the first three for-mants (F1, F2, F3) founded by the dynamic attractors. Schemes below : Words (a, e, i,o, u) labelling the vowels are obtained from these formants thanks to the associativememories (AM). Visual recognition tests (Bottom) Column 1 : Picture. Same sixdrawings as used in [3,14]. Columns 2 and 3 : Oriented Edges and Curvature maps.Column 4 : Dynamic attractors. The classification map is shown in blue and the posi-tions of the dynamic attractor in red. Column 5 : Bihistograms with oriented edges inabscissae and curvature in ordinate. Column 6 : Elementary semantic representationsfounded by the dynamic attractors. Last column: Words obtained by AM in Get mode.(Color figure online)

Perceptive Invariance and Associative Memory 283

compression of dynamic-range and amplification are three major signal process-ing functions of the cochlea [13]. After that, the auditory signal is converted intoa normalized sonagram. On a low time-scale, MAP1 selects a main blob (patch)with a single DA, that becomes the ROI for MAP2. Inside MAP2, the differentformants are found with additional DAs in order to obtain the sound’s semanticrepresentations.

4 Results from Dynamic Attractors

Visual Invariance: Figure 4 details the functioning initiated in Fig. 3c. MAP1is the context representation in environment. A single DA delivers the elementarysemantic representations: (h1, s1) for Gm, (0, 0) for Dm, (0, α1) for Sm, and (x1,y1, p1, q1) for Pm. Values translate into: “A red element at x1, y1, size p1, q1,with orientation α1, not moving”. MAP2 is the representation in standard size(perceptive invariance). Each DA is recruited by decreasing energy (rank ordercoding [18]). The first DA found in MAP2 detects a curvature (with its direction).Values then translate into: “A red curved and (bottom oriented) convex element,at x2, y2, with size p2, q2, not moving”. The second DA detects “A red horizontalline at bottom of MAP2” and the third one “A red oblique line in the centralpart of MAP2”. Altogether these three sentences uniquely characterize the word‘digit 2’. The bottom left part shows the associative memory function with, onone side these elementary semantic representations and, on the other side theequivalent word ‘digit 2’.

Visual and Audition Applications. Figure 5 (top) shows how the USERconcept deals with a simple auditory perception processed as a visual input. Fromthe sonagram, the perception of the three first formants represents a specificvowel. Figure 5 (bottom) shows the set of six drawings used in [3,14], rangingfrom exact circle to exact square. The result shows a clear-cut separation betweenthe semantic representations. The drawing A is clearly recognized as a circle, thedrawing B as a diamond, and other drawings, C to F, as squares.

5 Toward Multi-sensory Integration

A first multi-integration application has been performed on the Psikharpax rat[2] with a 220 nm chip technology names BIPS. Since, the new chip USER, in28 nm, is an ongoing process for first quarter 2017 delivery.

Figure 6 shows the integration of the different modules explained above:A. Detailed scheme between two different perceptions. B. Extension to multi-perceptions in parallel. C. Serial extension for hierarchical processing.

Each elementary semantic representation is the result of bihistogram com-putation with 22.w bits. A minimal configuration of a single dynamic attractortypically uses the three results of barycenter position (x, y), of structural char-acteristics (curvature and oriented gradient), and of the dynamic attractor’s size(p, q). Thus, the input data of memory is 23.2.w bits, and thus 230 bits for

284 P. Pirim

Fig. 6. Scalability in multi-sensory integration. In A, a first integrative levelshows that the same module can be used independently for two different sensory inputs,vision and audition, with only one simple link. This link consists in the common internaladdresses between the two associative memories. In B, the same process is generalizedto other sensory inputs in parallel. Three main generic properties are shared: 1. Thesame module is used for each sensory input. 2. Each time a feedback is sent back tothe sensor that allows the adaptation for invariant representation. 3. The link of thecommon internal addresses between associative memories described above is extendedto all sensory processes. In C, for each sensory input, the process is extended to ahierarchical serial processing along which the duration of temporal sequences increases.The grouping of USER modules with one associative memory forms one layer. Thelayers are serially connected bidirectionally [15]. In forward flow, the sensor data issent to the first layer. Its output ‘word’ is then sent as the input to a second layerwhose output ‘word’ will represent a set of previous words. In feedback flow, each layermay anticipate the previous layer processing. For example, in forward flow a first layermay extract phonemes, a second one syllables, and a third one spoken words. Whenthe beginning of a spoken word is heard, the feedback flow allows to anticipate the endthe spoken word.

Perceptive Invariance and Associative Memory 285

w=5 that is a range of only 32 values. An optimized memory size [16] of 230

is already a large size when embedded. With w=5 and a map of 0.3 Mp, theUSER module have inputs in 5 bits and includes 8 dynamic attractors, each ofthem integrating 4 bihistograms, each of 1024× 19 bits memory sizes. A 100 Hzsampling of the world is enough for human representation, leading to 100 MHzsilicon chip computation. With inputs in only 5 bits (32 levels), a USER moduleallows huge semantic structural elementary representations. Then an associativememory with a size of 256 K× 16 bits (standard package) is enough for a quitelarge brain-like memory.

6 Discussion

The USER concept is the result of the knowledge accumulation gained fromperceptive industrial applications using bio-inspired ASIC conceptions. An initialcombination of histogram chips allowed to make a DA [8] that became a generictool, initially used to detect color blobs (global perception). Then for videosensors, the dynamic perception was added [9]. Later the structural perceptionwas included in a new chip for Advanced Driver Assistance Systems (ADAS)[10]. These three features [5] and the DA [14] are supported by neurobiologicaldata, confirming the fact that the nature uses it on a daily basis.

The architecture generalizes real-time processes at different levels. First, theUSER module transforms incoming data into elementary semantic representa-tions. Secondly, a combination of USER modules and associative memories formsa layer with perceptive invariance ability (as soon as the first layer), followedby cascading processes mixing sensory data. Thirdly, the USER module con-tinuously performs forward and feedback flows. This latter feedback flow allowsthe previous layer to anticipate its perceptive process. As a consequence, thearchitecture allows fast single-trial learning.

The same module processes different perceptive functions at successive timesteps producing invariant representations. It is a compact and efficient solutionsince it allows the use of associative memories with reasonable size for real-timecomplex applications with low energy consumption (200 mW), small physicalsize (50 cm3) and cheap price with reasonable process (28 nm). IoT, ADAS, andDeepMind are clearly good targets for such applications. The semantic repre-sentation allows to build a complete application using processing elements thatmatch words in the human language, allowing an easy and efficient communica-tion between developers.

7 Conclusion

The concept of electronic USER modules is a new way of performing invariantperceptive processing and producing semantic representation with any type ofsensory input. We can now test and implement a low-cost and low-energy elec-tronic device that labels any multimodal sensory input in a set of words definingbasic useful categories, thanks to automatic extraction of invariant properties.

286 P. Pirim

The generic aspect of the USER’s layers will open new researches and applica-tions in robotics and in IoT, allowing to communicate through these universalsemantic representations. The USER devices being more aware of the worldaround them, will be more capable, more autonomous, safer and easier to use.

References

1. Bach-y-Rita, P.: Theoretical aspects of sensory substitution and ofneurotransmission-related reorganization in spinal cord injury. Spinal Cord37(7), 465–474 (1999)

2. Bernard, M., N’Guyen, S., Pirim, P., Guillot, A., Meyer, J.-A., Gas, B.: Asupramodal vibrissa tactile and auditory model for texture recognition. In: Don-cieux, S., Girard, B., Guillot, A., Hallam, J., Meyer, J.-A., Mouret, J.-B. (eds.)SAB 2010. LNCS, vol. 6226, pp. 188–198. Springer, Heidelberg (2010)

3. Colgin, L.L., Leutgeb, S., Jezek, K., Leutgeb, J.K., Moser, E.I., McNaughton, B.L.,Moser, M.B.: Attractor-map versus autoassociation based attractor dynamics inthe hippocampal network. J. Neurophysiol. 104, 35–50 (2010)

4. Goodale, M.A., Milner, A.D.: Separate visual pathways for perception and action.Trends Neurosci. 15(1), 20–25 (1992)

5. Hubel, D.H., Wiesel, T.N.: Receptive fields of single neurons in the cat’s striatecortex. J. Physiol. (London) 148, 574–591 (1959)

6. Nilsen, K.E., Russell, I.J.: The spatial and temporal representation of a tone on theguinea pig basilar membrane. Proc. Nat. Acad. Sci. 97(22), 11751–11758 (2000)

7. Pagiamtzis, K., Sheikholeslami, A.: Content-addressable memory (CAM) circuitsand architectures: a tutorial and survey. IEEE J. Solid-State Circuits 41(3), 712–727 (2006)

8. Pirim, P.: Method and device for real-time processing of a sequenced data flow, andapplication to the processing and digital video signal representing a video image,Patent FR2611063 (1987)

9. Pirim, P.: Method and device for real-time detection, localization and determina-tion of the speed and direction of movement of an area of relative movement in ascene, Patent FR2751772 (1996)

10. Pirim, P.: Automated method and device for perception associated with determi-nation and characterisation of borders and boundaries of an object of a space,contouring and applications, Patent WO2005010820 (2003)

11. Pirim, P.: Generic bio-inspired chip model-based on spatio-temporal histogramcomputation: application to car driving by gaze-like control. In: Lepora, N.F.,Mura, A., Krapp, H.G., Verschure, P.F.M.J., Prescott, T.J. (eds.) Living Machines2013. LNCS, vol. 8064, pp. 228–239. Springer, Heidelberg (2013)

12. Pirim, P.: Processeur de perception bio-inspire: une approche neuromorphique.Techn. Ingenieur. IN-220 (2015). (in French)

13. Rasetshwane, D.M., Gorga, M.P., Neely, S.T.: Signal-processing strategy forrestoration of cross-channel suppression in hearing-impaired listeners. IEEE Trans.Biomed. Eng. 61(1), 64–75 (2014)

14. Renno-Costa, C., Lisman, J.E., Verschure, P.F.: A signature of attractor dynamicsin the CA3 region of the hippocampus. PLoS Comput. Biol. 10(5), e1003641 (2014)

15. Touzet, C.F.: The theory of neural cognition applied to robotics. Int. J. Adv.Robot. Syst. 12, 74 (2015)

Perceptive Invariance and Associative Memory 287

16. Somasundaram, M.: Circuit to generate a sequential index for an input number ina pre-defined list of numbers, Patent US7155563 B1, 26 Dec 2006

17. Strasburger, H., Rentschler, I., Juttner, M.: Peripheral vision and pattern recogni-tion: a review. J. Vis. 11(5), 1–82 (2011)

18. Van Rullen, R., Gautrais, J., Delorme, A., Thorpe, S.: Face processing using onespike per neurone. Biosystems 48(1–3), 229–239 (1998)

Thrust-Assisted Perching and Climbingfor a Bioinspired UAV

Morgan T. Pope(B) and Mark R. Cutkosky

Stanford University, Stanford, CA 94305, USAmpope@stanford.edu

http://bdml.stanford.edu

Abstract. We present a multi-modal robot that flies, perches andclimbs on outdoor surfaces such as concrete or stucco walls. Althoughthe combination of flying and climbing mechanisms in a single platformextracts a weight penalty, it also provides synergies. In particular, a smallamount of aerodynamic thrust can substantially improve the reliabilityof perching and climbing, allowing the platform to maneuver on other-wise risky surfaces. The approach is inspired by thrust-assisted perchingand climbing observed in various animals including flightless birds.

1 Introduction

In the aftermath of an earthquake or other disaster, small unmanned air vehiclessuch as quadrotors are ideal for initial response. They are unaffected by debrison the ground and can fly rapidly to remote and cluttered sites. Unfortunately,they are typically hampered by a short mission life – often 20 min or fewerfor platforms weighing less than 0.5 kg. However, if these vehicles can perchand crawl on vertical surfaces, they can extend their missions to hours whileperforming surveillance, inspection or communication functions. Once perched,they are also relatively unaffected by poor weather; when conditions improve,they can take off and return home or fly elsewhere. Not surprisingly, many smallflying animals also perch frequently, resting and taking shelter between flights.

Although multi-modal flying, perching and crawling operation confers advan-tages, it also extracts a penalty. For small air vehicles, where every gram counts,the mechanisms used for clinging and crawling can easily double the weight of thevehicle, with a corresponding deterioration in flying performance. Fortunately,there are also synergies to be realized by combining climbing and flying capa-bilities. These synergies are explored in SCAMP (Stanford Climbing and AerialManeuvering Platform).

SCAMP is a small robot that combines a commercial Crazyflie 2.0TMquadrotorwith insect- and bird-inspired perching and climbing mechanisms. It can fly out-doors and land on unprepared vertical surfaces such as concrete or stucco walls.Like various flying and gliding animals, it also uses aerodynamic effects to enhanceperching and climbing.1

1 A video of SCAMP in operation is available at https://www.youtube.com/watch?v=bAhLW1eq8eM.

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 288–296, 2016.DOI: 10.1007/978-3-319-42417-0 26

Thrust-Assisted Perching and Climbing for a Bioinspired UAV 289

xz

AssemblyFoot Motion (Extend/Retract)

Foot Motion (In/Out)

x

yz

footwithspines

bowspring

T rod

tendon

tail

Fig. 1. (left) SCAMP climbing (inset: SCAMP in flight). (right) Climbing mechanismwith two servo motors, one to extend/retract feet along the wall, and one to pull feetin/out from wall surface, attaching or detaching spines.

2 Related Work

Steadily shrinking microprocessors, sensors and actuators have lead to an explo-sion in the popularity of micro air vehicles in general, and quadrotors in partic-ular. However, limits on battery life and small-scale aerodynamic efficiency leadto an acknowledged deficiency in mission life, which tends to be short – 30 min orless [1]. A solution to this problem is to fly to a target area and then perch, withthe rotors powered down to extend battery life. Motivated by this realization,the last decade has seen steady advances in climbing and perching robots. Oneparticular impetus was DARPA’s Perch and Stare competition, which, althoughit did not produce a drone that could complete all of the the target objectives,advanced the state of the art for future efforts to build on [2].

Other perching efforts have used darts [3], spines [4,5] or adhesives or suction[6–10] to perch on vertical and even overhanging surfaces. SCAMP also drawsparticularly on robots that use arrays of miniature spines to climb rough verticalwalls with two, four, or six legs [11–13]. Another related subject concerns multi-modal robots that combine airborne and vertical locomotion abilities [14–16].

Most of the previous work on maneuvers for perching has utilized off-boardvisual tracking and computation (e.g., [17,18]). However, for the smallest and

290 M.T. Pope and M.R. Cutkosky

lightest of the cited perching vehicles, it is possible to simply fly into a wall,utilizing the contact forces on impact to achieve a grip that resists reboundforces. As described in Sect. 3, SCAMP takes this approach, augmented by thrustfrom the rotors.

To our knowledge, no prior robot is capable of repeatedly perching and climb-ing outdoors using passive attachment technology.

2.1 Bioinspiration

The mechanisms and actions used for perching also draw inspiration from glidingand perching animals. In this section, we restrict ourselves to animals that, likeSCAMP, perch on vertical surfaces, such as tree trunks and walls.

The spines used in SCAMP are similar to those found on the legs of manyinsects, that catch on small asperities (bumps or pits) to provide purchase on allbut very smooth surfaces [19,20]. The distinguishing characteristic is that spinesengage asperities that are (i) larger than or equal to the spine tip radius and (ii)have a surface angle and friction coefficient such that the spine force vector fallsinside the resulting friction cone. This point is considered further in Sect. 3.1.

In contrast to most robotic approaches to flying and perching, animals main-tain aerodynamic control up to the instant of contact. Thrust-assisted perchingdraws inspiration from aerial strategies used by a wide range of species, fromants [21,22] to frogs [23], flying squirrels [24] and colugos [25], as well as manybirds that perch on tree trunks, and similar vertical surfaces. In particular, birdsthat climb tree trunks often use their tails to provide a positive contact force,reducing the “pull-in” force required at the front feet [26], a strategy similar tothat adopted in SCAMP. Other birds, including flightless birds, use their wingsto enable them to run up otherwise insurmountable surfaces [27,28], a strategyanalogous to thrust-assisted climbing in SCAMP.

From an energetic standpoint, the strategy of climbing up vertical surfacesand then gliding down to a nearby tree or other target can be favorable [29],which may explain its popularity across many species.

3 SCAMP Operations

When flying, SCAMP carries its perching and climbing apparatus above therotors (Fig. 1 inset), where they add 11 g to the 28 g Crazyflie platform. Withthis overhead addition, the platform’s flight time reduces from 5 min to 3.5 min.Hence, SCAMP relies on perching to achieve useful mission durations.

The behaviors programmed for SCAMP are depicted in Fig. 2, with labelsmatching those in the following text. SCAMP perches by flying, tail-first, intoa wall at speeds of ≈1–2 m/s. This brisk approach reduces the effects of windthat could make it difficult to hover next to a wall. As the tail impacts the wall(a. Contact) the internal accelerometer measures a large signal, which triggersthe pitch-up maneuver (2. Pitch up). At its simplest, this consists of disablingthe internal stabilization control and allowing the rotors to pull the climbing

Thrust-Assisted Perching and Climbing for a Bioinspired UAV 291

1. Fly

3. Perch 4. Climb

2. Pitch up

a. Contact

b. At rest

5. Take off

c. Command

c. Command

6. Fall

d. Failuree. Recovery

d. Failuree. Recovery

Fig. 2. State-event diagram for SCAMP – labels match numbers in text.

mechanism parallel to the wall. SCAMP then falls slightly until the feet engageasperities on the surface and the robot comes to rest (3. Perch). SCAMP detectsa failure to attach (d. Failure) by monitoring whether the vertical accelerationfalls below 0.8 g during the first few hundred milliseconds after impact.

Once attached, SCAMP can climb up or down (4. Climb), using servos thatmove the feet in the extend/retract (z) and in/out (x) directions (Fig. 1, right).The extend/retract servo rotates a carbon fiber rod back and forth in the (y, z)plane, pulling on tendons connected to the two feet. Carbon fiber bow springs,with roughly constant force over the range of travel (up to 95 mm), keep thetendons taut. A second servo rotates a T-shaped rod with wire guide loops at itstips, which pull the tendons and feet into or away from the wall surface. The feetuse small spines to catch on asperities and provide forces primarily parallel andslightly into the wall, supporting the robot’s weight and counteracting the pitch-back moment produced by the robot’s center of mass, which is approximately40 mm out from the wall surface. The spines are adapted from those used in [4].

Based on wireless commands (c. Command), SCAMP switches betweenperching and climbing up or down. At present, the external wireless commandsare issued by a human operator. However, the details of operation while perchingor climbing are computed and controlled using the onboard microprocessor. Ifthe accelerometer detects a fall, SCAMP attempts to arrest the fall (e. Recovery)by briefly turning on the rotors to generate thrust.

On command, SCAMP can position its servos, normally used for climbing,into a special configuration that causes its spring-loaded take-off arm, located atthe rear of the platform, to swing into the wall. The tip of the arm has a spinethat catches on the wall so that, as SCAMP loses its grip, it pitches backwardinto the horizontal flight orientation and resumes flight (1. Fly). When the servosreturn to their normal climbing configuration, a string attached to one of theservos pulls the take-off arm back into its cocked position, visible in Fig. 1.

3.1 Thrust-Assisted Perching and Climbing

Using thrust simplifies perching and increases climbing reliability in several ways.

292 M.T. Pope and M.R. Cutkosky

Fig. 3. (top left) Diagram of impact angle θI . (bottom left) Observed range of impactangles and velocities leading to successful perching. (A) Rotors and kinetic energy tiltSCAMP upward. (B) Rotors spin down as perch stabilizes.

Pitch-up Maneuver: Whereas previous fixed-wing airplanes and quadrotors(e.g. [4,17,18]) initiated a pitch-up maneuver immediately prior to perching – toreduce speed and orient their spines correctly for attachment – SCAMP is lightenough that it can fly directly into a wall and use aerodynamic thrust to tilt itsbody upward, aligning itself with the wall. As SCAMP tilts upward, its rotorthrust is enhanced by ground effects, which produce an inward force, normal tothe wall. This force increases rapidly as SCAMP reaches vertical, suppressingthe rebound that previous perching vehicles had to absorb with springs anddamping foam. When SCAMP is aligned vertically, the rotors need only 10 % oftheir maximum thrust to hold the platform against the wall.

This approach is robust within some angle and velocity constraints. At impactangles θI > 5◦, the point of impact is above the center of mass (Fig. 3) andtorque is initially applied in the wrong direction, increasing the chance of failure.The impact velocity must be also large enough to create an acceleration largerthan that experienced during normal flight conditions. Approximately 1 g is asafe acceleration threshold, corresponding to ≈ 0.8 m/s. Initial velocities above≈2.8 m/s produce large vibrations and a large initial rebound, which can leadto SCAMP aligning incorrectly. The perching process and empirical limits aresummarized in Fig. 3.

Climbing: Thrust is equally effective at increasing the robustness of climbing.Figure 4 shows how the application of a small amount of rotor thrust can dra-matically change the effective loading angle for a microspine engaging a surface.In the profile image, θLn represents the angle of the load force at the front feet,

Thrust-Assisted Perching and Climbing for a Bioinspired UAV 293

Usable ranges:

S> Ln-

S> Lt-

Load vectorn: no thrust t: with thrust

Lt

TFA

W

Ln

rs

Traced surface

Spine

Thrust (body weights)m

m /

aspe

rity

0 0.1 0.2 0.3 0.4 0.5 0.6 0.70

1

2

3

4

5

Prediction

Data

Fig. 4. (left) Traced concrete surface corresponding to 10µm spine tip. Adding thrustshifts load vector from θLn to θLt, increasing asperity density. (right) Stochastic asper-ity model with thrust compared to empirical data for concrete. (Color figure online)

which depends on the location of the center of mass and the length from the feetto the tail. Adding a small amount of thrust shifts the load vector from θLn toθLt. Only a small amount of thrust (10 % of the maximum) is needed to shiftthe load force from pointing slightly outward to slightly inward.

The surface shown in Fig. 4 is taken from [12] and represents a concretecinderblock profiled with a stylus. The spine tip (10µm radius) sliding over thissurface creates a traced surface [30], shown in yellow. On this traced surface, the“perchable regions” are those for which the surface orientation is such that

θs > θLt − θμ (1)

where θs is the angle of the local inward surface normal with respect to vertical,θμ = arctan μ and μ is the coefficient of friction.

Without thrust, there are just four regions (shown in magenta) that satisfyEq. 1; with a small amount of thrust there are eight regions (shown in green).More generally, the distribution of asperities on a surface can be modeled approx-imately as a spatial Poisson process. For a spine dragging along a region of thesurface, the distribution of lengths between asperities is approximately describedby an exponential random variable, with probability density function

fx(x;λ) = λe−λx x ≥ 0 (2)

where x is the distance traveled and λ is the number of asperities per unit length;λ in turn is a function of the traced surface, θLt, and μ.

The plot on the right side of Fig. 4 compares the predicted average distancebetween asperities with data obtained by dragging a loaded spine along a con-crete block and varying the load angle to correspond to different amounts ofthrust. The error bars represent the 95 % confidence intervals for our estimationof the average distance between asperities based on an exponential distribution.

294 M.T. Pope and M.R. Cutkosky

Recovery: A small amount of thrust can also make the difference between asmall loss in height and a catastrophic fall when SCAMP misses a step. Like theants studied in [21,22], SCAMP applies an aerodynamic force to return to thewall when it detects the drop in vertical acceleration resulting from unexpecteddetachment. Figure 5 illustrates this response, and plots acceleration, velocity,and position data from a typical misstep. Average fall distance is ≈14 cm witha standard deviation of ≈12 cm. If thrust is not used, the robot has never beenobserved to re-engage with the surface.

Fig. 5. Data, illustration, and images for climbing recovery. When vertical accelerationdrops below 0.5 g (A), SCAMP turns its rotors on to return its feet to the wall, wherethey engage with asperities and arrest downward motion (B).

4 Conclusions

A robot that functions in two distinct locomotory modes must make compromisesin weight and complexity to accommodate the different interaction mechanics,control approaches, and power requirements for the modes of interest. While thisusually means that the resulting design is heavier than a conventional single-purpose robot, the additional locomotory modes can act to complement eachother. In the case of SCAMP, the perching mechanism extends the effectivetime aloft for a flying machine that is interested in observing a fixed area; therotorcraft itself, on the other hand, aids climbing through both its sensitiveonboard accelerometers and its ability to create aerodynamic forces, allowingthe robot to recover from failures and to increase the asperity density.

Acknowledgements. Support for this work was provided by NSF IIS-1161679 andARL MAST MCE 15-4. We gratefully acknowledge the help of H. Jiang, C. Kimes, W.Roderick and C. Kerst in conducting experiments.

Thrust-Assisted Perching and Climbing for a Bioinspired UAV 295

References

1. Keennon, M.T., Grasmeyer, J.M.: Development of the black widow and microbatmavs and a vision of the future of mav design. In: AIAA International Air andSpace Symposium and Exposition: The Next 100 Years, pp. 14–17 (2003)

2. Prior, S.D., Shen, S.-T., Erbil, M.A., Brazinskas, M., Mielniczek, W.: HALO thewinning entry to the DARPA UAVForge challenge 2012. In: Marcus, A. (ed.)DUXU 2013, Part III. LNCS, vol. 8014, pp. 179–188. Springer, Heidelberg (2013)

3. Kovac, M., Germann, J., Hurzeler, C., Siegwart, R.Y., Floreano, D.: A perchingmechanism for micro aerial vehicles. J. Micro-Nano Mechatron. 5, 77–91 (2010)

4. Lussier Desbiens, A., Asbeck, A.T., Cutkosky, M.R.: Landing, perching and takingoff from vertical surfaces. Int. J. Rob. Res. 30(3), 355–370 (2011)

5. Kovac, M., Germann, J.M., Hurzeler, C., Siegwart, R., Floreano, D.: A perchingmechanism for micro aerial vehicles. J. Micro-Nano Mechatron. 5, 77–91 (2009)

6. Anderson, M.L., Perry, C.J., Hua, B.M., Olsen, D.S., Parcus, J.R., Pederson, K.M.,Jensen, D.D.: The sticky-pad plane and other innovative concepts for perchingUAVs. In: 47th AIAA Aerospace Sciences Meeting (2009)

7. Daler, L., Klaptocz, A., Briod, A., Sitti, M., Floreano, D.: A perching mechanismfor flying robots using a fibre-based adhesive. In: IEEE International Conferenceon Robotics and Automation (2013)

8. Kalantari, A., Mahajan, K., Ruffatto III., D., Spenko, M.: Autonomous perchingand take-off on vertical walls for a quadrotor micro air vehicle. In: 2015 IEEEInternational Conference on Robotics and Automation (ICRA), pp. 4669–4674.IEEE (2015)

9. Tsukagoshi, H., Watanabe, M., Hamada, T., Ashlih, D., Iizuka, R.: Aerial manip-ulator with perching and door-opening capability. In: 2015 IEEE InternationalConference on Robotics and Automation (ICRA), pp. 4663–4668. IEEE (2015)

10. Liu, Y., Sun, G., Chen, H.: Impedance control of a bio-inspired flying and adhe-sion robot. In: 2014 IEEE International Conference on Robotics and Automation(ICRA), pp. 3564–3569. IEEE (2014)

11. Clark, J., Goldman, D., Lin, P.C., Lynch, G., Chen, T., Komsuoglu, H., Full, R.J.,Koditschek, D.E.: Design of a bio-inspired dynamical vertical climbing robot. In:Robotics: Science and Systems (2007)

12. Asbeck, A., Kim, S., Cutkosky, M.R., Provancher, W.R., Lanzetta, M.: Scalinghard vertical surfaces with compliant microspine arrays. Int. J. Rob. Res. 25, 14(2006)

13. Spenko, M.J., Haynes, G.C., Saunders, J.A., Cutkosky, M.R., Rizzi, A.A., Full,R.J.: Biologically inspired climbing with a hexapedal robot. J. Field Rob. 25, 223–242 (2008)

14. Estrada, M., Hawkes, E.W., Christensen, D.L., Cutkosky, M.R., et al.: Perchingand vertical climbing: Design of a multimodal robot. In: 2014 IEEE InternationalConference on Robotics and Automation (ICRA), pp. 4215–4221. IEEE (2014)

15. Myeong, W., Jung, K., Jung, S., Jung, Y., Myung, H.: Development of a drone-type wall-sticking and climbing robot. In: 2015 12th International Conference onUbiquitous Robots and Ambient Intelligence (URAI), pp. 386–389. IEEE (2015)

16. Dickson, J.D., Clark, J.E.: Design of a multimodal climbing and gliding roboticplatform. IEEE/ASME Trans. Mechatron. 18(2), 494–505 (2013)

17. Mellinger, D., Shomin, M., Kumar, V.: Control of quadrotors for robust perchingand landing. In: Proceedings of the International Powered Lift Conference, pp.119–126 (2010)

296 M.T. Pope and M.R. Cutkosky

18. Thomas, J., Pope, M., Loianno, G., Hawkes, E.W., Estrada, M.A., Jiang, H.,Cutkosky, M.R., Kumar, V.: Aggressive flight for perching on inclined surfaces.J. Mech. Rob. 8(5), Article ID. 051007 (2016). doi:10.1115/1.4032250

19. Dai, Z., Gorb, S.N., Schwarz, U.: Roughness-dependent friction force of the tarsalclaw system in the beetle pachnoda marginata (coleoptera, scarabaeidae). J. Exp.Biol. 205(16), 2479–2488 (2002)

20. Gorb, S.N.: Biological attachment devices: exploring nature’s diversity for bio-mimetics. Philos. Trans. R. Soc. Lond. A: Math. Phys. Eng. Sci. 366(1870), 1557–1574 (2008)

21. Yanoviak, S.P., Dudley, R., Kaspari, M.: Directed aerial descent in canopy ants.Nature 433(7026), 624–626 (2005)

22. Yanoviak, S.P., Munk, Y., Kaspari, M., Dudley, R.: Aerial manoeuvrability inwingless gliding ants (cephalotes atratus). Proc. Roy. Soc. London B: Biol. Sci.277, 2199–2204 (2010). doi:10.1098/rspb.2010.0170

23. McCAY, M.G.: Aerodynamic stability and maneuverability of the gliding frogpolypedates dennysi. J. Exp. Biol. 204(16), 2817–2826 (2001)

24. Paskins, K.E., Bowyer, A., Megill, W.M., Scheibe, J.S.: Take-off and landing forcesand the evolution of controlled gliding in northern flying squirrels Glaucomys sabri-nus. J. Exp. Biol. 210(Pt. 8), 1413–1423 (2007)

25. Byrnes, G., Lim, N.T.L., Spence, A.J.: Take-off and landing kinetics of a free-ranging gliding mammal, the Malayan colugo (Galeopterus variegatus). Proc. Roy.Soc. B: Biol. Sci. 275, 1007–1013 (2008)

26. Fujita, M., Kawakami, K., Higuchi, H.: Hopping and climbing gait of japanesepygmy woodpeckers (picoides kizuki). Comp. Biochem. Physiol. Part A: Mol.Integr. Physiol. 148(4), 802–810 (2007)

27. Dial, K.P.: Wing-assisted incline running and the evolution of flight. Science299(5605), 402–404 (2003)

28. Bundle, M.W., Dial, K.P.: Mechanics of wing-assisted incline running (WAIR). J.Exp. Biol. 206, 4553–4564 (2003)

29. Norberg, R.A.: Why foraging birds in trees should climb and hop upwards ratherthan downwards. Ibis 123(3), 281–288 (1981)

30. Okamura, A.M., Cutkosky, M.R.: Feature detection for haptic exploration withrobotic fingers. Int. J. Rob. Res. 20(12), 925–938 (2001)

The EASEL Project: Towards EducationalHuman-Robot Symbiotic Interaction

Dennis Reidsma1(B), Vicky Charisi1, Daniel Davison1, Frances Wijnen1,Jan van der Meij1, Vanessa Evers1, David Cameron2, Samuel Fernando2,

Roger Moore2, Tony Prescott2, Daniele Mazzei3, Michael Pieroni3,Lorenzo Cominelli3, Roberto Garofalo3, Danilo De Rossi3, Vasiliki Vouloutsi4,Riccardo Zucca4, Klaudia Grechuta4, Maria Blancas4, and Paul Verschure4,5

1 Human Media Interaction/ELAN,University of Twente, Enschede, The Netherlands{d.reidsma,v.charisi,d.p.davison,f.m.wijnen,

j.vandermeij,v.evers}@utwente.nl2 Sheffield Robotics, University of Sheffield, Sheffield, UK

{d.s.cameron,s.fernando,r.k.moore,t.j.prescott}@sheffield.ac.uk3 E. Piaggio Research Center, University of Pisa, Pisa, Italy{daniele.mazzei,michael.pieroni,lorenzo.cominelli,roberto.garofalo,d.derossi}@centropiaggio.unipi.it

4 SPECS, DTIC, N-RAS, Universitat Pompeu Fabra, Barcelona, Spain{vicky.vouloutsi,riccardo.zucca,klaudia.grechuta,

maria.blancas,paul.verschure}@upf.edu5 Catalan Institute of Advanced Studies (ICREA), Barcelona, Spain

Abstract. This paper presents the EU EASEL project, which exploresthe potential impact and relevance of a robot in educational settings. Wepresent the project objectives and the theorectical background on whichthe project builds, briefly introduce the EASEL technological develop-ments, and end with a summary of what we have learned from the eval-uation studies carried out in the project so far.

Keywords: Synthetic Tutoring Assistant · DAC · Education · Childrobot interaction · Architecture · Evaluation

1 Introduction

This paper presents the EU EASEL project (“Expressive Agents for SymbioticEducation and Learning”), which explores the potential impact and relevance ofa robot in educational settings. EASEL targets Human Robot Symbiotic Inter-action (HRSI) in the domain of education and learning. Symbiosis is taken hereas the capacity of the robot and the person to mutually influence each other,

D. Reidsma—This project has received funding from the European Union SeventhFramework Programme (FP7-ICT-2013-10) as part of EASEL under grant agree-ment no. 611971. Coauthors are grouped by institute.

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 297–306, 2016.DOI: 10.1007/978-3-319-42417-0 27

298 D. Reidsma et al.

and alter each other’s behaviour over different time-scales (within encountersand across encounters). Based on perception of the social, communicative andeducational context, the robot responds to the student in order to influence theirlearning progress.

The impact of EASEL developments crucially depends on the combinationof two domains. The field of human robot interaction concerns conversationaland social interaction between humans and robots. This involves short terminteractions as well as the development of a relation over longer time throughrepeated interactions. The field of learning and education concerns principlesand practices of how a student learns in interaction with other people and learn-ing materials (Charisi et al. 2015). The theoretical work carried out in EASELconcerns the integration of insights from these two fields.

Clearly, the resulting tutoring assistant(s) need to be evaluated. EASELachieves this through a combination of lab studies and in-the-wild studies inschools, museums, and daycare centers. The studies carried out in EASEL rangeacross the combination of the two above domains, from studies focusing on thedevelopment of a longer term relation between human and robot to studies focus-ing on the exact effect of certain tasks on the learning process and outcome.

The paper is structured as follows. We introduce the EASEL objectives inSect. 2. Section 3 briefly discusses the interplay between the two above mentioneddomains of learning and social interaction. Section 4 focuses on the technologydeveloped in EASEL. Section 5 summarizes the EASEL evaluation studies car-ried out so far, placing them in the framework between HRI and education.Finally, we tie the results together in Sect. 6, looking at the future directionswe need to go to achieve the ultimate aim of EASEL: social robots that are atransformative contribution to the classroom of the future.

2 EASEL Objectives

EASEL aims to deliver a new set of Robotic Based Tutoring Solutions: a Syn-thetic Tutoring Assistant that incorporates key features of human tutors andother proven approaches capable of instructing a human user and learns fromtheir interactions during large time scales. To this end, new approaches aredeveloped for acquiring social context from sensor data, modeling the student’slearning process, and determining the appropriate and most effective strategiesfor delivering the learning material. The results are incorporated in a social dia-log carried out between student and robot, which supports the student in thelearning task. The end result of EASEL is a unique and beyond the state of theart social robot based tutoring system that comprises a learning model of theuser, a synthetic agent control system for symbiotic interaction establishment,a computational framework of social affordances and a multi modal analysissystem for subject’s social and affective state analysis. In addition, the projectyields guidelines for the design and development of the appropriate robot behav-iour toward children in various possible robot roles in the contexts of school andmuseum environments.

The EASEL Project: Towards Educational HRSI 299

3 Learning, Education, and Robots

The theoretical viewpoint from which we approach learning and education isdescribed by Charisi et al. (2015). Below, we briefly describe this view and showthe potential areas of contribution for social robots.

Vygotsky (1978) describes learning as a socio-cultural process. In this process,the student’s learning is mediated in two ways. Firstly, by “physical tools”: thelearning materials such as books, computers, and other tools. Secondly, learn-ing is mediated by “social tools”: other people who participate in the learningsituation. Figure 1 shows how learning happens in a triadic interaction betweenstudent, materials, and other persons.

Fig. 1. Learning as a triadic interaction between student, learning materials, andanother person (in this case: a robot)

When we look at the interaction between student and learning materials,there is the student interacting directly with the learning materials. We canobserve the actions the student takes in the learning task, the utterances andexpressions, and the performance in the task. Furthermore, there are the thingsgoing on in the mind of the student. These include attitude towards task (is itinteresting? is it hard? is it relevant?), self-efficacy with respect to the task (canI do this task? am I doing well right now?), mind set in learning (Dweck 2012)(e.g., directed at risk avoidance / aversive to failure? directed at growth andlearning? willingness to make mistakes? curiosity?), and other factors.

When we look at the role that an “other person” can play in the student’slearning process, we see three possibilities. The other can be more knowledgeable(e.g., teacher or more advanced fellow student), differently knowledgeable (a fellowstudent doing the task together with, or alongside of, the student, see Dillenbourg(1999)), or even less knowledgeable. Things the other person could do in their vari-ous roles are, for example, explain to student, be explained to by student, encour-age student, praise student in various ways, give good or bad example as fellowstudent, and many other things. In this way, the other person can influence boththe observable behavior and the mental state of the student. The effectiveness of

300 D. Reidsma et al.

these actions depends to a large extent on the relation between the student andthe other: factors such as trust or likability will influence to what extent the stu-dent is willing to modify his or her actions or perceptions in response to the other’ssuggestions and contributions.

Given this context, a robot in class can serve a unique mixed role. It is acomputer, and as such it can present learning materials to the student in asmart and adaptive way based on the student’s skills and progress: the robot assmart learning material. At the same time, a robot is a social entity, more so thana computer. As such, it can fulfill the role of a more or differently knowledgeable“other person” across the entire spectrum from teacher or more advanced fellowstudent to less knowledgeable peer being taught to by the student. It is thissocial role of the robot that we are concerned with here. Like many others, webelieve this makes the robot a very powerful tool for learning because learningtakes place in a social context (Vygotsky 1978). However, the effectiveness ofthe robot depends among other things on the social believability and the qualityof the relation between student and robot. This requires us to also look at shortand long term affective interactions between student and robot (Cameron et al.submitted).

4 EASEL Technology Developments

The EASEL architecture incorporates the novel technology developments of theproject, and is structured in four layers: acquisition, cognitive modules, dialogand behavior planning, and robot behavior realisation. Vouloutsi et al. (2016)describe how this architecture is a practical incarnation of the conceptual archi-tecture of the Distributed Adaptive Control (DAC) theory (Verschure 2012) ofthe design principles underlying perception, cognition and action. This sectionbriefly introduces the main components developed as part of EASEL.

4.1 Acquisition

The acquisition layer of EASEL consists of various modules integrating audio-visual analysis and acquisition of physiological signals from the user. Thespeech recognition is based on the open source Kaldi speech recognizer(Povey et al. 2011), with an EASEL specific vocabulary, language model andrecognition grammar. Its specific speaker adaptation solution makes it veryrobust with respect to interfering speech from the robot itself. Audiovisual sceneanalysis is done using the SceneAnalyzer (Zaraki et al. 2014), which builds uponseveral other libraries to deliver quick and robust integrated recognition of mul-timodal features of the users and their behaviour. Physiological signals can beacquired from the user using non-obtrusive sensor patches that can be integratedin the robot or the learning materials, which allows signal analysis to take placewithout sensors worn strapped to arm/head/body of user.

The EASEL Project: Towards Educational HRSI 301

4.2 Cognitive Modules

The cognition of the tutoring assistant consists of the memory and decision mod-ules. The memory module stores the current state of the user and the learningtask, and has the potential to store longer term memory in repeated interaction,which allows the tutor to build up a user model of a learner from all informationin the network. The allostatic control module (Vouloutsi et al. 2013) regulatesthe tutoring process, learning optimal strategies for delivering the learning con-tent in the right order and at the right difficulty levels, adapted to the student’scharacteristics and capabilities. The exercise generator, finally, delivers learningexercises of exactly the right nature and difficulty level that is requested by thetutoring models.

4.3 Dialog and Behavior Planning

The cognitive modules deliver the interaction goals of the tutoring agent. Theseare then translated by the dialog and behavior planning modules into actualutterances and expressions to be realised by the robot. We use the Flipper dia-log manager of ter Maat and Heylen (2011) for managing the dialog, and theBML realizer ASAPRealizer by van Welbergen et al. (2012). Flipper affers flexi-ble dialog specification via information state and rule based templates to triggerinformation state changes as well as behaviour requests. ASAPRealizer offerseasy, configurable, control of multimodal choreographed behaviours across sev-eral robots using the BML language, which abstracts away from specific motorcontrol by exposing more general behaviour specifications to the dialog man-ager. ASAPRealizer realizes the requested behaviours on robotic embodimentsby directly accessing the motion primitives of the embodiments (see below).

4.4 Content Presentation and Behavior Generation

The learning content is presented in two ways: through the EASEL Scope tablets,and through utterances and expressions of the robot. The EASEL Scope offersa mixed reality interface that allows the student to interact with the learningscenario materials. It can be used to present additional information to the childabout the learning materials. This allows the system to vary between differentways of scaffolding the learning of the user. The two main robots used in EASELare the Robokind Zeno R251 and the FACE robot (Lazzeri et al. 2013; Mazzeiet al. 2014). For both robots, controllers have been developed that offer access tothe robot’s motion control primitives in a comparable way, allowing the systemto present the same content through different robots.

The first versions of the complete EASEL architecture have been deployedalready in various settings and used in a number of studies; the final year ofthe project will see several deployments of the system in real life contexts overlonger periods of time.

1 http://www.robokindrobots.com/.

302 D. Reidsma et al.

5 EASEL Studies on Robots in Learning

In order to evaluate the integrated solutions developed in EASEL, an ongoingseries of experiments is conducted throughout the project. In this section wediscuss results we have achieved so far. We looked at the impact of robot behaviorand characteristics on the way students perceive the robot, the potential forimpact on the longer term (social) relationship between student and robot, theimpact of the robot on the learning outcome, and the impact of the robot onthe learning process that the student goes through. In most experiments thesestudents were children of primary school age since these are the main targetgroup for EASEL. Most studies were conducted with a Robokind Zeno R25robot; a few were carried out with an iCub.

5.1 Impact of Robot Behavior and Characteristics on Perceptionof Robot by Student

Regarding direct perception of the robot by the children interacting with it,EASEL studies have looked at the childs affective and social responses relatedto gender (Cameron et al. 2015b) and age (Cameron et al. 2015c). Childreninteracted with a robot during a Simon Says game in two studies. Results ofboth studies show a clear gender difference: boys respond more positively, show-ing more smiling and liking, while girls display less smiling with an expressiverobot. Furthermore, the second study revealed an age difference: on average olderchildren considered the robot to be significantly more like a machine than theyounger children did. Follow-up studies will look at the influence of age on per-ception of animacy and gender of the robot Zeno by boys and girls from differentage groups.

Regarding responses of users to the robot during a task, another EASELstudy found that children seem to respond differently to the robot dependingon whether it seems to be autonomously responsive to speech from the childrenor its speech understanding is visibly mediated by the experiment controller.In the former case children seem to display more anticipatory gaze awaitingthe robot’s responses, in contrast to more reactive gaze after the robot startedresponding (Cameron et al. 2016a). In several of the studies we saw that childrenattempted to engage the robot in social interactions, displaying behavior such asconversational turn-taking and socially oriented gaze and spontaneous utterancestowards the robot, and they would sometimes modify their speech utterances(following robot errors, participant corrects robot then repeats question moreslowly and clearly, emphasizing key words).

5.2 User Behavior Towards the Robot: Potential for Longer TermRelation Between Robot and Student

In the previous section we saw that children seem to be willing to treat theEASEL robot as a social partner, and respond socially to it, depending on its

The EASEL Project: Towards Educational HRSI 303

behavior. We are also interested in finding out to what extent the robot’s behav-ior could influence the forming of a longer term relationship.

One EASEL study focused on the impact of robot-stated phrases, relatingto its limitations or its intentions, on individuals liking, perceptions of robotcompetence, and willingness to assist the robot (Cameron et al. 2015a, 2016b).Results of this study showed that robot behavior can influence the user’s will-ingness to use the robot in the future. This study provides new evidence thatstrategies used by individuals in interpersonal relationship development can beextended to apply to social robotics HRI.

We also looked at the impact of the activities that robot and child shareon the potential for longer term relationship. Davison et al. (2016) looked atchildren who engage in both an educational task and a physical exercise witha peer-like robot, or only in educational tasks. Results, although not significantenough, suggest that sharing an additional physical and extra-curricular activitymight promote social perception of the robot.

For future EASEL studies we are planning to focus on the details of relation-ship development over a longer period of time and in repeated interaction withthe system.

5.3 Impact of Robot on Learning Outcome

We looked at the effect on learning performance related to different behaviortypes of the robot: tutor-like behavior vs peer-like behavior, and various waysof implementing the robot’s gaze behavior. The impact was measured on theperception and subjective experience of the user and on task performance andlearning outcome (Blancas et al. 2015; Vouloutsi et al. 2015). Regarding learningoutcome, experiments so far indicate that although we can measure improvementin performance successfully with a post-test, the improvement was not yet signif-icantly different between conditions of different robot behavior. We will addressthis aspect further in longer term studies, since we expect that this effect willdepend on longer term interactions.

5.4 Impact of Robot on Learning Process

The final aspect that the EASEL project targets is impact of the robot (behavior)on the learning process. In the final EASEL study discussed here, children hadto do an inquiry learning task in one of two conditions: with a social robot, orwith an interactive tablet. In both conditions the content of the task was thesame, including the spoken instructions issued by the robot or tablet. In bothconditions, the child was invited to verbally explain their thoughts. Importantsteps in an inquiry learning task are related to generating explanations. It is wellknown that explaining learning content to someone else is a powerful source oflearning (Bargh and Schul 1980); children often gain a deeper understanding ofthe material when they are asked to verbalize their thoughts and reasoning toothers (Coleman et al. 1997; Holmes 2007).

304 D. Reidsma et al.

The main hypothesis for this study was that the social nature of the robot,compared to the tablet, would trigger the child to verbalize their thoughts moreeasily (faster and/or longer responses). To measure this we looked at both theverbalization by the child, and their perception of the tablet or robot as a socialentity.

Results tentatively indicated that children tended to verbalize more andrespond faster to questions when working with the robot. It seems that therobot provided a more intuitive context for verbalization than the tablet. Theresults of the exit interview suggested that the robot was indeed seen as a moresocial entity compared to a tablet. For example, statements like: “I taught therobot”, or “the robot was curious” were given by the children in the exit inter-view, and children spontaneously addressed the robot as ‘robot’, asking it itsopinion, etcetera.

6 Discussion

We presented the EASEL project, which aims to deliver a new state of the art inSynthetic Tutoring Assistants. The conceptual and technological developmentsso far have resulted in an integrated system capable of deriving contextual infor-mation from audiovisual sensors, modeling students and learning, learn strategiesfor effective teaching and deliver the teaching material through social dialog.

An important observation in the evaluation studies is the fact that child-robotinteraction, be it in learning or in other domains, is challenging to evaluate. So farwe have applied a number of novel and traditional methods that gave us sensibleresults and significant differences between conditions, but not all methods workedequally well (or at all) (Charisi et al. submitted). The evaluation of child-robotinteraction is a topic that we will address in future work. Clearly, we are notalone in this position, as shown by the growing number of workshops, symposia,and panels dedicated to this topic.

Nevertheless, we feel that we managed to start exploring the potential impactof robots in education across the whole spectrum: the perception of the robotby the student, the student’s responses to the behavior and characteristics ofthe robot; the potential for longer term relationship; the possibility for robotto influence the learning process, and the learning outcome. We see increasingevidence toward the positive effects of the EASEL robots social behavior on howchildren approach learning tasks as well as on the learning outcome. We arenow preparing the final longer term evaluations in which we combine all theseaspects in one setup, with the aim of showing how the EASEL robots can be atransformative contribution to the classroom of the future.

The EASEL Project: Towards Educational HRSI 305

References

Bargh, J.A., Schul, Y.: On the cognitive benefits of teaching. J. Educ. Psychol. 72(5),593–604 (1980)

Blancas, M., Vouloutsi, V., Grechuta, K., Verschure, P.F.M.J.: Effects of the robot’s roleon human-robot interaction in an educational scenario. In: Wilson, S.P., Verschure,P.F.M.J., Mura, A., Prescott, T.J. (eds.) Living Machines 2015. LNCS, vol. 9222,pp. 391–402. Springer, Heidelberg (2015)

Cameron, D., Collins, E.C., Chua, A., Fernando, S., McAree, O., Martinez-Hernandez,U., Aitken, J.M., Boorman, L., Law, J.: Help! i can’t reach the buttons: facilitatinghelping behaviors towards robots. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A.,Prescott, T.J. (eds.) Living Machines 2015. LNCS, vol. 9222, pp. 354–358. Springer,Heidelberg (2015a)

Cameron, D., Fernando, S., Collins, E., Millings, A., Moore, R., Sharkey, A., Evers, V.,Prescott, T.: Presence of life-like robot expressions influences childrens enjoymentof human-robot interactions in the field. In: Salem, M., Weiss, A., Baxter, P., Daut-enhahn, K. (eds.) 4th International Symposium on New Frontiers in Human-RobotInteraction, Canterbury, UK (2015b)

Cameron, D., Fernando, S., Millings, A., Moore, R., Sharkey, A., Prescott, T.: Chil-dren’s age influences their perceptions of a humanoid robot as being like a personor machine. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A., Prescott, T.J. (eds.)Living Machines 2015. LNCS, vol. 9222, pp. 348–353. Springer, Heidelberg (2015c)

Cameron, D., Fernando, S., Collins, E., Millings, A., Moore, R., Sharkey, A., Prescott,T.: Impact of robot responsiveness and adult involvement on childrens social behav-iours in human-robot interaction. In: Salem, M., Weiss, A., Baxter, P., Dautenhahn,K. (eds.) 5th International Symposium on New Frontiers in Human-Robot Interac-tion, Sheffield, UK (2016a)

Cameron, D., Loh, E., Chua, A., Collins, E., Aitken, J., Law, J.: Robot-stated limita-tions but not intentions promote user assistance. In: Salem, M., Weiss, A., Baxter,P., Dautenhahn, K. (eds.) 5th International Symposium on New Frontiers in Human-Robot Interaction, Sheffield, UK (2016b)

Cameron, D., Millings, A., Davison, D., Reidsma, D., Fernando, S., Prescott, T.: Aframework for affect-led human-robot symbiotic interaction (submitted)

Charisi, V., Davison, D., Wijnen, F., Van Der Meij, J., Reidsma, D., Prescott, T.,Van Joolingen, W., Evers, V.: Towards a child-robot symbiotic co-development: atheoretical approach. In: New Frontiers in Human-Robot Interaction, AISB (2015)

Charisi, V., Davison, D., Reidsma, D., Evers, V.: Evaluation methods for user-centeredchild-robot interaction. In: Proceedings of the 25th IEEE International Symposiumon Robot and Human Interactive Communication, RO-MAN 2016, New York City,United States, August 26-31 (to appear, 2016)

Coleman, E.B., Brown, A.L., Rivkin, I.D.: The effect of instructional explanations onlearning from scientific texts. J. Learn. Sci. 6(4), 347–365 (1997)

Davison, D., Schindler, L., Reidsma, D.: Physical extracurricular activities in educa-tional child-robot interaction. In: Salem, M., Weiss, A., Baxter, P., Dautenhahn, K.(eds.) 5th International Symposium on New Frontiers in Human-Robot Interaction,Sheffield, UK (2016)

Dillenbourg, P.: What do you mean by collaborative learning? Collaborative-Learning:Cognitive and Computational Approaches, Chap. 1, pp. 1–19 (1999)

Dweck, C.: Mindset: how you can fulfill your potential. Constable and Robinson (2012).ISBN: 9781780333939

306 D. Reidsma et al.

Holmes, J.: Designing agents to support learning by explaining. Comput. Educ. 48,523–547 (2007)

Lazzeri, N., Mazzei, D., Zaraki, A., De Rossi, D.: Towards a believable social robot.In: Lepora, N.F., Mura, A., Krapp, H.G., Verschure, P.F.M.J., Prescott, T.J. (eds.)Living Machines 2013. LNCS(LNAI), vol. 8064, pp. 393–395. Springer, Heidelberg(2013). doi:10.1007/978-3-642-39802-5-45

Mazzei, D., Zaraki, A., Lazzeri, N., De Rossi, D.: Recognition and expression of emo-tions by a symbiotic android head. In: 2014 14th IEEE-RAS International Conferenceon Humanoid Robots (Humanoids), pp. 134–139. IEEE (2014)

Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goeln, N., Hanne-mann, M., Motlicek, P., Qian, Y., Schwarz, P., Silovsky, J., Stemmer, G., Vesely,K.: The Kaldi speech recognition toolkit. In: IEEE Workshop on Automatic SpeechRecognition and Understanding (2011). ISBN 978-1-4673-0366-8

ter Maat, M., Heylen, D.: Flipper: An Information State Component for Spoken Dia-logue Systems. In: Vilhjalmsson, H.H., Kopp, S., Marsella, S., Thorisson, K.R. (eds.)IVA 2011. LNCS, vol. 6895, pp. 470–472. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23974-8 67

van Welbergen, H., Reidsma, D., Kopp, S.: An incremental multimodal realizer forbehavior co-articulation and coordination. In: Nakano, Y., Neff, M., Paiva, A.,Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 175–188. Springer, Heidelberg(2012). ISBN: 978-3-642-33196-1, ISSN: 0302-9743

Verschure, P.F.: Distributed adaptive control: a theory of the mind,brain, body nexus. Biol. Inspired Cogn. Architectures 1, 55–72 (2012).doi:http://dx.doi.org/10.1016/j.bica.2012.04.005. ISSN: 2212-683X

Vouloutsi, V., Lallee, S., Verschure, P.F.M.J.: Modulating behaviors using allostaticcontrol. In: Lepora, N.F., Mura, A., Krapp, H.G., Verschure, P.F.M.J., Prescott,T.J. (eds.) Living Machines 2013. LNCS, vol. 8064, pp. 287–298. Springer, Heidelberg(2013)

Vouloutsi, V., Munoz, M., Grechuta, K., Lallee, S., Duff, A., Puigbo, J.Y.L., Verschure,P.F.M.J.: A new biomimetic approach towards educational robotics: the distributedadaptive control of a synthetic tutor assistant. In: Salem, M., Weiss, A., Baxter, P.,Dautenhahn, K. (eds.) 4th International Symposium on New Frontiers in Human-Robot Interaction, Canterbury, UK (2015)

Vouloutsi, V., Blancas, M., Zucca, R., Omedas, P., Reidsma, D., Davison, D.,Charisi, V., Wijnen, F., van der Meij, J., Evers, V., Cameron, D., Fernando, S.,Moore, R., Prescott, T., Mazzei, D., Pieroni, M., Cominelli, L., Garofalo, R., Rossi,D.D., Verschure, P.F.: Towards a synthetic tutor assistant: the EASEL project and itsarchitecture. In: Proceedings of the 5th International Conference on Biomimetic andBiohybrid Systems, Living Machines 2016, Edinburgh, Scotland, July 2016 (2016)

Vygotsky, L.: Mind in Society: The Development of Higher Psychological Processes.Harvard University Press, Cambridge (1978). ISBN: 0674076680

Zaraki, A., Mazzei, D., Giuliani, M., De Rossi, D.: Designing and evaluating a socialgaze-control system for a humanoid robot. IEEE Trans. Hum. Mach. Syst. 44(2),157–168 (2014)

Wasp-Inspired Needle Insertion with Low Net Push Force

Tim Sprang, Paul Breedveld, and Dimitra Dodou(✉)

Department of BioMechanical Engineering, Faculty of Mechanical,Maritime and Materials Engineering, Delft University of Technology,

Delft, the Netherlandsd.dodou@tudelft.nl

Abstract. This paper outlines the development of a four-part needle prototypeinspired by the ovipositor of parasitic wasps. In the wasp ovipositor, three longi‐tudinal segments called valves move reciprocally to gain depth in the substrate.It has been suggested that serrations located along the wasp ovipositor induce afriction difference between moving and anchoring valves that is needed for thisreciprocal motion. Such an anchoring mechanism may not be desired in a medicalsetting, as serrations can induce tissue damage. Our aim was to investigatewhether a multipart needle can penetrate tissue phantom material with near-zeronet push force while using needle parts devoid of surface gripping textures orserrations. Accordingly, a four-part needle prototype was developed and testedin gelatine substrates. The performance of the prototype was assessed in terms ofthe degree of slipping of the needle with respect to the gelatine, with less slipimplying better performance. Slip decreased with decreasing gelatine concentra‐tion and increasing offset between the needle parts. Motion through gelatine wasachieved with a maximum push force of 0.035 N. This study indicates the possi‐bility of needle propagation into a substrate with low net push force and withoutthe need of serrations on the needle surface.

Keywords: Percutaneous interventions · Wasp ovipositor · Biomimetics

1 Introduction

1.1 Percutaneous Needles: A Brief State-of-the-art

Blood sampling, biopsies, regional anaesthesia, neurosurgery, and brachytherapy, allrely on percutaneous needles [1]. Most needles used in these procedures are rigid andfollow straight trajectories. Accessibility of targets located deep inside the body isinhibited, and deviations from the desired path due to organ deformation are common[2]. These functional limitations hamper medical treatment when designated areascannot be reached [2] and compromise safety when undesired areas are penetrated [3].

Several flexible steerable needles have been developed in an effort to overcome thepath planning limitations encountered with rigid straight needles (e.g., [2, 4–9]). Someof these newly developed needles rely on concentric axial insertion of multiple pre-bentneedle parts, whereas others rely on reaction forces from the tissue to control the steeringcurvature (for reviews see [10, 11]). One drawback of flexible needles is that the axial

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 307–318, 2016.DOI: 10.1007/978-3-319-42417-0_28

load applied at the back of a needle increases with resistive forces on the needle tip andshaft when penetrating deeper into the tissue. The tissue can only support this axial loadup to a certain penetration depth; further needle advancement can lead to buckling ofthe needle and/or lateral slicing and damage of the tissue.

1.2 The Wasp Ovipositor and Biologically Inspired Steerable Needles

The wasp ovipositor is a needle-like structure, with no musculature. The ovipositorextends from the last abdominal segments of the wasp and serves the main purpose ofegg deposition. The ovipositor consists of three longitudinal segments called valves:two ventral valves and one dorsal valve. The dorsal valve is connected to the two ventralvalves along its length by means of a tongue-and-groove mechanism that allows forrelative movement of the valves in the direction of the ovipositor while preventing theirseparation. Abdominal musculature actuates the valves independently from each other[12, 13]. The wasp penetrates the substrate by antagonistically moving the ventral valveswhile the dorsal valve acts as sliding support. Serrations at the ovipositor tip are likelyto allow the valves anchor against the substrate (Fig. 1). It has been hypothesized thatthe wasp applies a pull force on the anchored valve, allowing to load the penetratingvalve with a push force higher than its critical buckling load [14, 15].

Fig. 1. Schematic representation of the hypothesized working principle of penetration of a waspovipositor (left). The SEM image (right) shows the tip of the ovipositor of Megarhyssa nortoninortoni, an Ichneumonoid parasitoid wasp (figures from [14]).

The wasp ovipositor has been used as an inspiration source for designing steerablepercutaneous needles [3, 15–19]. For example, Oldfield et al. [18] described a 6-mmdiameter four-part needle prototype with reciprocally moving segments. Tissue trans‐versal with no net axial force by means of microstructures on the needle surface hasbeen reported in [6, 16, 19]. Similarly, inspired by insertion methods observed inmosquito proboscis, Aoyagi et al. [20] indicated that substrate penetration with a micro‐needle comprising serrated stylets is possible without applying a net push force (see also[21] for a description of buckling prevention in mosquitos and wasps).

1.3 Goal of this Study

It has been suggested that the serrations located on the wasp ovipositor induce a frictiondifference between moving and anchoring valves [19]. Serrations may not be desired in

308 T. Sprang et al.

medical settings, as they can induce tissue damage. Our aim was to investigate whethera multipart needle can penetrate tissue phantom material with low net push force whilebeing devoid of any surface gripping textures or serrations.

2 Design of a Multipart Needle Prototype

A multipart needle prototype was developed, comprising a low-friction cart thatsuspended a four-part needle assembly, four linear actuators, and driving electronics tomove a needle assembly back and forth (Fig. 2). The needle assembly consisted of fourstainless steel square needle parts and a stainless steel guide tube. The needle assemblyhad a total diameter of 2 mm (cf. diameter range of common hypodermic needles: 0.2–4.3 mm [22]) and a length of 200 mm (in line with the penetration depths in [6]). Theneedle tip was conical (angle about 20o) and sharp. The needle parts had a smooth surfacefinish (Ra = 0.2); here we differentiate from [19] where saw-tooth microstructures wereused to induce friction.

1 2 & 3

Fig. 2. Image of the prototype, showing the needle (1), needle assembly (2), linear actuatorassembly (3), and the low friction cart with driving electronics (4).

Each needle part ended in a fin mounted to a leadscrew, each leadscrew being trans‐lated back and forth by a linear stepper motor (resolution: 1.8o, corresponding to0.0061 mm/step of leadscrew translation). The motors were mounted in an aluminiumbracket. Two alignment pins, extending from the bottom of the bracket, aligned theneedle and actuator assembly with the low-friction cart. The cart was supported by twoaxes, each containing two ball bearings with metal shields (624-ZZ, 4 × 13 × 5 mm).

Four stepper drivers were mounted behind the linear actuator module. The driverswere controlled with a microcontroller. A two-row LCD display showed a timer andcycle counter during actuation, and settings when standby. The settings were controlledby two push buttons and a rotary encoder. The system was battery powered.

The embedded code was written in C++ including Arduino libraries. The softwareconsisted of a user interface part containing a rotational encoder with a push button, a

Wasp-Inspired Needle Insertion with Low Net Push Force 309

green button, a red button, and the display driver, and an embedded part containing theregister, stepper sequence controller, stepper controller, and stepper driver.

Needle penetration was realized in cycles, each of which consisted of a protrusionand a retraction phase. In the protrusion phase the needle parts were protruded one-by-one (or two-by-two, depending on the sequence setting) with respect to the cart over adistance equal to a predefined offset. In the retraction phase the needle parts wereretracted altogether with respect to the cart, over a distance equal to the offset.

3 Hypotheses

In a four-part needle, zero net push force penetration relying on a friction difference dueto a difference between the number of stationary and protruding needle parts wouldrequire the advancement of one needle part at a time. Previous measurements in porcineliver showed that friction force on a needle with a diameter of 1.27 mm increased linearlywith penetration depth, whereas cutting forces remained about constant along the pene‐tration depth [23, 24]. It can be therefore expected that, when protruding one needle partat a time, an equilibrium between the friction force on the stationary needle parts andthe resistive force (i.e., sum of cutting force and friction force ) on the protrudingneedle part is reached at a certain penetration depth:

(1)

From this depth onwards, penetration with zero net push force is theoretically possible,since the friction force on the stationary needle parts is higher than the resistive forceon the protruding needle part.

The performance of the prototype was assessed in terms of the degree of slipping ofthe needle with respect to the gelatine, with less slip implying better performance. Thefollowing hypotheses were investigated:

• Protrusion sequence: When moving one needle part at the time, performance is notexpected to be affected by the protrusion sequence, as the surface area differencebetween moving and stationary needle parts is independent from the protrusionsequence. A two-by-two actuation is expected to result in no penetration, as in thatcase the friction force on the stationary needle parts is smaller than the resistive forceson the protruding needle parts.

• Needle-part velocity: Friction force increases with insertion velocity more steeplythan the corresponding cutting force [25]. Following Eq. 1, it is thus expected thatthe needle performs better at higher velocities.

• Gelatine concentration: In very low gelatine concentrations no penetration isexpected, as there is no sufficient needle-substrate friction to overcome the bearingfriction and inertia of the car. Resistive forces on the protruding needle parts are likelyto also be dominant in high concentrations, leading to poor performance. An optimumin gelatine concentration is thus expected for best performance.

310 T. Sprang et al.

• Offset: The friction force on the protruding needle part is expected to be higher fora larger offset, due to the larger surface area compared to the area for lower offsets.

4 Methods

Two experiments were conducted. In the first experiment (called henceforth prototypeexperiment), the effect of gelatine concentration, needle-part offset, needle-part velocity,and needle-part sequence on the performance of the prototype was measured. In thesecond experiment (force experiment), force measurements were conducted during theinsertion of the needle into gelatine to estimate the relation between cutting and frictionforce as a function of needle velocity and gelatine concentration.

4.1 Experimental Setup

Prototype Experiment. A platform consisting of an aluminium frame and a500 × 500 × 8 mm acrylic plate formed a level surface for the low-friction cart to travelback and forth between a proximity sensor and a gelatine sample. Adjustment screwsran through river nuts on the four corners of the frame for levelling. Gelatine substrateswere produced in 434 × 344 × 107-mm trays. One panel of each tray had a 20 × 2 gridof 15-mm diameter holes, each 40 mm apart. The tray was set to height with spacers,and oriented so that the panel with holes was positioned against the platform. A laserproximity sensor (Micro-Epsilon optoNCDT1302-200; range: 200 mm; resolution:0.1 mm) measured the travelled distance of the prototype. The sensor was mounted ona bracket and placed against the platform to ensure a constant distance between thesensor and gelatine tray. Sensor data were sampled at 50 Hz using LabVIEW 2010 anda National Instruments NI USB-6210 16-bit data acquisition system.

Force Experiment. The four needle parts and guide tube were clamped to a forcesensor assembly with the tip directed downwards. The needle was translated by meansof a linear stage. The axial force was measured during the insertion and retractionmovement of the needle in a gelatine container positioned underneath the needle. Theforce sensor assembly consisted of a mechanical decoupler holding a force transducer.The sampling rate was set at 1 kHz. The decoupler assembly was statically calibratedwith balance weights ranging between 0.25 and 1.0 kg in steps of 0.25 kg.

4.2 Variables

Independent Variables. The following variables were manipulated:

• Needle-part offset [mm]: The length of the protruding part of the moving needle partsrelative to the stationary needle parts varied between 3, 10, and 20 mm.

• Needle-part velocity [mm/s]: The protrusion and retraction velocity of the needleparts with respect to the cart, varied between 4, 8, and 13.5 mm/s, where 13.5 mm/sis the maximum velocity that the linear actuators were able to generate.

Wasp-Inspired Needle Insertion with Low Net Push Force 311

• Gelatine concentration [wt%]: The % weight of gelatine powder in water, variedbetween 3, 8, and 13 wt%, to simulate a variety of soft tissues, from brain and fat tobreast and liver tissue [26, 27].

• Needle-part sequence [no units]: The sequence of protruding needle parts, variedbetween circular, diagonal, and a two-by-two manner of actuation.

Dependent Variables (Measured). The following variables were measured:

• Sa [mm]: The distance travelled by the cart (prototype experiment).• Number of cycles (Nc) [no units]: The number of cycles performed by the cart to travel

a given distance (prototype experiment).• Fi [N]: The axial force acting on the needle during insertion, measured with the force

transducer. Fi represents the sum of the needle cutting force and the needle-substratefriction force (force experiment).

• Ff [N]: The axial force acting on the needle during retraction. Ff represents the needle-substrate friction force (force experiment).

Dependent Variables (Derived). The following variables were calculated:

• St [mm]: The number of cycles multiplied by the offset representing the theoreticaldistance that the cart would travel if there were zero slip (prototype experiment).

• Slippro [mm]: The slip during protrusion, equalling the backwards travelled distanceof the cart during the protrusion of the needle in one cycle (prototype experiment).

• Slipret [mm]: The slip during retraction, equalling the difference between the offsetand the actual travelled distance of the cart during retraction (prototype experiment).

• SRtot [no units]: The slip ratio over an entire measurement, defined as SRtot = 1 − Sa/St. Less slip means less dependence of the needle penetration on the net axial pushforce on the needle (prototype experiment).

• SRpro [no units]: The slip ratio in the protrusion phase, defined as: SRpro = Slippro/Offset (prototype experiment).

• SRret [no units]: The slip ratio in the retraction phase, defined as: SRret = Slipret/Offset(prototype experiment).

• Fc [N]: The cutting force defined as the axial force on the needle as a result of thesubtraction of the retraction force from the insertion force (force experiment).

• Depth of equilibrium [mm]: The depth at which Fc and Ff are in equilibrium accordingto Eq. 1 (force experiment).

4.3 Experimental Procedure

Prototype Experiment. Gelatine was prepared in trays one day in advance of theexperiment. The needle was positioned in front of the allocated hole of the tray and amachinist square was used to align the cart perpendicular to the platform. Before thestart of each measurement, the needle was manually pushed 35 mm into the gelatine(this initial depth was defined in pilot measurements as the minimum distance for whichneedle advancement was possible; for smaller distances, the contact area of the needlewith the gel was not sufficient and the needle slipped instead of advancing). Next, the

312 T. Sprang et al.

proximity sensor was placed against the base of the platform, aligned with the cart, andswitched on. About 1 s later, the actuation of the needle parts started and continued untilthe needle had travelled about 120 mm inside the gelatine.

All measurements were performed over a time span of one day. The four independentvariables were varied from a common baseline, see Table 1. Measurements wererepeated five times for each combination of parameters. Measurement allocation to thetray holes was done quasi-randomly, so that each set of five measurements in 8 wt%gelatine was distributed over the upper and lower rows of three trays. All measurementsin 3 wt% and 13 wt% gelatine were performed in a single tray and distributed randomlybetween the upper and lower rows of the tray.

Force Experiment. Gelatine samples were prepared one day in advance of the experi‐ment. A drilling spot along the edge of the gelatine container at least 30 mm apart fromother drillings and from the edge of the container was chosen and the needle was posi‐tioned 1 mm above the gelatine. The waiting time between needle insertion and retrac‐tion was set to 3 s, to allow the gelatine around the needle to settle.

All force measurements were performed over a time span of one day. Needle velocity(4, 8, and 13.5 mm/s) and gelatine concentration (3, 8, and 13 % wt) were varied in afully crossed manner. Each combination of parameters was repeated five times. Temper‐ature of the gelatine was kept between 4 and 8 Co in both experiments, as the insertionforce of a needle in gelatine decreases for temperatures above 8°C [28].

4.4 Data Processing

Prototype Experiment. The raw signal of the cart position was filtered using a movingaverage filter over five samples. Sa was calculated by subtracting the mean value of thelast 10 data samples from the first 10 data samples. Nc was determined by the numberof local maxima in the signal. Slippro was determined by subtracting the cart positionafter the protrusion phase of a given cycle from the cart position before the protrusionphase of the cycle. Slipret was determined by subtracting the difference in cart positionbefore and after the retraction phase from the offset.

Force Experiment. The raw force signal was filtered using a moving average filterover 100 samples and shifted so that the first 1,000 data points (i.e., 1 s) averaged atzero. The retraction phase was defined as the part of the curve from maximum retractionto substrate exit. The protrusion phase was defined as the part of the curve until maximumprotrusion, with the same length as the retraction part. The protrusion and retractionforce profiles were aligned at their maximum force value. The resultant of the retraction

Table 1. Parameter variation in prototype experiment.

Sequence [no unit] Offset [mm] Gelatine concentration [%wt] Velocity [mm/s]Circular 3 3 4Diagonal 10 8 8Two-by-two 20 13 13.5

Note. The baseline condition is annotated in bold.

Wasp-Inspired Needle Insertion with Low Net Push Force 313

and protrusion force is the cutting force. The depth at which cutting and friction forceswere in equilibrium (Eq. 1) was determined by the intersection between a linearly fittedretraction force and the mean cutting force.

4.5 Statistical Analysis

For the prototype experiment, a one-way ANOVA was performed to compare the means ofSRpro, SRret, and SRtot as a function of the parameters in Table 1. For the force experiment, aone-way ANOVA was performed to compare the means of the maximum friction andaverage cutting force as a function of needle velocity and gelatine concentration. Data anal‐ysis was performed in MATLAB R2013a (The MathWorks, Inc., Natick, Massachusetts).

5 Results

5.1 Prototype Experiment

SRtot was not significantly different between circular and diagonal actuation in 8 wt%gelatine at 4 mm/s (F(1,8) = 0.56, p = .476; Fig. 3). Two-by-two actuation of needleparts resulted in no penetration at 4 mm/s and in penetration with a SRtot higher than 0.9at 13.5 mm/s. A 4-mm offset was associated with a higher SRtot as compared to SRtot for10-mm and 20-mm offsets (F(2,12) = 160.54, p < .001; the latter two also being signif‐icantly different from each other; post-hoc paired t-test: p = .009). SRtot increased withgelatine concentration (F(2,12) = 154.71, p = 2.70.10−9). SRtot did not differ betweenthe three tested velocities (F(2,12) = 1.43, p = .278). SRpro decreased with offset andincreased with gelatine concentration, in line with the effects observed for SRtot, whereasSRret decreased with offset and gelatine concentration.

Circular Diagonal

SRtot[-]

0

0.2

0.4

0.6

0.8

4 mm 10 mm20 mm0

0.2

0.4

0.6

0.8

3 wt% 8 wt%13 wt%0

0.2

0.4

0.6

0.8

4 mm/s 8 mm/s 13 mm/s0

0.2

0.4

0.6

0.8

Circular Diagonal

S Rpro

[-]

0

0.2

0.4

0.6

0.8

4 mm 10 mm20 mm0

0.2

0.4

0.6

0.8

3 wt% 8 wt%13 wt%0

0.2

0.4

0.6

0.8

4 mm/s 8 mm/s 13 mm/s0

0.2

0.4

0.6

0.8

SequenceCircular Diagonal

SRret[-]

0

0.2

0.4

0.6

Offset4 mm 10 mm20 mm

0

0.2

0.4

0.6

Substrate3 wt% 8 wt%13 wt%

0

0.2

0.4

0.6

Velocity4 mm/s 8 mm/s 13 mm/s

0

0.2

0.4

0.6

Fig. 3. Prototype experiment. Slip ratio as a function of each of the independent variables (fromleft to right: sequence, offset, gelatine concentration, velocity) (top row) per cycle (SRtot); (middlerow) in the protrusion phase (SRpro); and (bottom row) in the retraction phase (SRret).

314 T. Sprang et al.

5.2 Force Experiment

The maximum friction force increased with needle velocity for all three gelatine concen‐trations (3 wt%: F(2,12) = 17.6, p < .001; 8 wt%: F(2,12) = 10.33, p = .003; 13 wt%:F(2,12) = 47.83, p = < .001). Also the average cutting force increased with needlevelocity (3 wt%: F(2,12) = 40.62, p < .001; 8 wt%: F(2,12) = 131.3, p < .001; 13 wt%:F(2,12) = 50.91, p < .001). The equilibrium depth increased with velocity and decreasedwith gelatine concentration.

6 Discussion

We tested the principle of needle advancement through tissue phantom material withlow net push force while using a four-part needle devoid of surface gripping textures.We found that slip ratio decreased with decreasing gelatine concentration and increasingoffset. Friction force increased with velocity, gelatine concentration, and penetrationdepth, whereas cutting force increased with velocity and gelatine concentration, andremained constant with penetration depth.

6.1 Main Findings

Gelatine Concentration. We found a proportional relation between SRtot and gelatineconcentration, which is not in line with our hypothesis and Eq. 1 but is consistent withParittotokkaporn et al. (2009) who reported substrate traversal with slip in 6 wt% gelatineand no substrate traversal at all in 8 wt% gelatine. A parameter possibly contributing tothe observed proportional relation is the elastic deformation of the gelatine: when allfour needle parts are protruded, the cart moves slightly backwards due to the gelatinespringing back. It is possible that this effect increases with gelatine concentration, asfriction increases with gelatine concentration more than elasticity does [29–31]. Cartinertia and bearing friction possibly also contribute to the observed proportional relationbetween SRtot and gelatine concentration. Specifically, the resistance against movementof the cart may act as a pushing force on the protruding needle part, preventing the cartfrom moving backwards. Cutting force increases with gelatine concentration (in linewith [25, 28, 30]), leaving us to believe that the effect of cart inertia and bearing frictionon Slippro decreases with gelatine concentration.

Offset. Whereas the absolute amount of slip per cycle was proportional to the offset inagreement with our hypothesis, the total amount of slip decreased with offset, meaningthat the best overall performance was achieved with the largest offset. The elastic defor‐mation described in the previous paragraph could contribute to the relatively high SRtotfor small offsets. As the absolute effect of elasticity should be equal for the three offsets,the relative contribution to the SRpro is higher for small offsets. This is supported by anexploratory test with an offset of 1 mm, which resulted in no penetration at all, possiblybecause in that case only elastic deformation was achieved.

Sequence. No performance difference between one-by-one circular and diagonalactuation was found, which is expected, as the surface area subjected to resistive forces

Wasp-Inspired Needle Insertion with Low Net Push Force 315

is independent from the actuation order of the needle parts. Two-by-two actuationyielded no penetration at 4 mm/s, suggesting that the difference in friction betweenprotruding and stationary needle parts is the dominant factor for needle penetration.Two-by-two actuation at 13.5 mm/s resulted in penetration with a slip ratio over 0.9,probably due to cart inertia: due to the relatively large mass of the cart, the protrudingneedle part accelerated forwards faster than the cart was able to accelerate backwards.

Velocity. No slip reduction was observed for increasing needle-part velocity, contra‐dicting our hypothesis. Force measurements indicated that the cutting force increasedmore steeply with needle velocity than the friction force did, opposite to [25]. Thedifference in maximum acceleration for different velocities in the prototype experimentmakes the validation of the performance results difficult. Future experiments withvarying velocity require a trade-off between maximum acceleration and accelerationtime to reach a certain needle-part velocity, as a reduction of acceleration results in alonger acceleration time, whereas for high acceleration differences between velocitysettings, inertia confounds the effect of velocity on performance.

6.2 Limitations

Cart inertia and bearing friction possibly prevented slip in the protrusion phase in whichthe resultant force on the cart was directed out of the gelatine. The cart weighed 0.73 kgand accelerated with a maximum of 27 mm/s2 in the 4-mm/s velocity setting, thereforerequiring a force of about 0.02 N to overcome its inertia. To overcome bearing friction,a force of about 0.015 N would be needed (assuming a friction coefficient of 0.002 forlubricated ball bearings; [32]). Thus, in the protrusion phase, where the resultant forceon the needle is likely to be lower than 0.035 N, an effect of inertia and bearing frictioncannot be ruled out.

To validate the method used to define friction and cutting force, supplementarymeasurements were conducted, in which friction force was measured when the entireneedle was inserted 5 times in the same hole in 3, 8, and 13 wt% gelatine. Results showedthat in all three concentrations the retraction force declined after the first insertion byabout 10 %. These results indicate that friction force both in our force experiments andin the literature is generally underestimated and therefore the cutting force is overesti‐mated, due to the fact that the retraction (thereby friction) force is measured after thehole is already made during the insertion movement of the needle.

In this work, we only focused on straight trajectories. Frasson et al. [7] showed thatsteering with a multipart needle can be achieved by controlling the relative offsetbetween needle parts. Asymmetric forces acting from the substrate on the (bevelled) tipcause the needle to cut a curved trajectory. Wasps are also capable of bending theirovipositors in several ways (e.g., [33]). Further research is required to find the mostsuitable way of steering a multipart needle.

Acknowledgements. This research is supported by the Dutch Technology Foundation STW,which is part of the Netherlands Organization for Scientific Research (NWO), and which is partlyfunded by the Ministry of Economic Affairs (project 12712, STW Perspectief Program iMIT-Instruments for Minimally Invasive Techniques).

316 T. Sprang et al.

References

1. Abolhassani, N., Patel, R., Moallem, M.: Needle insertion into soft tissue: a survey. Med.Eng. Phys. 29, 413–431 (2007)

2. Misra, S., Reed, K.B., Schafer, B.W., Ramesh, K.T., Okamura, A.M.: Observations andmodels for needle-tissue interactions. In: IEEE International Conference on Robotics andAutomation, pp. 2687–2692 (2009)

3. Frasson, L., Parittotokkaporn, T., Davies, B.L., Rodriguez y Baena, F.: Early developmentsof a novel smart actuator inspired by nature. In: 15th International Conference onMechatronics and Machine Vision in Practice, pp. 163–168 (2008)

4. Ebrahimi, R., Okazawa, S., Rohling, R., Salcudean, S.E.: Hand-held steerable needle device.In: Ellis, R.E., Peters, T.M. (eds.) MICCAI 2003. LNCS, vol. 2879, pp. 223–230. Springer,Heidelberg (2003)

5. Engh, J.A., Podnar, G., Khoo, S.Y., Riviere, C.N.: Flexible needle steering system forpercutaneous access to deep zones of the brain. In: IEEE 32nd Annual Northeast Conferenceon Bioengineering, pp. 103–104 (2006)

6. Frasson, L., Ko, S.Y., Turner, A., Parittotokkaporn, T., Vincent, J.F., Rodriguez y Baena, F.:STING: a soft-tissue intervention and neurosurgical guide to access deep brain lesionsthrough curved trajectories. J. Eng. Med. 224, 775–788 (2010)

7. Frasson, L., Ferroni, F., Ko, S.Y., Dogangil, G., Rodriguez y Baena, F.: Experimentalevaluation of a novel steerable probe with a programmable bevel tip inspired by nature. J.Robot. Surg. 6, 189–197 (2012)

8. Walsh, C.J., Franklin, J.C., Slocum, A., Gupta, R.: Material selection and force requirementsfor the use of pre curved needles in distal tip manipulation mechanisms. In: Design of MedicalDevices Conference, 3913 (2010).

9. Webster, R.J., Okamura, A.M., Cowan, N.J.: Toward active cannulas: miniature snake-likesurgical robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems,pp. 2857–2863 (2006)

10. Scali, M., Pusch, T.P., Breedveld, P., Dodou, D.: Needle-Like Instruments for SteeringThrough Solid Organs: a Review of the Scientific and Patent Literature (submitted)

11. Van de Berg, N.J., Van Gerwen, D.J., Dankelman, J., Van den Dobbelsteen, J.J.: Designchoices in needle steering—a review. IEEE-ASME T. Mech. 20, 2172–2183 (2015)

12. Rahman, M.H., Fitton, M.G., Quicke, D.L.: Ovipositor internal microsculpture in thebraconidae (insecta, hymenoptera). Zool. Scr. 27, 319–332 (1998)

13. Scudder, G.G.E.: Comparative morphology of insect genitalia. Annu. Rev. Entomol. 16, 379–406 (1971)

14. Vincent, J.F.V., King, M.J.: The mechanism of drilling by wood wasp ovipositors.Biomimetics 3, 187–201 (1995)

15. Burrows, C., Secoli, R., Rodriguez y Baena, F.: Experimental characterisation of abiologically inspired 3D steering needle. In: 13th IEEE International Conference on Control,Automation and Systems, pp. 1252–1257 (2013)

16. Frasson, L., Parittotokkaporn, T., Schneider, A., Davies, B.L., Vincent, J.F.V., Huq, S.E.,Degenaar, P., Rodriguez y Baena, F.R.: Biologically inspired microtexturing: investigation into thesurface topography of next-generation neurosurgical probes. In: 30th IEEE Annual InternationalConference of the Engineering in Medicine and Biology Society, pp. 5611–5614 (2008)

17. Leibinger, A., Oldfield, M., Rodriguez y Baena, F.R.: Multi-objective design optimizationfor a steerable needle for soft tissue surgery. In: 15th International Conference on BiomedicalEngineering, pp. 420–423 (2014)

Wasp-Inspired Needle Insertion with Low Net Push Force 317

18. Oldfield, M.J., Burrows, C., Kerl, J., Frasson, L., Parittotokkaporn, T., Beyrau, F., Rodriguezy Baena, F.: Highly resolved strain imaging during needle insertion: results with a novelbiologically inspired device. J. Mech. Beh. Biomed. Mat. 30, 50–60 (2014)

19. Parittotokkaporn, T., Frasson, L., Schneider, A., Huq, E., Davies, B.L., Degenaar, P.,Biesenack, J., Rodriguez y Baena, F.R.: Soft tissue traversal with zero net push force:feasibility study of a biologically inspired design based on reciprocal motion. In: IEEEInternational Conference on Robotics and Biomimetics, pp. 80–85 (2009)

20. Aoyagi, S., Takaoki, Y., Takayanagi, H., Huang, C.-h., Tanaka, T., Suzuki, M., Takahashi,T., Kanzaki, T., Matsumoto, T.: Equivalent negative stiffness mechanism using three bundledneedles inspired by mosquito for achieving easy insertion. In: 2012 IEEE/RSJ InternationalConference on Intelligent Robots and Systems, pp. 2295–2300 (2012)

21. Sakes, A., Dodou, D., Breedveld, P.: Buckling prevention strategies in nature as inspiration forimproving percutaneous instruments: a review. Bioinsp. Biomim. 11, 021001-1–021001-26(2016)

22. Syringe Needle Gauge Chart (2014). http://www.sigmaaldrich.com/chemistry/stockroom-reagents/learning-center/technical-library/needle-gauge-chart.html

23. Hing, J.T., Brooks, A.D., Desai, J.P.: Reality-based needle insertion simulation for hapticfeedback in prostate brachytherapy. IEEE International Conference on Robotics andAutomation, pp. 619–624 (2006)

24. Simone, C., Okamura, A.M.: Modeling of needle insertion forces for robot-assistedpercutaneous therapy. In: IEEE International Conference on Robotics and Automation, vol.2, pp. 2085–2091 (2002)

25. Van Gerwen, D.J., Dankelman, J., Van den Dobbelsteen, J.J.: Needle-tissue interactionforces-a survey of experimental data. Med. Eng. Phys. 34, 665–680 (2012)

26. Nguyen, M.M., Zhou, S., Robert, J.L., Shamdasani, V., Xie, H.: Development of oil-in-gelatinphantoms for viscoelasticity measurement in ultrasound shear wave elastography. UltrasoundMed. Biol. 40, 168–176 (2014)

27. Moreira, P., Peterlik, I., Herink, M., Duriez, C., Cotin, S., Misra, S.: Modelling prostatedeformation: SOFA versus experiments. Mech. Eng. Res. 3, 64–72 (2013)

28. Ng, K.W., Goh, J.Q., Foo, S.L., Ting, P.H., Lee, T.K.: Needle insertion forces studies foroptimal surgical modeling. Int. J. Bioscience Biochem. Bioinformatics 3, 187–191 (2013)

29. Zhang, X., Qiang, B., Greenleaf, J.: Comparison of the surface wave method and theindentation method for measuring the elasticity of gelatine phantoms of differentconcentrations. Ultrasonics 51, 157–164 (2011)

30. De Lorenzo, D.: Force Sensing and Display in Robotic Driven Needles for Minimally InvasiveSurgery. Doctoral dissertation, Politecnico di Milano, Milano (2012)

31. Crouch, J.R., Schneider, C.M., Wainer, J., Okamura, A.M.: A velocity-dependent model forneedle insertion in soft tissue. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol.3750, pp. 624–632. Springer, Heidelberg (2005)

32. Van Beek, A.: Advanced Engineering Design: Lifetime Performance and Reliability. DelftUniversity of Technology, Delft (2006)

33. Quicke, D.L.: Ovipositor mechanics of the braconine wasp genus zaglyptogastra and theichneumonid genus pristomerus. J. Nat. Hist. 25, 971–977 (1991)

318 T. Sprang et al.

Use of Bifocal Objective Lens and ScanningMotion in Robotic Imaging Systemsfor Simultaneous Peripheral and HighResolution Observation of Objects

Gasper Skulj(B) and Drago Bracun

Faculty of Mechanical Engineering, University of Ljubljana,1000 Askerceva 6, Ljubljana, Slovenia

{gasper.skulj,drago.bracun}@fs.uni-lj.sihttp://www.fs.uni-lj.si/

Abstract. Imaging systems are widely used in robotic systems to detectfeatures of their surroundings and their position in order to guide therobot’s motion. In the paper, a variant of an artificial imaging systembased on the unique bifocal eye of sunburst diving beetle (Thermonec-tus marmoratus) larvae is proposed. The biologically inspired imagingsystem of a single sensor and a coaxial lens form a superposition of twofocused narrow and wide view angle images. The output image contains ahigh resolution area of interest and its periphery. The scanning motion ofthe bifocal imaging system is also imitated and provides positional rela-tions between objects. Acquired images are used for distance assessment.The intended use of the proposed imaging system is in a guidance systemof an autonomously moving robot with biologically inspired locomotion.

Keywords: Biomimetics · Imaging system · Bifocal lens · Scanningmotion · Image disparity

1 Introduction

Imaging systems are useful for autonomous robots in order for them to observetheir surroundings and navigate around obstacles. An advantage in comparisonto other sensors is that imaging systems can work on considerable distance andproduce an output with plenty of information. Robotic imaging systems need tobe able to observe region of interest with high resolution and wide surroundingarea with lower resolution. This is usually realised with complex combinationof several imaging systems with different view angles. To reduce the complexityof autonomous robots new design of an imaging system is needed that providesthe desired capabilities without the use for numerous cameras or special costlyoptical components.

Nature provides us with many examples of different eye designs. The mam-malian eye for example has an adaptable deformable lens that focuses the imagec© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 319–328, 2016.DOI: 10.1007/978-3-319-42417-0 29

320 G. Skulj and D. Bracun

on the retina on the curved surface on the back of the eye. Mimicking the mam-malian eye design with standard optical components is problematic because arti-ficial lenses are rigid and image sensors are flat with uniform pixel density [1,2].Insects, on the other hand, usually have a rigid compound eye design that is alsostudied and imitated, but again has to be created with non-standard opticalparts [3,4].

A possible inspiration that could solve the need for simultaneous high res-olution observation of an area of interest and a peripheral area is the recentlydiscovered bifocal eye design of sunburst diving beetle larvae. The larvae frontaleyes have a fully featured bifocal function. The lenses are rigid and because ofasymmetry focus the two images onto distinct distal and proximal retinas. Thelarvae scan their surroundings with these principal eyes by oscillating their headsdorso-ventrally as they approach potential prey [5,6]. Based on the sunburst div-ing beetle larvae eye and the idea of two images in one eye a robotic imagingsystem design of standard optical components is proposed.

2 Imaging System Design

The proposed imaging system (Fig. 1) that mimics the larvae bifocal eye functionsis composed of a Galilean telescope that is mounted in front of a machine visioncamera lens. The Galilean telescope is assembled from a convex lens 1 and con-cave lens 2. The telescope diameter is smaller than the camera lens diameter. Thisway the light can pass through the telescope and further on through the cameralens as it is marked with A light rays in Fig. 1. However, the light can also passbesides the telescope directly to the camera lens, as it is marked with B light rays.A combination of A and B light paths forms a bifocal optical system, that createstwo images with a different magnification on the same image detector. The lightpath A has a greater magnification because of the Galilean telescope.

Fig. 1. Bifocal optical system configuration with a fully opened iris. The Galileantelescope (1 and 2), camera lens (3) with iris (4), image detector (5).

Use of Bifocal Objective Lens and Scanning Motion 321

Let us denote the light path A as the central vision and light path B as theperipheral vision. Switching between these two systems is carried out by changingthe iris opening in the camera lens. If the iris is fully opened (Fig. 1), the lightpasses through both optical systems to the image detector and two differentlymagnified images overlap. Consequently the complete imaging system has twodifferent view angles. If the iris is fully closed as is demonstrated in Fig. 2, theperipheral vision is switched off.

Fig. 2. Bifocal optical system configuration with fully closed iris. The Galilean tele-scope (1 and 2), camera lens (3) with iris (4), image detector (5).

In order to avoid overlapping problem at the fully opened iris, and to achieve asmooth transition between only the central or only the peripheral or both images,a suitable ratio of telescope and camera lens diameters should be selected. Thecamera lens should have a much greater light power capacity then the telescope.This way, when the iris is fully opened, the peripheral image is much brighterthan the central image. Consequently the image detector acquires mostly periph-eral image. The central image is in this case barely noticeable. If the iris is pre-cisely tuned, both equally bright images can be acquired. However, if the irisis fully closed, the peripheral image vanishes and only the central narrow angleimage remains on the image detector. The telescope magnification defines themagnification of the central image.

As a possibility, a similar imaging system can also be achieved if the telescopeis reversed. In this case the light first encounters the concave lens 2 and thenthe convex lens 1 of the telescope. The complete magnification of the light pathA is therefore smaller than the one of the light path B. The light path A thusbecomes the wide angle peripheral vision and the light path B the narrow anglecentral vision. The iris effect is consequently also turned around. When the irisis fully closed only the peripheral wide angle view remains visible.

The difference between the proposed imaging system and the larvae bifocaleye in terms of the visual function are image A & B alignment and the numberof image sensors. The larvae eye has two distinct retinas that correspond to two

322 G. Skulj and D. Bracun

image sensors where images are focused. The proposed imaging sensor uses onlya single image sensor. The larvae eye also produces images A and B that have anoffset equal to the distance between the two retinas. Images A and B produced inthe proposed system are centrally aligned and completely overlap. The offset ofthe two images could be achieved by offsetting the telescope axis to the cameraimaging system axis. The proposed system employs the function of the iris thatis not present in the larvae eye, to compensate for these differences by possibleselection between the central or the peripheral images.

3 Experimental System

The proposed imaging system concept was studied with the experimental setupshown in Fig. 3. The diameter of the camera lens is 45 mm and the diameter ofthe telescope is 15 mm. The focal length of the camera lens is 50 mm, the focallength of the telescope convex lens is 25 mm and the focal length of the telescopeconcave lens is -12.7 mm. The image detector of the system has 744× 480 pixels.The imaging system is mounted on two rotational actuators that are used topoint the imaging system in the desired direction with the resolution of 0.01◦.

Fig. 3. The experimental setup.

The scene consists of two bolts of different sizes (Fig. 4). Bolts are usedbecause of their simple but at the same time recognisable shape. And also whenthe bolt is observed in high enough resolution its thread is visible. A bolt as anobserved object therefore offerers a lot of distinct features that can be used inimage analysis. The bolts are positioned 120 cm from the imaging system andspaced apart approximately 5 cm. The smaller bolt is the main observed objectin the experiment. The bigger bolt serves as a reference point in the scene and ispositioned further away form the imaging system and to the right of the smallerbolt.

First of all, the function of the iris is demonstrated. Figure 5 (a) shows a scenewith a fully opened iris. The image where the light passes besides the telescope

Use of Bifocal Objective Lens and Scanning Motion 323

Fig. 4. The experimental scene.

directly to the camera lens prevails. This is the wide angle view image with lowermagnification. The image formed with the light path A is barely visible. In Fig. 5(b) the iris is partially closed and both images are sufficiently represented. Inthe last case shown by the Fig. 5 (c), the iris is fully closed and only the narrowangle view image with higher magnification is visible. The image formed withthe light path B is not visible. It was also found that the telescope works asa central iris for a light path B when the camera lens iris is fully opened. Theimage formed by a light path B is consequently high-pass filtered. This enhancessmall details, which can be an advantage in disparity searching algorithms.

Fig. 5. Effect of the iris settings. Fully open iris (a); partially closed iris (b); fully closediris (c).

Secondly, the use of image disparity to estimate distance to the object is ver-ified. Image disparity of close objects is larger than the disparity of far objects.Figure 6 shows the difference in disparity between two distant points L1 and

324 G. Skulj and D. Bracun

Fig. 6. The image disparity in dependence of the object distance from the imagingsystem.

L2. The point L1 closer to the imaging system has a greater disparity D1 thanthe more distant point L2 with disparity D2. In general, disparity exponentiallydecreases with larger distances. If the observed point has no disparity, it is posi-tioned directly in the imaging system optical axis or it is very far away.

Image disparity in the proposed bifocal imaging system results in two over-lapping images with different magnifications. The magnification for the imageA is denoted as MA and for the image B as MB. Ratio between MA and MB isdenoted as Z. With the use of equations for thin lens optics (1, 2 and 3) therelation between Z and the observed object’s distance L from the image detectoris derived. A condition for the given equations is that the image of the observedobject must be focused.

L = S1 + S2 (1)

1S1

+1S2

=1F

(2)

M = − I

O= −S2

S1(3)

Distance L is the sum of distances S1 and S2. S1 is the distance between theobject and the lens. S2 is the distance between the lens and the image detector.Focal distance F for the optical system A is denoted as FA and focal distancefor the optical system B as FB. Magnification is the ratio between the image sizeI and the object size O. The derived theoretical relation Z(L) is shown in (4).

Z(L) =IA(L)IB(L)

=MA(L)MB(L)

=

(L− √

L(L− 4FA)) (

L +√L(L− 4FB)

)(L +

√L(L− 4FA)

) (L− √

L(L− 4FB)) (4)

Use of Bifocal Objective Lens and Scanning Motion 325

Fig. 7. The image used to measure ratio Z at position 1200mm.

Fig. 8. Comparison of measured values (red dots) with theoretical model for Z(L).(Color figure online)

During the experiment ratio Z was determined on three locations 1200 mm,1800 mm and 2600 mm. The measurements (Fig. 7) are compared to the theo-retical model Z(L) in Fig. 8. The measurements sufficiently correspond to themodel to confirm that the proposed imaging system can be used for distanceassessment of an object. However, for a precise distance measurement the exper-imental system is not yet sufficient. The images taken are not sharp enough toprecisely determine the edges of the object. This can be improved with betterimage focusing and with an image detector of higher resolution.

Lastly, the use of the scanning motion of the imaging system is explained.The scanning motion is achieved with a rotational actuator that locally swingsthe whole imaging system predominantly around one axis. Alternatively thescanning motion can also be achieved by rigidly mounting the imaging system

326 G. Skulj and D. Bracun

Fig. 9. Example of image disparity D in relation to the imaging system rotation (a)left of the object and (b) centrally aligned with the object.

on the robot, whose movement is then used to swing the system. The scanningrotation angle depends on the ratio between the narrow and wide angle of view.The observed object must appear on both images for its relative position to beestimated using disparity D of its images A and B (Fig. 9).

Search for minimal disparity of an object of interest is used to point theimaging system directly at it. This is necessary because the distance assessmentof an object should ideally take place when the centre of the object lies onthe imaging system optical axis. The imaging system scans its surroundingsidentifying objects of interest, aligning with them and estimating their distance.

Objects of interest are determined based on background elimination usingoptical flow image analysis. Optical flow image analysis is used to detect movingobjects in the image (Fig. 10). If the object is moving on its own, it can bedetected without the scanning motion. Otherwise scanning motion is employed

Fig. 10. (a) Projection of the same point of the scanned object at different locationson to the sensor of the bifocal imaging system. (b) Optical flow image analysis of thescanned object. (Color figure online)

Use of Bifocal Objective Lens and Scanning Motion 327

to move the image of the stationary object across the out-of-focus background.An example of optical flow algorithm use is shown in Fig. 10 (b). The result ofthe optical flow algorithm are movement vectors shown as red lines that depictthe direction and the amplitude of the detected movement. It can be seen thatthe movement of the object xO is detected on both parts of the image (for A andB light path). The resulting vectors for the object’s image A and B are markedas xA and xB, respectively. The amplitude ratio between the movement vectorscorresponds to ratio Z(L). The information provided by the optical flow analysiscan therefore be used to estimate the distance of the observed object.

4 Discussion

The proposed imaging system design successfully mimics the main functionalityof the bifocal eye of sunburst diving beetle (Thermonectus marmoratus) larvae.The imaging system was realized as a working experimental prototype. Thebifocal imaging system produces two images on a single imaging sensor that areused to determine the object’s distance from the system. The scanning motionof the system is used to locate the object of interest and distinguish them fromthe background. With the bifocal imaging system it is possible to simultaneouslyobserve the object in relation to its surroundings and in high resolution.

Future work will continue the development based upon the bifocal eye inspi-ration with the goal to improve image quality and increase the system’s distancemeasurement resolution. To enable an efficient use of the system, decision algo-rithms based on image processing will be developed. It is also desirable to reducethe size of the imaging system, because of the intention to integrate the imagingsystem on a robot with biologically inspired locomotion. The integration willprovide an opportunity for co-evolution of individual robot’s subsystems andfurther development of the artificial bifocal imaging system. For a biologicallyinspired robotic system to be truly successful, all of its parts have to be insynergy and build upon their strengths.

An example that can provide an insight into future development of the pro-posed system is a snake like robot. An advantage of a snake like robot is itslarge length-to-width ratio. A small width and flexible body enable it to slitherthrough tight spaces and corners. The improved version of the proposed imagingsystem would be placed centrally on the robot’s head in order to preserve itsdimensions and at the same time provide depth perception. Mimicry of a snake’smotion would be used in synergy with the imaging system to passively scan thesurroundings and detect objects of interest. The distance assessment function ofthe imaging system would provide the robot with enough data to construct amap of its local environment and enable it to autonomously navigate towardsthe desired location. A potential use for a snake-like robot would be autonomoussurveillance on rough terrain or underwater. A specially interesting case wouldbe a search for survivors in demolished buildings after natural disasters.

328 G. Skulj and D. Bracun

References

1. Lee, L.P., Szema, R.: Inspirations from biological optics for advanced photonic sys-tems. Science 310, 1148–1150 (2005)

2. Rim, S.B., Catrysse, P.B., Dinyari, R., Huang, K., Peumans, P.: The optical advan-tages of curved focal plane arrays. Opt. Express 16, 4965–4971 (2008)

3. Jeong, K.H., Kim, J., Lee, L.P.: Biologically inspired artificial compound eyes. Sci-ence 312, 557–561 (2006)

4. Song, Y.M., Xie, Y., Malyarchuk, V., Xiao, J., Jung, I., Choi, K.J., Li, R.: Digitalcameras with designs inspired by the arthropod eye. Nature 497, 95–99 (2013)

5. Stowasser, A., Rapaport, A., Layne, J.E., Morgan, R.C., Buschbeck, E.K.: Biologicalbifocal lenses with image separation. Curr. Biol. 20, 1482–1486 (2010)

6. Bland, K., Revetta, N.P., Stowasser, A., Buschbeck, E.K.: Unilateral range findingin diving beetle larvae. J. Exp. Biol. 217, 327–330 (2014)

MantisBot Uses Minimal Descending Commandsto Pursue Prey as Observed

in Tenodera Sinensis

Nicholas S. Szczecinski(B), Andrew P. Getsy, Jacob W. Bosse,Joshua P. Martin, Roy E. Ritzmann, and Roger D. Quinn

Case Western Reserve University, Cleveland, OH 44106, USAnss36@case.edu

http://biorobots.case.edu/

Abstract. Praying mantises are excellent models for studying directedmotion. They may track prey with rapid saccades of the head, prothorax,and legs, or actively pursue prey, using visual input to modulate theirwalking patterns. Here we present a conductance-based neural controllerfor MantisBot, a 28 degree-of-freedom robot, which enables it to usefaux-visual information from a head sensor to either track or pursueprey with its prothorax and appropriate movements of one of its legs.The controller can switch between saccades and smooth tracking, as seenin pursuit, modulating only two neurons in its model thoracic ganglia viadescending commands. Similarly, the neural leg controller redirects thedirection of locomotion, and automatically produce reflex reversals seenin other insects when they change direction, via two simple descendingcommands.

Keywords: Real-time neural control · Praying mantis · CPG ·Descending commands

1 Introduction

Praying mantises are ambush hunters who primarily use ballistic, visually-guidedsaccades of the head and body to track prey while standing still [7,9,12]. Starv-ing them, however, will cause them to actively pursue prey, smoothly adjustingtheir gaze with their body joints as well as changes in their walking patterns.This behavior is of particular interest to roboticists, to understand how visualinformation affects the rhythms and reflexes that produce walking.

Walking is made of two phases, stance and swing (See [2] for a thoroughreview of how animals coordinate their walking). In stance phase, muscles mustactivate to support the body and move it in the intended direction. Dependingon the linear and angular velocity of the animal in the ground plane, reflexes

N.S. Szczecinski—This work was supported by a NASA Office of the Chief Technol-ogists Space Technology Research Fellowship (Grant Number NNX12AN24H).

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 329–340, 2016.DOI: 10.1007/978-3-319-42417-0 30

330 N.S. Szczecinski et al.

are modulated to excite the appropriate muscles [1,8]. The networks that con-trol locomotion are distributed throughout the nervous system, so presumablythe brain does not directly modulate these reflexes. Populations in the centralcomplex (CX) code for intended direction [6], and stimulating these same neu-rons will modulate inter-joint reflexes [8]. But it is also known that descendingneurons in the ventral nerve cord carry information about visual targets, pre-sumably to the thoracic networks [16]. This suggests that the intended directionof motion is communicated as descending commands to the thoracic ganglia,which use this information to modulate reflexes locally.

In this paper we present a conductance-based neural controller for MantisBot,shown in Fig. 1 and described in [14], that can reproduce visually-guided saccadeslike those seen in the animal, actively “pursue” prey with appropriate movementsof one leg, and adjust the direction and load-specificity of walking based onsimple descending commands. The robot autonomously switches between thesemodes depending on the distance and location of the prey in its visual field.The walking controller requires only minimal parameter tuning based on thekinematics of the leg. The tuning is completely automated and takes less thanfive minutes to design the entire robot’s controller. The resulting controller isa hypothesis of how low-level locomotion control networks may be structuredto readily implement the reflex reversals observed in animals via only minimaldescending commands.

Fig. 1. MantisBot supporting its own weight on four legs, shown with its head sensor.

2 Methods

2.1 Animal Experimental Procedures

Adult female praying mantises, Tenodera sinensis, were starved for 3 to 5 daysprior to the experiment. Individual animals were placed in a clear acrylic arena,

MantisBot Uses Minimal Descending Commands to Pursue Prey 331

which was then placed on top of a computer monitor. High speed video (120 fps)was taken as the mantis was presented with an artificial stimulus, a 1× 2 cmblack oval on a white background, which moved at 4 cm/s through a 60◦ arc inthe center of the praying mantis’s visual field. The animal was presented withthis stimulus two times each at 3, 6, 9, 12, 15, and 18 cm away from the head.

2.2 Animal Data Analysis

Using the software WINanalyze (Mikromak, Berlin, Germany), the angle of theartificial stimulus and the praying mantis’s head were tracked. For the artificialstimulus this angle was the angle between a line going from the center of thearc to the origin (line A), and another line from the origin to the stimulus’sposition. For the praying mantis’s head, this angle was between line A and aline projecting perpendicularly out of the center of the animal’s, representingthe direction that the animal was focusing in any given frame of video. If theanimal did not move significantly from its starting position, then these angleswere used verbatim, but if the animal moved closer to the stimulus, they wereadjusted according to a factor that varied with the distance that the prayingmantis moved from its starting position to account for the fact that the visualangle to the stimulus will change if the origin moves. The animal’s behaviorswere also quantified by observing the video and tracking the frames in which thebehavior occurred.

2.3 Robot Experimental Procedures

MantisBot’s thorax was fixed in all six dimensions by bolting it to a rigid armsuspended from a wheeled frame. When the trial began, all leg joints were movedto their zero positions, and the strain gage in the leg was tared. Once the legmoved from its initial posture, an 11.4 cm block topped with a sheet of TeflonTM

was placed under the foot. This low-friction surface enabled the leg to registerleg strain when in stance phase, but allowed the foot to slip laterally. A 1600 lm,23 W compact fluorescent light bulb was then presented in front of the robot asa faux-visual prey-like signal. An array of solar cells is used to detect the angularposition and distance of the light source [5].

3 Animal Behavior

Mantises track prey with saccadic motions of the head, prothorax and thorax[7,9,12]. When starved, they may actively pursue prey, walking toward the tar-get. Figure 2 shows the orientation of the head when presented with a prey-likestimulus. When the stimulus starts far from the mantis, it approaches, smoothlyadjusting its heading over time. Once the mantis comes within reach of the prey,it strikes, and later tracks with saccades. Starting the stimulus nearer to themantis, as in the second trial, evokes typical saccades.

332 N.S. Szczecinski et al.

Fig. 2. Traces of mantis gaze compared to an artificial stimulus. The mantis’s state isencoded by the bars below the traces. (Color figure online)

4 Network Structure

All neurons in this controller are modeled as nonspiking leaky integrators. Mostneurons have no additional ion channels, but those in the CPGs possess a per-sistent sodium channel to provide additional nonlinear dynamics [4]. A neuron’smembrane voltage V fluctuates according to

CmemdV

dt= Gmem ·(Erest−V )+

n∑i=1

gsyn ·(Esyn−V )+GNam∞h·(ENa−V ) (1)

in which C is a constant capacitance, G is a constant conductance, g is a vari-able conductance, E is a constant voltage, and the subscript mem stands formembrane, syn stands for synaptic, and Na stands for persistent sodium. mand h are dynamical variables that describe the sodium channel activation anddeactivation, respectively. m is fast, and thus is simulated as its steady statevalue m∞. h changes according to

h = (h∞ − h)/τh(V ). (2)

Both m∞ and h∞ are sigmoids, and have the same form,

z∞(V ) =1

1 + Az · exp(Sz · (V − Ez)), (3)

the primary difference being that Sm > 0 and Sh < 0. The full simulation,including all parameter values, is available for download at http://biorobots.case.edu/projects/legconnet/mantisbot/.

MantisBot Uses Minimal Descending Commands to Pursue Prey 333

4.1 Head Networks

The robot uses the position of the light to decide whether to remain stationaryand track or actively pursue it. It then uses this information to generate motionthat moves the prey toward the center of its field of view. The hierarchicalnetwork in Fig. 3 enacts these behaviors.

The top section in Fig. 3 models the function of networks in the lobula andother visual processing centers, identifying the distance, angular position, andvelocity of visual targets. The Left neuron receives a current proportional to thehead sensor’s left/right comparison. The Right neuron rests at a voltage thatsignifies centered prey. The comparison between the Right and Left neuronsactivates the Error Right and Error Left neurons. When in the Track state,the Down Bias neuron is tonically active, inhibiting the Error Right and ErrorLeft neurons. This ensures that no error is detected as the target moves within10◦ of center, mimicking the “dead zone” observed in mantises as they saccadetoward prey [12]. If either Error neuron becomes active, it will excite both theError Large and Slow neurons. These neurons are identical, except that theSlow neuron has a larger capacitance, and thus larger time constant. The Triggerneuron then computes the difference between these, forming a Reichardt detector[11]. Thus Trigger detects when the prey is leaving the “dead zone,” requiringcorrective action.

The visual center also computes the distance to the target by mapping thehead’s center panel reading to a 1/r2 relationship, which controls Dist.’s activity.When the distance to the target is small enough, Too Close is disinhibited,halting pursuit. The visual center also computes the velocity of the visual inputvia another Reichardt detector [11].

Saccades are by their nature brief bursts of motion. Therefore the Sacc. neu-ron is configured to produce brief, stereotypical pulses when Trigger becomesactive. Sacc. and No Sacc. include persistent sodium channels, which are fre-quently used to model CPGs in locomotion models [4]. By biasing No Sacc.upward with a 1 nA tonic current, the equilibrium state of the Sacc./No Sacc.system is for No Sacc. to be tonically active, and Sacc. to be inactive. WhenTrigger inhibits the interneuron, it disinhibits Sacc., causing it to temporarilyinhibit No Sacc., producing a temporary pulse which disinhibits the Sacc. Rightand Sacc. Left neurons. These pulses can be observed in Fig. 6.

Flashing targets at mantises reduces the latency of subsequent saccades inthe same direction [7]. This suggests that there is some memory of the previoussaccade direction, which primes the system to move in that direction again. Thusthe Left Mem. and Right Mem. neurons mutually inhibit one another such thatexactly one is always tonically active while the other is inactive. The Error Rightand Error Left neurons can change the state of this memory, which then gate theSacc. Right and Sacc. Left neurons. These, in turn, gate the Right Vel. and LeftVel. neurons, which ultimately stimulate the motor neurons of the prothoraxand leg joints.

334 N.S. Szczecinski et al.

Fig. 3. Network that processes visual information for MantisBot and communicatesdescending commands to the thoracic ganglia model, partially shown in Fig. 4.

4.2 Thoracic Networks

Insects’ nervous systems are highly distributed. How, then, are individual jointscontrolled to direct posture or locomotion based on descending input? Signalsfrom load sensors signal the onset of stance phase, causing supportive and propul-sive muscles to contract. Which muscles are activated depends on the directionof travel [1]. Signals from movement sensors signal the end of stance phase, caus-ing the leg to lift from the ground and return to its position prior to stance [3].Figure 4 shows a control network for one joint based on these observations, whichunderpins the leg controller in this work. Every joint’s controller is identical,except for the weight of the four synapses drawn in bold, and the resting poten-tial of the Des. PEP and Des. AEP neurons. The network’s function directlyinforms the tuning of these values, as shown below.

Our joint network receives descending commands about the position andvelocity of the prey signal, as well as the state of the robot, whether staticallytracking or actively pursuing prey. The labels PEP (posterior extreme position)

MantisBot Uses Minimal Descending Commands to Pursue Prey 335

Fig. 4. Network that determines the timing and amplitude of the motion of each joint.With only minimal tuning and limited descending commands, the network produceswalking motion and reflex reversals to change the travel.

and AEP (anterior extreme position) refer only to the intended position of thejoint at the end of stance and swing, respectively. When Track is active, thetransition to swing is inhibited, and the PEP is simply the desired position of thatjoint. If Pursue is active, then the AEP and PEP are different, establishing thebeginning of stance and swing, respectively. This is manifested by the excitationof Lin. PEP and the inhibition of Lin. AEP. Visual information affects therotation of the PEP (Rot. PEP) and AEP (Rot. AEP). Even though this paperonly presents results from one leg of MantisBot, our design method is general,and is intended to control all the legs simultaneously. Here we describe how totune the network in Fig. 4 for a single joint. If we were to translate and rotate

336 N.S. Szczecinski et al.

some point on the robot distance p and angle θ, as shown in Fig. 5, then aparticular foot would have to move distance d at angle φ from straight ahead tostay in the original position. For each φ, the Moore-Penrose generalized inverse ofthe leg’s Jacobian matrix can be used to find the joint velocities required to movethe foot in the d direction, which minimize the norm of the joint velocity vector[10]. The ratio between joint velocities in the leg specify the relative proportionof motor neuron activation required to move the foot in a straight line in thedirection specified. If negative position feedback is present at each joint, thenscaling this proportion will specify an equilibrium position.

Each joint has a map of its deflection from rest as a function of the body’s rota-tion and translation (for example, Fig. 5C). We can draw a line showing the bodymotions that would require no change in the joint’s position. Points above thatline represent body motions that would require the joint to flex, and those belowcorrespond to motions that require extension. Because this map is specifying the

Fig. 5. Inverse kinematics used to design the joint network. To move the robot’s bodyin a particular direction (A), each leg must move in a different direction (B). The inversekinematics reveal at which body rotations and translations the joint must change itsdirection of travel (C). (Color figure online)

MantisBot Uses Minimal Descending Commands to Pursue Prey 337

PEP of the leg, the line predicts at what locomotion speeds and turning radii areflex reversal should take place, a consideration of prior studies [8,13].

Using this method, the four synapses in bold in Fig. 4 are designed such thatthe descending commands, which encode intended body translation and rotation,can be used to command the joint’s rotation. This network is duplicated, once tocommand the PEP and once to command the AEP. If the desired PEP, encodedby Des. PEP is greater than the resting posture, then the joint needs to extend instance phase, and the Ext. PEP neuron will be active. When active, it disinhibitsthe pathway from the leg’s strain gage, exciting Ext. IN and inhibiting Flx. HCduring stance, causing the joint to extend. Changing the descending commandsmay instead cause Flx. PEP to be active, which will route load information tothe opposite half-center of the CPG. The AEP network works the same way, butwith a signal that the leg is unloaded. In this way the kinematics of the robotare used to not only cause desired foot trajectories, but also enforce known reflexreversals with minimal descending commands.

5 Results

Figure 6 shows that the network in Fig. 3 can accurately orient the robot towardprey using the same strategy as the animal. The target is tracked smoothly whenin active pursuit, and with saccades when tracking. Locomotion also halts whenthis change takes place, and produces saccadic motions afterward. When thetarget strays from the center of view, the Error Left or Error Right neuron’s

Fig. 6. Traces from MantisBot tracking a stimulus and autonomously changing betweentracking and pursuit. (Color figure online)

338 N.S. Szczecinski et al.

Fig. 7. Foot positions as MantisBot walks when presented with stimuli at differentangles. (Color figure online)

activity increases over the permissible threshold, causing the Trigger neuron’sactivity to fluctuate rapidly (red arrows). These in turn cause the Sacc. neuronto pulse and generate the saccades seen in the top plot. The transition betweentracking and pursuit is mediated by the Dist. and Small Error neurons. Whenthe distance to the target is decreased, the pursuit has been “successful,” and therobot begins to track the prey with saccades. When the distance again decreasesand the prey is located within the field of view, the robot again pursues.

MantisBot Uses Minimal Descending Commands to Pursue Prey 339

Fig. 8. The network in Fig. 4 automatically produces the reflex reversals observed inother insects.

Figure 7 shows that the network in Fig. 4 can control the direction of the footin stance. Leg kinematics were recorded as the robot took 12 steps toward thelight in five different positions. The joint angles read from the servomotors werefed through the kinematic model to create 3D plots of the leg motion over time,as well as the foot placement on the ground. Calculating the angle of the linethat passes through the AEP and PEP with respect to straight ahead for eachstep shows that the stance direction is statistically significantly different for eachtarget presented (p < 0.001, 1 sided ANOVA). This includes both inward andoutward stance directions, suggesting that reflex reversals must take place tochange the stepping so drastically. Figure 8 shows that this is the case; the signof the load information fed to the femur-tibia joint is reversed by the networkin Fig. 4. Rather than guessing when this transition should occur, our networkautomatically changes the sign of this reflex with minimal descending commands.

6 Conclusion

The data shows that this joint controller is capable of producing basic stanceand swing motions, and modifying them with minimal descending commands.Insects use a host of reflexes to counteract external forces and transition fromstance to swing, and vice versa (for a review, see [2]). Reflexes in response todecreasing or excessive load, as well as searching for missing footholds, will beadded in the future [15]. In the future, this controller will be expanded to useall of the legs of MantisBot to orient towards and pursue faux-visual targets.

340 N.S. Szczecinski et al.

References

1. Akay, T., Ludwar, B.C., Goritz, M.L., Schmitz, J., Buschges, A.: Segment speci-ficity of load signal processing depends on walking direction in the stick insect legmuscle control system. J. Neurosci. 27(12), 3285–3294 (2007)

2. Buschmann, T., Ewald, A., Twickel, A.V., Buschges, A.: Controlling legs for loco-motion insights from robotics and neurobiology. Bioinspiration Biomimetics 10(4),41001 (2015)

3. Cruse, H.: Which parameters control the leg movement of a walking insect? II: thestart of the swing phase. J. Exp. Biol. 116, 357–362 (1985)

4. Daun-Gruhn, S.: A mathematical modeling study of inter-segmental coordinationduring stick insect walking. J. Comput. Neurosci. 30(2), 255–278 (2010)

5. Getsy, A.P., Szczecinski, N.S., Quinn, R.D.: MantisBot: the implementation ofa photonic vision system. In: Lepora, N.F., et al. (eds.) Living Machines 2016.LNCS(LNAI), vol. 9793, pp. 429–435. Springer, Switzerland (2016)

6. Guo, P., Ritzmann, R.E.: Neural activity in the central complex of the cockroachbrain is linked to turning behaviors. J. Exp. Biol. 216(Pt 6), 992–1002 (2013)

7. Lea, J.Y., Mueller, C.G.: Saccadic head movements in mantids. J. Comp. Physiol.A 114(1), 115–128 (1977)

8. Martin, J.P., Guo, P., Mu, L., Harley, C.M., Ritzmann, R.E.: Central-complexcontrol of movement in the freely walking cockroach. Curr. Biol. 25(21), 2795–2803 (2015)

9. Mittelstaedt, H.: Prey capture in mantids. In: Scheer, B.T. (ed.) Recent Advancesin Invertebrate Physiology, pp. 51–72. University of Oregon, Eugene (1957)

10. Murray, R.M., Li, Z., Sastry, S.S.: A Mathematical Introduction to Robotic Manip-ulation. CRC Press, Boca Raton (1994)

11. Reichardt, W.: Autocorrelation, a principle for the evaluation of sensory informa-tion by the central nervous system. In: Rosenblith, W.A. (ed.) Sensory Communi-cation, pp. 303–317. MIT Press, Cambridge, MA (1961)

12. Rossel, S.: Foveal fixation and tracking in the praying mantis. J. Comp. Physiol.A Neuroethology Sens. Neural Behav. Physiol. 139, 307–331 (1980)

13. Szczecinski, N.S., Brown, A.E., Bender, J.A., Quinn, R.D., Ritzmann, R.E.: A neu-romechanical simulation of insect walking and transition to turning of the cock-roach blaberus discoidalis. Biol. Cybern. 108(1), 1–21 (2013)

14. Szczecinski, N.S., Chrzanowski, D.M., Cofer, D.W., Moore, D.R., Terrasi, A.S.,Martin, J.P., Ritzmann, R.E., Quinn, R.D.: MantisBot: a platform for investigatingmantis behavior via real-time neural control. In: Wilson, S.P., Verschure, P.F.M.J.,Mura, A., Prescott, T.J. (eds.) Living Machines 2015. LNCS, vol. 9222, pp. 175–186. Springer, Heidelberg (2015)

15. Szczecinski, N.S., Chrzanowski, D.M., Cofer, D.W., Terrasi, A.S., Moore, D.R.,Martin, J.P., Ritzmann, R.E., Quinn, R.D.: Introducing MantisBot: hexapod robotcontrolled by a high- fidelity, real-time neural simulation. In: IEEE InternationalConference on Intelligent Robots and Systems. pp. 3875–3881. Hamburg, DE(2015)

16. Yamawaki, Y., Toh, Y.: A descending contralateral directionally selective move-ment detector in the praying mantis tenodera aridifolia. J. Comp. Physiol. A195(12), 1131–1139 (2009)

Eye-Head Stabilization Mechanismfor a Humanoid Robot Tested on Human

Inertial Data

Lorenzo Vannucci1(B), Egidio Falotico1, Silvia Tolu2, Paolo Dario1,Henrik Hautop Lund2, and Cecilia Laschi1

1 The BioRobotics Institute, Scuola Superiore Sant’Anna,Viale Rinaldo Piaggio 34, 56025 Pontedera, Pisa, Italy

{l.vannucci,e.falotico,p.dario,c.laschi}@sssup.it2 Department of Electrical Engineering, The Center for Playware,

Technical University of Denmark, Elektrovej Building 326,2800 Kongens Lyngby, Copenhagen, Denmark

{stolu,hhl}@elektro.dtu.dk

Abstract. Two main classes of reflexes relying on the vestibular systemare involved in the stabilization of the human gaze: the vestibulocollicreflex (VCR), which stabilizes the head in space and the vestibulo-ocularreflex (VOR), which stabilizes the visual axis to minimize retinal imagemotion. Together they keep the image stationary on the retina.

In this work we present the first complete model of eye-head stabiliza-tion based on the coordination of VCR and VOR. The model is providedwith learning and adaptation capabilities based on internal models. Testson a simulated humanoid platform replicating torso disturbance acquiredon human subject performing various locomotion tasks confirm the effec-tiveness of our approach.

Keywords: Head stabilization · VOR · VCR · Eye-head coordination ·Humanoid robotics

1 Introduction

Several neuroscientific studies focus on eye-head behaviour during locomotion.Results about head and eyes during walking mostly come from two-dimensionalstudies on linear overground, turning, treadmill locomotion, running and walkingon compliant surface [1–6]. These studies have shown that the body, head, andeyes rotate in response to the up-down and side-to-side motion to maintain stablehead pointing and gaze in space. This is achieved through the joint effect of twomain classes of reflexes, which rely on the output of the inertial system: 1. thevestibulo-ocular reflex (VOR), which stabilizes the visual axis to minimize retinalimage motion; 2. the vestibulocollic reflex (VCR), which stabilizes the head inspace through the activation of the neck musculature in response to vestibularinputs. The VOR compensates for head movements that would perturb visionc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 341–352, 2016.DOI: 10.1007/978-3-319-42417-0 31

342 L. Vannucci et al.

by turning the eye in the orbit in the opposite direction of the head movements[7]. Several approaches have been used to model the VOR depending on the aimof the study. In robotics literature we found some controllers inspired by theVOR [8–11]. The VCR stabilizes the head based on the inertial input space bygenerating a command that moves the head in the opposite direction to that ofthe current head in space displacement. When the head is rotated in the planeof a semicircular canal, the canal is stimulated and the muscles are activated.This stimulation produces a compensatory rotation of the head in the sameplane. If more than one canal is activated, then an appropriate reflex responseis produced. Unlike the VOR, the VCR controls a complex musculature. TheVOR involves six extraocular muscles, each pair acts around a single rotationaxis. On the other hand, the neck has more than 30 muscles controlling pitch,roll and yaw rotations.

In robotics, some head stabilization models already exist implemented onhumanoid robots. Gay et al. [12] proposed a head stabilization system for abipedal robot during locomotion controlled by the optical flow information. It isbased on Adaptive Frequency Oscillators to learn the frequency and phase shiftof the optical flow. Although the system can successfully stabilize the head ofthe robot during its locomotion, it does not take in consideration the vestibularinputs. The most close to the neuroscientific findings of the VCR are the worksproposed by Kryczka et al. [13–15]. They proposed an inverse jacobian controller[13,14] based on neuroscientific results [16] and an adaptive model based on afeedback error learning (FEL) [15] able to compensate the disturbance repre-sented by the trunk rotations. All the presented models try to reproduce specificaspects of the gaze stabilization behaviour, but none of them can provide acomprehensive model of gaze stabilization, integrating eye stabilization (VOR)together with head stabilization (VCR).

By considering the analysis of neuroscience findings, we can conclude thatin order to replicate eye-head stabilization behaviours found in humans it isnecessary to be able to replicate the joint effect of VCR for the head and VORfor the eye. This work goes in this direction by presenting a model that replicatesthe coordination of VCR and VOR and is suitable for the implementation on arobotic platform. We used, as a disturbance motion, inertial data acquired on ahuman subject performing various locomotion tasks (straight walking, running,walking a curved path on normal and soft ground) and replicated by the torso ofa humanoid robot. The purpose of these tests is to assess the effectiveness of thestabilization capabilities of the proposed model rejecting the torso disturbancemeasured in real walking tasks through the joint stabilizing effect of head andeye of the simulated iCub robot.

2 Eye-Head Stabilization Model

In order to implement the VOR-VCR system, a bio-inspired feed-forward con-trol architecture was used. The model uses classic feedforward controllers thatgenerate motor commands purely based on the current error. Each controller

Eye-Head Stabilization Mechanism for a Humanoid Robot 343

is coupled with a learning network that generates predictions based on internalmodels that are used to fine tune motor commands. An overview of the modelcan be seen in Fig. 1.

Fig. 1. The proposed model of eye-head stabilization. Dashed lines represent encoderreadings, dotted lines show inertial values.

2.1 Head Stabilization System

Inside the head stabilization system, the output of the VCR internal model (uvcr)is added to the output of the feedforward controller (evcr) in order to generatemotor commands that stabilized the head against the disturbance originatingfrom the torso movements. The VCR Feedforward Controller is implemented asa PD controller, and its output is computed as a function of the inertial readings(In, ˙In):

evcr = kp · In + kd · ˙In. (1)

The inputs to the learning network are the current and desired position andvelocity of the robotic head, and the network is trained with newly generatedmotor commands. In order to provide a proper reference to the VCR internalmodel, the current value of the external disturbance must be estimated. Using thereadings coming from the inertial measurement unit and the encoder values, thedisturbance vector (d) can be estimated using only direct kinematics functions,by computing d = In − In, i.e. by subtracting the expected angular rotationsgiven by the encoder values (In) from the inertial readings (In). In = [ϕ, ϑ, ψ]are the Euler angles for the rigid roto-translation matrix K(θh) from the rootreference frame to the inertial frame, computed as:

ϕ = atan2(−K(θh)2,1,K(θh)2,2), (2)ϑ = asin(K(θh)2,0), (3)ψ = atan2(−K(θh)1,0,K(θh)0,0). (4)

Likewise, the same procedure can be followed in order to estimate the velocityof the disturbance:

d = ˙In − ˙In = ˙In − J(θh) · θh, (5)

where J is the geometric Jacobian from the root reference frame to the inertialframe.

344 L. Vannucci et al.

2.2 Eye Stabilization System

The eye stabilization system implements the VOR and, similarly to the headstabilization system, produces a motor command for the eyes that is the sum ofthe feedforward controller output (evor) and the VOR internal model one (uvor).Given that the eye should stabilize the image against the relative rotation of thehead, the error is computed as the difference between inertial measurements andthe current eye encoders (θe, θe). Thus, the output of the VOR feedforwardcontroller is computed as

evor = kp · (−In) + kd · (− ˙In). (6)

The VOR internal model receives in input the head position and velocity signal asreferences, acquired through the vestibular system, along with the proprioceptivefeedback, and uses the generated motor command as a training signal.

2.3 Learning Network

Prediction of the internal model is provided by a learning network that is imple-mented with a machine learning approach, Locally Weighted Projection Regres-sion (LWPR) [17]. This algorithm has been proved to provide a representationof cerebellar layers that in humans are responsible for the generation of predic-tive motor signals that produce more accurate movements [18,19]. The LWPRspatially exploits localized linear models at a low computational cost throughan online incremental learning. Therefore, the prediction process is quite fast,allowing real-time learning. LWPR incrementally divides the input space into aset of receptive fields defined by a centre ck and a Gaussian area characterizedby a positive definite distance matrix Dk. The activation on each receptive fieldk in response to an input x is expressed by

pk(x) = exp

(−1

2(x − ck)TDk(x − ck)

), (7)

while the output is yk(x) = wk ·x+εk, where wk and εk are the weight vector andbias associated with the k-th linear model. For each iteration, the new input, x, isassigned to the closest RF based on its weight activation, and consequently, thecentre, the weights and the kernel width are updated proportionally to a trainingsignal. Moreover, the number of local models increases with the complexity ofthe input space.

The global output of the LWPR is given by the weighted mean of all thepredictions yk of the linear local models created:

u(x) =∑N

k=1 pk(x)yk(x)∑Nk=1 pk(x)

. (8)

Eye-Head Stabilization Mechanism for a Humanoid Robot 345

3 Experimental Procedure

In order to collect human inertial data relative to locomotion tasks, experimentswere conducted on a human subject with no visual and vestibular impairments.An inertial measurement unit (IMU) was placed on the back of the subject, nearT10, the tenth vertebra of the thoracic spine, as depicted in Fig. 2.

Fig. 2. Placement of the inertial measurement unit on the subject.

The IMU used was an Xsens MTi orientation sensor1, that incorporates anon-board sensor fusion algorithm and Kalman filtering. The inertial unit is ableto produce the current orientation of the torso at a frequency of 100 Hz.

Three different tasks were performed by the subject: straight walking (25 m),circular walking and straight running (25 m). The circular walking was carriedout by asking the subject to walk with a circular pattern, without any indicationof the pattern on the ground. Such task was executed both on normal and softground, provided by placing a foam rubber sheet on the ground. The foam hada density of 40 kg/m3 and the sheet measured 103× 160× 15 cm. All tasks wereperformed with bare feet.

Due to the fact that the inertial readings relative to the yaw rotational axis(rotation around z) can often be inaccurate because of drifting, we decided notto use such readings. Moreover, in order to prevent drifts of the sensor measure-ments on the other two rotational axis (pitch and roll, rotations around y and xrespectively), each trial lasted less than one minute with a reset of the rotationalangle at the beginning of the trial [20].

4 Robotic Platform

The proposed model was implemented for the iCub robot simulator [21], a soft-ware included with the iCub libraries. The iCub head contains a total of 6 degreesof freedom: 3 for the neck (pan, tilt and swing) and 3 for the eyes (a common

1 https://www.xsens.com/products/mti/.

346 L. Vannucci et al.

tilt, version and vergence), while the torso has 3 degrees of freedom (pan, tiltand swing). The visual stereo system consists of 2 cameras with a resolution of320× 240 pixels.

In order to assess the repeatability of the experiments on the iCub sim-ulator, first test were conducted to evaluate whether the measurements of thesimulated robot inertial rotations were compatible with the collected data. Thus,the collected torso rotations were given as motor commands to the robot torso.A graphical comparison can be seen in Fig. 3, where the actual IMU data isshown alongside the robot one. It can be observed that the simulation is accu-rate enough to reproduce the data, even if with a delay of 50 ms. The errorbetween the two signals was then computed after a temporal alignment and itsRoot Mean Squared value is 0.21 deg for the pitch rotational axis and 0.12 degfor the roll.

0 5 10 15 20 25−10

−5

0

5

10

time (s)

iner

tial p

ositi

on (d

eg)

real IMU pitchrobot IMU pitcherror

0 5 10 15 20 25−5

0

5

10

time (s)

iner

tial p

ositi

on (d

eg)

real IMU rollrobot IMU rollerror

Fig. 3. Comparison between collected IMU readings and simulated ones, on pitch rota-tional axis (top) and roll (bottom). (Color figure online)

5 Results

The stabilization model was tested on the data coming from the three differentlocomotion tasks (straight walking, circular walking and straight running). Dueto the fact that the collected inertial data related to the yaw rotational axis wasnot considered, the eye-head stabilization model has been simplified, so that nostabilization on the yaw axis was performed. Moreover, given that the roboteyes cannot influence stabilization on the roll rotational axis, due to the factthat only tilt and pan motors are present, only disturbance on the pitch axiswas compensated by the VOR model.

Eye-Head Stabilization Mechanism for a Humanoid Robot 347

The main measure of error during a stabilization task is the movement of thecamera image. In particular, human vision is considered stable if the retinal slip(the speed of the image on the retina) is under 4 deg/s [22]. In order to computethe error from the camera image, a target was placed in front of the simulatedrobot and its position was tracked from the camera images via a colour filteringalgorithm during the execution of the task. Another measure of performanceconsidered is the inertial orientation and speed of the head. As already statedbefore, no movement on the yaw rotational axis was considered, thus only thecamera error on the vertical axis is relevant for the evaluation.

For each task, a comparison between the same task performed with andwithout the stabilization model will be presented. The values of the gains ofthe PD controllers were set to kp = 5.0, kd = 0.1 for the VCR model and tokp = 1.0, kd = 0.1 for the VOR model, for all trials.

5.1 Straight Walking

Results for the compensation of the disturbance of straight walking inertial datacan be found in Table 1, where the Root Mean Square (RMS) values for inertialreadings and target position and speed are presented. In this and subsequenttables, Inp, ˙Inp are the inertial readings for rotation (deg) and rotation speed(deg/s) on the pitch axis, Inr, ˙Inr are the inertial readings for rotation (deg)and rotation speed (deg/s) on the roll axis, v, v are the position of the target onthe camera image (deg) and its speed (retinal slip, deg/s).

Table 1. Results for straight walking data.

Trial Inp˙Inp Inr

˙Inr v v

No stabilization 2.06 11.36 2.05 11.56 3.66 8.88

Stabilization 0.84 14.71 0.22 3.45 2.71 3.30

Figures 4, 5 and 6 show the behaviour of the task, showing the target positionand retinal slip, inertial data for the pitch rotational axis and inertial data for theroll axis, respectively. From these results it can be noticed that while the roll dis-turbance is almost completely compensated by the VCR model, the magnitudeof the rotational velocity on the pitch axis is too high to be fully compensated bythe said model, that only provides an improvement in the position space. Nev-ertheless, the VOR subsystem is still able to maintain the camera image stable,with a mean vertical retinal slip lower than 4 deg/s. Moreover, Fig. 4 also showsa comparison between the full stabilization model and a simplified model withonly the PD controllers. While the PD only implementation is able to reducethe error on the camera, it is outperformed by the complete model, thus provingthe effectiveness of the latter.

348 L. Vannucci et al.

5.2 Circular Walking

Two sets of data were collected for circular walking tasks: one for normal groundand one for soft ground. Results for both cases are presented in Table 2, whereit can be observed that walking on soft ground produces a greater disturbance,especially in the velocity space. Despite the higher disturbance the model is stillable to stabilize the head and the camera image, achieving stable vision in bothcases. As in the straight walking case, the disturbance on the pitch axis cannot be

0 2 4 6 8 10 12 14 16 18 20time (s)

-5

0

5

10

targ

et p

ositi

on (d

eg)

no stabilizationPD onlystabilization

0 2 4 6 8 10 12 14 16 18 20time (s)

-20

-10

0

10

20

retin

al s

lip (d

eg/s

) no stabilizationPD onlystabilization

Fig. 4. Stabilization task with data from a straight walking task, target position (top)and retinal slip (bottom). (Color figure online)

0 2 4 6 8 10 12 14 16 18 20time (s)

-10

-5

0

5

iner

tial p

ositi

on, p

itch

(deg

)

no stabilizationstabilization

0 2 4 6 8 10 12 14 16 18 20time (s)

-40

-20

0

20

40

iner

tial v

eloc

ity, p

itch

(deg

/s)

no stabilizationstabilization

Fig. 5. Stabilization task with data froma straight walking task, inertial positiondata (top) and inertial velocity (bottom),pitch axis. (Color figure online)

0 2 4 6 8 10 12 14 16 18 20time (s)

-5

0

5

10

iner

tial p

ositi

on, r

oll (

deg)

no stabilizationstabilization

0 2 4 6 8 10 12 14 16 18 20time (s)

-20

-10

0

10

20

iner

tial v

eloc

ity, r

oll (

deg/

s)

no stabilizationstabilization

Fig. 6. Stabilization task with datafrom a straight walking task, inertialposition data (top) and inertial velocity(bottom), roll axis. (Color figure online)

Eye-Head Stabilization Mechanism for a Humanoid Robot 349

fully compensated by the VCR alone, but thanks to the VOR module, the visionremains stable. The behaviour on the soft ground task can be observed in Fig. 7.

Table 2. Results for circular walking data on normal and soft ground.

Ground Trial Inp˙Inp Inr

˙Inr v v

Normal no stabilization 3.35 14.03 2.11 6.36 3.61 12.86

Normal stabilization 0.49 6.34 0.17 2.84 0.88 1.85

Soft no stabilization 3.09 16.00 3.89 10.95 4.23 15.15

Soft stabilization 0.46 6.35 0.24 3.07 2.74 3.56

0 10 20 30 40 50 60time (s)

-10

-5

0

5

10

15

targ

et p

ositi

on (d

eg)

no stabilizationstabilization

0 10 20 30 40 50 60time (s)

-40

-20

0

20

40

retin

al s

lip (d

eg/s

)

no stabilizationstabilization

Fig. 7. Stabilization task with data from a circular walking task on soft ground, targetposition (top) and retinal slip (bottom). (Color figure online)

5.3 Straight Running

During the last experiment, data from the straight running was used to move therobot torso. From Table 3 it can be observed that the model is not able to achievea complete compensation of the disturbance, due to the high rotational velocitieson the two axes. Nevertheless, the mean retinal slip is reduced to a quarter of theone of the trial with no stabilization. Thus, the model provides a viable solutioneven for disturbances of this magnitude, as it is also shown in Fig. 8.

350 L. Vannucci et al.

Table 3. Results for straight running data.

Trial Inp˙Inp Inr

˙Inr v v

No stabilization 3.14 45.36 1.77 16.28 3.49 25.96

Stabilization 1.06 43.03 0.53 10.78 1.61 6.36

0 2 4 6 8 10 12time (s)

-10

-5

0

5

10

targ

et p

ositi

on (d

eg)

no stabilizationstabilization

0 2 4 6 8 10 12time (s)

-50

0

50

retin

al s

lip (d

eg/s

)

no stabilizationstabilization

Fig. 8. Stabilization task with data from a straight running task, target position (top)and retinal slip (bottom). (Color figure online)

6 Conclusions

In this work we present the first complete model of gaze stabilization based onthe coordination of VCR and VOR and we validate it through an implemen-tation on a simulated humanoid robotic platform. We tested the model using,as a disturbance motion, inertial data acquired on a human subject performingvarious locomotion tasks that we replicated with the torso of the simulated iCubrobot. Results show that the model is able to perform well in almost all tri-als, with the exception of the straight running task, by reducing the retinal slipbelow 4 deg/s, thus achieving stable vision. In the running task, the model wasstill able to improve the stabilization by reducing the retinal slip to a quarter ofthe one from the task were no stabilization was present. As such, this model hasproven suitable to be used on humanoid robotic platforms, where it could helpduring visually guided locomotion tasks by stabilizing the camera view againstthe disturbance produced by walking.

Eye-Head Stabilization Mechanism for a Humanoid Robot 351

Acknowledgment. The research leading to these results has received funding fromthe European Union Seventh Framework Programme (FP7/2007-2013) under grantagreement no. 604102 (Human Brain Project). The authors would like to thank theItalian Ministry of Foreign Affairs, General Directorate for the Promotion of the “Coun-try System”, Bilateral and Multilateral Scientific and Technological Cooperation Unit,for the support through the Joint Laboratory on Biorobotics Engineering project.

References

1. Imai, T., Moore, S.T., Raphan, T., Cohen, B.: Interaction of the body, head, andeyes during walking and turning. Exp. Brain Res. 136(1), 1–18 (2001)

2. Hirasaki, E., Moore, S.T., Raphan, T., Cohen, B.: Effects of walking velocity onvertical head and body movements during locomotion. Exp. Brain Res. 127(2),117–130 (1999)

3. Pozzo, T., Berthoz, A., Lefort, L., Vitte, E.: Head stabilization during variouslocomotor tasks in humans. Exp. Brain Res. 85(1), 208–217 (1991)

4. Nadeau, S., Amblard, B., Mesure, S., Bourbonnais, D.: Head and trunk stabi-lization strategies during forward and backward walking in healthy adults. GaitPosture 18(3), 134–142 (2003)

5. Hashimoto, K., Kang, H.J., Nakamura, M., Falotico, E., Lim, H.O., Takanishi, A.,Laschi, C., Dario, P., Berthoz, A.: Realization of biped walking on soft ground withstabilization control based on gait analysis. In: IEEE International Conference onIntelligent Robots and Systems, pp. 2064–2069 (2012)

6. Kang, H.J., Hashimoto, K., Nishikawa, K., Falotico, E., Lim, H.O., Takanishi,A., Laschi, C., Dario, P., Berthoz, A.: Biped walking stabilization on soft groundbased on gait analysis. In: Proceedings of the IEEE RAS and EMBS InternationalConference on Biomedical Robotics and Biomechatronics, pp. 669–674 (2012)

7. Barnes, G.: Visual-vestibular interaction in the control of head and eye movement:the role of visual feedback and predictive mechanisms. Prog. Neurobiol. 41(4),435–472 (1993)

8. Shibata, T., Schaal, S.: Biomimetic gaze stabilization based on feedback-error-learning with nonparametric regression networks. Neural Netw. 14(2), 201–216(2001)

9. Viollet, S., Franceschini, N.: A high speed gaze control system based on thevestibulo-ocular reflex. Robot. Auton. Syst. 50(4), 147–161 (2005)

10. Porrill, J., Dean, P., Stone, J.V.: Recurrent cerebellar architecture solves the motor-error problem. Proc. Roy. Soc. Lond. B 271(1541), 789–796 (2004)

11. Franchi, E., Falotico, E., Zambrano, D., Muscolo, G., Marazzato, L., Dario, P.,Laschi, C.: A comparison between two bio-inspired adaptive models of vestibulo-ocular reflex (VOR) implemented on the iCub robot. In: 2010 10th IEEE-RASInternational Conference on Humanoid Robots, Humanoids 2010, pp. 251–256(2010)

12. Gay, S., Santos-Victor, J., Ijspeert, A.: Learning robot gait stability using neuralnetworks as sensory feedback function for central pattern generators. In: 2013IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),pp. 194–201, November 2013

13. Kryczka, P., Falotico, E., Hashimoto, K., Lim, H., Takanishi, A., Laschi, C., Dario,P., Berthoz, A.: Implementation of a human model for head stabilization on ahumanoid platform. In: Proceedings of the IEEE RAS and EMBS InternationalConference on Biomedical Robotics and Biomechatronics, pp. 675–680 (2012)

352 L. Vannucci et al.

14. Kryczka, P., Falotico, E., Hashimoto, K., Lim, H.O., Takanishi, A., Laschi, C.,Dario, P., Berthoz, A.: A robotic implementation of a bio-inspired head motionstabilization model on a humanoid platform. In: IEEE International Conferenceon Intelligent Robots and Systems, pp. 2076–2081 (2012)

15. Falotico, E., Cauli, N., Hashimoto, K., Kryczka, P., Takanishi, A., Dario, P.,Berthoz, A., Laschi, C.: Head stabilization based on a feedback error learningin a humanoid robot. In: Proceedings - IEEE International Workshop on Robotand Human Interactive Communication, pp. 449–454 (2012)

16. Falotico, E., Laschi, C., Dario, P., Bernardin, D., Berthoz, A.: Using trunk compen-sation to model head stabilization during locomotion. In: IEEE-RAS InternationalConference on Humanoid Robots, pp. 440–445 (2011)

17. Vijayakumar, S., Schaal, S.: Locally weighted projection regression: incrementalreal time learning in high dimensional space. In: ICML 2000: Proceedings of theSeventeenth International Conference on Machine Learning. Morgan KaufmannPublishers Inc., San Francisco, pp. 1079–1086 (2000)

18. Tolu, S., Vanegas, M., Luque, N.R., Garrido, J.A., Ros, E.: Bio-inspired adaptivefeedback error learning architecture for motor control. Biol. Cybern. 106(8–9),507–522 (2012)

19. Tolu, S., Vanegas, M., Garrido, J.A., Luque, N.R., Ros, E.: Adaptive and predictivecontrol of a simulated robot arm. Int. J. Neural Syst. 23(3), 1350010 (2013)

20. Bergamini, E., Ligorio, G., Summa, A., Vannozzi, G., Cappozzo, A., Sabatini, A.:Estimating orientation using magnetic and inertial sensors and different sensorfusion approaches: accuracy assessment in manual and locomotion tasks. Sens.(Switz.) 14(10), 18625–18649 (2014)

21. Tikhanoff, V., Cangelosi, A., Fitzpatrick, P., Metta, G., Natale, L., Nori, F.: Anopen-source simulator for cognitive robotics research: the prototype of the icubhumanoid robot simulator. In: Proceedings of the 8th Workshop on PerformanceMetrics for Intelligent Systems, PerMIS 2008, pp. 57–61. ACM, New York (2008)

22. Collewijn, H., Martins, A., Steinman, R.: Natural retinal image motion: origin andchange. Ann. N. Y. Acad. Sci. 374(1), 312–329 (1981)

Towards a Synthetic Tutor Assistant:The EASEL Project and its Architecture

Vasiliki Vouloutsi1(B), Maria Blancas1, Riccardo Zucca1, Pedro Omedas1,Dennis Reidsma3, Daniel Davison3, Vicky Charisi3, Frances Wijnen3,

Jan van der Meij3, Vanessa Evers3, David Cameron4, Samuel Fernando4,Roger Moore4, Tony Prescott4, Daniele Mazzei5, Michael Pieroni5,

Lorenzo Cominelli5, Roberto Garofalo5, Danilo De Rossi5,and Paul F.M.J. Verschure1,2

1 SPECS, N-RAS, DTIC, Universitat Pompeu Fabra (UPF), Barcelona, Spain{vicky.vouloutsi,maria.blancas,riccardo.zucca,pedro.omedas,

paul.verschure}@upf.edu2 Catalan Institute of Advanced Studies (ICREA), Barcelona, Spain

3 Human Media Interaction/ELAN, University of Twente, Enschede, Netherlands{d.reidsma,d.p.davison,v.charisi,f.m.wijnen,j.vandermeij,

v.evers}@utwente.nl4 Sheffield Robotics, University of Sheffield, Sheffield, UK

{d.s.cameron,s.fernando,r.k.moore,t.j.prescott}@sheffield.ac.uk5 University of Pisa, Pisa, Italy

{daniele.mazzei,michael.pieroni,roberto.garofalo,danilo.derossi}@centropiaggio.unipi.it, lorenzo.cominelli@for.unipi.it

Abstract. Robots are gradually but steadily being introduced in ourdaily lives. A paramount application is that of education, where robotscan assume the role of a tutor, a peer or simply a tool to help learnersin a specific knowledge domain. Such endeavor posits specific challenges:affective social behavior, proper modelling of the learner’s progress, dis-crimination of the learner’s utterances, expressions and mental states,which, in turn, require an integrated architecture combining perception,cognition and action. In this paper we present an attempt to improvethe current state of robots in the educational domain by introducing theEASEL EU project. Specifically, we introduce the EASEL’s unified robotarchitecture, an innovative Synthetic Tutor Assistant (STA) whose goalis to interactively guide learners in a science-based learning paradigm,allowing us to achieve such rich multimodal interactions.

Keywords: Education · Robotic tutor assistant · Pedagogical models ·Cognitive architecture · Distributed Adaptive Control

1 Introduction

Robot technology has become so advanced that automated systems are gradu-ally taking on a greater role in our society. Robots are starting to make theirc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 353–364, 2016.DOI: 10.1007/978-3-319-42417-0 32

354 V. Vouloutsi et al.

mark in many domains ranging from health-care and medicine, to automotivetechnologies, to service robots and robot companions.

Here our interest is in educational robots and, specifically, how automatedtechnologies can assist learners with their intellectual growth in the classroom.

Despite being still disputed, the use of robots in education has been shown topositively affect students’ concentration, learning interest and academic achieve-ments [1]. The introduction of assisting robots as part of a course has been provedeffective for integration, real-world issues, interdisciplinary work as well as criti-cal thinking [2]. Moreover, educational robots can be flexible, as they can assumethe role of mere tools [3], peers [4,5] or even tutors [6]. Although the preferredcharacterization is not yet conclusive [7], when used as a tutor or a peer, thusimplying a continuous robot–learner interaction, the robot’s design and behaviorbecome a crucial aspect [6,8,9]. Typically, robots in education are employed inscenarios ranging from technical education [10] (usually related to robotics ortechnology), to science [11] and learning of a foreign language [5], just to cite afew – see also [12] for a recent review and discussion about the use of robots ineducation.

Although a comprehensive examination of pedagogical theories in educationalrobotics is still lacking, two main approaches have been mostly influential. On oneside, Piaget’s theory of constructivism, which defines knowledge as the processin which an individual creates a meaning out of his own experiences [13]. On theother side, Papert’s theory of constructionism, stating that learning is the resultof building knowledge structures through progressive internalization of actionsand conscious engagement through making [14]. More recently, the work of theRussian psychologist Lev Vygotsky gained more and more attention, as it intro-duced the principle of scaffolding and the one of Zone of Proximal Development(ZPD) [15]. The former refers to the usage of tools or strategies providing help,whereas the latter corresponds to the distance between what a learner can doby himself and what he may do under the guidance of an effective mediator.All these pedagogical approaches are highly relevant to robotic applications ineducation, and a review of related studies can be found in [16].

The work we present here constitutes an effort to move one step forward in thedomain of robots in education by introducing the EASEL EU–project. EASEL isthe acronym of Expressive Agents for Symbiotic Education and Learning and it isa collaborative project aimed to explore and develop a theoretical understandingof the Human-Robot Symbiotic Interaction (HRSI) realized in the domain oftutoring. The final outcome of EASEL is the delivery of an innovative SyntheticTutor Assistant (STA), whose goal is to interactively guide learners (e.g., childrenin the age 8–11) using a science-based learning paradigm. The main focus of thispaper is in describing the underlying STA’s architecture that will allow the agentto interact with a human user in an educational task, whereas the theoreticalapproach and expected impact can be found in an accompanying paper (Reidsmaet al. [in preparation]).

The paper’s structure is as follows: in Sect. 2 we present the DistributedAdaptive Control (DAC) model of human and animal learning that serves as

Towards a Synthetic Tutor Assistant 355

both a pedagogical model and the control architecture of the STA. In Sect. 3 wepresent the developed educational scenarios that serve as both the test case ofthe architecture as well as an evaluation method. Finally, in Sect. 4 we introducethe overall architecture and present its individual components.

2 The DAC Architecture and Pedagogical Model

The STA’s main goal is to guide the learner through the science–based educa-tional scenario in order to maximize learning. Based on the perception of thesocial, communicative and educational context, the robot will respond accord-ingly within the educational scenario and the specific learning task. The STA’sreasoning and memory components need to continuously extract relevant knowl-edge from sequences of behavioural interactions over prolonged periods to learna model of the user and adapt its actions to the partner. To successfully interactwith the user, the STA thus requires an integrated architecture combining per-ception, cognition, and action. Here, we adopt the Distributed Adaptive Control(DAC) cognitive architecture (Fig. 1) as the basis to control the STA’s behavior.

Fig. 1. Schematic illustration of the Distributed Adaptive Control architecture (DAC)and its four layers: somatic, reactive, adaptive and contextual. Across the layers we candistinguish three main functional columns of organization: world related (exosensing,red), self related (endosensing, blue) and the interface to the world through action(green). The arrows indicate the flow of information. Adapted from [17]. (Color figureonline)

356 V. Vouloutsi et al.

DAC [17–19] is a robot-based neuronal model of perception, cognition andbehavior that is a standard in the domains of new artificial intelligence andbehavior-based robotics. It is biologically constrained and fully grounded since itautonomously generates representations of its primary sensory input [18,20]. TheDAC architecture is organized around four tightly coupled layers: Soma, Reac-tive, Adaptive and Contextual. Across these layers, three functional columnsof organization can be distinguished: exosensing, defined as the sensation andperception of the world, endosensing which detects and signals the states of theself, and the interface to the world through action.

The Soma represents the body itself and the information acquired from sensa-tion, needs and actuation. In the Reactive Layer, predefined sensorimotor reflexesare triggered by low complexity signals and are coupled to specific affectivestates of the agent. The Adaptive Layer extends these sensorimotor loops withacquired sensor and action states, allowing the agent to escape the predefinedreflexes through learning. Finally, the Contextual Layer develops the state–spaceacquired by the Adaptive Layer to generate behavioral plans and policies thatcan be expressed through actions.

Furthermore, DAC postulates that learning is organized along a similar hier-archy of complexity. In order to learn and consolidate new knowledge, the learnerundergoes three different learning stages: resistance, confusion and abduction.Resistance is the mechanism that results from defending one’s own world-modeland is highly related to maintaining the balance of one’s feeling of agency. Allagents possess an internalized world-model, which they need to reconsider whenexposed to new knowledge or experience. However, learners tend to hold overlyoptimistic and confused views about their level of knowledge: those with goodexpertise have a tendency to underestimate their overall capabilities, whereasthose who don’t, tend to overestimate their abilities [21]. Resistance is what con-sequently leads to the state of confusion, which actually generates the necessityto resolve the problem and learn through re-adapting. Adjusting to individual’sskills and progress helps the process of learning acquisition: it is therefore essentialto maintain a challenging enough task by adjusting the level of confusion to theskills and progress of the learner. Monitoring, controlling and adjusting confusionis what we define as shaping the landscape of success. Such approach is compa-rable to instructional scaffolding, a learning process intended to help the studentto cross what Vygotsky called the Zone of Proximal Development [15]. Confusionneeds to be controlled such that the task to learn is not too easy to become boringand, at the same time, it should not be too challenging, thus leading to a com-plete loss of motivation or development of learned helplessness [22]. The learnerneeds to believe that he can be effective in controlling the relevant events withinthe learning process [23]. Confusion is needed in order to discover and generatetheories to asses them later, that is, to be able to perform abduction, which is thevery process of acquiring the new knowledge.

In the EASEL project, the role of DAC is thus twofold: on the one hand, itacts as a control system to define and drive the STA’s actions; on the other hand,it serves as the pedagogical model through which learning can be achieved.

Towards a Synthetic Tutor Assistant 357

3 Science–Based Educational Scenarios

For the EASEL project we designed two interaction scenarios in real–life settingsbased on inquiry–based learning tasks. Typically, inquiry–based learning tasksinvolve active exploration of the world, asking questions, making discoveries andtesting hypotheses. The first scenario aims at teaching children about physicsconcepts based on the Piagetian balance-beam experiments. The second scenariois meant to help children learn about healthy living and physical exercise. Thesetwo situations allow to exploit two different perspectives: a formal teaching sce-nario compared to a more “voluntary free exploration”, and a language orientedversus more bodily involvement.

3.1 The Balance Beam

The balance beam problem was first described by Inhelder and Piaget to charac-terize and explain children’s stages of cognitive development [24]. Following thePiagetian work, Siegler developed a methodology which allowed him to classifychildren’s cognitive developmental stages on the base of four rules of increasingcomplexity that children of different ages would apply while solving the balancebeam task (Fig. 2) [25,26].

Briefly, in the balance beam scenario different numbers of weights are placedat varying distances from the fulcrum on the equally spaced pegs positioned onboth arms of the scale. Children explore the physics of the balance problem usingtangible materials and guided by an artificial agent (e.g., a robot or a virtualavatar) that serves as the bodily manifestation of the STA. Children are thenasked to predict the behavior of the beam given the configuration provided: ifit will stay in equilibrium, tip to the left or tip to the right. To succeed in thistask children have to identify the relevant physical concepts (i.e., weight anddistance) and understand the underlying multiplicative relation between the twovariables (i.e., the “torque rule”). The goal of the interaction is that the childlearns about balance and momentum by going through a series of puzzle taskswith the balance beam. The artificial agent is there to encourage the students, tohelp them get through the different tasks and to provide feedback; thus, learningimproves by constantly monitoring the learner’s progresses.

Fig. 2. Schematic illustration of the four rules assessed by Siegler [25]. At each devel-opmental stage one or both dimensions (i.e., weight and distance) are considered.For instance, Rule I exclusively considers the weight, whereas Rule IV considers bothweights and distance from the fulcrum.

358 V. Vouloutsi et al.

3.2 Healthy-Living

The second interaction scenario designed is that of healthy-living. This scenarioinvolves a synthetic agent assisting learners in an inquiry-based learning taskabout the benefits of physical exercise. Specifically, the children will investigatethe effects of different types of exercise on a number of physiological variables(e.g., heart rate, blood pressure, respiration rate, etc.). Two types of sessionsare considered: in the first one, the artificial agent will encourage the children toperform exercises at varying speeds. It will then provide information about theoutcome of the exercise in a friendly, accessible way, either via voice or througha video display allowing the children to have immediate feedback about theirown state. In the second type of session, the child sits down with a preparedworksheet about healthy-living, which is used to prompt a spoken dialogue withthe artificial agent. The child reads through the text on the worksheet, whichcomprises instructions and questions to the artificial agent about the previousinteraction. For instance, it explains how the physiological values are related toother forms of energy, such as the calories contained in different foods, and howmuch energy is burnt during an exercise. The worksheet prompts the child tostart off the interaction providing cues about the kind of things that can be saidto the artificial agent.

4 Architecture Overview

The proposed interaction scenarios consist of a humanoid robot, a handhelddevice (e.g., tablet) called the EASELscope and in the case of the balance beamtask, the Smart Balance Beam (SBB), a motorized scale equipped with sensorsto detect the object’s weight and position. Both scenarios imply a detailed inter-action between the learner and the robot which create a series of challenges suchas effective social behaviour to get a good social relation with the child, propermodelling of the students’ learning progress, various kinds of perception of thechilds utterances, expressions, bodily and mental states, as well as bodily andfacial expressions from the robot.

Rich multimodal interactions require an integrated architecture which com-bines perception, reasoning, and action (Sect. 2). Such integration is more thanjust technically running modules side-by-side and exchanging messages throughthe YARP middleware communication platform [27]. It also requires alignmenton the content, parameters, knowledge bases, and rule systems of all modules.In what follows, we present the unified EASEL architecture that makes all theinteraction possible.

All modules communicate and exchange messages through YARP via namedcommunication channels. This middleware platform allows us to not only dis-tribute the system over different machines, but also permits abstraction fromdifferent Operating Systems.

Towards a Synthetic Tutor Assistant 359

4.1 Modules and Mapping to the DAC Architecture

The integrated EASEL architecture is a practical incarnation of the conceptualarchitecture of the Distributed Adaptive Control (DAC) theory of the designprinciples underlying perception, cognition and action. Each implemented mod-ule in the EASEL architecture can be mapped to one or more of the core compo-nents of the DAC model that they embody and the specific modules are schemat-ically illustrated in Fig. 3.

Fig. 3. Overview of the EASEL architecture, where each implemented module ismapped to the core components of the DAC architecture.

The Speech recognition (ASR module), the SceneAnalyzer, the PhysioReaderand the EASELScope sensors embed the exosensing component of the Soma layerthrough which the states of the world are acquired and the internal drives areestablished. More precisely, the ASR module is based on the open-source Kaldispeech recognition toolkit [28] with an EASEL specific vocabulary, languagemodel and recognition grammar.

The SceneAnalyzer builds upon several other libraries to deliver integratedrecognition of multimodal features of the users and their behaviour [29,30]. Thephysiological signal acquisition module uses non-obtrusive and robust methodsfor obtaining information about the users physiological state: by integratingsensors in the robot or in the EASELscope tablet, information can be unob-trusively obtained without sensors worn or strapped to the body of user. TheEASELScope sensors allow detection of the current state of the balance beam

360 V. Vouloutsi et al.

allowing the EASEL system to respond to the actions of the user with respectto the learning materials.

In the reactive layer, ASAPRealizer module [31,32] is responsible for thechoreography of the behavior (verbal and non-verbal) of the STA using thegeneric robot-independent Behavior Markup Language (BML). We also use aneasily configurable XML binding between the BML and the motion primitives ofeach robotic platform (that serves as the physical instantiation of the STA). Suchapproach abstracts away from specific motor control by exposing more generalbehaviour specifications to the dialog manager and provides generalization acrossembodiments. The ASAPRealizer maps to the behavior component of the DACarchitecture by directly controlling the actuators of the somatic layer.

The Allostatic Control (AC) module currently implemented in the EASELarchitecture embraces both the Reactive and the Adaptive layers of DAC. Anhomeostatic controller continuously classifies the current state of each driveby sending fast requests for corrective actions to keep drives within optimalboundaries. The allostatic controller maintains consistency between drives in anadaptive way by assigning priorities to the different drives and making the appro-priate corrections to maintain coherence (e.g., by adapting the difficulty of thetask to the learners behavior). The learning algorithms of the allostatic con-troller [33,34] allow the STA to adapt its drives and homeostatic boundaries toa specific student’s behaviour and skills. Successful interactions (i.e., contextualcues and actions) are then stored as memory segments in the Object PropertiesCollector (OPC) module to build a model of the user.

The Exercise Generator contains the collection of learning exercises with alltheir different properties and difficulty levels. It selects the appropriate exercisegiven the current state of the tutoring model, the student model, and the outputof the AC. This information is shared with the Flipper Dialog Manager to allowthe robotic assistant to discuss with the child the progress of the exercise.

The Object Properties Collector (OPC) embodies the Contextual Layer’smemory components of DAC. At this stage of development, the OPC imple-ments the memory for events that can be stored and distributed to the otherSTA’s modules as well as its short-term memory component. At each instantof the interaction, the OPC can temporarily retain ongoing perceptions, actionsand values (i.e., outcomes of the current interaction) as segments of memory(relations). These relations allow the definition of rules for specific interac-tions that can be further stored as long-term memories if a high-level goal issuccessfully achieved. Between the OPC and the sensors lies the multimodalunderstanding module, a light and simple interpreter of speech, emotions andspeaker’s probabilities, which simplifies the requirements for the Flipper Dia-log Manager’s scripts. Flipper offers flexible dialog specification via informationstate and rule-based templates to trigger information state changes as well asbehaviour requests [35].

The robots (Zeno (Hanson Robotics, Hong Kong) and FACE [36]), the vir-tual robot avatars and the EASEL-scope hidden state visualizer correspond tothe DAC effectors and represent the main interface of the STA with the world

Towards a Synthetic Tutor Assistant 361

(Fig. 4). The EASELscope offers an augmented reality (AR) interface that allowsthe learner to interact with the task materials. It can be used to present extrainformation to the child about the learning content. This allows the system tovary between different ways of scaffolding the learning of the user. For instance,using the EASELscope we can present “hidden information” about the balancebeam, such as the weights of the pots, or the forces acting on the arms of thebeam; both are types of scaffolds in learning that would not be possible withoutthe scope (Fig. 4).

Fig. 4. The Zeno robot, the EASELscope and the Smart Balance Beam (SBB). Zenoacts as the embodiment of the STA and continuously interacts with the learner. Infor-mation about the task is displayed as augmented reality on top of the physical SBB.

5 Discussion and Conclusions

In this paper we presented the DAC based EASEL architecture, designed toguide learners through learning in two science-based educational scenarios. Eachmodule within the framework has been integrated in a cohesive setup and theconfiguration options, models, behaviour repertoires and dialog scripts will allowus to validate the EASEL system through specific experiments with child-robotinteraction in the proposed learning scenarios.

The way the architecture is organized gives us three key advantages: scala-bility, configurability and abstraction. This allows us to easily add sensory com-ponents with negligible changes to the main core of the system: it is sufficient toadd the input to the multimodal understanding module that, in turn, will storethe new information with an appropriate format in the OPC module. Further-more, all modules are fully configurable: for instance, we can add new behaviors(ASAPRealizer), drives (Allostatic Control) as well as dialogues (Flipper) in aneasy way with the usage of configuration files such as XML scripts. Thus, anyadditional implementation for the needs of the EASEL architecture (in terms ofscenarios or sensory inputs) can be done in a flexible way. Finally, the proposed

362 V. Vouloutsi et al.

architecture permits abstraction from the physical manifestation of the STA, ina way that using the same scenario, we can choose the robotic platform (or evenavatar) with small changes in the main core of the system.

At this stage, we are now ready to start validating our educational architec-ture and focus on concrete long-term studies on human-robot symbiotic interac-tions in learning tasks. For instance, by looking at the types of hypotheses thatthe child frames and the outcome in solving exercises, the STA can continuouslybuild and refine a model of the student’s understanding of the task. By varyingthe exercises, the scaffolding provided by the robot or the hidden-states informa-tion provided through the EASELScope, the STA can explore the most effectivelearning strategies for a specific task. By modifying the social and conversationalstrategies of the robot, the STA can extract the best “personality” and style ofthe robot leading to a good relationship between the child and the robot itselfand their impact on the overall learning process.

Acknowledgments. This work is supported by grants from the European ResearchCouncil under the European Union’s 7th Framework Programme FP7/2007-2013/ERCgrant agreement n. 611971 (EASEL) and n. 341196 (CDAC) to Paul F. M. J. Verschure.

References

1. Han, J.-H., Jo, M.-H., Jones, V., Jo, J.-H.: Comparative study on the educationaluse of home robots for children. J. Inf. Process. Syst. 4(4), 159–168 (2008)

2. Beer, R.D., Chiel, H.J., Drushel, R.F.: Using autonomous robotics to teach scienceand engineering. Commun. ACM 42(6), 85–92 (1999)

3. Mondada, F., Bonani, M., Raemy, X., Pugh, J., Cianci, C., Klaptocz, A., Magnenat,S., Zufferey, J.-C., Floreano, D., Martinoli, A.: The e-puck, a robot designed foreducation in engineering. In: Proceedings of the 9th Conference on AutonomousRobot Systems, Competitions, vol. 1, pp. 59–65. Instituto Politecnico de CasteloBranco (2009)

4. Wijnen, F., Charisi, V., Davison, D., van der Meij, J., Reidsma, D., Evers, V.:Inquiry learning with a social robot: can you explain that to me? In: Heerink,M., de Jong, M. (eds.) Proceedings of New Friends 2015: The 1st internationalconference on social robotics in therapy and education, pp. 24–25, WindesheimFlevoland, Almere (2015)

5. Kanda, T., Hirano, T., Eaton, D., Ishiguro, H.: Interactive robots as social partnersand peer tutors for children: a field trial. Hum.-Comput. Interact. 19(1), 61–84(2004)

6. Saerbeck, M., Schut, T., Bartneck, C., Janse, M.D.: Expressive robots in education:varying the degree of social supportive behavior of a robotic tutor. In: Proceedingsof the SIGCHI Conference on Human Factors in Computing Systems, pp. 1613–1622. ACM (2010)

7. Shin, N., Kim, S.: Learning about, from, with robots: students’ perspectives. In:The 16th IEEE International Symposium on Robot and Human Interactive Com-munication, RO-MAN 2007, pp. 1040–1045. IEEE (2007)

Towards a Synthetic Tutor Assistant 363

8. Vouloutsi, V., Munoz, M.B., Grechuta, K., Lallee, S., Duff, A., Llobet, J.-Y.P.,Verschure, P.F.M.J.: A new biomimetic approach towards educational robotics:the distributed adaptive control of a synthetic tutor assistant. New Frontiers inHuman-Robot Interaction, p. 22 (2015)

9. Blancas, M., Vouloutsi, V., Grechuta, K., Verschure, P.F.M.J.: Effects of therobot’s role on human-robot interaction in an educational scenario. In: Wilson,S.P., Verschure, P.F.M.J., Mura, A., Prescott, T.J. (eds.) Living Machines 2015.LNCS, vol. 9222, pp. 391–402. Springer, Heidelberg (2015)

10. Balch, T., et al.: Designing personal robots for education: hardware, software andcurriculum. IEEE Pervasive Comput. 7(2), 5–9 (2008)

11. Highfield, K., Mulligan, J., Hedberg, J.: Early mathematics learning through explo-ration with programmable toys. In: Proceedings of the Joint Meeting of PME 32and PME-NA, pp. 169–176. Citeseer (2008)

12. Mubin, O., Stevens, C.J., Shahid, S., Al Mahmud, A., Dong, J.-J.: A review of theapplicability of robots in education. J. Technol. Educ. Learn. 1, 209–215 (2013)

13. Piaget, J., Inhelder, B.: The Psychology of the Child. Basic Books, New York(1972)

14. Papert, S., Harel, I.: Situating constructionism. Constructionism 36, 1–11 (1991)15. Vygotsky, L.S.: Mind in Society: The Development of Higher Psychological

Processes. Harvard University Press, Cambridge (1980)16. Charisi, V., Davison, D., Wijnen, F., Van Der Meij, J., Reidsma, D., Prescott, T.,

Van Joolingen, W., Evers, V.: Towards a child-robot symbiotic co-development:a theoretical approach. In: AISB Convention 2015, The Society for the Study ofArtificial Intelligence and the Simulation of Behaviour (AISB) (2015)

17. Verschure, P.F.M.J.: Distributed adaptive control: a theory of the mind, brain,body nexus. Biologically Inspired Cogn. Architectures 1, 55–72 (2012)

18. Verschure, P.F.M.J., Voegtlin, T., Douglas, R.J.: Environmentally mediated syn-ergy between perception and behaviour in mobile robots. Nature 425, 620–624(2003)

19. Verschure, P.F.M.J., Pennartz, C.M., Pezzulo, G.: The why, what, where, when andhow of goal-directed choice: neuronal and computational principles. Phil. Trans.R. Soc. B 369(1655), 20130483 (2014)

20. Maffei, G., Santos-Pata, D., Marcos, E., Sanchez-Fibla, M., Verschure, P.F.M.J.:An embodied biologically constrained model of foraging: from classical and operantconditioning to adaptive real-world behavior in DAC-X. Neural Netw. 72, 88–108(2015)

21. Kruger, J., Dunning, D.: Unskilled and unaware of it: how difficulties in recognizingone’s own incompetence lead to inflated self-assessments. J. Pers. Soc. Psychol.77(6), 1121 (1999)

22. Abramson, L.Y., Seligman, M.E., Teasdale, J.D.: Learned helplessness in humans:critique and reformulation. J. Abnorm. Psychol. 87(1), 49 (1978)

23. Seligman, M.E.: Learned helplessness. Annu. Rev. Med. 23(1), 407–412 (1972)24. Inhelder, B., Piaget, J.: The Growth of Logical Thinking from Childhood to Ado-

lescence: An Essay on the Construction of Formal Operational Structures. BasicBooks, New York (1958)

25. Siegler, R.S.: Three aspects of cognitive development. Cogn. Psychol. 8(4), 481–520(1976)

26. Siegler, R.S., Strauss, S., Levin, I.: Developmental sequences within and betweenconcepts. Monogr. Soc. Res. Child Dev. 46, 631–683 (1981)

27. Metta, G., Fitzpatrick, P., Natale, L.: Yarp: yet another robot platform. Int. J.Adv. Rob. Syst. 3(1), 43–48 (2006)

364 V. Vouloutsi et al.

28. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N.,Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P., et al.: The kaldi speechrecognition toolkit. In: IEEE 2011 workshop on automatic speech recognition andunderstanding, no. EPFL-CONF-192584. IEEE Signal Processing Society (2011)

29. Zaraki, A., Mazzei, D., Giuliani, M., De Rossi, D.: Designing and evaluating asocial gaze-control system for a humanoid robot. IEEE Trans. Hum.-Mach. Syst.44(2), 157–168 (2014)

30. Zaraki, A., Mazzei, D., Lazzeri, N., Pieroni, M., De Rossi, D.: Preliminary imple-mentation of context-aware attention system for humanoid robots. In: Lepora,N.F., Mura, A., Krapp, H.G., Verschure, P.F.M.J., Prescott, T.J. (eds.) LivingMachines 2013. LNCS, vol. 8064, pp. 457–459. Springer, Heidelberg (2013)

31. Reidsma, D., van Welbergen, H.: AsapRealizer in practice - a modular and exten-sible architecture for a bml realizer. Entertainment Comput. 4(3), 157–169 (2013)

32. van Welbergen, H., Yaghoubzadeh, R., Kopp, S.: AsapRealizer 2.0: the next stepsin fluent behavior realization for ECAs. In: Bickmore, T., Marsella, S., Sidner, C.(eds.) IVA 2014. LNCS, vol. 8637, pp. 449–462. Springer, Heidelberg (2014)

33. Vouloutsi, V., Lallee, S., Verschure, P.F.M.J.: Modulating behaviors using allo-static control. In: Lepora, N.F., Mura, A., Krapp, H.G., Verschure, P.F.M.J.,Prescott, T.J. (eds.) Living Machines 2013. LNCS, vol. 8064, pp. 287–298. Springer,Heidelberg (2013)

34. Lallee, S., Vouloutsi, V., Wierenga, S., Pattacini, U., Verschure, P.F.M.J: Efaa: acompanion emerges from integrating a layered cognitive architecture. In: Proceed-ings of the 2014 ACM/IEEE International Conference on Human-Robot Interac-tion, pp. 105–105. ACM (2014)

35. ter Maat, M., Heylen, D.: Flipper: an information state component for spokendialogue systems. In: Vilhjalmsson, H.H., Kopp, S., Marsella, S., Thorisson, K.R.(eds.) IVA 2011. LNCS, vol. 6895, pp. 470–472. Springer, Heidelberg (2011)

36. Lazzeri, N., Mazzei, D., Zaraki, A., De Rossi, D.: Towards a believable social robot.In: Lepora, N.F., Mura, A., Krapp, H.G., Verschure, P.F.M.J., Prescott, T.J. (eds.)Living Machines 2013. LNCS, vol. 8064, pp. 393–395. Springer, Heidelberg (2013)

Aplysia Californica as a Novel Sourceof Material for Biohybrid Robots

and Organic Machines

Victoria A. Webster(B), Katherine J. Chapin, Emma L. Hawley, Jill M. Patel,Ozan Akkus, Hillel J. Chiel, and Roger D. Quinn

Case Western Reserve University, 10900 Euclid Ave, Cleveland, OH 44106, USAvaw4@case.edu

http://www.case.edu

Abstract. Aplysia californica is presented as a novel source of actu-ator and scaffold material for biohybrid robots. Collagen isolated fromthe Aplysia skin has been fabricated into gels and electrocompacted scaf-folds. Additionally, the I2 muscle from the Aplysia buccal mass had beenisolated for use as an organic actuator. This muscle has been character-ized and the maximum force was found to be 58.5 mN with a maximummuscle strain of 12 ± 3 %. Finally, a flexible 3D printed biohybrid robothas been fabricated which is powered by the I2 muscle and is capable oflocomotion at 0.43 cm/min under field stimulation.

Keywords: Biohybrid devices · Organic actuators · Biorobotics ·Aplysia californica

1 Introduction

The field of biohybrid devices involves the combination of synthetic substrateswith living tissue in order to produce a functioning device. A common applicationof such devices is in the area of biohybrid robotics. By seeding contractile cellson synthetic or organic polymers, researchers are able to produce devices usingthin films for lab-on-chip style testing [1–4], or even devices capable of swimming[5,6], and crawling [7–11]. Recent efforts in the area of biohybrid robotics and thedevelopment of organic actuators have focused on mammalian [5–10] or avian[11] contractile cells to serve as actuators. However, both mammalian and aviancells require precise pH, temperature, osmotic balances, and CO2 levels. Suchrequirements make these devices difficult to fabricate and maintain, and limittheir applicability beyond the lab.

A few studies have investigated the possibility of using frog [12] or insect [13,14]muscle to power biohybrid devices. Instead, we propose Aplysia californica as apotential source for contractile tissue using the I2 muscle and scaffold material,using isolated collagen-likematerial from the skin.Aplysia californica is a species ofsea slug native to the southern coast of California whose range extends into Mexico.

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 365–374, 2016.DOI: 10.1007/978-3-319-42417-0 33

366 V.A. Webster et al.

These animals live in the intertidal and sub-littoral zones of the photic zone andas a consequence can be subjected to large temperature changes as well as changesin the surrounding salt water environment [15]. As a result, Aplysia californica arevery robust, making them an ideal candidate for use in biohybrid robotics.

In order to build biohybrid devices, suitable scaffolds are needed. Such scaf-folds should be strong enough to allow handling and assembly, while beingdeformable enough to allow the muscle to deflect the substrate. In this study,we present a basic characterization of the I2 muscle as well as a protocol forisolating collagen from the skin. Additionally, biobots powered by the I2 musclehave been 3D printed using flexible polymeric resin.

Fig. 1. Right: Aplysia californica. Left: a diagram of the relevant anatomy of theAplysia. The I2 muscle (red) has been highlighted on the buccal mass (white). (Colorfigure online)

2 Methods

2.1 Materials

Aplysia californica were obtained from Marinus Scientific and South Coast Bio-marine and maintained in artificial seawater (Instant Ocean, Aquarium Systems)in aerated aquariums at a temperature of 16oC. Animals weighing 200–450 g wereanesthetized by injecting isotonic MgCl2 (333 mM) at a volume in millilitersequal to 50 % of the animal’s body weight in grams. For isolation of collagenlike materials, the skin was harvested and stored at −200C until processed.A diagram of relevant sea slug anatomy is presented in Fig. 1.

2.2 I2 Muscle Isolation

Using tweezers and surgical scissors, a horizontal cut was made behind therinophores, followed by a perpendicular cut towards the jaws of the slug. Thesecuts are only as deep as the skin. Any connective tissue limiting access to thebuccal mass of the slug was cut away from the skin. The esophagus was iden-tified, exposed and cut above the cerebral ganglia. Nerves or connective tis-sue connected to the buccal mass were removed and the scissors were used tocut the buccal mass at the jaws of the slug. The buccal mass was then placedin a small dish in order to hold it stationary for the removal of the I2 mus-cle. The buccal ganglia was removed to reveal the entirety of the I2 muscle

I2 Powered Biohybrid Devices and Organic Actuators 367

which was then removed by carefully cutting it away from the tissue at boththe origin and insertion lines (Fig. 2). The I2 muscle was stored in Aplysia saline(0.46 M NaCl, 0.01 M KCl, 0.01 M MOPS, 0.01 M C6H12O6, 0.02 M MgCl2·6H2O,0.033 M MgSO4·7H2O, 0.01 M CaCl2·2H2O, pH 7.5) until used.

Fig. 2. A diagram of the animal dissection and I2 isolation process.

2.3 I2 Muscle Characterization

Cantilever-Based Actuator Tension Measurement: In order to character-ize the muscle force and strain capabilities, flexible cantilevers were printed using aFormlabs Form 1+ stereolithographic (SLA) printer with photocurable resin. TheI2 muscles were attached to the cantilever and base using gel based cyanoacry-late (Fig. 3). In order to produce contractions, the muscle was stimulated usingtwo platinum electrodes spaced 3 cm apart. Stimulation patterns were based onthe stimulation activity seen in Aplysia during biting behavior. The muscle wasstimulated at 20 Hz for a duration of 3 s followed by a 3 s rest [16]. In order toinvestigate the effect of stimulation current on muscle force, the muscle was stimu-lated at 0.25 V/mm, 0.3 V/mm, and 0.42 V/mm. At each voltage, the muscle wasstimulated 10 times and then allowed to rest for 5 min. The muscle force was thendetermined based on the deflection of the cantilever.

Muscle Dimensions: The dimensions of the Y-shaped I2 muscle (Fig. 4) weremeasured first across the two arms to measure the width of the Y, and sec-ond, along the base of the Y starting from the cleft between the two arms tothe connective tissue at the center of the edge which connected to the I1/I3complex (Fig. 4).

368 V.A. Webster et al.

Fig. 3. The cantilever setup used for muscle characterization. Initial bending of thecantilever was due to the cyanoacrylate and was accounted for in force calculations.

Mechanical Testing: Mechanical tests were performed using a 5 lb. loadcell (Omega Engineering, Connecticut, USA) calibrated for use with the TestResources universal testing machine (Test Resources, Minne sota, USA). Sam-ples were tested using strain controlled monotonic tensile loading at a rate of1 mm/min. Failure load and displacement were recorded by Test Resources,and were used with width analyzed in ImageJ and thickness measurementstaken using a FemtoTools MicroRobotics Testing and Handling Module with aFT-S1000 Microforce sensing probe to calculate modulus.

2.4 Aplysia Collagen Isolation and Scaffold Fabrication

The Aplysia collagen-like material isolation procedure was developed based on apreviously reported procedure for isolating collagen from the oyster Crassostreagigas [17]. Aplysia skin was thawed at room temperature and minced with scis-sors before being suspended in 0.1 M NaOH (10:1 volume:weight) for 24 h at 4oC.After 24 h, the mixture was centrifuged at 10000 x g for 20 min. All centrifugingwas performed at 4oC. The supernatant was discarded and the remaining residuewas washed by gently stirring in distilled water for 20 min. The washed solutionwas then centrifuged again and the supernatant discarded. The residue was thensuspended in disaggregating solution composed of 0.1 M Tris-HCL (pH 8.0) con-taining 0.5M NaCl, 0.05 M EDTA, and 0.2 M 2-mercaptoethanol [18]. This extrac-tion occurred over 7 days with gentle stirring at 4oC, after which the solution wascentrifuged and the supernatant discarded. The residue was washed as before andresuspended in 4 M guanidine hydrochloride in 0.05 M Tris-HCL (pH 7.5) at a vol-ume to weight ratio of 1/2 the original tissue weight. After 24 h at 4oC, the solutionwas again centrifuged, the supernatant was discarded and the residue was washed.The residue was further processed in 0.5 M acetic acid containing porcine pepsin(enzyme to substrate ratio of 1:20) for 48 h at 4oC. After 48 h, the solution wasfractionated by salt precipitation by adding NaCl to a final concentration of 2 M.

I2 Powered Biohybrid Devices and Organic Actuators 369

Fig. 4. A diagram of I2 muscle length measurements. The first measurement (M1) ismade between the arms of the Y while the second measurement (M2) measures fromthe base of the Y to the cleft between the arms.

The solution was again centrifuged and the supernatant discarded. The collectedcollagen was dialyzed for 24 h against 0.02 M Na2HPO4.

Aplysia collagen was gelled by incubation at 25oC for 3 h. Additionally, col-lagen was dialyzed against distilled water for 18 h with water changes every 2 hfor the first 6 h in order to produce stock for electrocompaction. This stock waselectrocompacted into aligned threads using a dual wire setup [19] and into com-pacted, unaligned sheets using plate electrodes [11]. Compaction was achievedby application of 15 V for 30 s.

2.5 Flexible 3D-printed Devices

Flexible 3D-printed devices have been fabricated using a Formlabs Form 1+SLA printer with photocurable resin. The resulting structures had a modulus of2.29 - 2.76 MPa (as reported on the Formlabs datasheet), and are strong enoughto be manipulated but flexible enough that the I2 muscle was able to deflect thinmembers. The I2 muscle was attached to each arm half way along the lengthand to the rear of the body using cyanoacrylate (Fig. 8 A and B). In order toprovide surface traction, devices were tested in a dish with a sandpaper base.

370 V.A. Webster et al.

3 Results and Discussion

3.1 I2 Muscle Characterization

The Effect of Animal Size on Muscle Dimensions: Muscle measure-ments were made and compared to the mass of the animal for animals weighing280–420 g. Neither the muscle width nor the base length correlated strongly tothe animal mass (Fig. 5) for the range tested. This presents one possible draw-back to the use of the I2 muscle as an actuator because it is not possible topredict the size of the muscle with a high degree of certainty prior to dissection.However, all animals tested were mature adult animals and only a narrow sizerange was checked. If the size range was widened to include small juveniles aswell as larger adults a correlation may emerge. However, for the purposes ofdesigning a biohybrid device, the observed variability can be accommodated inthe design of the structure.

Fig. 5. Measurements for M1 (left) and M2 (right). Neither measurement correspondedstrongly to the animal mass.

Mechanical Testing: Tensile testing of 4 I2 muscle samples in the muscle fiberdirection resulted in a Young’s modulus of 0.49 ± 0.99 MPa.

The Effect of Stimulation Voltage on Muscle Tension: During this studythe I2 muscle had a maximum actuating force of 58.5 mN which occurred whenthe muscle was stimulated at 0.3 V/mm. However, previous literature indicatesthat the muscle may be capable of an active tension of up to 120 mN [20]. Thelower force output may be due to the use of field stimulation, rather than suctionelectrodes as were used by Yu et al. [20]. However, suction electrodes are stiffand may interfere with muscle motion as well as biohybrid device performance,making field stimulation preferable.

Noticeable deflection did not occur at or below 0.2 V/mm (data not shown).Noticeable deflection began at 0.25 V/mm and increased with increasing voltage.Additionally, no noticeable difference in active tension was recorded between 0.3

I2 Powered Biohybrid Devices and Organic Actuators 371

Fig. 6. The effect of stimulation voltage on muscle force. 0.25 V/mm resulted in the low-est muscle force while there was no appreciable difference between 0.3 and 0.42 V/mm.

and 0.42 V/mm (Fig. 6). Therefore a voltage of 0.3 V/mm is recommended forstimulation in order to maximize actuator force while minimizing fatigue andpower requirements.

Muscle Strain: The I2 muscle was able to reach considerable strains whencontracting with an average maximum strain of 12 ± 3%.

3.2 Aplysia Collagen

Isolated collagen was capable of forming robust gels which retained their moldedshape (Fig. 7.A). Additionally, the compacted threads showed strong alignmentalong the thread axis (Fig. 7.B). These structures could be suitable for use asscaffolds for cell based organic machines [11].

3.3 Flexible 3D-printed Devices

Flexible 3D-printed devices were capable of locomotion under the actuating forceof the I2 muscle. Devices designed with frictionally anisotropic legs (Fig. 8.A)were capable of locomotion at speeds of approximately 0.5 cm/min. A charac-teristic example of locomotion is presented in Fig. 8 B and C. All stimulationtrials began with 30–35 s without stimulation in order to verify that the devicewas not moving due to residual fluid flow in the surrounding medium. Duringeach step the device experienced some degree of slip, indicating that groundcontact and frictional anisotropy could be improved (Fig. 8.D). Over the course

372 V.A. Webster et al.

Fig. 7. A: Aplysia collagen gel. Gels were robust to manipulation and held theirmolded shape. B: Electrocompacted Aplysia collagen thread imaged using polarizedlight microscopy. The collagen shows strong alignment along the thread axis (bluecoloration, direction indicated by the double headed arrow) (Color figure online)

Fig. 8. A: A rendering of the 3D printed biohybrid robot. The muscle was attached tothe two arms and between the legs. Angled legs served to provide frictional anisotropy toallow forward motion. B: The starting position of the biohybrid device. C. The positionof the biohybrid device after 2 min of stimulation. The device moved approximately1 cm. D: A plot of the devices motion with respect to time. Electrical stimulationbegan as indicated by the black arrow.

of each trial, fatigue is observed in the device, with initial steps producing moreforward motion than later steps. In order to improve the device lifetime, alter-native stimulation techniques, such as including the buccal ganglia, which con-tains the motor neurons, and using chemical stimulation, should be investigated.

I2 Powered Biohybrid Devices and Organic Actuators 373

Additionally, the device was observed to become caught on larger features of therough dish, which stalled motion until later steps freed the device. In order toreduce the occurrence of such events, the geometry of the underside of the devicecan be improved.

4 Conclusion

In this study we have demonstrated several ways in which Aplysia californicacan be used as a novel source of materials for biohybrid robots. Additionally,we have provided one application for the use of the I2 muscle to powered 3Dprinted biohybrid robots. The collagen-like material and muscle from the Aplysiaoffer a robust alternative to mammalian or avian sources. In the future, gangliaor individual neurons may be isolated from the animal to provide an organiccontrol system. Such devices could have applications in environmental sensingand aquatic surveillance.

Acknowledgments. This material is based upon work supported by the NationalScience Foundation Graduate Research Fellowship under Grant No. DGE-0951783 anda GAANN Fellowship (Grant No. P200A150316). This study was also funded in partby grants from the National Science Foundation (Grant No. DMR-1306665), and theNational Institute of Health (Grant No. R01 AR063701). Any opinion, findings, andconclusions or recommendations expressed in this material are those of the authors anddo not necessarily reflect the views of the National Science Foundation.

References

1. Feinberg, A.W., Feigel, A., Shevkoplyas, S.S., Sheehy, S., Whitesides, G.M., Parker,K.K.: Muscular thin films for building actuators and powering devices. Science80(317), 1366–1370 (2007)

2. Grosberg, A., Alford, P.W., McCain, M.L., Parker, K.K.: Ensembles of engineeredcardiac tissues for physiological and pharmacological study: Heart on a chip. LabChip 11(24), 4165 (2011)

3. Vannozzi, L., Ricotti, L., Cianchetti, M., Bearzi, C., Gargioli, C., Rizzi, R., Dario,P., Menciassi, A.: Self-assembly of polydimethylsiloxane structures from 2D to 3Dforbio-hybrid actuation. Bioinspir. Biomim. 10(5), 056001 (2015)

4. Nagamine, K., Kawashima, T., Sekine, S., Ido, Y., Kanzaki, M., Nishizawa, M.:Spatiotemporally controlled contraction of micropatterned skeletal muscle cells ona hydrogel sheet. Lab Chip 11(3), 513–7 (2011)

5. Nawroth, J.C., Lee, H., Feinberg, A.W., Ripplinger, C.M., McCain, M.L.,Grosberg, A., Dabiri, J.O., Parker, K.K.: A tissue-engineered jellyfish with bio-mimetic propulsion. Nat. Biotechnol. 30(8), 792–797 (2012)

6. Williams, B.J., Anand, S.V., Rajagopalan, J., Saif, M.T.A.: A self-propelled bio-hybrid swimmer at low Reynolds number. Nat. Commun. 5, 3081 (2014)

7. Xi, J., Schmidt, J.J., Montemagno, C.D.: Self-assembled microdevices driven bymuscle. Nat. Mater. 4, 180–184 (2005)

8. Chan, V., Park, K., Collens, M.B., Kong, H., Saif, T.A., Bashir, R.: Developmentof miniaturized walking biological machines. Sci. Rep. 2, 857 (2012)

374 V.A. Webster et al.

9. Cvetkovic, C., Raman, R., Chan, V., Williams, B.J., Tolish, M., Bajaj, P., Sakar,M.S., Asada, H.H., Taher, A., Taher A Saif, M., Bashir, R.: Three-dimensionallyprinted biological machines powered by skeletalmuscle. PNAS 111(28), 10125–10130 (2014)

10. Kim, J., Park, J., Yang, S., Baek, J., Kim, B., Lee, S.H., Yoon, E.-S., Chun, K.,Park, S.: Establishment of a fabrication method for a long-term actuated hybridcell robot. Lab Chip 7, 1504–1508 (2007)

11. Webster, V.A., Hawley, E.L., Akkus, O., Chiel, H.J., Quinn, R.D.: Fabricationof electrocompacted aligned collagen morphs for cardiomyocyte powered livingmachines. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A., Prescott, T.J. (eds.)Living Machines 2015. LNCS, vol. 9222, pp. 429–440. Springer, Heidelberg (2015)

12. Herr, H., Dennis, R.G.: A swimming robot actuated by living muscle tissue. J.Neuroeng. Rehabil. 1, 6 (2004)

13. Baryshyan, A.L., Woods, W., Trimmer, B.A., Kaplan, D.L.: Isolation andmaintenance-free culture of contractile myotubes from Manduca sexta embryos.PLoS One 7(2), e31598 (2012)

14. Uesugi, K., Shimizu, K., Akiyama, Y., Hoshino, T., Iwabuchi, K., Morishima,K.: Contractile performance and controllability of insect muscle-powered bioactu-ator with different stimulation strategies for soft robotics. Soft Robot. 3(1), 13–22(2016)

15. Kandel, E.: Behavioral Biology of Aplysia: Contribution to the Comparative Studyof Opistobranch Molluscs. W.H. Freeman and Company, San Francisco (1979)

16. Hurwitz, I., Neustadter, D., Morton, D.W., Chiel, H.J., Susswein, A.J.: Activitypatterns of the B31/B32 pattern initiators innervating the I2 muscle of the buccalmass during normal feeding movements in Aplysia californica. J. Neurophysiol.75(4), 1309–26 (1996)

17. Mizuta, S., Miyagi, T., Yoshinaka, R.: Characterization of the quantitatively majorcollagen in the mantle of oyster Crassostrea gigas. Fish. Sci. 71(1), 229–235 (2005)

18. Matsumura, T.: Shape, size and amino acid composition of collagen fibril of thestarfish asterias amurensis

19. Cheng, X.: Umut a Gurkan, Christopher J Dehen, Michael P Tate, Hugh WHillhouse, Garth J Simpson, and Ozan Akkus.: An electrochemical fabricationprocess for the assembly of anisotropically oriented collagen bundles. Biomaterials29(22), 3278–3288 (2008)

20. Yu, S.N., Crago, P.E., Chiel, H.J.: Biomechanical properties and a kinetic simula-tion model of the smooth muscle I2 in the buccal mass of Aplysia. Biol. Cybern.81, 505–513 (1999)

A Soft Pneumatic Maggot Robot

Tianqi Wei1(B), Adam Stokes2, and Barbara Webb1

1 School of Informatics, Institute of Perception, Action and Behaviour,University of Edinburgh, Edinburgh EH8 9AB, UK

chitianqilin@163.com, B.Webb@ed.ac.uk2 School of Engineering, Scottish Microelectronics Centre, University of Edinburgh,

Edinburgh EH9 3FF, UKA.a.stokes@ed.ac.uk

Abstract. Drosophila melanogaster has been studied to gain insightinto relationships between neural circuits and learning behaviour. Totest models of their neural circuits, a robot that mimics D. melanogasterlarvae has been designed. The robot is made from silicone by casting in3D printed moulds with a pattern simplified from the larval muscle sys-tem. The pattern forms air chambers that function as pneumatic musclesto actuate the robot. A pneumatic control system has been designed toenable control of the multiple degrees of freedom. With the flexible bodyand multiple degrees of freedom, the robot has the potential to resemblemotions of D. melanogaster larvae, although it remains difficult to obtainaccurate control of deformation.

1 Introduction

We have designed a robot to mimic Drosophila melanogaster larvae (maggots),as a platform to test and verify their learning and chemotaxis models. Drosophilaas a model system has a useful balance between relatively small number of neu-rons yet interestingly complex behaviours [10]. Many genetic techniques, such asGAL4/UAS systems developed by Brand and Perrimon [2], facilitate research onthe connectivity and dynamics of the circuits. As a result, a number of necessarycomponents of neural circuits for sensorimotor control and learning are beingfound and modelled. Currently, the models are tested by comparing betweenwildtype and genetic mutation lines, or using simulations of neural circuits andcomparing output with biological experimental recordings. To test models in awider environment, more similar to a larva, a physical agent that copies proper-ties of the larval body is important.

Larvae have high degrees of freedom (DOFs) and flexible bodies. As a result,they are able to do delicate and spatially continuous motion. Simplified inmechanics, a larval body consists of body wall attached to the muscles andbody fluids inside the body wall. The 2 parts works together as a hydrosta-tic skeleton [5]. The skin has regular repeating symmetrical folds, which areessential for its deformation and friction, forming its segments. The muscles ofDrosophila larvae are in 3 orientations: dorso-ventral, anterioro-posterior and

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 375–386, 2016.DOI: 10.1007/978-3-319-42417-0 34

376 T. Wei et al.

oblique. Antero-posterior muscles are located nearer the interior than dorso-ventral muscles. The body wall muscles are segmentally repeated, and in eachabdominal half segment there are approximately 30 of them ([1]) (Fig. 1).

Fig. 1. A Drosophila larva expressing mCherry (a type of photoactivatable fluorescentproteins [14]) in its muscles. From Balapagos (2012).

Based on the property of their bodies, Drosophila larvae are able to do sev-eral motions, such as peristaltic crawling, body bending and rolling. Forwardperistaltic motion is best described. In the centre of the body, viscera suspendedin hemolymph is essential for limiting body wall deformation and produces pis-ton motion. During the ‘piston phase’ of peristalsis, muscles on the tail contractand push the viscera forward. The second ‘wave phase’ involves a wave of musclecontraction travelling through the bodywall segments from tail to head [4]. Tomimic various and motions of a Drosophila larval, it is important to utilize thisanatomical structure and avoid oversimplifying the high DOFs.

Some soft robots have been developed as bionic robots. The main materialsare silicone, rubber, or other flexible and stretchable materials. They are usuallyactuated by Shape-Memory Alloy (SMA) or pneumatically, such as BiomimeticMiniature Robotic Crawler [7], GoQBot [6], Multigait soft robot [13], and a flu-idic soft robot [11]. These robots only have several degrees-of-freedom (DOFs)and usually only have one type of motion, which is not sufficient to mimic larvalmotion. Although SMA is widely applied on soft robots, it has a significant short-coming. As SMAs deform according to temperature, their response is limited bycontrol of temperature. Because soft robots are usually not sufficient in heatdissipation, heat accumulates inside the robots, and response times of SMAs gettoo long so that continuous actuation is infeasible. The shortcoming does notexist on pneumatic actuation. Hence, pneumatic actuators are a feasible optionas they have a faster response and longer effective working time. However, themain action most of soft pneumatic actuators is off-axis bending, and the axialelongation and contraction are only side effects. For examples: Micro PneumaticCurling Actuator- Nematode Actuator [9], Pneu-net [13]), and Robot Air Mus-cles made from Oogoo [8]. As axial contraction is necessary for some motion(such as peristalsis), we designed a new type of pneumatic actuators.

2 Methods

The robot is made from soft silicone rubber, instead of rigid material, because:(1) motions of Drosophila larva are based on continous body deformation; (2)

A Soft Pneumatic Maggot Robot 377

soft materials have more similar properies to biological tissue than rigid material,such as nolinear elasticity and hysteresis, which are suitable to simulate dynamiccharacteristics of the muscle; and (3) defomation of Drosophila larval body wallis one method to control friction between body and contacted surface.

Figure 2 shows a sketch of a possible structure of a maggot robot. The robothas repeating modular body wall segments, with a water bag or air bag inside.Here we described the construction and control of 4 body segments. At present,the control system and pneumatic system are placed off board because of limitedspace and load.

Fig. 2. Sketch of the soft maggot robot. A central bag of fluid is surrounded by musclesegments.

2.1 Design of the Actuator and Body Wall of the Soft Robot

Pneu-nets (Fig. 3(a) and (b)) are usually made from 2 different soft materials: (1)flexible and stretchable material, such as Ecoflex, to form chambers to inflate andexpand; (2) flexible but less or not stretchable material, such as Polydimethyl-siloxane (PDMS). Thus, when pneu-nets are inflated, the actuator bends to theside made from less stretchable material. Pneu-nets are not suitable for tubu-lar body wall because the stretchable layer limits axial bending, hence we havemodifeid the design to produce a new actuator type, which we called ExtensiblePneu-nets.

Fig. 3. Structure of Pneu-nets and Extensible Pneu-nets (a) and (b) are longitudinaland transverse sections of Pneu-nets, respectively; (c) and (d) are longitudinal andtransverse sections of Extensible Pneu-nets, respectively.

378 T. Wei et al.

Extensible Pneu-nets (Fig. 3(c) and (d)) use only 1 stretchable material.Small air chambers are connected by air tunnels to form a muscle. Different mus-cles are isolated. When an air chamber is inflated, it expands in all directions,and the direction with maximum expansion is the direction with the maximumcross sectional area. To limit deformation in the unwanted direction, thickness ofthe inner walls between chambers and thickness of the outer walls are carefullyselected and tested. As stretchable material allows not only bending but alsoexpansion along the surface, tubular body wall based on Extensible Pneu-netsare possible to axially bend.

To make the air chambers and tunnels inside, the actuator is divided into 2layers which are cast separately. The moulds can be manufactured in conven-tional machining process or by 3D printing. Then the 2 layers are glued togetherwith the same material. Finally, tubes for injecting pressed compressed air areinserted and glued. By including more air chambers and tunnels on a model, abody wall with multiple pneumatic actuators can be cast.

The first attempt at a muscle pattern was designed according to real mus-cle pattern on dissected and flattened body wall of Drosophila larva (Fig. 4).Dorsal oblique (DO) muscles, lateral transverse (LT) muscles, oblique lateral(LO) muscles, ventral longitudinal (VL) muscles and ventral acute (VA) musclesare simplified and mapped on the muscle pattern of the body wall. However, theadjacent muscles limited each others motions, especially when they have differ-ent orientations. The cause of limitation is that inner walls between air chamberslimit transverse deformation, which is the direction that the adjacent muscles aredesigned to deform. Thus adjacent muscles should either be parallel, or shouldnot be contiguous.

Fig. 4. Body wall of a body segment with Extensible Pneu-nets designed according toreal muscle pattern on dissected and flattened body wall of Drosophila larva.

The design of the prototype evaluated in this paper is a body wall with 4body segments (Fig. 5, left). Each body segment has 3 transverse muscles and3 longitudinal muscles. These 2 types of muscles are connected perpendicularlyand only connected on corners, leaving gaps between them to avoid limitationof deformation between each other (Fig. 5, right). Body segments are connectedin series by longitudinal muscles. Figure 6 shows the mould for the body wall.After a flat body wall was made, it was folded end to end and clamped by 2specially cut boards. Through the window of the board, the end was carefullyaligned and glued together. By this process, the flat body wall is formed into ahollow cylinder shape (Fig. 7).

In this 4 body segment version, because the limited resolution of the 3Dprinter we use (Wanhao Duplicator with 0.4 mm nozzle) and resistance of air

A Soft Pneumatic Maggot Robot 379

flow in tube, the dimensions of air chamber, as shown in Fig. 3(c) and (d), are:a = 1.2 mm, b = 3.0 mm, c = 0.8 mm, d = 1.2 mm, e = 18 mm (longitudinalmuscles) or 28 mm (transverse muscles), f = 2.0 mm, g = 3.0 mm. In a curvedsingle body segment, the longitudinal length is 40 mm, the diameter is of therobot is about 50 mm. The total length of the 4 body segment body wall isabout 175 mm.

Fig. 5. (left) Prototype design of a body wall with perpendicular arrangement of mus-cles. Transverse muscles and longitudinal muscles of the first body segment are high-lighted in red and green, respectively. (centre) A closer view of the flatten body wallshows gaps and spaces between the muscles to allow expansion. (right) The gaps andspaces when the body wall curved. (Color figure online)

Fig. 6. The 3D printed mould for body wall casting.

2.2 Pneumatic Actuation and Control System

The pneumatic actuation and control system controls the robot by controllingair pressures of air chambers. Air pressure sensors measures pressure in everymuscle, pumps and valves control the air flow.

380 T. Wei et al.

Fig. 7. (left) The body wall is clamped and glued. (centre) Formed into a hollowcylinder. (right) Names of muscles: body segments are numbered, longitudinal musclesnamed in capital letters, transverse muscles named in lower case letters.

Pneumatic Control System. A pneumatic control system has been designedfor the robot. The system is located off board and connects to the robot withrubber tubes. As the robot has more DOFs than previous pneumatic soft robotsmentioned above, the size of the pneumatic control system is designed to becompact.

The main component of the system is a valve island with 24 pairs of miniature 2way solenoid valves (Fig. 8). The size of solenoid valves is 10 mm× 11 mm× 23 mm.Overall, the size of the valve island is 120 mm× 91 mm× 60 mm. The valves areinstalled the 3D main structure by interference fit. The main structure of the valveisland consists of layers of 3D printed parts. The upper layer made form Acry-lonitrile Butadiene Styrene (ABS), which offers Mechanical strength to fix valves,and lower layer made from Thermoplastic Elastomer (TPE), which has build in airchannels with air-tightness. Every channel connects 4 ways, which are 2 valves, apressure sensor, and an air chamber on the robot. The other 2 ways of each pair ofvalves are connected to compressed air and open to air, respectively.As the solenoidvalves speed up to 100 Hz, the air flow can be finely controlled.

Fig. 8. (left) A valve and pump in the system. (centre) Structure of the 3D printedvalve island. (right) Pneumatic valve island with 24 pairs of valves

EmbeddedControl System. An Embedded Control system has been designedfor control and actuation. The control system is a hierarchical control system con-sisting of 1 main controller and 3 slave controllers. Their micro controllers are

A Soft Pneumatic Maggot Robot 381

STM32F411RE by STMicroelectronics. They are based on Cortex-M4 by ARMwith digital signal processor (DSP) and floating-point unit (FPU). The maincontroller receives commands from a computer, and distributes them among theslave controllers by Universal Synchronous/Asynchronous Receiver/Transmitters(USART). On each of the slave controllers, 16 hardware Pulse-width modulation(PWM) channels and 8 Analog-to-digital converters (ADC) are configured to con-trol 8 muscles. The PWMs control Darlington transistor arrays (ULx2003 by TexasInstruments). On each slave control board, 3 of them are adopted to drive valves.MPS20N0040D-D, which is an air pressure sensor to measure pressure in air cham-bers, is adopted to measure the pressures.

Algorithm. At present stage, the robot is controlled by feedforward prepro-grammed motion. According to a approximate linearization between deforma-tion and pressure at the initial state of equilibrium, the pressure is utilized asfeedback of motion of muscle.

3 Experiments

The robot was tested for individual control of every muscle and coordinationbetween them. Three motions are programmed and tested on the robot (A videoof the experiments: https://youtu.be/aFE9dANHowk). The muscles are namedbased on their location. As show in (Fig. 7), body segments are numbered, lon-gitudinal muscles named in capital letters, transverse muscles named in lowercase letters.

Turn. In this motion, muscle a and B on every body segment was actuated atthe same time, then pressure released. To minimize friction and show relevancebetween pressure and deformation, the robot is tested while floating on water.Figure 9 shows the motion of the robot. The pressure of the muscles is shownin Fig. 10.

Fig. 9. Turning left. The black lines show the initial central axis.

382 T. Wei et al.

0 10 200

10

20

30

40

50

Pres

ure

(kPa

)

1B

0 10 200

10

20

30

40

50

Pres

ure

(kPa

)

1A

0 10 200

10

20

30

40

502A

0 10 200

10

20

30

40

502B

0 10 200

10

20

30

40

503A

0 10 200

10

20

30

40

504A

0 10 200

10

20

30

40

504B

Target PressureFeedback

0 10 200

10

20

30

40

503B

0 10 20Time (s)

0

10

20

30

40

501b

0 10 20Time (s)

0

10

20

30

40

501c

0 10 20Time (s)

0

10

20

30

40

50

Pres

ure

(kPa

)

1C

0 10 20Time (s)

0

10

20

30

40

501a

Fig. 10. Pressure of muscles in the first body segment, muscles A and muscles B duringturning. (Color figure online)

Fig. 11. Roll.

A Soft Pneumatic Maggot Robot 383

Roll. In this motion, muscle a and B, b and C, c and A, are inflated alternately.As the bundle of the tubes flowing the robot impact the rolling on a surface inwater, the robot is hold on its tail vertically during test. Figure 11 shows themotion of the robot. The pressure of muscles show in Fig. 12.

Peristalsis. In this simplified peristalsis, all the muscles on a body seg-ment inflate at same time and muscles of different segments inflate alternately.

0 10 20 300

10

20

30

40

50

Pres

ure

(kPa

)

1A

0 10 20 300

10

20

30

40

502A

0 10 20 300

10

20

30

40

503A

0 10 20 300

10

20

30

40

504A

0 10 20 300

10

20

30

40

50

Pres

ure

(kPa

)

1B

0 10 20 300

10

20

30

40

502B

0 10 20 300

10

20

30

40

503B

0 10 20 300

10

20

30

40

504B

Target PressureFeedback

0 10 20 30Time (s)

0

10

20

30

40

501b

0 10 20 30Time (s)

0

10

20

30

40

501a

0 10 20 30Time (s)

0

10

20

30

40

50

Pres

ure

(kPa

)

1C

0 10 20 30Time (s)

0

10

20

30

40

501c

Fig. 12. Pressure of muscles in the first body segment, muscles A and muscles B duringrolling. (Color figure online)

Fig. 13. Peristalsis. The parts on the blue lines were expanding. (Color figure online)

384 T. Wei et al.

0 20 400

10

20

30

40

50

Pres

ure

(kPa

)

1A

0 20 400

10

20

30

40

502A

0 20 400

10

20

30

40

503A

0 20 400

10

20

30

40

504A

0 20 400

10

20

30

40

50

Pres

ure

(kPa

)

1B

0 20 400

10

20

30

40

502B

0 20 400

10

20

30

40

503B

0 20 400

10

20

30

40

504B

Target PressureFeedback

0 20 40Time (s)

0

10

20

30

40

50

Pres

ure

(kPa

)

1C

0 20 40Time (s)

0

10

20

30

40

501a

0 20 40Time (s)

0

10

20

30

40

501b

0 20 40Time (s)

0

10

20

30

40

501c

Fig. 14. Pressure of muscles in the first body segment, muscles A and muscles B duringperistalsis. (Color figure online)

The motion is tested on water. Figure 13 shows the motion of the robot. Thepressure of muscles show in Fig. 14.

4 Discussion

In the tests above, the robot produced three different motions from musclesactuated individually in different orders. The system was able to control thepressures according to the control signal, although the pressures have some noise.However, the three motions are not accurate. Deformation for the same pressureis different between the muscles. That is because muscles are slightly differentand the relationship between deformation and pressure is not ideally linear.When an air chamber is inflated to a given range, the pressure does not changemuch even with obvious deformation. Hence, applying the same pressure todifferent muscles can result in different deformations. Thus, deformation sensorswill be important to precise control of the robot.

Hence our immediate aim for future work is to develop and install sensors onthe robot for deformation feedback. As the sampling density of the deformation

A Soft Pneumatic Maggot Robot 385

sensors is limited, different deformations may map to the same output, hence amodel or method to learn the relationship between sensor outputs and postureis necessary. We should then be able to explore more thoroughly the movementcapabilities of the current design. Some additional redesign of the pneumaticmuscle and body wall may be necessary, for example, surface processes to mimicdenticles on Drosophila larval skin which generate asymmetric friction so thatperistalsis produces forward locomotion [12].

5 Conclusion

Our longer term aim for this robot is to use it as a platform to test neural circuitmodels of Drosophila larvae. Initially this could focus on the motor circuits thatgenerate and control peristalsis and bending. In particular these circuits couldform the basis of an adaptative method for learning the control signals needed toadjust to the irregularities and non-linearities in the actuators and their interac-tions, in the same way that maggots are able to adapt to rapid change and growthin their body while maintaining efficient locomotion. Ultimately we would like toadd sensors for environmental signals and investigate the sensorimotor controland associative learning involved in, e.g., odour search [3].

References

1. Bate, M.: The mesoderm and its derivatives. The Development of DrosophilaMelanogaster, pp. 1013–1090. Cold Spring Harbor Laboratory Press, New York(1993)

2. Brand, A.H., Perrimon, N.: Targeted gene expression as a means of altering cellfates and generating dominant phenotypes. Development (Cambridge, England)118(2), 401–415 (1993)

3. Gomez-Marin, A., Louis, M.: Multilevel control of run orientation in Drosophilalarval chemotaxis. Front. Behav. Neurosci. 8, 38 (2014)

4. Heckscher, E.S., Lockery, S.R., Doe, C.Q.: Characterization of Drosophila larvalcrawling at the level of organism, segment, and somatic body wall musculature. J.Neurosci. 32(36), 12460–12471 (2012)

5. Kohsaka, H., Okusawa, S., Itakura, Y., Fushiki, A., Nose, A.: Development of larvalmotor circuits in Drosophila. Dev. Growth Differ. 54(3), 408–419 (2012)

6. Lin, H.T., Leisk, G.G., Trimmer, B.: GoQBot: a caterpillar-inspired soft-bodiedrolling robot. Bioinspir. Biomim. 6(2), 026007 (2011)

7. Menciassi, A., Accoto, D., Gorini, S., Dario, P.: Development of a biomimeticminiature robotic crawler. Auton. Robots 21(2), 155–163 (2006)

8. mikey77: Soft robots: Making robot air muscles. Webpage (2012). http://www.instructables.com/id/Soft-Robots-Making-Robot-Air-Muscles/

9. Ogura, K., Wakimoto, S., Suzumori, K., Nishioka, Y.: Micro pneumatic curlingactuator - Nematode actuator. In: 2008 IEEE International Conference on Roboticsand Biomimetics, pp. 462–467 (2009)

10. Olsen, S.R., Wilson, R.I.: Cracking neural circuits in a tiny brain: new approachesfor understanding the neural circuitry of Drosophila. Trends Neurosci. 31(10),512–520 (2008)

386 T. Wei et al.

11. Onal, C.D., Rus, D.: Autonomous undulatory serpentine locomotion utilizingbody dynamics of a fluidic soft robot. Bioinspir. Biomim. 8(2), 026003 (2013).http://www.ncbi.nlm.nih.gov/pubmed/23524383

12. Ross, D., Lagogiannis, K., Webb, B.: A model of larval biomechanics revealsexploitable passive properties for efficient locomotion. In: Wilson, S.P., Verschure,P.F.M.J., Mura, A., Prescott, T.J. (eds.) Living Machines 2015. LNCS, vol. 9222,pp. 1–12. Springer, Heidelberg (2015)

13. Shepherd, R.F., Ilievski, F., Choi, W., Morin, S.A., Stokes, A.A., Mazzeo, A.D.,Chen, X., Wang, M., Whitesides, G.M.: From the cover: multigait soft robot. Proc.Nat. Acad. Sci. 108(51), 20400–20403 (2011)

14. Subach, F.V., Patterson, G.H., Manley, S., Gillette, J.M., Lippincott-Schwartz, J.,Verkhusha, V.V.: Photoactivatable mCherry for high-resolution two-color fluores-cence microscopy. Nat. Methods 6(2), 153–159 (2009)

Short Papers

On Three Categories of Conscious Machines

Xerxes D. Arsiwalla1(B), Ivan Herreros1, and Paul Verschure1,2

1 Synthetic Perceptive Emotive and Cognitive Systems (SPECS) Lab,Center of Autonomous Systems and Neurorobotics, Universitat Pompeu Fabra,

Barcelona, Spainx.d.arsiwalla@gmail.com

2 Institucio Catalana de Recerca i Estudis Avancats (ICREA), Barcelona, Spain

Abstract. Reviewing recent closely related developments at the cross-roads of biomedical engineering, artificial intelligence and biomimetictechnology, in this paper, we attempt to distinguish phenomenologicalconsciousness into three categories based on embodiment: one that isembodied by biological agents, another by artificial agents and a thirdthat results from collective phenomena in complex dynamical systems.Though this distinction by itself is not new, such a classification is usefulfor understanding differences in design principles and technology nec-essary to engineer conscious machines. It also allows one to zero-in onminimal features of phenomenological consciousness in one domain andmap on to their counterparts in another. For instance, awareness andmetabolic arousal are used as clinical measures to assess levels of con-sciousness in patients in coma or in a vegetative state. We discuss anal-ogous abstractions of these measures relevant to artificial systems andtheir manifestations. This is particularly relevant in the light of recentdevelopments in deep learning and artificial life.

Keywords: Conscious agents · Artificial intelligence · Complex sys-tems · Bioethics

1 Introduction

The topic of this discussion addresses the question of what it takes to engineerconscious processes in an artificial machine and measures useful for describingsuch processes. By conscious processes we mean phenomenological aspects ofconsciousness. This concerns an epistemically objective description of a problemthat may well have an ontologically subjective element. Drawing from what isknown (howsoever little) about biological systems, we build a parallel argumentfor artificial and collective systems.

2 Biological Consciousness

In patients with disorders of consciousness ranging from coma, locked-in syn-drome to those in vegetative states, levels of consciousness are assessed throughc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 389–392, 2016.DOI: 10.1007/978-3-319-42417-0 35

390 X.D. Arsiwalla et al.

a battery of behavioral tests as well as physiological recordings. More generally,these measurements are grouped into those referring to awareness and thoseto arousal [5,6]. In fact, for clinical purposes, closely associated states of con-sciousness can be grouped into clusters on a plane with awareness and arousalrepresenting orthogonal axes [5,6]. Scales of awareness target both higher- andlower-order cognitive functions enabling complex behavior (see [1,2] for a dis-cussion on measures of consciousness). Arousal results from biochemical home-ostatic mechanism regulating survival drives and is clinically measured in termsof glucose metabolism in the brain. Clinical consciousness is thus assessed as abivariate measure on these two axes. In fact, in all known organic life forms,biochemical arousal is a necessary precursor supporting the hardware necessaryfor cognition. In turn, evolution has shaped cognition in such a way so as tosupport the organism’s basic survival as well as higher-order drives associatedto cooperation and competition in a multi-agent environment. Awareness andarousal thus form a closed-loop, resulting in phenomenological consciousness.

How can we then abstract and map arousal and awareness to non-biologicalsystem to set-up a comparative measure of consciousness? As noted above,arousal results from autonomous homeostatic mechanisms necessary for the self-preservation of an organism’s germ line in a given environment. In other words,arousal results from self-sustaining life processes necessary for basic survival,whereas awareness can be functionally abstracted as general forms of intelli-gence, necessary for higher-order functions in a complex social world. If biologi-cal consciousness as we know it, is phenomenologically an amalgamation of lifeand intelligence, how can we use this argument to conceive a functional notionof consciousness in artificial systems?

3 Artificial Consciousness

The reason why the aforementioned question has now become very relevantis due to some remarkable recent advances in two seemingly unrelated fields:artificial intelligence and synthetic life. The most recent example of the former isGoogle DeepMind’s AlphaGo system [8] and recent examples of the latter includesynthesis of artificial DNA with six nucleotide bases (the naturally occurringadenine (A), cytosine (C), guanine (G) and thymine (T) plus a new syntheticpair), which was engineered into the cell of the bacterium Escherichia coli andsuccessfully passed from one generation to the next [7]; synthetic proto-cellscapable of replicating themselves [4]; and a fully functioning synthetic genomeimplanted into the cell of the bacterium Mycoplasma genitalium that convertsit into a novel bacterium species [3].

Of course the abstract functional criterion we have defined above for con-sciousness is not even remotely satisfied by any of these systems as they eitherhave some limited form of intelligence or life but not yet both. However,AlphaGo’s feat in beating the top human Go champion was noteworthy fora couple of reasons. Unlike Chess, possible combinations in Go run into millionsand when played using a timer any brute-force algorithm trying to scan the entire

On Three Categories of Conscious Machines 391

search space would simply run out of computational capacity or time. Hence,a pattern recognition algorithm mimicking intuition was crucial to the devel-opment of AlphaGo, where the machine learned complex strategies by playingitself over and over again while reinforcing successful sequence of plays throughthe weights of its deep neural networks [8]. Most remarkably, it played counter-intuitive moves that shocked the best human players and the only game thatLee Sedol, the human champion won out of five, was possible after he himselfadopted a brilliant counterintuitive strategy.

Hence asking whether a machine can think like a human is similar to askingwhether a submarine can swim. It just does it differently. If the goal of a systemis to learn and solve complex tasks close to human performance or better, currentmachines are already doing that in specific domains. However, these machines arestill far from learning and solving problems in generic domains and in ways thatwould couple its problem solving abilities to its own survival drives. On the otherside, neither have the above-mentioned synthetic cells yet been used to buildcomplex structures with computing or cognitive capabilities. Nevertheless, thisdoes suggest that the synthesis between artificial life forms and AI would satisfythe same criterion of functional consciousness as that used by known biologicalsystems. This form of consciousness, if based on a life form with different survivaldrives/mechanisms and non-human forms of intelligence, would also likely leadto non-human behavioral outcomes.

4 Collective Consciousness

The notion of collective consciousness is not really a distinct type of conscious-ness by itself when one thinks of biological consciousness as a manifestation ofthis collective phenomenon where individual cells making up an organism arenot considered conscious by themselves even though the organism as a wholeis. As such collective intelligence is a phenomenon widely studied in systemsranging from ant colonies to social networks to the internet. But these are gen-erally not regarded as conscious systems. As a whole they are not considered tobe life forms with survival drives that compete or cooperate with other similaragents. But these considerations begin to get blurred at least during transientepochs when collective survival comes under threat. For example, when a beecolony comes under attack by hornets, it demonstrates a prototypical survivaldrive, similar to lower-order organisms and which is transiently implementedalongside its existing social intelligence. This does suggest a third category ofconscious machines, which possess a distributed architecture along with collec-tive survival drives and learning abilities, needed for competition or cooperationin a multi-agent environment.

5 Societal Impact, Ethics and Conclusions

Both, the societal impact and ethics of any form of intelligent machine, especiallyconscious machines, constitutes a very serious issue. For example, the impact of

392 X.D. Arsiwalla et al.

medical nanobots for removing tumors, attacking viruses or non-surgical organreconstruction has the potential to change medicine forever. Or intelligent sys-tems to clear pollutants from the atmosphere or the rivers are absolutely essen-tial for some of the biggest problems that humanity faces. However, as discussedabove, purely increasing the performance of a machine along the intelligence axiswill not constitute consciousness as along as this intelligence is not accessible bythe system to autonomously regulate or enhance its survival drives. On the otherhand, whenever the latter is indeed made possible, issues of societal interactions ofmachines with humans and the ecosystem, becomes an imminent ethical respon-sibility. It becomes important to understand the kind of cooperation-competitiondynamics that a futuristic human society will face. The early stages of designingthese machines are probably the best times to regulate future behavioral outcomesof their interaction dynamics. This analogy might not be surprising to any parentthat has a child. Hence, a serious effort into understanding the evolution of com-plex social traits is crucial alongside the engineering that goes into building suchsystems.

Acknowledgments. This work has been supported by the European Research Coun-cil’s CDAC project: “The Role of Consciousness in Adaptive Behavior: A CombinedEmpirical, Computational and Robot based Approach” (ERC-2013-ADG 341196).

References

1. Arsiwalla, X.D., Verschure, P.: Computing information integration in brain net-works. In: Wierzbicki, A., Brandes, U., Schweitzer, F., Pedreschi, D., et al. (eds.)NetSci-X 2016. LNCS, vol. 9564, pp. 136–146. Springer, Heidelberg (2016). doi:10.1007/978-3-319-28361-6 11

2. Arsiwalla, X.D., Verschure, P.F.: Integrated information for large complex net-works. In: The 2013 International Joint Conference on Neural Networks (IJCNN),pp. 1–7. IEEE (2013)

3. Hutchison, C.A., Chuang, R.Y., Noskov, V.N., Assad-Garcia, N., Deerinck, T.J.,Ellisman, M.H., Gill, J., Kannan, K., Karas, B.J., Ma, L., et al.: Design and syn-thesis of a minimal bacterial genome. Science 351(6280), aad6253 (2016)

4. Kurihara, K., Okura, Y., Matsuo, M., Toyota, T., Suzuki, K., Sugawara, T.: Arecursive vesicle-based model protocell with a primitive model cell cycle. Nat. Com-mun. 6, 8352 (2015)

5. Laureys, S.: The neural correlate of (un) awareness: lessons from the vegetativestate. Trends Cogn. Sci. 9(12), 556–559 (2005)

6. Laureys, S., Owen, A.M., Schiff, N.D.: Brain function in coma, vegetative state,and related disorders. Lancet Neurol. 3(9), 537–546 (2004)

7. Malyshev, D.A., Dhami, K., Lavergne, T., Chen, T., Dai, N., Foster, J.M., Correa,I.R., Romesberg, F.E.: A semi-synthetic organism with an expanded genetic alpha-bet. Nature 509(7500), 385–388 (2014)

8. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G.,Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al.: Master-ing the game of go with deep neural networks and tree search. Nature 529(7587),484–489 (2016)

Gaussian Process Regression for a BiomimeticTactile Sensor

Kirsty Aquilina1,2(B), David A.W. Barton1,2, and Nathan F. Lepora1,2

1 Department of Engineering Mathematics, University of Bristol, Bristol, UK{ka14187,david.barton,n.lepora}@bristol.ac.uk

2 Bristol Robotics Laboratory, University of Bristol, Bristol, UK

Abstract. The aim of this paper is to investigate a new approach todecode sensor information into spatial information. The tactile fingertip(TacTip) considered in this work is inspired from the operation of dermalpapillae in the human fingertip. We propose an approach for interpretingtactile data consisting of a preprocessing dimensionality reduction stepusing principal component analysis and subsequently a regression modelusing a Gaussian process. Our results are compared with a classificationmethod based on a biomimetic approach for Bayesian perception. Theproposed method obtains comparable performance with the classificationmethod whilst providing a framework that facilitates integration withcontrol strategies, for example to perform controlled manipulation.

Keywords: Tactile sensors ·Gaussian process regression · Bayesian per-ception

1 Introduction

Touch is essential in everyday life as it provides the required information to prop-erly perform manipulation tasks [1]. This kind of sensory input should therebyalso be provided to robots in order to make them capable of performing complexmanipulation tasks. In consequence, it is important to have robust tactile sensorsand algorithms that are able to decode all the important sensor information.

The aim of this paper is to investigate a new approach to decode tactilesensor information into spatial information. The standard approach is to use aclassification framework in which percepts are discrete classes and then the bestclass is chosen, for example via Bayesian inference as in the biomimetic activeperception method of refs [2,3]. In the present paper, another Bayesian approachis considered where the input signal is preprocessed using Principal ComponentAnalysis (PCA) and then given as input to a Gaussian Process (GP) [4] thatperforms regression.

The sensor used here is a biologically-inspired tactile fingertip, the TacTip [5]:a dome-shaped sensor with internal artificial papillae inspired by dermal papil-lae, i.e. protrusions on the inside of the sensor skin. These artificial papillae willbe here referred to as taxels (the sensing elements) or pins. The motion of eachc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 393–399, 2016.DOI: 10.1007/978-3-319-42417-0 36

394 K. Aquilina et al.

individual pin is tracked using an internally-encased camera and image process-ing techniques. The algorithm proposed in this work also has a biological inspi-ration. The dimensionality reduction preprocessing step is analogous to humanvision, where retinal cone photoreceptors reduce the dimensionality of the incom-ing light to three dimensions depending on the wavelength range [6]. Additionally,second-order statistics such as PCA have been shown to have similarities withreceptive fields in the primary visual cortex [7]. The idea of using a regressionmodel is also rooted in human behaviour where it has been proposed that peo-ple learn continuous-valued functions by a regression framework [8]. Furthermore,Griffiths et al. [9] show that GPs provide a possible explanation to how humanslearn continuous functions.

This paper is organised as follows: the Methods will provide a descriptionof the decoding of the sensor data, then the Results will compare this decodingwith that of the Bayesian perception method from [2,3] that has been previouslyused for interpreting data from the TacTip sensor.

2 Methods

The tactile data analysed in this paper was collected from the TacTip mountedon a 6-DOF robotic arm (ABB IRB120) that performed vertical taps on an edgestimulus as shown in Fig. 1. The sensor is rotated over this edge, between taps, by1◦ collecting data ranging from 0–359◦. The sensor readings for 0◦ and 180◦ areopposites of each other, as in each case a different subset of the pins is in contactwith the object, thereby permitting to distinguish between all the angles overthe considered range. The pin arrangement of the TacTip can be seen in Fig. 2a.This version of the TacTip [10] has 127 pins leading to a total of 254 dimensions,as each pin (taxel) gives measurements in the x and y direction. Furthermore,each taxel produces a time series of pixel deflections in the x direction and inthe y direction during each tap.

Two datasets were collected, one being used as a training set and the otherone used as a test set. Each set contained a time series of taxel positions for eachdimension for each tap, having in total 360 taps covering the whole range ofinterest. In order to obtain one value for each of these 254 dimensions for everytap, the first principal component of each time series is computed using PCA.This was done by collecting all the data over all 360 taps for each taxel dimension,then finding the most dominant eigenvector and projecting the data on it. Theresult of doing this dimensionality reduction can be seen in Fig. 2b which is ascaled down version (0.4 times) of the principal components superimposed on theposition of each taxel. This means that the time series provided by each tap ismapped to one single value per taxel dimension computed using the whole timeseries. This transformed the training dataset into a matrix of 360× 254 which wasthen further preprocessed by obtaining the principal components that explainthe motion of the taxels when different edge orientations were sensed. The firsttwo principal components were considered for the rest of the analysis which canbe seen in Fig. 2c. The image shows that considering these two dimensions is

Gaussian Process Regression for a Biomimetic Tactile Sensor 395

a valid approach as this representation encodes each orientation differently butsimilar orientations are related to each other. This PCA was performed so asto reduce the computational expense incurred in this approach by optimising 4hyperparameters instead of 256 hyperparameters.

The training set was composed of 2 dimensions and 360 observations. Thisdata was then used to optimise the GP hyperparameters built using a squaredexponential (SE) covariance function. Two GPs were used, as angles which areclose to 0◦ or 360◦ are actually the same angle. Therefore in order not to causeinterference between these two extremes a GP was used to learn the mappingfor angles between 0◦ and 179◦ and another for the rest of the range to 359◦.The use of multiple local GPs was inspired from the approach presented in [11].However, in the present work the training points for each model are chosen beforethe model learning and the Euclidean distance is used as a measure to choosewhich model is to be associated with a new test point. The predictions are madeusing only one model and are not a merged version of both models.

Fig. 1. The setup used for data gathering.

The hyperparameters were optimised using the MATLAB optimisation tool-box by minimizing the negative log marginal likelihood of the GP [4] therebyobtaining values for the signal noise standard deviation σn, the signal standarddeviation σf , and the length-scale l (one for each dimension) required by the SEcovariance. These are the respective values for model 1 (0◦–179◦) 0.26, 66.23,121.57, 144.5 and model 2 (180◦–359◦) 0.21, 59.56, 151.78, 95.98. The length-scale can be seen to be related to the circle depicted in Fig. 2c which has a radiusof approximately 106 in the principal components dimension.

396 K. Aquilina et al.

0 100 200 300 4000

50

100

150

200

250

300

350

400

x position(px)

y po

sitio

n(px

)

40 m

m

Pin Positions

(a) All the taxels present in the TacTip

0 100 200 300 4000

50

100

150

200

250

300

350

400

x position(px)

y po

sitio

n(px

)

PCA taxel visualisation

(b) Taxel contribution visualisation

−150 −100 −50 0 50 100 150−150

−100

−50

0

50

100

150

First principal component

Seco

nd p

rinci

pal c

ompo

nent

start end

PCA visualisation

(c) Overall Principal Components visualisation

Fig. 2. The pins or taxels of the sensor are seen in (a). These are then displaced overthe range of interest (b) showing the first principal component in the x and y directionscaled by 0.4. Finally the first and second principal components over all taxels areshown in (c) where ‘start’ denotes 0◦ and ‘end’ denotes 359◦.

3 Results

The framework described in the Methods section was tested using the testdataset. The regression results can be seen in Fig. 3. The mean predicted valueshad an average orientation error of 0.33◦ as shown in Fig. 4a. It is interestingto note that the point that had the largest error also showed a larger varianceimplying that the model is less sure about the predicted value.

The passive Bayesian perception approach explained in [2] having 1◦ perclass and using one tap to make a decision was then used to analyse the sametwo datasets. When using these parameters the passive Bayesian perception is

Gaussian Process Regression for a Biomimetic Tactile Sensor 397

0 50 100 150 200 250 300 3500

50

100

150

200

250

300

350

Regression test results

Edge orientation, θ (deg)

Pred

icte

d O

rient

atio

n,(θ

) (de

g)

116 118 120 122 124116

118

120

122

124

Fig. 3. Regression results for the two joined GP models, along with a zoomed in plotsection. The blue line depicts the computed mean and the grey bands show ± 3 standarddeviations. (Color figure online)

0 60 120 180 240 300 3600

2.5

5

Edge orientation, θ (deg)

Orie

ntat

ion

erro

r, e θ(θ

) (de

g)

Orientation accuracy

Average error: eθ = 0.33 deg

(a) Regression Error

0 60 120 180 240 300 3600

2.5

5

Edge orientation, θ (deg)

Orie

ntat

ion

erro

r, e θ( θ

) (de

g)

Orientation accuracy

Average error: eθ = 0.88 deg

(b) Classification error

Fig. 4. Comparing the error obtained using the proposed regression method and thecurrently used classification method.

equivalent to using maximum likelihood which resulted in an average error of 0.88◦

as shown in Fig. 4b. Therefore the proposed method managed to obtain resultscomparable to the classification method and achieved an improvement in accu-racy due to the averaging implicit in a GP. Additionally, the proposed method hasthe advantage of providing a measure of variance on the prediction, which mightbe useful for a decision making process. Another advantage is that a regressionapproach requires fewer training points while still covering the whole range. Forinstance using 92 training points over the whole range which include 0◦, 179◦,180◦ and 359◦ lead to the same average error of 0.33◦. Most importantly, havinga regression method to infer sensor readings is important in order to facilitate theintegration of control strategies in tactile robots.

398 K. Aquilina et al.

4 Conclusions

The results show that the proposed decoding method using PCA and a GP canlearn the required mapping in the particular experiment considered here (anedge over a range of angles). This preliminary work showed that this methodcan provide an alternative to the classification approach with discrete percepts.Additionally, it provides more flexibility as continuous predictions are computeddue to the interpolation capabilities of a GP, along with a measure of uncertaintyvia the computed variances. We expect that fewer training points will also berequired. Obtaining a good regression model to infer sensor data is important asthis can give an appropriate perceptual component to integrate within controlloops, which is important for attaining complex manipulation behaviour.

The work investigated in this paper is based on a biomimetic sensor inspiredfrom the human tactile sensory system and the considered inference approachis biologically inspired. Natural organisms such as humans are a great source ofinspiration for artificial systems as they have been perfected over many millen-nia of evolution, which can inspire solutions to hard engineering problems in amanner with superior robustness and adaptability.

Acknowledgements. I would like to thank Benjamin Ward-Cherrier for providingthe training and test dataset used in this paper. This work was supported by theEPSRC Centre for Doctoral Training in Future Autonomous and Robotic Systems(FARSCOPE) at Bristol Robotics Laboratory. This work was also supported by a grantfrom the Engineering and Physical Sciences Research Council (EPSRC) on ‘TactileSuperresolution Sensing’ (EP/M02993X/1).

References

1. Johansson, R.S., Flanagan, J.R.: Coding and use of tactile signals from the finger-tips in object manipulation tasks. Nat. Rev. Neurosci. 10(5), 345–359 (2009)

2. Lepora, N.F., Ward-Cherrier, B.: Superresolution with an optical tactile sensor.In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS), pp. 2686–2691, September 2015

3. Lepora, N.F.: Biomimetic active touch with fingertips and whiskers. IEEE Trans.Haptics 9(2), 170–183 (2016)

4. Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning. AdaptativeComputation and Machine Learning Series. University Press Group Limited, NewEra Estate (2006)

5. Chorley, C., Melhuish, C., Pipe, T., Rossiter, J.: Development of a tactile sen-sor based on biologically inspired edge encoding. In: International Conference onAdvanced Robotics, ICAR 2009, pp. 1–6, June 2009

6. Gegenfurtner, K.R., Kiper, D.C.: Color vision. Annu. Rev. Neurosci. 26(1),181–206 (2003)

7. Baddeley, R.J., Hancock, P.J.B.: A statistical analysis of natural images matchespsychophysically derived orientation tuning curves. Proc. R. Soc. Lond. B: Biol.Sci. 246(1317), 219–223 (1991)

Gaussian Process Regression for a Biomimetic Tactile Sensor 399

8. Koh, K., Meyer, D.E.: Function learning: induction of continuous stimulus-responserelations. J. Exp. Psychol. Learn. Mem. Cogn. 17(5), 811 (1991)

9. Griffiths, T.L., Lucas, C., Williams, J., Kalish, M.L.: Modeling human functionlearning with gaussian processes. In: Koller, D., Schuurmans, D., Bengio, Y.,Bottou, L. (eds.) Advances in Neural Information Processing Systems 21,pp. 553–560. Curran Associates Inc. (2009)

10. Ward-Cherrier, B., Cramphorn, L., Lepora, N.F.: Exploiting sensor symmetry togeneralize biomimetic touch. In: Ohwada, H., Yoshida, K. (eds.) Living Machines2016. LNCS, vol. 9793, pp. 540–544. Springer, Switzerland (2016)

11. Nguyen-Tuong, D., Peters, J.: Local gaussian process regression for real-timemodel-based robot control. In: IEEE/RSJ International Conference on IntelligentRobots and Systems, IROS 2008, pp. 380–385, September 2008

Modulating Learning Through Expectationin a Simulated Robotic Setup

Maria Blancas1(B), Riccardo Zucca1,Vasiliki Vouloutsi1, and Paul F.M.J. Verschure1,2

1 Laboratory of Synthetic Perceptive Emotive Cognitive Systems (SPECS), DTIC,N-RAS, Universitat Pompeu Fabra, Barcelona, Spain

{maria.blancas,riccardo.zucca,vicky.vouloutsi,paul.verschure}@upf.edu2 Catalan Institute of Advanced Studies (ICREA), Barcelona, Spain

Abstract. In order to survive in an unpredictable and changing envi-ronment, an agent has to continuously make sense and adapt to theincoming sensory information and extract those that are behaviorallyrelevant. At the same time, it has to be able to learn to associate spe-cific actions to these different percepts through reinforcement. Using thebiologically grounded Distributed Adaptive Control (DAC) robot-basedneuronal model, we have previously shown how these two learning mech-anisms (perceptual and behavioral) should not be considered separatelybut are tightly coupled and interact synergistically via the environment.Through the use of a simulated setup and the unified framework of theDAC architecture, which offers a pedagogical model of the phases thatform a learning process, we aim to analyze this perceptual-behavioralbinomial and its effects on learning.

Keywords: Distributed Adaptive Control · Autonomous syntheticagents · Rule learning

1 Introduction

You wake up at night and you want to reach a small (turned on) lamp at theother side of the room. As you see the light, reaching it will not be a problem.But, what would happen if you were in complete darkness and wanted to turnon that light? You could first try a random path and probably crash with theobjects in the room. You would try to find another path, and then another one,and so on, until you finally find the lamp. This strategy, although probablyleading you to a correct solution at the end, would not be really optimal. Anoptimal strategy would be to use the objects you run into as cues and use thedistance between them to learn the whole sequence that allows you to arrivefrom your bed to the lamp. In order to correctly reach your final goal, you haveto make accurate predictions of the position of the objects in your path.

This example illustrates the critical interplay occurring between perceptualand behavioral learning. If you learned the room only through perception, you

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 400–408, 2016.DOI: 10.1007/978-3-319-42417-0 37

Modulating Learning Through Expectation in a Simulated Robotic Setup 401

would have to perceive as many objects in the room as possible, to form a generalidea of the room. That would provide you with a large amount of information ofthe surrounding environment, but the most of it would not be needed for yourgoal. If you learnt to reach the lamp only through behavior, you would haveto try different paths until finding the right one, but it would have been easierif you had some references of the environment. Thus, perceptual learning givesyou cues of the environment, the sensory events; and behavioral learning allowsyou to build associations between perceptions and actions, thus reshaping yourinternal model of the world and improve your accuracy. In that sense, whenyou have enough information of the environment that allows you to form quasi-correct expectations of the cues you are going to receive, you can start to learnfrom context the most optimal solution that leads you to reach your goal.

The positive effect of the interplay between perceptual and behavioral learn-ing on accuracy and performance has been previously investigated in a robotforaging task in the context of the Distributed Adaptive Control (DAC) the-ory of mind, brain and body nexus [1–3]. Similarly to the example describedpreviously, in the foraging task, the agent first explores the world to learn itsregularities using the simple reflexes it is equipped with; then, through adap-tation, those behaviors can be reshaped and internal representations can beformed. When the difference between the stimulus expected (prediction) andthe stimulus received (perception) is low enough, that is, when the outcome ofexpectations is close to reality, the agent begins to learn from context by learn-ing the sequences of actions that lead to a goal. This form of learning does notfocus only on the appropriate actions but also on the sequences that lead to agoal state, shaping the world model of the agent and leading to a stabilizationof the agent’s behavior and internal representations.

The use of the DAC cognitive architecture offers an already defined frame-work to monitor and control this behavior. Furthermore, DAC predicts thatlearning is organized along a hierarchy of complexity and in order to learn andconsolidate new material, the learner undergoes a sequence of learning phases:resistance to new information when that threatens an agent’s inner model ofthe world, confusion that creates the necessity to resolve the problem and learnthrough re-adapting and abduction which is the process of acquiring the newknowledge. Given the interplay between perceptual and behavioral learning andthe three phases of learning according to DAC, we propose a first approachto map these two ideas. That is: how, on the one hand, the accuracy of one’sexpectations allows for a more optimal learning (perceptual) while contextuallearning stabilizes behavior and internal representations and on the other hand,how learning occurs based on the phases suggested by DAC. Here, we focus onintroducing the idea of stage transitions, and how the different layers of theDAC architecture contribute to the stabilization of behavior. This paper willlater serve as the basis for our main goal: map these stage transitions to thephases of learning and understand how they occur.

In the following sections, we first present the DAC architecture (see Sect. 1.1)as a model of cognitive development. In Sect. 2 we describe the experimental

402 M. Blancas et al.

setup (simulated environment) used to test the proposed stage transition (Sect. 3).Finally, in the last section we discuss how we plan to proceed in order to map thecurrent stage transition to the phases of learning.

1.1 DAC as a Model of Cognitive Development

The Distributed Adaptive Control (DAC) architecture views the learner asa complex individual whose behavior derives from the interplay of sensation,reaction, adaptation and contextualization. This biologically grounded cogni-tive architecture is defined in the context of classical and operant conditioningand represents the association between mind, brain and behavior [4]. DAC isorganized across four different layers: Soma, Reactive, Adaptive and Contex-tual. The Soma Layer represents the body itself and the information acquiredfrom sensation, needs and actuation. The Reactive Layer enables the interac-tion of the system with its environment through a set of pre-wired behaviors inresponse to specific stimuli. The Adaptive Layer uses the cues provided by theReactive Layer to form sensory representations and their associated behaviors.These internal representations of the world, or prototypes, are later used by theContextual Layer, a behavioral learning system based on short and long mem-ory systems, to construct goal-oriented behavioral plans based on mechanismsof operant conditioning.

The Reactive and Adaptive layers of the DAC architecture follow the par-adigm of classical conditioning, where a stimulus that was initially neutralbecomes conditioned (CS) by relating it to an intrinsically motivational orunconditioned stimulus (US), allowing the CS to trigger actions, or conditionedresponses (CR). In this sense, the acquired sensor and motor states are asso-ciated through the valence states triggered by the Reactive Layer. The US isdefined through internal states and the CS, through sensory modalities.

The Contextual Layer is a behavioral learning system that uses the senso-rimotor representations acquired by the Adaptive Layer as inputs to constructgoal-oriented behavioral plans. It comprises systems for memorizing and recallingbehavioral sequences, consisting of two structures: short-term memory (STM)and long-term memory (LTM), which allow for the formation (conditional onthe goal achievement of the agent as signaled by the Reactive and the AdaptiveLayers) of sequential representations of states of the actions generated by theagent and of the environment. What triggers the activation of the ContextualLayer is the Discrepancy (D) measure. When this measure, calculated as thediscrepancy between expected and actual CS events, reaches a value below apre-specified Discrepancy threshold (θD), the Contextual Layer is enabled.

Learning is defined as a process where the learner moves through a hierar-chically organized progression of stages to acquire new knowledge. Therefore,an agent’s actions result from merging the perceived events coming from a con-stantly changing environment and the agent’s own internal needs. DAC predictsthat learning is organized along a hierarchy of complexity. The Reactive Layerallows the organism to explore the world and gain experiences based on whichthe Adaptive Layer would learn the states of the world and their associations.

Modulating Learning Through Expectation in a Simulated Robotic Setup 403

Only after the states are well consolidated the Contextual Layer can extractconsistent rules and regularities.

DAC predicts that in order to learn and consolidate new material the learnerundergoes a sequence of learning phases: resistance, confusion, and abductionthat a learner has to go through in order to acquire and consolidate new knowl-edge. Resistance occurs when the learner overestimates his knowledge and skillsby defending his own world’s model when facing a new condition [5]. This resis-tance to accept new knowledge drives the learner to confusion. When feelingconfused, the learner has the need to readjust his internal model to cope withthe new one. Nevertheless, this confusion has to stay in a controllable range sothat he does not feel frustrated or not challenged enough. Lastly, the develop-ment of new theories to overcome confusion and their posterior assessment iswhat is called abduction. This is what allows the learner to acquire new knowl-edge. The agent begins forming his inner world model in the Reactive Layer,through reflexes, but it is through adaptation and experience that this worldmodel is shaped, to reach its maximum accuracy in the Contextual Layer.

This process is not completely parallel to the layers of the DAC architecture,but results from their combination.

2 Materials and Methods

2.1 Experimental Setup and Procedure

We tested the DAC architecture using a simulated robot solving a foraging taskin a restricted arena as in [6]. The arena contains a light source used as a tar-get (unconditioned stimulus, US) and six patches of different color on the floorserving as contextual cues (conditioned stimuli, CSs) (Fig. 1). Colored patcheson the floor are distinguished by their hue value. On each trial, the robot is ran-domly placed in one of the three starting positions (grey patches on the bottomof the arena) and the goal of the robot is to maximize the number of times it cancorrectly find the light. To learn the task, the robot has to form adequate rep-resentations of the environmental cues and stable associations of sensory eventsand actions (CS-UR associations).

We considered two experimental conditions in which the DAC contextuallayer was either “Diasbled” or “Enabled”. For each condition the experimentalsession is divided in two main cycles, each of them beginning with a stimulationperiod (2000 time steps), where the US signal from the target is activated, andfollowed by a recall period (5000 time steps), where the US signal from the targetis deactivated.

The foraging task is implemented in the Gazebo robot simulator (v4.0) [7]whereas the DAC neuronal model is implemented with the iqr neuronal networksimulator (v4.2) [8] running on a Linux machine.

2.2 Predictive Correlative Subspace Learning Rule

In DAC the adaptive layer forms internal representations of the environment andlearns sensory-motor contingencies based on the classical conditioning paradigm

404 M. Blancas et al.

Robot

Starting points

Cues (CS)

Light (US)

Lightsensors

Camerasensor

Fig. 1. The experimental setup. The restricted arena foraging task used for the exper-iments. The unconditioned stimulus target (US, white circle) corresponds to the lightsource, colored patches on the floor represent the contextual cues (conditional stimuli,CSs) whereas grey patches correspond to the starting positions. The wheeled simulatedrobot is equipped with a set of light sensors to detect the source of light (US sensors)and a camera to detect the patches on the floor (CS sensor).

[1,3,6]. The slow dynamics describing the change of synaptic weights (W) followthe predictive correlative subspace learning rule (PCSL) [2].

ΔW = η((x − Wy)((1 − ζ)y + (1 + ζ)r)T + ρe(r − y)T )

The parameter η represents the learning rate, the ρ parameter varies theinfluence of the behavioral error term on learning. The ζ parameter regulatesthe balance between perceptual (ζ ∼ −1) and behavioral learning (ζ ∼ 1), thatis, how much the system relies on perception or behavior to define the prototypes.An extended study on the individual effects of these components can be foundin [2].

The contextual layer relies on the internal representations formed by theadaptive layer and is activated only when the CS prototypes approximate the CS.This transition is controlled by a discrepancy measure D, which, whenever fallsbelow 0.2, enables the contextual layer. D is a running average of the differencebetween the perceived and the expected CS (or prototype) defined as:

d(x, e) =1N

N∑j=1

∣∣∣∣ xj

max(x)− ej

max(e)

∣∣∣∣

3 Results

In this study, we tested the differences in the behavior of a simulated robot dur-ing a foraging task. In this task, the robot had to navigate in an environment

Modulating Learning Through Expectation in a Simulated Robotic Setup 405

and learn to associate sensory events (CS) to actions (UR), based on classicalconditioning. Two conditions (with 10 repetitions each) were tested: the “Dis-abled” condition (where the only DAC layers that are enabled are the Reactiveand the Adaptive), and the “Enabled” condition (where, as soon as D falls below0.2, the Contextual layer is enabled). We analysed the temporal evolution of thediscrepancy measure across simulations for the two conditions. In the Enabledcondition, the activation of the Contextual Layer makes the Discrepancy fallbelow the value observed for the Disabled condition (Fig. 2 left). This is causedby the reduction of the discrepancy between the predicted and actual CS eventsand the stabilization of the synaptic weights inside the Adaptive Layer, whichleads to a transition to the Contextual Layer. The graph at the right representsthe distribution of the changes in discrepancy, where we can see that the dis-crepancy of the “Enabled” condition is lower than the one of the “Disabled”condition.

Stimulation Stimulation

Recall Recall

Fig. 2. (Left) Evolution of the discrepancy measure (D). The dashed vertical linesindicate the beginning of the periods inside each cycle. The dashed horizontal lineindicates the transition value of D. (Right) Distribution of discrepancy changes forthe two experimental conditions (enabled and disabled contextual layer) (Color figureonline)

Figure 3 represents the evolution of the change in the weights. During thesecond stimulation and recall periods, there is a significantly lower value of theaverage absolute change in synaptic weights: enabled |ΔW | = 1.6x10−3, disabled|ΔW | = 1.8x10−3 (Kolmogorov-Smirnov p � 0.001).

Verschure et al. [3] showed that enabling the Contextual Layer results ina less variable behavior, what, in turn, lowers the variability of the perceivedsensory inputs that the system has to classify. This leads to a more reduced setof input states to adjust the structures for perceptual learning; a change that, atthe same time, can be seen in the reduction and stabilization of the Discrepancy

406 M. Blancas et al.

Fig. 3. Evolution of the change in synaptic weights during the experimental session).(Color figure online)

measure. Thus, the transition to the Contextual Layer may be facilitated bybehavioral feedback, as it lowers the discrepancy between expected and actualCS events.

4 Discussion and Conclusion

In this study, we implemented a learning task with a simulated autonomousrobotic system controlled by the Distributed Adaptive Control (DAC) architec-ture, a neuronal based model that includes perceptual and behavioral learningmechanisms.

Our aim was to define a first approach that would allow us to operationallyidentify and explain the transition between the three stages of learning postu-lated by the pedagogical model of DAC.

The first time the robot faces the arena, it does not have any knowledge ofit either. It only has the schemes coming from its reflexes, provided from theReactive Layer to interact with the environment. These schemes are similar tothe neonatal schemes postulated by Piaget which come from the cognitive struc-tures underlying innate reflexes, even when they have not had much experienceabout the world [9].

This system performs through a balanced combination between perceptualand behavioral learning. On the one hand, perceptual learning permits to acquiremore knowledge about the environment, but can lead to the acquisition of erro-neous behaviors. On the other hand, behavioral learning allows for a betterperformance but drives the system to an acquisition of a smaller state space.Only when the confidence between perceived and expected stimuli is high (i.e.,the discrepancy variable D is lower than a specified threshold), the ContextualLayer is enabled and begins to store the correct sequences in long-term memory.Enabling the Contextual Layer, then, ensures the generation of actions mostsupported by the knowledge stored in the LTM.

Modulating Learning Through Expectation in a Simulated Robotic Setup 407

In this setup, the balance measure, responsible for the interplay betweenperceptual and behavioral learning, was set to achieve a balanced combinationof the two kinds of learning. Nevertheless, if this variable was set to enable onlyone of the two kinds of learning, the behavior of the robot would be affected. Duffet al. [6] shown how, on the one side, enabling only perceptual learning causesthe robot to associate arbitrary actions to the patches, what complicates itspossibilities to reach the target. On the other side, when there is only behaviorallearning, the robot learns to associate patches and correct actions that lead it tothe light. When there is an interplay between perceptual and behavioral learning,the robot not only associates some of the patches to the correct actions that leadit to the light, but also learns to associate some others to the actions that do notlead to it. Moreover, the results of Verschure et al. [3] confirmed that behavioralfeedback directly enhances performance.

Considering Peirce theory [10], every agent has his own idea of the world(defined as ego), in contrast with the real way the world is (defined as non-ego).When the agent is faced with a situation that does not fit his inner world model(ego), he would be surprised. DAC proposes an equivalent paradigm, wherethis difference between the inner world of the learner derived from his previousknowledge (similar to Peirce’s ego) and the world itself, which presents him newknowledge (Peirce’s non-ego) causes confusion, what the learner has to overcomein order to abduct and acquire knowledge. A similar difference can be found inthe Discrepancy measure, that is, the difference between the expected CS and theactual one. Whenever this measure has reached a pre-defined value, the confusiongenerated by the difference between expectation and reality disappears.

Moreover, following DAC model, it has been previously demonstrated that isnot the specific use of perceptual or behavioral learning what leads to the bestresults, but the interplay among them [3]. In the same way, we can extract fromPeirce that the best results of this difference between ego and non-ego would becoming from interplay between Perception and Imagination.

4.1 Future Work

Further steps will be focused on continuing exploring the similarities betweenthe phases of learning based on DAC’s pedagogical model and the dichotomy ofperceptual and behavioral learning. One possibility would be the study of theresistance phase as being reflected on the system when it is more dependent ofperceptual learning and less prone to adapt to changes into the environment.

Moreover, another plausible experiment would be testing the behavior of thesimulated robot when the distribution of the cue patches is changed after theContextual Layer has been enabled and how it affects to its behavior and learn-ing, with the aim of studying the modulation of the transition from confusionphase to the abduction one.

Acknowledgments. This work is supported by the EU FP7 project WYSIWYD(FP7-ICT-612139) and EASEL (FP7-ICT- 611971).

408 M. Blancas et al.

References

1. Marcos, E., Sanchez-Fibla, M., Verschure, P.F.M.J.: The complementary rolesof allostatic and contextual control systems in foraging tasks. In: Doncieux, S.,Girard, B., Guillot, A., Hallam, J., Meyer, J.-A., Mouret, J.-B. (eds.) SAB 2010.LNCS(LNAI), vol. 6226, pp. 370–379. Springer, Heidelberg (2010)

2. Duff, A.: The dynamics of perceptual and behavioral learning in executive control.Ph.D. thesis, Diss., Eidgenossische Technische Hochschule ETH Zurich, Nr. 18766,2009 (2009)

3. Verschure, P.F.M.J., Voegtlin, T., Douglas, R.J.: Environmentally mediated syn-ergy between perception and behaviour in mobile robots. Nature 425(6958), 620–624 (2003)

4. Verschure, P.F.M.J.: Distributed adaptive control: a theory of the mind, brain,body nexus. Biologically Inspired Cogn. Architectures 1, 55–72 (2012)

5. Kruger, J., Dunning, D.: Unskilled and unaware of it: how difficulties in recognizingone’s own incompetence lead to inflated self-assessments. J. Pers. Soc. Psychol.77(6), 1121 (1999)

6. Duff, A., Verschure, P.F.M.J.: Unifying perceptual and behavioral learning with acorrelative subspace learning rule. Neurocomputing 73(10–12), 1818–1830 (2010)

7. Koenig, N., Howard, A.: Design and use paradigms for gazebo, an open-sourcemulti-robot simulator. In: Proceedings. 2004 IEEE/RSJ International Conferenceon Intelligent Robots and Systems, (IROS 2004). vol. 3, pp. 2149–2154. IEEE(2004)

8. Bernardet, U., Verschure, P.F.M.J.: iqr: A tool for the construction of multi-levelsimulations of brain and behaviour. Neuroinformatics 8(2), 113–134 (2010)

9. Piaget, J.: The Origins of Intelligence in Children, vol. 8(5), pp. 18–1952. Interna-tional Universities Press, New York (1952)

10. Peirce, C.S., Turrisi, P.A.: Pragmatism as a Principle and Method of Right Think-ing: The 1903 Harvard Lectures on Pragmatism. SUNY Press, New York (1997)

Don’t Worry, We’ll Get There:Developing Robot Personalities to Maintain

User Interaction After Robot Error

David Cameron(B), Emily Collins, Hugo Cheung, Adriel Chua,Jonathan M. Aitken, and James Law

Sheffield Robotics, University of Sheffield, Sheffield, UK{d.s.cameron,e.c.collins,mcheung3,

dxachua1,jonathan.aitken,j.law}@sheffield.ac.ukhttp://www.sheffieldrobotics.ac.uk/

Abstract. Human robot interaction (HRI) often considers the humanimpact of a robot serving to assist a human in achieving their goal or ashared task. There are many circumstances though during HRI in whicha robot may make errors that are inconvenient or even detrimental tohuman partners. Using the ROBOtic GUidance and Interaction DEvel-opment (ROBO-GUIDE) model on the Pioneer LX platform as a casestudy, and insights from social psychology, we examine key factors fora robot that has made such a mistake, ensuring preservation of individ-uals’ perceived competence of the robot, and individuals’ trust towardsthe robot. We outline an experimental approach to test these proposals.

Keywords: Human-robot interaction · Design · Guidance · Psychology

1 Background

Human-robot interaction (HRI) research typically explores interactions in whichthe robot plays a supportive or collaborative role for the human user [4]. However,there are circumstances in which robots may fail to meet these requirements,either through errors in processing the interaction scenario, or failure to adaptto changing HRI scenario circumstances. Furthermore, reliability and error ratesof robots have both been identified as important factors in user trust towardsrobots [4]. Recent work has explored the social impact of a robot’s fault orerror, in terms of user cooperation [9], and whether an apology from a robot canmitigate the negative impact of the mistake [10]. However, there still remainsmuch to be explored: first in terms of the negative impact even simple robotmistakes can have on user trust and their willingness to engage in HRI; andsecond, if the methods by which a robot acknowledges a mistake, and thenpotentially corrects for it, have differential impacts on HRI.

Recent work identifies that the means by which a socially adaptive robotasks for help can impact on: users’ attitudes towards the robot, clarity in thesupport the robot needs, and people’s willingness to use the robot in collaborativec© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 409–412, 2016.DOI: 10.1007/978-3-319-42417-0 38

410 D. Cameron et al.

tasks [2]. The context for that interaction does not concern robot error butrather robots requiring user intervention (completing a task outside of the robot’scapability) to progress towards a goal. Nevertheless, the principles on which thatinteraction is based can be drawn upon to identify means by which robots canuse particular social interactions to recover, in part, from mistakes.

The synthetic personality a robot exhibits can have substantial impact onthe user’s experience of, and their engagement with, HRI [3]. The relatively newsocial domain of HRI is unfamiliar to many, but principles of social psychologyhave been applied to social HRI scenarios with promising results. These include:accurate recognition of synthetic personality types, even in non-humanoid robots[6]; development of classification of social ‘rules’ for robots to adhere to [3]; andparticipant response towards synthetic personalities corresponding with theo-retical models of interpersonal cooperation [2]. This paper draws on social psy-chological theories to develop a model of social factors for robots that supportrecovery from robot error and maintain user interaction.

2 ROBO-GUIDE Interactive Scenario

To explore the social factors that support a robot in recovery from error, andmaintain user interaction, it is useful to consider an interactive scenario in whichthese circumstances might arise. The ROBOtic GUidance and Interaction DEvel-opment (ROBO-GUIDE) project [1,8] is an ideal scenario to consider the impactof such social factors as it requires humans to place trust in a robot, even in theinstance where an error might occur.

ROBO-GUIDE is embodied on the Pioneer LX mobile platform. The plat-form is capable of autonomously navigating a multistory building and leadingusers from their arrival point to their desired destination. Our focus here is a crit-ical point in building navigation: floor determination whilst using the elevator tonavigate between floors. Each floor in the building is similar and ROBO-GUIDErelies on subtle cues to differentiate between them; in noisy environments errorscan occur [8]. For example, during busy periods the corridor structural featuresor floor-indication signs may be obscured, noise may mask lift announcements, orthe ROBO-GUIDE might be misinformed or misled by a member of the public.

We identify the disembarkation of the elevator on the wrong floor as a simpleand, critically, natural type of error for users to encounter (for the purposesof the experiment, errors would be staged). This tour guide scenario, in whichindividuals use a robot to navigate an unfamiliar building, means any error wouldmildly inconvenience the user. Moreover, it gives opportunity for the robot torecover from the error and allows testing of different means for the robot tosocially communicate error and attempts to correct it.

3 Social-Recovery and Experimental Proposal

Previous HRI work has identified a social psychological model of cooperation [7]as a useful framework for exploring the impact different personalities can have on

Don’t Worry, We’ll Get There: Developing Robot Personalities 411

user willingness to engage with robots [1]. This model, when applied in an HRIcontext, contrasts the impact of friendly-oriented statements to build user-liking,and goal-oriented statements to suggest the robot’s task competency. Findingsindicate that individuals are more willing to use a robot they like than a robotsuggesting task competency; they also regard the interaction with a friendly robotto be less ambiguous [2]. Individuals further report trust towards the robot acrossthe dimensions offered in the model: affective (from personable interactions) andcognitive (from evidence of competency). While results do not show substantivedifferences in trust between conditions, this may be due to a ceiling effect becausethere was no challenge made to the competency of the robot.

The error scenario (Sect. 2) provides such a challenge to the robot’s apparentcompetency; it further provides opportunity for the robot to attempt to sociallyrecover from the error and work to restore use perceptions of competency. Again,using the framework developed in [1], friendly-oriented and goal-oriented syn-thetic personalities, as means to socially communicate error, may result in dif-ferences in individuals’ views towards the robot.

The framework may be further enhanced by considering social psychologicalunderstanding on the impact of acknowledging and apologising for error. Apolo-gies for errors in competency are observed to raise an individual’s trustworthinessbut not their apparent competency [5]. Simple apologies may therefore supporta user’s affective trust towards a robot but not their cognitive trust. In contrast,identification of the error and communication of means to resolve it, to maintainprogress towards a goal, may restore users cognitive trust.

We outline a brief experimental proposal to test the impact of statements pro-moting affective and cognitive trust for the user in the error scenario (Sect. 2).Acknowledgments of the error by the robot are planned to be manipulatedin a 2× 2 experimental between-subjects design: inclusion or absence of thecompetency-oriented apology-oriented statements following error. These will becommunicated using the on-board speech synthesizer1. We anticipate that thesestatements will impact on participant willingness to use the robot through thekey channels of affective and cognitive trust.

Participants will experience one of the four conditions: (1) the control con-dition comprising simple instructions for the user to follow after making anerror, (e.g., ‘Follow me back to the lift’); (2) inclusion of competency-orientedstatements that emphasise the robot’s ability to recognise the environment, iden-tifying the cues used to orient, and reaffirming the goal (e.g., ‘That sign said weare on C floor and we need to go to B floor. Follow me back to the lift’); (3)inclusion of apology-oriented statements that emphasise attempts to relate tousers but do not indicate competency (e.g., ‘Sorry about the error; we can allmake mistakes sometimes. Follow me back to the lift’); (4) inclusion of both thecompetency- and apology-oriented statements.

Outcomes are measured using the prior measures of affective and cognitivetrust, clarity of interaction and willingess to use the robot [2]. We anticipate that

1 The viable alternatives of pre-recorded spoken phrases or an on-screen display areacknowledged.

412 D. Cameron et al.

the apology and competency statements will support affective and cognitive trustrespectively, although affective trust better predict willingness to use the robotin future. Findings from this work will support the development of a sociallyadaptive robots for HRI and further reveal the social models users draw uponwhen interacting with socially engaging robots.

Acknowledgments. This work was supported by European Union Seventh Frame-work Programme (FP7-ICT-2013-10) under grant agreement no. 611971.

References

1. Cameron, D., et al.: Help! I cant reach the buttons: facilitating helping behav-iors towards robots. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A., Prescott,T.J. (eds.) Living Machines 2015. LNCS(LNAI), vol. 9222, pp. 354–358. Springer,Heidelberg (2015)

2. Cameron, D., Loh, E.J., Chua, A., Collins, E.C., Aitken, J.M., Law, J.: Robot-stated limitations but not intentions promote user assistance. In: Salem, M., Weiss,A., Baxter, P., Dautenhahn, K. (eds.) Proceedings of the 5th International Sym-posium on New Frontiers in Human-Robot Interaction. AISB (In Press)

3. Dautenhahn, K.: Socially intelligent robots: dimensions of human-robot interac-tion. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 679–704 (2007)

4. Hancock, P.A., Billings, D.R., Schaefer, K.E., Chen, J., De Visser, E., Parasura-man, R.: A meta-analysis of factors affecting trust in human-robot interaction. J.Hum. Factors Ergon. Soc. 53, 517–527 (2011)

5. Kim, P., Ferrin, D., Cooper, C., Dirks, K.: Removing the shadow of suspicion: theeffects of apology versus denial for repairing competence- versus integrity-basedtrust violations. J. Appl. Psychol. 89, 104–118 (2004)

6. Lee, K.M., Peng, W., Jin, S.A., Yan, C.: Can robots manifest personality?: Anempirical test of personality recognition, social responses, and social presence inhuman-robot interaction. J. Communication 56, 754–772 (2006)

7. McAllister, D.J.: Affect-and cognition-based trust as foundations for interpersonalcooperation in organizations. Acad. Manage. J. 38, 24–59 (1995)

8. McAree, O., Aitken, J.M., Boorman, L., Cameron, D., Chua, A., Collins, E.C.,Fernando, S., Law, J., Martinez-Hernandez, U.: Floor Determination in the opera-tion of a lift by a mobile guide robot. In: Proceedings of the European Conferenceon Mobile Robots (2015)

9. Salem, M., Lakatos, G., Amirabdollahian, F., Dautenhahn, K.: Would you trust a(faulty) robot? Effects of error, task type and personality on human-robot cooper-ation and trust. In: Proceedings of the 10th ACM/IEEE International Conferenceon Human-Robot Interaction (HRI 2015), Portland (2015)

10. Snijders, D.: Robots recovery from invading personal space. In: 23rd Twente Stu-dent Conference on IT, Enschede, The Netherlands (2015)

Designing Robot Personalities for Human-RobotSymbiotic Interaction in an Educational Context

David Cameron(B), Samuel Fernando, Abigail Millings, Michael Szollosy,Emily Collins, Roger Moore, Amanda Sharkey, and Tony Prescott

Sheffield Robotics, University of Sheffield, Sheffield, UK{d.s.cameron,s.fernando,a.millings,m.szollosy,

e.c.collins,r.k.moore,a.sharkey,t.j.prescott}@sheffield.ac.ukhttp://easel.upf.edu/project

Abstract. The Expressive Agents for Symbiotic Education and Learn-ing project explores human-robot symbiotic interaction with the aim tounderstand the development of symbiosis over long-term tutoring inter-actions. The final EASEL system will be built upon the neurobiologicallygrounded architecture - Distributed Adaptive Control. In this paper, wepresent the design of an interaction scenario to support development ofthe DAC, in the context of a synthetic tutoring assistant. Our humanoidrobot, capable of life-like simulated facial expressions, will interact withchildren in a public setting to teach them about exercise and energy. Wediscuss the range of measurements used to explore children’s responsesduring, and experiences of, interaction with a social, expressive robot.

Keywords: Human-robot interaction · Humanoid · Psychology ·Symbiosis · Facial expression

1 Background

A key challenge in Human-Robot symbiotic interaction (HRSI) is to developrobots that can adapt to users during interactions. The Expressive Agentsfor Symbiotic Education and Learning (EASEL) project seeks to develop abiologically-grounded robot [9] that is responsive to users and capable of adap-tation to user needs within, and across, interactions. One particular challengefor long-term or repeat human-robot interactions is maintaining successful socialengagement with the user. A means to address this issue is ensuring the syn-thetic personality for a social robot is engaging and responsive to users [1]. Acornerstone of the EASEL project is the development of a synthetic personalityfor a social robot, which promotes sustained user engagement.

Robots that can effectively achieve the process of sustained social engage-ment and further act to positively shape user behavior (such as communicatingto inform further user interactions) could be considered to behave symbioti-cally. HRSI extends beyond standard human-robot interaction by consideringthe: ‘[D]ynamic process of working towards a common goal by responding andc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 413–417, 2016.DOI: 10.1007/978-3-319-42417-0 39

414 D. Cameron et al.

adapting to a partners actions, while affording a partner to do the same. Thisterm suggests a mutually beneficial relationship between various parties’ [5]. TheEASEL project explores HRSI in the context of one-to-one tutoring interactionsbetween a socially-adaptive humanoid robot and children.

A tutoring scenario (described in Sect. 2) gives context for the robot to adaptin response to a child’s behaviour, updating its communication based on thedeveloping interaction. The scenario also serves to direct children’s interactionswith the robot and gives goals for children to work in concert with the SyntheticTutoring Assistant (STA). Effectiveness of the STA as an engaging personality,educational device, and model for an HRSI theoretical framework can be assessedthrough development of field interaction scenarios.

2 Field Interaction Scenario

2.1 The STA

The STA is to be developed upon the neurobiologically grounded architectureDistributed Adaptive Control (DAC) [9]. The DAC is a tiered, self-regulation sys-tem, structured to manage low- and high-level behaviours for synthetic agents,comprising: allostatic control, adaptation to sensory information, and the acqui-sition and expression of contextual plans. Behavioural outputs of the DAC areguided through synthetic motivation and emotion states. Behavioural interactionis scheduled using the AsapRealiser (e.g., [8]) interaction manager, which deter-mines processing of sensory inputs to the robot and how content and feedbackare delivered to the user.

This complete system is embodied in the Hanson Robokind Zeno R25 [7](Fig. 1). A distinctive feature of the Zeno model robot is the platform’s realisticface, capable of displaying a range of life-like simulated facial expressions. Thismodel enables the conveying of synthetic emotions from the DAC in immediatelyrecognizable ways that are minimally obtrusive to ongoing interactions [6].

Fig. 1. The Robokind Zeno R25 platform (humanoid figure approximately 60 cm tall)

Designing Robot Personalities for Human-Robot Symbiotic Interaction 415

2.2 Tutoring

HRSI between the user and STA will be explored in the context of a healthy-living tutoring scenario. Children will be invited to interact with the STA at atwo-week special exhibit on robotics at a natural history museum.

The interaction comprises of four key stages. First, the robot briefly intro-duces itself and the activities for the interaction; in this period, the robot alsocalibrates its automatic speech recognition (ASR) to brief prompted responsesfrom the child. Second, children engage in robot-led physical activities of differ-ent intensities and duration. After each activity, the robot provides responsivefeedback about the energy used by the child to perform each action. Third,children complete a quiz based on their activity (i.e., which exercise used themost/least energy) and a similar quiz based on exercise and energy in general(i.e., identifying low- and high-energy activities). Finally, children can give Zenovoice commands to perform facial expressions or other physical actions. Theinteraction lasts approximately ten minutes in total.

The scenario will be delivered autonomously by the STA1. Development ofthe robot’s personality is drawn from prior HRSI research to best ensure userengagement throughout the interaction. This includes regular confirmation fromthe robot that it is responsive to the individual user [4] and showing context-appropriate simulated facial expressions when delivering feedback [2]. Withinthe context of a tutoring scenario, supportive interactions, such as these, areanticipated to not just encourage interaction with the robot but also promoteuser engagement with the learning activity.

3 Exploring User Engagement, Learning, and HRSI

User Engagement with the scenario and the robot will be measured acrossmultiple means, including: video recording, motion tracking, self-report andparental-report of child. User engagement will be examined using a between-subjects design. The standard scenario (Sect. 2.2) will be contrasted againstan ‘enhanced’ scenario, which includes a brief rapport-building introduction bythe robot (experimental condition) or researcher (active-control condition). Weanticipate that the rapport-building introduction by the robot will promote userengagement, particularly at the start of the interaction.

The interaction scenario will be filmed throughout and children’s facialexpressions coded, as done in a prior study [2]. Video data will be further used todetermine children’s gaze throughout interaction; our prior work indicates thatchildren turn towards a socially-responsive robot, in anticipation of its speak-ing, more often than towards a less-responsive robot [4]. Children turning theirattention towards parents or the researcher will be coded as measures of com-fort seeking and clarity seeking respectively. Children looking towards the STA,

1 Wizard of Oz style control is available should the ASR fail to adapt to an individualchild’s voice.

416 D. Cameron et al.

particularly with positive facial expressions, rather than towards other figures,will be regarded as an indication of engagement with the STA.

Physical activity (in terms of joules) can be calculated automatically by theSTA. This provides an additional measure of engagement; children more engagedin a prior scenario performed more physical activity at the robot’s direction [4].Interpersonal distance between the user and the robot during the physical activitywill further be used as an index of social engagement with the robot: a smallerdistance between user and robot indicating greater comfort in the interaction.

After the interaction, children will complete a brief survey on their affect feltwhen working with Zeno and their enjoyment of the scenario. Parents will alsocomplete a brief survey on their views of their child’s engagement and enjoymentof the scenario.

User learning will be examined using between-subjects design experiment.The standard scenario schedules general questions on the relationship betweenenergy usage and both exercise intensity and duration after children have com-pleted (1) physical activity of varying intensities and duration and (2) questionson their activity consolidating this relationship. Baseline measures of children’sunderstanding of this relationship can be determined by scheduling the criticalquestions before the physical activity and consolidation questions. The numberof questions answered correctly by children in each condition will be comparedas a measure of learning.

The learning activity is more appropriate for seven-year-old children becauseit addresses topics covered in the UK National Curriculum for Lower Key StageTwo (ages seven to nine). We anticipate that the supportive social interactionand learning activity provided will provide scaffold for children to understandthe relationship between exercise and energy. Given that the interaction is partof a public exhibit and open to all ages, a wide age range is anticipated in ourdata collection. Children’s age will be co-varied in analysis to make account forchildren’s prior education on the subject of exercise and energy.

HRSI development will be informed by the above factors. It will further beexplored both in terms of children’s perceptions of the robot and the individ-ual differences in personality that may facilitate interaction. After interaction,children will complete a brief survey of their perceptions of Zeno’s ‘status’ asbeing like a machine or person [3] and familiarity (i.e., like a stranger, acquain-tance, friend, best friend). They further rate Zeno’s ‘knowledge’ about exerciseand skill at teaching. The above measures are anticipated to correlate with userengagement and user learning and further inform our understand of the socialcontext that children use to understand HRSI (e.g., working with a teacher versusco-operation with a fellow friendly learner).

This study offers a potentially large-scale sample so that we may examineindividual differences in personality and the impact this can have on children’sperceptions of, and interaction with, a socially-responsive humanoid robot. Par-ents will complete a brief five-factor personality questionnaire about their child.We anticipate that openness and agreeableness will positively correlate with

Designing Robot Personalities for Human-Robot Symbiotic Interaction 417

user engagement with the robot, conscientiousness with scenario engagementand learning, and extroversion with positive expression towards the robot.

This interaction scenario enables the further development of an engaging STAand extension of HRSI research, so that robot personalities can better adapt touser requirements throughout interaction and be tailored to meet individualneeds.

Acknowledgments. This work was supported by European Union Seventh Frame-work Programme (FP7-ICT-2013-10) under grant agreement no. 611971. We thankTheo Botsford and Jenny Harding for their assistance in scenario development.

References

1. Breazeal, C., Scassellati, B.: How to build robots that make friends and influencepeople. In: IEEE/RSJ International Conference on Intelligent Robots and Systems,IROS Proceedings, vol. 2, pp. 858–863 (1999)

2. Cameron, D., Fernando, S., Collins, E.C., Millings, A., Moore, R.K., Sharkey, A.,Evers, V., Prescott, T.: Presence of life-like robot expressions influences children’senjoyment of human-robot interactions in the field. In: Salem, M., Weiss, A., Baxter,P., Dautenhahn, K. (eds.) 4th International Symposium on New Frontiers in Human-Robot Interaction, pp. 36–41 (2015)

3. Cameron, D., Fernando, S., Millings, A., Moore, R.K., Sharkey, A., Prescott, T.:Children’s age influences their perceptions of a humanoid robot as being like aperson or machine. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A., Prescott, T.J.(eds.) Living Machines 2015. LNCS, vol. 9222, pp. 348–353. Springer, Heidelberg(2015)

4. Cameron, D., Fernando, S., Collins, E.C., Millings, A., Moore, R.K., Sharkey, A.,Prescott, T.: Impact of robot responsiveness and adult involvement on children’ssocial behaviours in human-robot interaction. In: Salem, M., Weiss, A., Baxter, P.,Dautenhahn, K. (eds.) Proceedings of the 5th International Symposium on NewFrontiers in Human-Robot Interaction. AISB (in press)

5. Charisi, V., Davison, D., Wijnen, F., van der Meij, J., Reidsma, D., Prescott, T.,van Joolingen, W., Evers, V.:Towards a child-robot symbiotic co-development: atheoretical approach. In: Salem, M., Weiss, A., Baxter, P., Dautenhahn, K. (eds.)in 4th International Symposium on New Frontiers in Human-Robot Interaction, pp.30-35 (2015)

6. Costa, S., Soares, F., Santos, C.: Facial expressions and gestures to convey emotionswith a humanoid robot. In: Herrmann, G., Pearson, M.J., Lenz, A., Bremner, P.,Spiers, A., Leonards, U. (eds.) ICSR 2013. LNCS, vol. 8239, pp. 542–551. Springer,Heidelberg (2013)

7. Hanson, D., Baurmann, S., Riccio, T., Margolin, R., Dockins, T., Tavares, M.,Carpenter, K.: Zeno: A cognitive character. In: AI Magazine, and Special Proceed-ings of AAAI National Conference, Chicago (2009)

8. Reidsma, D., van Welbergen, H.: AsapRealizer in practice - a modular and extensiblearchitecture for a BML Realizer. Entertainment Comput. 4, 157–169 (2013)

9. Verschure, P.F.: Distributed adaptive control: a theory of the mind, brain, bodynexus. Biologically Inspired Cogn. Architectures 1, 55–72 (2012)

A Biomimetic Fingerprint Improves SpatialTactile Perception

Luke Cramphorn1,2(B), Benjamin Ward-Cherrier1,2, and Nathan F. Lepora1,2

1 Department of Engineering Mathematics, University of Bristol, Bristol, UK{ll14468,bw14452,n.lepora}@bristol.ac.uk

2 Bristol Robotics Laboratory, University of Bristol, Bristol, UK

Abstract. The function of the human fingertip has been often debated.There have been studies focused on how the fingerprint affects the per-ception of high temporal frequencies, such as for improved texture per-ception. In contrast, here we focus on the effects of the fingerprint onthe spatial aspects of tactile perception. We compare two fingertips, onewith a biomimetic fingerprint and the other having a smooth surface.Tactile data was collected on a sharp edged stimulus over a range oflocations and orientations, and also over a smooth (cylindrical) object.The perceptual capabilities of both (fingerprint and smooth) sensor typeswere compared with a probabilistic classification method. We find thatthe fingerprint increases the perceptual acuity of high spatial frequencyfeatures (edges) by 30–40% whilst not influencing the tactile acuity oflow spatial frequency features (cylinder). Therefore the biomimetic fin-gerprint acts as an amplifier of high spatial frequencies, and provides uswith evidence to suggest that the perception of high spatial frequenciesis also one of the functions of the human fingertip.

Keywords: Tactile sensors · Biomimetic

1 Introduction

The human sense of touch defines our ability as a species to manipulate theworld in unparalleled ways. There are many morphological aspects that aid inthe performance of this sense. In particular, the fingerprint subtly enhances thetemporal frequency of stimuli, providing us with the ability to detect and identifyfine textures [1,2].

Here we look at the effects of the fingerprint for spatial perception with a bio-mimetic fingertip. Our inspiration for the design comes from the mechanical cou-pling between the surface of the skin and the touch receptor cells, created by thedermal ridges that make up the human fingerprint (Fig. 1a). This coupling occursthrough the dermal papillae, protrusions of the dermis into the epidermis in whichthe touch receptor cells are found [3]. These papillae create a non-planar layer inthe dermis that transfers the kinetic energy from the surface to the sensory cell. Theexact function of the fingerprint is uncertain in the biological community.Hypothe-ses include that the fingerprint exists to assist grasping by acting as a high frictionc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 418–423, 2016.DOI: 10.1007/978-3-319-42417-0 40

A Biomimetic Fingerprint Improves Spatial Tactile Perception 419

Fig. 1. (a) The epidermal ridges overlay protrusion in the dermis (inner layer of skin)called dermal ridges creating a mechanical coupling between the surface and the touchreceptor cells (Merkel cells) that sit within the dermal ridges. (b) Cross sectional com-parison of the two TacTip tips compared in this paper. The fingertip enhanced TacTip(left) has solid plastic cores that run from the inside of the artificial dermal ridges(nodules) to the top of the pins. The original structure (right) has a featureless skinand no plastic cores.

surface [4], and that the fingerprint aids tactile perception by acting as a smallmechanical lever which magnifies the response to tactile contact [5]. It was alsoshown later that a lensing effect amplifies the signal/noise ratio due to the pres-ence of a fingerprint [6]. Due to previous biomimetic studies, there is evidence thatsupports the idea that the fingerprint aids in perception of textures (and other hightemporal frequency profiles) [1,2]. Finite element modelling of the effect of dermalridges shows that there is an increase in stress contrast between adjacent measure-ment points, and thus better accuracy and consistency in edge encoding [7]. Herewe present the evidence for the fingerprint also aiding in the perception of high spa-tial frequency (measured from the components of a spatial Fourier transform) suchas edges. To demonstrate this effect of the fingerprint, a biomimetic optical tactilefingertip is used with two different tips, one with the fingerprint and one without.By comparing the accuracy of these tips on an edge (high spatial frequency) vialocalisation and orientation and over a cylinder (low spatial frequency) we showthat the sensitivity and accuracy of the sensor is improved by the inclusion of thefingerprint over the large spatial frequency with no change to the accuracy or sen-sitivity on the low spatial frequencies. Thus we demonstrate that the fingerprinthelps with the perception of high spatial features as well as the previously demon-strated high temporal features.

2 Methods

The sensor used is the TacTip optical tactile sensor developed at Bristol RoboticsLaboratory [8–11]. The TacTip sensor is relativity cheap and easy to modify asit is 3D printed. The input of the sensor uses an off-the-shelf CCD (LifeCamCinema HD, Microsoft) where an array of pins on the internal side of a 3D

420 L. Cramphorn et al.

Fig. 2. (a) Image of the TacTip sensor used for the experiment, highlighting the nod-ules, that simulate the fingerprint, on the surface of the sensor. (b) Graphical repre-sentation of the pin layout on the interior of the sensor. Each pin is coloured and canbe matched to deflections on data sets in Fig. 3.

Fig. 3. The displayed data shows the time-series recordings over 600 taps spanning a20 mm location range across the edge. Pin displacements are shown in units of pix-els, where each colour represents the displacement of a corresponding pin (shown inFig. 2b). A sharpening of the features (highlighted as an increase in rate of pin deflec-tion) due to the fingerprint (b) in comparison to the set without (a) reduces ambiguityand thus improves classification.

printed rubber-like skin are tracked from frame to frame. The pins are inspiredby the dermal papillae that are linked to tactile sensing in the human finger. Asthe sensor uses a plug and play web cam, the sensor is easy to install. The skinis filled with an optically clear gel, which gives the sensor compliance as wellas structure. The sensor has interchangeable tips which can be produced at arelatively low cost and keep any contact far from any delicate electronics. Thismeans the sensor is robust and any tip can be replaced with ease if damaged. Theplatform for the system was a six degree-of-freedom robot arm (IRB 120, ABB

A Biomimetic Fingerprint Improves Spatial Tactile Perception 421

Robotics), which precisely positions the TacTip with an absolute repeatabilityof 0.01 mm.

To produce a tip with the biomimetic fingerprint, domes are included on theexterior of the skin and mirror the interior locations of the pins. This createsmechanics akin to dermal ridges. Each dome has a plastic core that starts inthe interior of the dome and connects to the pin tip providing direct couplingbetween the stimuli and the pin tips (Fig. 1b).

The validation of the two tips considered here was performed by analysing theaccuracy in location and orientation of various stimuli. The task was performedcollecting tactile data during taps on the surface of each stimulus. Each taprecords a time series over the press (approximately 2 s), with ∼50 recorded framesper series. The data used for the results in this paper was collected in two distinctsets, ensuring that the training and test sets are different, and that the validationof the results is based on sampling from an independent data set to that usedto train the classifier.

3 Results

Varying sensor location (y) and orientation (θ) over the stimulating featureschanges the extent to which the pins displace, both in magnitude and direc-tion, this creates unique time-series for each contact feature of the object. Thisuniqueness is what permits classification of the feature. Sets with lower ambigu-ity therefore have greater classification accuracy. The data series (Fig. 3) showsthe sx and sy displacements of the recordings, where the key information fromthe contact is the region of increasing displacements in the central region.

The data over the edge feature shows a low rate of change in pin deflection forthe smooth tip (Fig. 3a) contrasted with a high rate of change for the fingerprinttip (Fig. 3b). This contrast creates a less ambiguous representation of the featurein the direction of travel (y) for the tip with the fingerprint.

The validation and comparison of the effectiveness of the two different tipsis done by using probabilistic methods for localization that have been describedextensively in refs [9,12]. The collected training data sets are used such thateach of the move increments are treated as distinct location classes. A likelihoodmodel is constructed using the tactile data at each location using a histogrammethod. Given a sample of test data with an unknown location, this model canbe used to determine the likelihood of which location class it originates from.

The two training sets taken from each of the stimuli are used with the MonteCarlo procedure where randomly sampled data from one of the sets are used toclassify locations in the other (10000 iterations). Each increment of test data usesa maximum likelihood approach to classify the position classes. The maximumlikelihood method means that the lower the ambiguity at each point the betterthe accuracy of the classification.

The errors in location acuity determined from validation show improvementsin accuracy for the tip with the fingerprint: the correct class is classified more fre-quently. The average error with a fingerprint is eloc(y) = 0.067 mm and without

422 L. Cramphorn et al.

is eloc(y) = 0.091 mm which represents a 32 % improvement in location acuity.The validation for orientation of the edge is classified in the same way. Onceagain the number of iterations classifying correctly is higher in the fingerprintversion. The average error of the fingerprint version was eloc(θ) = 0.412 and forthe smooth version was eloc(θ) = 0.693 which represents an improvement of 41 %in orientation accuracy. In the third experiment, the error in location perceptionis examined for a smooth cylinder, similar to the experiment on location acuitypreviously performed in [9]. The results show that the fingerprint neither addednor subtracted in any substantial accuracy with errors of eloc(x) = 0.081 for thefingerprint and eloc(x) = 0.076 without the fingerprint.

4 Conclusions

This study has demonstrated that the inclusion of a biomimetic fingerprintimproves tactile acuity of a biomimetic tactile fingertip to features with a highspatial frequency: the accuracy of the location perception of an edge increasedby 30–40 % with no notable decrease or increase in the accuracy of the locationperception on the low spatial frequency features of a cylinder. This provides uswith evidence to suggest that the perception of high spatial frequencies is alsoone of the functions of the human fingerprint.

Acknowledgment. We thank Sam Coupland, Gareth Griffiths and Samuel Forbesfor their assistance with 3d-printing. LC was supported by the EPSRC Centre forDoctoral Training in Future Autonomous and Robotic Systems (FARSCOPE). BWCwas supported by an EPSRC DTP studentship and NL was supported in part by anEPSRC grant on ‘Tactile Superresolution Sensing’ (EP/M02993X/1).

References

1. Scheibert, J., Leurent, S., Prevost, A., Debregeas, G.: The role of fingerprints in thecoding of tactile information probed with a biomimetic sensor. Science 323(5920),1503–1506 (2009)

2. Winstone, B., Griffiths, G., Pipe, T., Melhuish, C., Rossiter, J.: TACTIP - tac-tile fingertip device, texture analysis through optical tracking of skin features. In:Lepora, N.F., Mura, A., Krapp, H.G., Verschure, P.F.M.J., Prescott, T.J. (eds.)Living Machines 2013. LNCS, vol. 8064, pp. 323–334. Springer, Heidelberg (2013)

3. Dahiya, R., Metta, G., Valle, M., Sandini, G.: Tactile sensing from humans tohumanoids. IEEE Trans. Rob. 26(1), 1–20 (2010)

4. Jones, L., Lederman, S.: Human Hand Function. Oxford University Press,New York (2006)

5. Cauna, N.: Nature and functions of the papillary ridges of the digital skin. Anat.Rec. 119(4), 449–468 (1954)

6. Fearing, R., Hollerbach, J.: Basic solid mechanics for tactile sensing. Int. J. Robot.Res. 4(3), 40–54 (1985)

A Biomimetic Fingerprint Improves Spatial Tactile Perception 423

7. Gerling, G., Thomas, G.: The effect of fingertip microstructures on tactile edgeperception. In: Eurohaptics Conference, 2005 and Symposium on Haptic Interfacesfor Virtual Environment and Teleoperator Systems, World Haptics 2005, FirstJoint, pp. 63–72. IEEE (2005)

8. Chorley, C., Melhuish, C., Pipe, T., Rossiter, J.: Development of a tactile sensorbased on biologically inspired edge encoding. In: ICAR International ConferenceAdvanced Robotics, pp. 1–6. IEEE (2009)

9. Lepora, N., Ward-Cherrier, B.: Superresolution with an optical tactile sensor. In:IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),pp. 2686–2691 (2015)

10. Ward-Cherrier, B., Cramphorn, L., Lepora, N.: Tactile manipulation with a Tac-thumb integrated on the open-hand M2 gripper. Rob. Autom. Lett. 1(1), 169–175(2016)

11. Cramphorn, L., Ward-Cherrier, B., Lepora, N.: Tactile manipulation with bio-mimetic active touch. In: IEEE International Conference on Robotics and Automa-tion (ICRA), pp. 123–129

12. Lepora, N.: Biomimetic active touch with fingertips and whiskers. IEEE Trans.Haptics 9(2), 170–183 (2016)

Anticipating Synchronisation for Robot Control

Henry Eberle(B), Slawomir Nasuto, and Yoshikatsu Hayashi

Brain Embodiment Lab, Reading University, Berkshire RG6 6UR, UKnm010080@reading.ac.uk

Abstract. Anticipating synchronisation (AS) is an interesting phenom-enon whereby a dynamical system can trace the future evolution of ananalogous ‘master’ system using delayed feedback. Although some havetheorised that AS (or a similar phenomenon) has a role in human motorcontrol, research has primarily focused on demonstrating it in novel sys-tems rather than its practical applications. In this paper we explore onesuch application: by coupling the dynamics of the joints in a simulatedrobot arm, we seek to demonstrate that AS can have a functional role inmotor control by reducing instability during a reaching task.

Keywords: Anticipating synchronisation · Motor control · Jointsynergy · Robotics

1 Introduction

Because sensing and actuation are not instantaneous (and cannot be), somedegree of anticipation is necessary for an organism to execute any form of move-ment accurately. Typically this is considered the domain of computational for-ward models that accept the parameters of an action and calculate its result. Analternative formulation is that of strong anticipation where the prediction arisesfrom the organism’s continuous coupling with the environment rather than aninternal model [1].

Anticipation Synchronisation (AS), the phenomenon where a ‘slave’ dynam-ical system can predict the behaviour of the qualitatively similar master drivingit [2], is the closest realisation of this concept that yet exists. However, researchhas focused primarily on its applications in communication rather than motorcontrol or cognition.

This paper offers a 1st step demonstrating that an AS coupling can play astabilising role in motor control without the need for explicit kinematic models,using the example of a two-joint planar robot arm.

Closed-loop control of a robot arm generally requires the use of a kinematicmodel in order to transform target coordinates in the external work space intoappropriate joint angles. However, the arm can also be treated as a dynamicalsystem (and is, when analysing the stability of control laws). Kinematics is thusirrelevant so long as the robot’s state space trajectory converges on the correcttarget.

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 424–428, 2016.DOI: 10.1007/978-3-319-42417-0 41

Anticipating Synchronisation for Robot Control 425

Dispensing with kinematics means that the future course of the arm’s motioncannot be computationally modelled. Since a deviation from a desired path mustthus be sensed before it can be corrected, any delay in the joints’ reactionswill create delay in the control loop that can cause instability. Such a delay isinevitable due to the fact that the joints must move links with mass, and thusinertia, which prevents the reaction from being instantaneous.

However, because of the arm’s control law being treated as a dynamicalprocess, there is scope to utilise the phenomenon of anticipating synchronisation(AS) to counteract this instability.

2 The Problem of Delay

In order to demonstrate the tangible benefits of an AS coupling in motor con-trol, we use visual control based on a linear transformation we term the jointrelationship matrix (JRM) [3] (In preparation):

τ = JRM(Kp(xd − x) − Kvx) (1)

τ is the vector of joint torques, x is the work space position of the end effector.xd is the target in the work space. Kp is a proportional gain matrix for the workspace error, while Kv is a velocity gain matrix. JRM is a static transformationapplied to the work space feedback, defining each joint’s reaction each joint’sresponse to the feedback terms.

The JRM is used to associate each joint in the simulated robot arm witha direction vector in the work space. Although this is not an accurate kine-matic transformation, the resulting control law can correct any deviations froma straight motion as they occur. Since a deviation must be detected before itcan be corrected, the correction always lags behind the error, causing unstable‘bumps’ in the end effector’s trajectory.

For the purposes of this paper JRM is fixed at the value(

0 1−1 0

), which

relates the torque at the first and second joints to the vertical and horizontalwork space error, respectively.

3 Anticipating Synchronisation

Instead of using explicit forward modelling, which would greatly increase thecomplexity of the control, we seek to introduce an element of anticipating syn-chronisation into Eq. 1.

The AS paradigm we base this paper on is that of anticipation via delayedself-feedback as described by Voss [2]: a ‘slave’ system anticipates the evolutionof a ‘master’ through the addition of the coupling shown in Eq. 3.

x(t) = f(x(t)) (2)y(t) = f(y(t)) + K[x(t) − y(t− tdelay)] (3)

So long as y(t) lags x(t+tdelay) the coupling drives the slave to evolve at a fasterrate until it is in advance of the master. When y(t) is equal to x(t+ tdelay), thecoupling disappears and the two systems become independent.

426 H. Eberle et al.

Fig. 1. Comparison of anticipation periods where tdelay is varied and k = 150. Kp =( 100 0

0 100 ) and Kv = ( 75 00 75 ). Master’s angular velocity is multiplied by −1.

4 Applying Anticipating Synchronisation

In order to demonstrate the effect of adding an AS coupling to Eq. 1, the JRM isfixed at the value

(0 1−1 0

). This allows the arm’s end effector perform a horizontal

reaching movement towards a target in the work space, but does not producestraight trajectories as the two joints’ reactions to each others’ movement isdelayed.

To reduce this effect, we designated the second joint the ‘slave’ and addedan additional delay coupling term to Eq. 1. By causing the slave to anticipatethe negative of the master’s angular velocity, we aim to cause the end effector to

Fig. 2. Comparison of paths taken by the end effector taken between (1, 0) and (7, 0)in the non-coupled (controlled by Eq. 1) and coupled (controlled by Eq. 4 with k = 250and tdelay = 0.2 s) cases. Kp and Kv are kept constant at ( 100 0

0 100 ) and ( 150 00 150 ),

respectively.

Anticipating Synchronisation for Robot Control 427

trace a straight line. This results in a modified control law with the below form:

τ = JRM(Kp(xd − x) − Kvx) − ( 0k[x2(t)−x1(t−tdelay)]

)(4)

where k is the coupling strength constant and tdelay is the coupling delay. Thiscoupling creates a stable, anticipatory phase relationship between the angularvelocities of the two joints, as measured by the time difference between thenearest peaks in the two joints’ angular velocity time series (Fig. 1).

The addition of this coupling can also be used to significantly increase thestraightness of the path taken by the end effector. As seen in Fig. 2, the master-slave coupling effectively counteracts the deviations caused by inertial delay,leading to a notably straighter path.

Similarly, the modified control law makes the arm much more robust toperturbation. As seen in Fig. 3, the deviation caused by a torque pulse appliedto the master joint is quickly corrected, while the unmodified control law tracesan unstable spiral when subject to the same disturbance.

Fig. 3. Comparison of paths taken by the end effector taken between (1, 0) and (7, 0)in the non-coupled (controlled by Eq. 1) and coupled (controlled by Eq. 4 with k = 250and tdelay = 0.2 s) cases. A torque pulse is applied to the master joint between 0.5 and1 s. Kp and Kv are kept constant at ( 100 0

0 100 ) and ( 150 00 150 ), respectively.

5 Conclusions

Although delay would appear to be an unavoidable consequence of controllinga robotic arm without explicit kinematics, the phenomenon of AS allows us torun counter to this normal temporal relationship. This opens the possibility ofeffective motor control without the need for computational modelling. We believethis is a solid 1st step in showing that AS can perform an integral role in taskssuch as reaching.

428 H. Eberle et al.

References

1. Stepp, N., Turvey, M.T.: On strong anticipation. Cogn. Syst. Res. 11(2), 148–164(2010)

2. Voss, H.U.: Anticipating chaotic synchronization. Phys. Rev. E 61(5), 5115–5119(2000)

3. Eberle, Y., Nasuto, S.J., Hayashi, Y.: Integration of visual and joint information toenable linear reaching motions (2016, forthcoming)

MantisBot: The Implementationof a Photonic Vision System

Andrew P. Getsy(B), Nicholas S. Szczecinski, and Roger D. Quinn

Case Western Reserve University, Euclid Ave., 10900, Cleveland, OH 44106, USAapg26@case.edu

Abstract. This paper presents the design of a vision system for Man-tisBot, a robot designed to model a male Tenodera sinensis. The visionsystem mounted on the head of the robot consists of five oriented voltaicpanels that distinguish the location of the most dominant light sourcein reference to the head’s location. The purpose of this vision system isto provide the neural control system of the robot with descending com-mands, allowing for the investigation of transitions between targetedmotions. In order to mimic the vision system of the insect, the voltaicpanels were positioned in a particular orientation that when combinedwith a designed electric circuit yields an output that can be used bythe neural controller to control the robot’s motion. This paper summa-rizes the design of the vision system and its outputs. It also presentscalibration data revealing the system’s ability to encode a light source’selevation and azimuth angle as well as its distance.

Keywords: Descending commands · Robot · Mantis

1 Introduction

Praying mantises are ambush predators that primarily rely on vision to orienttheir bodies toward prey before capture [3]. While large amounts of data havebeen collected regarding the kinematics of their motion prior to strike [2,6], theneural systems that control this motion are largely unknown. Our group hasconstructed hypothetical models of visually guided control of the legs [5], butsimulations have the disadvantage of taking a long time to calculate, and arenever a complete picture of the real world. Therefore, we constructed the robotMantisBot, which until now had no head or head sensors [4].

In this paper we present the design of a simple head sensor for MantisBot,which provides the descending commands needed to explore how these mightmodulate local control systems in the thoracic ganglia that orient the robottoward prey. The sensor is composed of five photovoltaic cells arranged in aconvex pattern on a head made of balsa wood. The cell’s output are measured,amplified, and compared to calculate the centroid and distance of a known lightsource in the robot’s field of view. Here we describe the design and presentcalibration data.c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 429–435, 2016.DOI: 10.1007/978-3-319-42417-0 42

430 A.P. Getsy et al.

2 Methods and Results

2.1 Mechanical Design

The design for the head, that holds the vision system, was constructed from6.35 mm (0.25 in.) thick Balsa wood. Wood was chosen as the structural materialdue to its low density. The strength of this material is sufficient due to the lackof external and inertial loads present in the structure. The components of thisstructure were cut with an Epilog Legend 36EXT laser cutter, at the Larry Searsand Sally Zlotnick Sears think[box]. The general dimensions of the structure werechosen in order to fit on the MantisBot robot which was built with a 13.3:1 scaleto the insect.

Fig. 1. Left: constructed head design with vision system positioned within the firsttesting fixture made from a cardboard box. Right: the second test fixture made of1/16 in. steel rod.

Experiments were performed to calibrate the sensor. In all experiments, a50 lumen light was shone onto the head from different known locations, andthe locations were correlated with sensor readings. Two test fixtures were usedfor calibration. The first test fixture was a 45 cm by 45 cm by 30 cm cardboardbox with twenty-five holes at measured locations, shown in Fig. 1. Testing wasconducted by covering all holes except one and shining light, directed at thecenter panel, for a duration of five seconds through the only opening. The holeswere utilized in a known order, and the location of each in all three dimensionswas measured and recorded. Shadows and reflections within the box led to thedevelopment of a second test fixture, a cage constructed from 1.59 mm (1/16 in.)steel wire brazed together to form a 23 cm dome, shown in Fig. 1. This fixtureallows for the vision system to be positioned at the center of the dome while, ina dark room, a light source is held at the measured locations. This test methodeliminated the creation of unwanted reflections from the light source and externalinterferences.

MantisBot: The Implementation of a Photonic Vision System 431

Fig. 2. Average solar panel readings with approximation function. Drop off angle (*)from linear fits of the segmented approximation function.

The collected readings were then analyzed in MATLAB (Mathworks, Natick,MA). Periods of zero volt readings from the center sensor were used to demarcatereadings, and then the mean sensor values for each reading were recorded. Leastsquares curve fitting was conducted using a genetic algorithm and then a Newtonoptimizer. Initial test results showed that the head could only accurately anduniquely predict the position of the light source over a narrow range of angularpositions due to the changing flux on each sensor (Fig. 3). Therefore, individualpanels were tested by presenting a light to the panel at a variety of orientations

Fig. 3. Collected data using the cardboard box testing fixture. Left: graph showingcollected reading versus light source azimuth angle. Right: graph showing collectedreading versus light source elevation angle.

432 A.P. Getsy et al.

Fig. 4. Collected data using the steel dome testing fixture. Left: graph showing col-lected reading versus light source azimuth angle. Right: graph showing collected readingversus light source elevation angle.

ranging from orthogonal to parallel with respect to both the short and long axisof the panel while keeping the distance from the panel constant.

The collected data revealed that the solar panel reacted in a similar fash-ion with regard to the short and long axis. This allowed the average of the twotrails to represent how the light source angle effects the solar panel’s output. Anapproximation function was constructed to fit the average of the solar panel read-ings (Fig. 2). This function was then used to find the angle at which the panel’soutput began to drop off at a faster rate. Segmenting the function and performinglinear fits on the two portions led to an intersection at an angle of forty degrees.This was the chosen angle to offset the panels in both the azimuth and eleva-tion directions in order to achieve a field of view of one hundred eighty degrees(Fig. 5). Making this change produced a linear, monotonic mapping between theangular position of the light source and the sensor readings, as shown in Fig. 4.This was deemed sufficient.

2.2 Electrical Design

The designed photonic vision system uses voltaic panels to distinguish the loca-tion of the most dominant light source in reference to its own location. Thecircuit consists of five TLV274 operational amplifiers. These amplifiers operateon a zero to five volt power rail, so they were chosen due to the Arbotix-MRobocontroller’s zero to five volt analog inputs. Another important feature ofthis amplifier is the rail-to-rail output swing feature, which allows the use ofthe complete span between the supply voltages [1]. This feature also increasesthe signal-to-noise ratio and dynamic range, which allows for a larger range ofcleaner signals.

The circuit is connected to five Sundance Solar calibrated solar cells orientedwith two on the left side of the head, two on the right side of the head, and onein the center, as seen in Fig. 5. These solar panels receive incoming light while

MantisBot: The Implementation of a Photonic Vision System 433

Fig. 5. 3D rendering of MantisBot’s updated vision system with voltaic panel orienta-tion angles.

outputting a maximum of 0.5 V at 200 mA based on the intensity of the lightsource. The objective of the circuit is to gather these signals and produce twooutputs that reveal which of the panels are receiving the most light; the two up-most, down-most, left-most, or right-most panels, and one output revealing thegeneral intensity of the light source. These outputs inform the control systemwhen a change in direction needs to occur in order to keep the light sourcecentered. The vision system also contains an isolated digital LED circuit (Fig. 6)which provides visual feedback to the operator showing which solar panels arereceiving the most light.

The production of these visual signals was accomplished by the combinationof three steps within the electronic circuit shown in Fig. 6. First, the signals of thefour solar panels, not including the center panel, are each sent through a unitygain buffer to prevent the solar cells from loading the rest of the circuit (Fig. 6a).The signals are then added in four individual sums; the two left panels, two rightpanels, two top panels, and two bottom panels. These sums are performed bythe operational amplifier summing circuits represented in Fig. 6b. These combinethe voltages of the input signals (Fig. 6b Point I), which creates the average ofthe two signals, and then amplify that combined voltage with a gain of positiveeight (Fig. 6b Point II). This is equivalent to the addition of the two signalswith a gain of four. These four outputs are passed through individual unity gainbuffers to ensure signal isolation (Fig. 6c).

The second step involves a unity gain summing amplifier which biases thesesignals upwards to ensure the output is between zero and five Volts. During thisstep a voltage divider is used to supply two and a half volts to the right andupward sum signals, shown in Fig. 6d Point III. This stage also introduces thecenter panel, whose signal is amplified by a factor of eleven (Fig. 6d Point IV).The increased gain for the center panel is necessary to distinguish light sources atfarther distances once it was centered in the field of view. The gain also enhances

434 A.P. Getsy et al.

Fig. 6. Designed vision system circuit with labeled signal inputs UL (Up Left), UR(Up Right), DL (Down Left), DR (Down Right); LED circuit; and numbered TLV274operational amplifiers.

the sensitivity of the signal making slight position changes of the light sourcemore distinguished.

The final stage computes the centroid of the light source, and buffers theoutput. In this step the summed left signal is subtracted from the summedright signal, which has been offset by two and a half volts, and the summeddown signal is subtracted from the offset summed upward signal (Fig. 6e). Thesesignal manipulations result in two output analog signals ranging from 0.5 V to4.5 V. Even though the TLV274 operational amplifier supports rail to rail outputsignals, a half volt safety region was incorporated for conservative operations.

These output signals are supplied to the Arbotix-M Robocontroller via threeanalog signal pins. The analog inputs are run through a ten-bit analog-to-digitalconverter, therefore the input signal voltage is divided into a range from zeroto one thousand twenty three, which corresponds to the zero to five volt powersupply. Since the supplied power is only positive voltage, a median voltage of

MantisBot: The Implementation of a Photonic Vision System 435

zero to distinguish left from right and up from down would not be possible; itwas for this reason that two and a half volts were added to the summed right andupward signals. The addition of this magnitude resulted in the median voltagebeing two and half volts, or five hundred twelve in the digital system. What thismeans is that when the centroid of the light source is left of the center of view,the first output signal will be some number larger than five hundred twelve, andwhen right the value will be below. When the light source is below the centerof view, the output will be larger than five hundred twelve, and smaller whenabove the head. The center panel outputs a signal that will range from zeroto one thousand twenty four. This signal decreases as the light source distanceincreases. The solar panel is also sensitive to the angle at which the light is held.The signal will be strongest when the light source is directly in front of the panel.

3 Conclusion

The vision system for MantisBot is capable of providing high level feedback tothe robot’s control system from an exterior light source. This feedback is in theform of serial outputs that reveal the location of a luminous target. This willallow the further investigation of how descending commands communicated tothe local neural system can initiate reflexes that change the orientation of ananimal or robot.

References

1. Family of 550-uA/Ch 3-MHz Rail-to-Rail Output OperationalAmplifiers (Rev. D)(2004). http://www.ti.com/lit/ds/symlink/tlv274.pdf

2. Cleal, K.S., Prete, F.R.: The predatory strike of free ranging praying mantises,sphodromantis lineola (Burmeister). II: strikes in the horizontal plane. Brain Behav.Evol. 48, 191–204 (1996)

3. Mittelstaedt, H.: Prey capture in mantids. In: Recent Advances in InvertebratePhysiology, pp. 51–72 (1957)

4. Szczecinski, N.S., Chrzanowski, D.M., Cofer, D.W., Terrasi, A.S., Moore, D.R.,Martin, J.P., Ritzmann, R.E., Quinn, R.D.: Introducing MantisBot: hexapod robotcontrolled by a high-fidelity, real-time neural simulation. In: IEEE InternationalConference on Intelligent Robots and Systems, Hamburg, DE, pp. 3875–3881 (2015)

5. Szczecinski, N.S., Martin, J.P., Bertsch, D.J., Ritzmann, R.E., Quinn, R.D.:Neuromechanical model of praying mantis explores the role of descending com-mands in pre-strike pivots. Bioinspiration and Biomimetics 10(6), 1–18 (2015).http://dx.doi.org/10.1088/1748-3190/10/6/065005

6. Yamawaki, Y., Uno, K., Ikeda, R., Toh, Y.: Coordinated movements of the headand body during orienting behaviour in the praying mantis Tenodera aridifolia. J.Insect Physiol. 57(7), 1010–1016 (2011)

Force Sensing with a Biomimetic Fingertip

Maria Elena Giannaccini1,2(B), Stuart Whyle1,2, and Nathan F. Lepora1,2

1 Department of Engineering Mathematics, University of Bristol, Bristol, UK2 Bristol Robotics Laboratory, University of Bristol, Bristol, UK

{maria.elena.giannaccini,n.lepora}@bristol.ac.uk

Abstract. Advanced tactile capabilities could help new generations ofrobots work co-operatively with people in a wider sphere than thesedevices have hitherto experienced. Robots could perform autonomousmanipulation tasks and exploration of their environment. These appli-cations require a thorough characterisation of the force measurementcapabilities of tactile sensors. For this reason, this work focuses on thecharacterisation of the force envelope of the biomimetic, low-cost androbust TacTip sensor. Comparison with a traditional load cell shows thatwhen identifying low forces and changes in position the TacTip provessignificantly less noisy.

Keywords: Tactile sensing · Bioinspiration · Force sensing

1 Introduction

The sense of touch is one of the key human modalities. It is of vital importancein everyday tasks like object manipulation, grasp control and exploration. Inorder to develop an effective robotic sense of touch, inspiration is sought fromthe human tactile sensory system. This is chosen as a source of inspiration givenits ability to process and transmit detailed information [1]. Here we consider theTacTip sensor, a biomimetic, dome-shaped device based on the human fingertip.Specifically, TacTip relies on the movement of internal pins inspired by humanfingertip papillae: internal projections of the dermis that contain specializednerve endings and are arranged in ridgelike lines.

Despite advancements in methods for tactile perception with the TacTipin localization [2–4], in shape perception [5,6] and texture discrimination [7],the characterisation of the force response of the sensor has not been tackledbeyond an initial exploration [8]. In this work, we present a characterisation ofthe force sensing capability of the TacTip sensor, aimed at determining the forceenvelope the sensor can measure. This is a necessary step in order to identifythe usability of TacTip in applications such as dexterous robot manipulationand high performance industrial quality control.

2 Method

Chorley et al. [8] were inspired to consider the behaviour of human glabrousskin. They built on previous research showing that the Meissner’s Corpusclesc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 436–440, 2016.DOI: 10.1007/978-3-319-42417-0 43

Force Sensing with a Biomimetic Fingertip 437

work in tandem with the dermal papillae, see Fig. 1A, to provide edge encodingof a touched surface. When the human finger makes contact with an object orsurface, deformation results in the epidermal layers of the skin and the changeis detected and reported by the mechanoreceptors. The human sensory responseis one the TacTip, see Fig. 1B, device seeks to replicate. In the design of TacTipthe epidermal layers are represented by internal papillae pins on the inside ofits skin. These move when the device contacts an object and the resulting pindeformation is optically tracked by a camera.

Fig. 1. A: Mechanoreceptors present on the top of the dermal papillae in the glabrousskin (cross section), B: The TacTip sensor with its pins, protrusions mechanicallycoupled with the outer skin surface, similarly to the dermal papillae.

For the force characterisation, a Tedea-Huntleigh, Model No. 1022 3 Kg beamload cell connected to a Novatech SY011 amplifier is utilised. In the experiment,the TacTip is pressed vertically against the load cell that outputs the magnitudeof the pressing force. The calibration of the load cell is performed by placingweights on the load cell and then recording its voltage output.

In preparation for the experiments, the maximum pressing magnitude is iden-tified to avoid damaging the sensor. It is found that beyond 7 mm the TacTipbegins to deform irreversibly with its filling, a clear silicone elastomer gel, begin-ning to separate from the outer skin. This permanently degrades the TacTipperformance. Hence, the maximum press magnitude is set, conservatively, at5 mm from initial contact. At 5 mm the maximum output of the load cell isapproximately 700 mV. A variety of weights are then used to give the range ofoutputs from 0 to 900 mV, which includes the 700 mV value we need.

After calibrating the beam load cell, the TacTip is fixed to the end-effectorof an ABB IRB 120 robot arm and pressed onto the beam load cell for a range of0.5–5 mm with 0.5 mm increments. In the interest of thoroughness it is importantto clarify the effect of the load cell width on the calibration and experimentmeasurements. Due to its dome shape, the diameter of the TacTip increasesgradually to its maximum (36 mm) while the width of the load cell is 25.4 mm.This implies that, due to the 25.4 mm limit, for final presses, the contact areamagnitude is smaller than what it would be were the load cell as wide as theTacTip maximum diameter. Given that in the experiment the press displacement

438 M.E. Giannaccini et al.

increments are kept constant (0.5 mm) and it takes less force to obtain the samepenetration displacement given a smaller area of contact then the accuracy ofthe force measurements for the final presses could be affected. Nevertheless, it isimportant to keep in mind that given the conservative maximum press magnitudeselected the effects of this phenomenon are very limited.

3 Results and Discussion

The results of the press experiments can be seen in Figs. 2 and 3. The data showthat for each subsequent press increment, the voltage difference is increasing.The average voltage is derived by selecting from immediately behind the initialpeak across the reading until the next press, utilising approximately 150 datapoints. These voltage values are then used to calculate the weight equivalentusing the calibration relationship.

Fig. 2. The voltage output of the beam load cell generated from the TacTip pressingon the load beam, with 0.5 mm increments across a 0.5 to 5 mm range.

For the initial presses, Fig. 2, the voltage output from the load beam cell issimilar to the noise level of the device. For the first press, the voltage change isbarely discernible; whereas for the TacTip the step change at the initial pressesis clearly evident and remains so throughout further presses, see Fig. 3. Thisshows that when identifying low forces and small changes in position the TacTipproves significantly less susceptible to noise than the load cell.

Inspection of the graph in Fig. 2 shows that, immediately after the press, thevoltage peaks then subsides with a number of oscillations. The magnitude ofthe oscillation increases as the amount of presses increases. Silicone elastomerslike the TacTip filling material do not have linear stress-strain behaviour. Theinitial peak in the load cell output is likely due to the viscoelastic response of thesilicone elastomer. This would be dependent on the strain rate of the incrementalend-effector loading.

Force Sensing with a Biomimetic Fingertip 439

Fig. 3. Tactile data from TacTip, showing pin deflection against each press. Each pressis a 0.5 mm increment across a range of 0.5–5 mm.

Fig. 4. Mapping the load beam outputs relative to the TacTip position. The dottedline shows the relationship to the function y = 0.2147x0.5236. A common effect ofviscoelasticity is strain stiffening at large stretches. Hence the fitted function may notbe valid for larger deformations. At any rate, TacTip deformations are limited to avoidsensor damage.

Looking at Fig. 4 it can be seen that as the press magnitude increases, therelative voltage step and therefore the force, also increases. This covers a rangeof weights from 0 to 450 g which is equivalent to 0 to 4.413 N.

These results show that the TacTip is able to reliably detect changes inposition of 0.5 mm. The experiments conducted yield a characterisation of therelationship between changes in position and force for the TacTip sensor alongits full working range. In addition, the TacTip proves significantly less noisy thana load cell at identifying low forces and changes in position.

440 M.E. Giannaccini et al.

Acknowledgments. NL and MEG were supported by the EP/M02993X/1 EPSRCgrant on Tactile Superresolution Sensing.

References

1. Dahiya, R.S., Valle, M.: Tactile sensing for robotic applications. In: Rocha,J.G., Lanceros-Mendez, J.G. (eds.) Sensors: Focus on Tactile Force andStress Sensors, pp. 289–304 (2008). http://www.intechopen.com/books/sensors-focus-on-tactile-force-and-stress-sensors/tactile sensing for robotic applications

2. Lepora, N.F., Martinez-Hernandez, U., Evans, M., Natale, L., Metta, G., Prescott,T.J.: Tactile superresolution and biomimetic hyperacuity. Trans. Robot. 31(3), 605–618 (2015)

3. Lepora, N.F., Ward-Cherrier, B.: Superresolution with an optical tactile sensor. In:Intelligent Robots and Systems (IROS), pp. 2686–2691, September 2015

4. Cramphorn, L., Ward-Cherrier, B., Lepora, N.: Tactile manipulation with bio-mimetic active touch. In: International Conference on Robotics and Automation(ICRA) (2016)

5. Lepora, L.: Biomimetic active touch with tactile fingertips and whiskers. IEEETrans. Haptics (2016)

6. Lepora, N., Ward-Cherrier, B.: Tactile quality control with biomimetic active touch.Robot. Autom. Lett. (2016)

7. Winstone, B., Griffiths, G., Pipe, T., Melhuish, C., Rossiter, J.: TACTIP - tactilefingertip device, texture analysis through optical tracking of skin features. In: Lep-ora, N.F., Mura, A., Krapp, H.G., Verschure, P.F.M.J., Prescott, T.J. (eds.) LivingMachines 2013. LNCS, vol. 8064, pp. 323–334. Springer, Heidelberg (2013)

8. Chorley, C., Melhuish, C., Pipe, T., Rossiter, J.: Development of a tactile sen-sor based on biologically inspired edge encoding. In: International Conference onAdvanced Robotics (ICAR), pp. 1–6 (2009)

Understanding Interlimb CoordinationMechanism of Hexapod Locomotion via

“TEGOTAE”-Based Control

Masashi Goda1(B), Sakiko Miyazawa1, Susumu Itayama1, Dai Owaki1,Takeshi Kano1, and Akio Ishiguro1,2

1 Research Institute of Electrical Communication, Tohoku University,2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan

{m.goda,saki,s.itayama,owaki,tkano,ishiguro}@riec.tohoku.ac.jp2 Japan Science and Technology Agency, CREST, 4-1-8 Honcho,

Kawaguchi, Saitama 332-0012, Japanhttp://www.cmplx.riec.tohoku.ac.jp/

Abstract. Insects exhibit surprisingly adaptive and versatile locomo-tion despite their limited computational resources. Such locomotor pat-terns are generated via coordination between leg movements, i.e., aninterlimb coordination mechanism. The clarification of this mechanismwill lead us to elucidate the fundamental control principle of animal loco-motion as well as to realize truly adaptive legged robots that could not bedeveloped solely by conventional control theory. In this study, we tried tomodel the interlimb coordination mechanism underlying hexapod loco-motion on the basis of a concept called “TEGOTAE,” a Japanese con-cept describing how well a perceived reaction matches an expectation.Preliminary experimental results show that our proposed TEGOTAE-based control scheme allows us to systematically design a decentralizedinterlimb coordination mechanism that can well-reproduce insects’ gaitpatterns.

Keywords: Hexapod locomotion · Interlimb coordination · Local forcefeedback · Central pattern generator (CPG) · TEGOTAE

1 Introduction

Insects exhibit versatile gait patterns in response to their locomotion speed andenvironmental conditions [1,2]. These locomotor patterns are generated via theinterlimb coordination mechanism. Although knowledge of these patterns at thebehavioral and functional levels has been reported [3], the essential mechanismfor interlimb coordination is still unknown. Thus, further clarification of thismechanism is required in order to design an adaptable and multifunctional leggedrobot as well as to understand the fundamental mechanism responsible for theremarkable abilities of legged animals.

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 441–448, 2016.DOI: 10.1007/978-3-319-42417-0 44

442 M. Goda et al.

To tackle this issue, we introduce a unique approach in this paper. We tryto model the interlimb coordination mechanism underlying hexapod locomo-tion from the viewpoint of TEGOTAE, describing how well a perceived reactionmatches an expectation. We introduce a TEGOTAE function, which is a functionthat quantitatively measures TEGOTAE, from which we can design a decen-tralized interlimb coordination mechanism in a systematic manner. Preliminaryexperimental results using an insect-like hexapod robot suggest that the designedinterlimb coordination mechanism based on TEGOTAE well-reproduces the gaitpatterns of insects.

2 Model

Biological findings support that thoracic ganglia play a crucial role in the inter-limb coordination in insect locomotion [4,5]. This strongly suggests that a decen-tralized control scheme, in which the coordination of simple individual compo-nents yields the nontrivial macroscopic behavior or functionalities, could be thekey to understanding the interlimb coordination mechanism of insects. Thus far,various studies have been devoted to elucidating this mechanism [2,6] as wellas to designing hexapod robots [7,8]. Here, we employ a “central pattern gen-erator” (CPG)-based approach. For simplicity, we implement a phase oscillatorinto each leg as a decentralized controller.

The time evolution of the oscillator phase is described as follows:

φi = ω +∂Ti

∂φi, (1)

where ω is the intrinsic angular velocity, φi is the phase of oscillator implementedinto ith leg, and Ti is a TEGOTAE function. The ith leg is actively controlledaccording to φi such that the ith leg is in the swing phase when 0 ≤ φi < π,i.e., sin φi > 0, and in the stance phase when π ≤ φi < 2π, i.e., sinφi < 0.Ti represents the key concept of our approach. Ti quantifies TEGOTAE, i.e.,to what extent a perceived reaction matches an expectation, in real time. Forconvenience, we define Ti such that its value increases when good TEGOTAEis received at the ith leg. Thus, Eq. (1) means that φi is modified so that goodTEGOTAE increases.

Now, the question is how to define Ti to well-reproduce the interlimb coor-dination observed in insect locomotion. In this study, we define Ti as follows:

Ti = σ1Ti,1 + σ2Ti,2, (2)

Ti,1 = Ni(− sin φi), (3)

Ti,2 =

⎛⎝ 1

nL

nL∑j∈L(i)

Nj

⎞⎠ sin φi. (4)

As Eq.(2) indicates, Ti consists of two TEGOTAE functions, Ti,1 and Ti,2,both of which are linearly coupled via the positive constants σ1 and σ2. Ni is the

Understanding Interlimb Coordination Mechanism of Hexapod Locomotion 443

Fig. 1. (a) Developed hexapod robot. (b) The definition of L(i), describing a set con-sisting of the legs neighboring the ith leg.

Fig. 2. Definition of the “TEGOTAE” functions.

ground reaction force acting on the ith leg. L(i) denotes a set consisting of thelegs neighboring the ith leg, and nL is the number of elements of L(i) (Fig. 1(b)).In what follows, we explain how we have designed these two TEGOTAE functionsin detail.

Ti,1 quantifies TEGOTAE on the basis of the information only locally avail-able at the corresponding leg; when the local controller intends to be in the stanceleg (− sin φi > 0), and results in receiving a ground reaction force (Ni > 0) (Fig. 2top), Ti,1 evaluates this situation as a good TEGOTAE, and returns a positivevalue. Note that the TEGOTAE function is described as the product of “what alocal controller wants to do, i.e., − sin φi,” and “its resulting reaction, i.e., Ni.”

On the other hand, Ti,2 quantifies TEGOTAE on the basis of the relation-ship between the movements of the corresponding leg and its neighboring legs;when the local controller intends to be in the swing phase (sinφi > 0) and itsneighboring legs support the body well at that time ( 1

nL

∑nL

j∈L(i) Nj > 0) (Fig. 2

444 M. Goda et al.

Fig. 3. Gait diagrams for (a) ω = 6.0 and (b) ω = 7.5. The colored regions in theupper graphs in (a) and (b) represent the stance phase, where sin φi < 0. The differencebetween these results and biological data [1] was the overlapping swing movements inthe tetrapod gait. (Color figure online)

Understanding Interlimb Coordination Mechanism of Hexapod Locomotion 445

Fig. 4. Top: Octopod robot model. Bottom: The definition of L(i), describing a setconsisting of the legs neighboring the ith leg in the octopod model

bottom), Ti,2 evaluates that the corresponding leg well-establishes a relationshipwith its neighboring legs and returns a positive value.

By substituting Eqs. (2), (3) and (4) into Eq. (1), we obtain our CPG modelas follows:

φi = ω − σ1Ni cos φi + σ2

⎛⎝ 1

nL

nL∑j∈L(i)

Nj

⎞⎠ cos φi. (5)

3 Preliminary Experimental Results

To verify the proposed control scheme in the real world, we developed a hexapodrobot, as shown in Fig. 1(a). Each leg has two servo motors (Futaba, RS303MR)to generate the swing and stance phase motions according to the correspondingoscillator phase. We conducted experiments with the use of the parameter valuesω = 6.0 and 7.5. Here, the initial phases of all oscillators were φi = 3π/2, andthe parameters for the feedback gains were σ1 = 25 and σ2 = 11.

Figure 3 shows the gait diagrams for (a) ω = 6.0 and (b) ω = 7.5. In thesegraphs, the colored regions represent the stance phase, where sinφi < 0. Thegait pattern observed in Fig. 3(a) corresponds to a tetrapod gait, where the feettouch the ground in the order of the hind, middle, and fore legs, whereas the gait

446 M. Goda et al.

Fig. 5. Gait diagrams from the simulation (top) and in animal locomotion (bottom).The definitions of L1, L2, L3, L4, R1, R2, R3, and R4 are indicated at the bottom ofeach subfigure. The typical gait patterns for a (a) spider and (b) scorpion were drawnby authors after [9,10], respectively. (Color figure online)

Understanding Interlimb Coordination Mechanism of Hexapod Locomotion 447

pattern in Fig. 3(b) corresponds to a tripod gait, where the (L1, R2, L3) and(R1, L2, R3) feet alternately touch the ground in anti-phase. The results suggestthat our model explains part of the observations of the gait patterns in insects.The difference between these results and biological data [1] is the overlappingswing movements in the tetrapod gait. Furthermore, we tested the effect of thevariation in the initial oscillator phases on the gait patterns. As a result, weconfirmed that more than 50 % of the initial conditions converged to the typicalgait patterns in insects.

4 Applicability to Octopod Locomotion

Our long-term goal is to develop a systematic design scheme of interlimb coordi-nation for multilegged robots by elucidating the interlimb coordination mecha-nism of various legged animals. No past studies tried to elucidate the mechanismunderlying legged locomotion from a unified viewpoint: they focused on a specificlocomotion generated via the particular number of legs. Toward the achievementof our goal, we investigate the applicability of the TEGOTAE-based controlscheme to octopod locomotion, which is the locomotion observed in spiders orscorpions.

To this end, we modeled a simple octopod robot, as shown in Fig. 4. We con-ducted dynamical simulations using Open Dynamics Engine (ODE). To verifythe applicability of the proposed control scheme, we implemented the decentral-ized controller designed in Sect. 2 in the octopod robot model. We comparedthe obtained gait patterns by changing the parameters of the feedback gains σ1

and σ2 in Eq. (5). Here, the initial phases of all oscillators were φi = 0, and theparameter of the intrinsic angular velocity was ω = 2π.

Figure 5 shows the simulation results for octopod locomotion. The uppergraph in Fig. 5(a) shows the gait diagram in the case σ1 = 3 < σ2 = 80. Theobtained gait pattern corresponds to a typical gait pattern in a spider [9], asshown in the lower graph in Fig. 5(a). On the other hand, the upper graphin Fig. 5(b) shows the gait diagram in the case σ1 = 217 > σ2 = 70. Thegait pattern corresponds to a typical gait pattern in scorpions [10], as shownin the lower graph in Fig. 5(b). We found that the spider gait was observedwhen Ti.2 was dominant over Ti,1, whereas the scorpion gait was observed whenTi.1 was dominant over Ti,2. Surprisingly, our model also well-reproduced spiderand scorpion locomotion with the same control scheme by only changing themagnitude of the local sensory feedback terms.

5 Conclusion and Future Work

In this study, we modeled the interlimb coordination mechanism underlyinghexapod locomotion on the basis of the concept of TEGOTAE. Using a hexa-pod robot, we demonstrated that our proposed TEGOTAE-based control schemewell-reproduces insects’ gait patterns. Furthermore, we confirmed that our modelalso well-reproduces spider and scorpion locomotion with the same control

448 M. Goda et al.

scheme. The applicability of the design scheme based on the TEGOTAE func-tion for the locomotion of various animals seems to be of great interest and willalso be studied in further investigations.

Acknowledgements. We acknowledge the support of a JSPS KAKENHI Grant-in-Aid for Scientific Research (B) (16H04381).

References

1. Graham, D.: A behavioural analysis of the temporal organisation of walking move-ments in the 1st instar and adult stick insect (Carausius morosus). J. Comp. Pysiol.81, 23–52 (1972)

2. Schilling, M., Hoinville, H., Schmitz, J., Cruse, H.: Walknet, a bio-inspired con-troller for hexapod walking. Biol. Cybern. 107, 397–419 (2013)

3. Cruse, H.: What mechanisms coordinate leg movement in walking arthopods?Trends Neurosci. 13, 15–21 (1990)

4. Jeffrey, D.: Leg coordination in the stick insect carausius morosus: effects of cuttingthoracic connectives. J. Exp. Biol. 145, 103–131 (1989)

5. Bassler, U., Buschges, A.: Pattern generation for stick insect walking movements-multisensory control of a locomotor program. Brain Res. Rev. 27, 65–88 (1998)

6. Kimura, S., Yano, M., Shimizu, H.: A self-organizing model of walking patterns ofinsects. Biol. Cybern. 69, 183–193 (1993)

7. Beer, R.D., Quinn, R., Chiel, H.: Biologically inspired approaches to robotics.Commun. ACM 40, 30–38 (1997)

8. Ambe, Y., Nachstedt, T., Manoonpong, P., Worgotter, F., Aoi, S., Matsuno, F.:Stability analysis of a hexapod robot driven by distributed nonlinear oscillatorswith a phase modulation mechanism. In: IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS 2013), pp. 5087–5092 (2013)

9. Biancardi, C.M., Fabrica, C.G., Polero, P., Loss, J.F., Minetti, A.E.: Biomechanicsof octopedal locomotion: kinematic and kinetic analysis of the spider Grammostolamollicoma. J. Exp. Biol. 214, 3433–3442 (2011)

10. Bowerman, R.F.: The control of walking in the Scorpion I. J. Comp. Physiol. A100, 183–196 (1975)

Decentralized Control Scheme for MyriapodLocomotion That Exploits Local Force Feedback

Takeshi Kano1(B), Kotaro Yasui1, Dai Owaki1, and Akio Ishiguro1,2

1 Research Institute of Electrical Communication, Tohoku University,2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan

{tkano,k.yasui,owaki,ishiguro}@riec.tohoku.ac.jp2 Japan Science and Technology Agency, CREST, 4-1-8 Honcho,

Kawaguchi, Saitama 332-0012, Japanhttp://www.cmplx.riec.tohoku.ac.jp/

Abstract. Legged animals exhibit adaptive and resilient locomotionthrough their inter-limb coordination. Our long-term goal of this studyis to develop a systematic design scheme for legged robots by elucidatingthe inter-limb coordination mechanism of various legged animals from aunified viewpoint. As a preliminary step towards this, we here focus onmillipedes. We performed behavioral experiments on a terrain with gap,and found that legs do not tend to move without the ground contact.Based on this qualitative finding, we proposed a decentralized controlscheme using local force feedback.

Keywords: Millipede · Inter-limb coordination · Local force feedback

1 Introduction

In the animal kingdom, there are animals having various number of legs suchas humans, quadrupeds, insects, and myriapods. What is interesting is thatthey can adapt to changes in the environment as well as changes in their ownmorphologies, e.g. leg amputation, in real time [1]. This adaptive and resilientlocomotion is achieved by coordinating their limbs in a decentralized manner.Clarifying this inter-limb coordination mechanism will help develop legged robotsthat can move in harsh environments such as disaster areas.

Our challenge is to develop a systematic design scheme for legged robotsby elucidating the inter-limb coordination mechanism of various legged animalsfrom a unified viewpoint. Towards this goal, we started our research by focusingon individual animal species, particularly on animals with few number of legs,i.e., two, four, and six legged animals [2–4]. However, animals having many legshave been less studied.

Accordingly, we here focus on millipedes. In fact, millipedes are suitablemodel because they have the largest number of legs on earth [5] and can moveon unstructured terrain by propagating density waves of the leg tips from thetail to head appropriately [6]. In this study, we first performed behavioral exper-iments on a terrain with a gap. Then, based on the qualitative findings fromc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 449–453, 2016.DOI: 10.1007/978-3-319-42417-0 45

450 T. Kano et al.

Fig. 1. Experiment setup.

Fig. 2. Millipede locomotion on a terrain with a gap.

the experiments, we proposed a simple decentralized control scheme using localforce feedback.

2 Behavioral Experiments

We performed behavioral experiments using millipedes (Spirosteptus giganteus).To investigate the effect of the ground contact, we observed locomotion on aterrain with a gap (Fig. 1). The result is shown in Fig. 2. On the ground, theleg tips form density waves that propagate forward, as reported in the previousstudy [6]. However, this behavior changed on the gap: When a leg entered thegap, it exhibited searching movement [7] for a while (Fig. 2A), after which itstopped (Fig. 2B). It started to swing again when its several anterior legs contactthe ground (Fig. 2C). Thus, each leg is likely driven by the ground contact ofitself and its several anterior legs.

Decentralized Control Scheme for Myriapod Locomotion 451

Fig. 3. Schematic for the proposed control scheme.

3 Model

We proposed a decentralized control scheme for millipede locomotion on thebasis of the qualitative findings that the motion of each leg is likely triggeredby the ground contact of itself and its anterior legs. The schematic is shown inFig. 3. Legs are connected via the body trunk. Each leg can move both forward-backward and upward-downward. A phase oscillator is implemented in each leg.The motion of the ith leg on the right- and left-hand side is controlled accordingto the oscillator phase φi,r and φi,l, respectively. Legs tend to be in the swingand stance phase when the oscillator phase is between 0 and π and between πand 2π, respectively.

The time evolution of the oscillator phase is described as follows:

φij = ω − (a − σ1Ni,j) cos φi,j + σ2Ni−1,j cos φi,j , (j = r, l) (1)

where ω is the intrinsic angular velocity, a, σ1, σ2 are positive constants, andNi,j is the ground reaction force acting on the leg.

The first and second terms on the right-hand side of Eq. (1) are based on thefinding that the leg motion stops on the gap in the above behavioral experiment(Fig. 2B). When ω < a and the leg does not contact the ground (Ni,j = 0),ω− (a−σ1Ni,j) cos φi,j = 0 has two solutions, where one is stable and the otheris unstable (Fig. 3). Thus, the leg tends to stay at the position corresponding tothe stable solution. When the leg contacts the ground and Ni,j increases, thestable and unstable solutions vanish, and thus, the leg moves periodically.

The third term on the right-hand side of Eq. (1) is based on the findingthat the leg begins to move when its anterior legs contact the ground (Fig. 2C).

452 T. Kano et al.

Fig. 4. Myriapod-like robot we are now developing.

When the i − 1th leg contacts the ground, the ith oscillator phase converges toπ/2, which makes the leg begin to move. Further, because the ith leg waits forits ground contact until the i − 1th leg lifts off the ground, it is expected thatthe density waves of the leg tips generate in a decentralized manner.

4 Conclusion and Future Work

We focused on millipedes and proposed a decentralized control scheme based onthe results of the behavioral experiments. We are now developing a myriapod-like robot (Fig. 4) to investigate the validity of the proposed control scheme. Theexperimental results will be shown in the presentation.

Acknowledgments. This work was supported by Japan Science and TechnologyAgency, CREST. The authors would like to thank Kazuhiko Sakai of Research Instituteof Electrical Communication of Tohoku University.

References

1. Schilling, M., Hoinville, T., Schmitz, J., Cruse, H.: Walknet, a bio-inspired controllerfor hexapod walking. Biol. Cybern. 107, 397–419 (2013)

2. Owaki, D., Kano, T., Nagasawa, K., Tero, A., Ishiguro, A.: Simple robot suggestsphysical interlimb communication is essential for quadruped walking. J. Roy. Soc.Interface 10, 20120669 (2013)

3. Owaki, D., Ishiguro, A.: CPG-based control of bipedal walking by exploiting plantarsensation. In: Proceedings of the CLAWAR 2014, pp. 335–342 (2014)

Decentralized Control Scheme for Myriapod Locomotion 453

4. Ishiguro, A., Nakamura, K., Kano, T., Owaki, D.: Neural communication vs. physicalcommunication between limbs: which is essential for hexapod walking? In: DynamicWalking 2014, T 406 (2014)

5. Marek, P.E., Bond, J.E.: Rediscovery of the world’s leggiest animal. Nature 441,707 (2006)

6. Kuroda, S., Tanaka, Y., Kunita, I., Ishiguro, A., Kobayashi, R., Nakagaki, T.: Com-mon mechanics of mode switching in locomotion of limbless and legged animals. J.Roy. Soc. Interface 11, 20140205 (2014)

7. Durr, V.: Stereotypic leg searching-movements in the stick insect: kinematic analysis,behavioural context and simulation. J. Exp. Biol. 204, 1589–1604 (2001)

TEGOTAE-Based Control Schemefor Snake-Like Robots That Enables

Scaffold-Based Locomotion

Takeshi Kano(B), Ryo Yoshizawa, and Akio Ishiguro

Research Institute of Electrical Communication, Tohoku University,2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan{tkano,r-yoshi,ishiguro}@riec.tohoku.ac.jp

http://www.cmplx.riec.tohoku.ac.jp/

Abstract. Snakes exhibit “scaffold-based locomotion” wherein theyactively utilize terrain irregularities and move effectively by pushingtheir body against the scaffolds that they encounter. Implementing theunderlying mechanism in snake-like robots will enable them to work wellin unstructured real-world environments. In this study, we proposed adecentralized control scheme for snake-like robots based on TEGOTAE,a Japanese concept describing how well a perceived reaction matchesan expectation, to reproduce scaffold-based locomotion. A preliminaryexperimental result showed that reaction forces from environment areevaluated based on TEGOTAE in real time and those beneficial forpropulsion of the body are selectively exploited.

Keywords: Snake-like robot · TEGOTAE-based control · Scaffold-based locomotion

1 Introduction

Snakes actively utilize terrain irregularities and move effectively by pushing theirbody against the scaffolds that they encounter [1,3]. This behavior, which we call“scaffold-based locomotion,” is achieved by appropriately coordinating large num-ber of bodily degrees of freedom in a decentralized manner. Clarifying the under-lying mechanism will help develop robots that can work well in unstructured real-world environments. Unfortunately, however, previous models for the scaffold-based locomotion could not reproduce the innate behavior of snakes [2–4].

To address this issue, here we propose a decentralized control scheme forscaffold-based locomotion from the viewpoint of TEGOTAE, a Japanese con-cept describing how well a perceived reaction matches an expectation. In thiscontrol mechanism, reaction forces from environment are evaluated based onTEGOTAE in real time and those beneficial for propulsion of the body areselectively exploited. The validity of the proposed control scheme is investigatedvia a preliminary experiment using a snake-like robot developed (Fig. 2).

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 454–458, 2016.DOI: 10.1007/978-3-319-42417-0 46

TEGOTAE-Based Control Scheme for Snake-Like Robots 455

Fig. 1. Scaffold-based locomotion of real snake (Elaphe climacophora).

Fig. 2. Schematic of the proposed model.

2 Model

The schematic of the proposed model is shown in Fig. 1. Several segments areconcatenated one-dimensionally via joints. Frictional anisotropy is implementedin each segment like real snakes, and the friction coefficient along the tangentialdirection is smaller than that along the normal direction. Each joint is controlledaccording to proportional control, and the target and real angle of the ith jointis denoted by φi and φi, respectively. We assume that each segment can detectforces from the environment, and the contact forces detected at the ith segmentfrom the right- and left-hand side are denoted by fr,i and fl,i, respectively.

456 T. Kano et al.

The target joint angle φi is updated each time step as follows:

φ1(t + 1) = hd(t) + σ

i+nf∑j=i−nb

τj(t)(fl,j(t) + fr,j(t)), (1)

φi(t + 1) = φi−1(t) + σ

i+nf∑j=i−nb

τj(t)(fl,j(t) + fr,j(t)), (i > 1) (2)

where t is a time step, hd(t) is the motor command from a operator, σ is a positiveconstant, τj(t) is a torque generated at the jth joint. The first term on the right-hand side of Eq. (2) denotes curvature derivative control [5] wherein torquesproportional to the curvature derivative of the body curve are generated so thatbodily waves propagate from the head to the tail.

Fig. 3. Mechanism of TEGOTAE-based control. (Color figure online)

Fig. 4. Overview of the snake-like robot HAUBOT VI.

The second term on the right-hand side of Eqs. (1) and (2) denotes a sensoryfeedback based on TEGOTAE. Here, TEGOTAE is defined as a quantity thatexpresses “how well a perceived reaction matches an expectation” and described

TEGOTAE-Based Control Scheme for Snake-Like Robots 457

Fig. 5. Preliminary experiment.

as the product of the torque, i.e., intended action, and the contact forces, i.e.,reaction. This feedback works as shown in Fig. 3. When a torque generated tobend the body leftward (red arrow in Fig. 3(a)(1)) resulted in receiving a con-tact force from the right (purple arrow in Fig. 3(a)(1)), the contact force assistspropulsion, and the feedback works to generate further torques (yellow arrow inFig. 3(a)(1)) so that the contact force increases (Fig. 3(a)(2)). On the other hand,when a torque generated to bend the body leftward (red arrow in Fig. 3(b)(1))resulted in receiving a contact force from the left (purple arrow in Fig. 3(b)(1)),the contact force impedes propulsion, and the feedback works to generate fur-ther torques (yellow arrow in Fig. 3(b)(1)) so that the contact force decreases(Fig. 3(b)(2)). Thus, the TEGOTAE-based control scheme enables the robot to“selectively” exploit reaction forces from environment, unlike previous controlschemes based on simple local reflexive mechanisms [2–4].

3 Preliminary Experiment

We developed a snake-like robot HAUBOT VI (Fig. 4) and performed a prelim-inary experiment on a terrain with two pegs as shown in Fig. 5. Then, the robotmoved by selectively exploiting a peg that assists propulsion.

Acknowledgments. This work was supported by NEDO (Core Technology Develop-ment Program for Next Generation Robot). The authors would like to thank DaikiNakashima of Research Institute of Electrical Communication of Tohoku University.

458 T. Kano et al.

References

1. Moon, B.R., Gans, C.: Kinematics, muscular activity and propulsion in gophersnakes. J. Exp. Biol. 201, 2669–2684 (1998)

2. Liljeback, P., Pettersen, K.Y., Stavdahl, Ø., Gravdahl, J.T.: Snake Robots - Mod-elling, Mechatronics, and Control: Advances in Industrial Control. Springer, London(2012)

3. Hirose, S.: Biologically Inspired Robots (Snake-like Locomotor and Manipulator).Oxford University Press, Oxford (1993)

4. Kano, T., Ishiguro, A.: Obstacles are beneficial to me! Scaffold-based locomotion of asnake-like robot using decentralized control. In: IEEE/RSJ International Conferenceon Intelligent Robots and Systems (IROS), pp. 3273–3278 (2013)

5. Date, H., Takita, Y.: Adaptive locomotion of a snake like robot based on curva-ture derivatives. In: IEEE/RSJ International Conference on Intelligent Robots andSystems (IROS), pp. 3554–3559 (2007)

Modelling the Effect of Cognitive Load on EyeSaccades and Reportability: The Validation Gate

Sock C. Low1(B), Joeri B.G. van Wijngaarden1, and Paul F.M.J. Verschure1,2

1 Synthetic Perceptive Emotive and Cognitive Systems Group (SPECS),Universitat Pompeu Fabra, Barcelona, Spain

sockching.low@upf.edu2 Institucio Catalana de Recerca i Estudis Avancats (ICREA), Barcelona, Spain

Abstract. Being able to selectively attend to stimuli is essential forany agent operating in noisy environments. Humans have developed,or are inherently equipped, with mechanisms to do so; these mecha-nisms are a rich source of debate. Here, we contribute to it by buildinga functional model to describe the findings of an existing psychophysi-ological experiment demonstrating how early and late saccades as wellas “conscious report” are affected by varying levels of cognitive load.The model adheres to the established principles of neurophysiology. In atask focused on the monitoring of moving stimuli, where objects usuallymove predictably but randomly deviate from it every 0.2 s, change incognitive load is reflected in the proportion of late saccades and behav-ioural reports in response to the task. It also provides evidence that thephysiological structure of the brain is capable of implementing selectiveattention using a method other than the attentional spotlight.

Keywords: Attention · Model · Validation gate · Attentionalmechanisms · Moving targets

1 Introduction

The human brain receives an overwhelming amount of sensory input and isunable to process all this information at a constant level of awareness. Instead,the brain chunks and selects specific parts of this sensory stream to attend tobased on relevance and saliency. Over the years, many different theories forattention have been proposed, with one general consensus — that attentionis not solely dependent on sensory input, commonly referred to as bottom-upattention (BU), but also relies heavily on contextual information (e.g., memoryor higher-level cognitive areas), known as top-down attention (TD). How thesesystems interact is, however, a matter of contention [1].

Currently, one of the most widely accepted views is that attention uses a so-called attentional spotlight: the saliency of objects from the bottom-up streamis modulated by top-down processes such that it boosts the saliency of goal- ortask-relevant stimuli. This occurs either through an increase in their saliency, a

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 459–466, 2016.DOI: 10.1007/978-3-319-42417-0 47

460 S.C. Low et al.

suppression of irrelevant stimuli, or a combination of both. This notion receivessupport on both a behavioural (e.g., [2]) and neurophysiological (e.g., [3]) level.However, this view is unable to comprehensively explain specific attentionalshortcomings such as the attentional blink.

An alternative theory is that of the validation gate, one which originatedfrom the field of aircraft position tracking. It proposes that rather than sam-pling the input for relevant stimuli and increasing their saliency respectively,higher-level cognitive regions use predictive coding in a feed-forward manner topredict upcoming states of objects. It then exerts an inhibitory influence on thesepredicted states such that only violations of the predictions become salient. Withlow cognitive load, there are sufficient cognitive resources to attend to a largeramount of objects, thus a person is able to observe small violations with highsensitivity. As cognitive load increases, and resources decline, this sensitivity tochange decreases and the accuracy at which disruptions are observed diminish.

Mathews et al. designed a visual detection task to study this interactionbetween BU and TD influences [4]. During this task, ten objects (non-filledwhite circles) were presented on a screen, each moving at a constant speed in arandomly initialised direction and bouncing off the edges of the screen, givingthem a predictable path. For each trial, the participant was allowed to freelylook around the screen as a randomly chosen object is displaced from its lineartrajectory, thus violating the next predicted state. Their task was to report thedisplacement as quickly as possible while simultaneously performing a secondarydistractor task that modulates their cognitive load. Difficulty for this secondtask increased over three conditions: low (no second task), medium (reciting thealphabet normally) and high (reversed alphabet, skipping each second letter).The measures used were gaze movement, split into early (<250 ms) and late(250 ms–800 ms) saccades, and a button press to report a deviant translation.

The authors concluded that cognitive load did not affect the number of fast orslow saccades, but that the number of slow saccades followed by conscious detec-tion did decrease as cognitive load increased. In this short paper, we developeda functional model of the visual circuit that can account for the validation gateand is consistent with existing literature. It aims to describe the findings of [4],specifically the interaction between cognitive load, early and late saccades, andbehavioural reportability.

2 Method

2.1 Design

A series of frames were generated to simulate the visual input that the partici-pants were exposed to. Assuming a frame rate of 50 Hz, 500× 500 pixels frameswere generated to last a duration of 20 s. For simplicity, the validation gate wastaken to be a circular region around the stimuli rather than an oval. As thisimplies an equal probability of an object moving in any direction, instead ofkeeping a constant trajectory as in [4], objects were set to move randomly toone of eight points on a square four pixels in length that was centred on the

Modelling the Effect of Cognitive Load on Eye Saccades and Reportability 461

circle (Fig. 1). In line with the heuristic of [5], eight filled circles with a radiusof three pixels were used as stimuli. Jumps occurred every 10 frames (i.e. 0.2 s),and in order to generate them a random circle would use a square of randomlength between 20–100 pixels, while non-displaced objects continued with theoriginal “rule” of four pixels. Simulations were performed over several runs, eachwith varying levels of cognitive load that ranged between 0–1 at intervals of 0.1;a value of 1 represented the maximal cognitive load, comparable to a person whois unable to pay attention to anything else other than the primary task.

Fig. 1. Crosshair indicating centre of a circle, with black lines indicating the pointswhere the circle may move to in the next time step.

2.2 Model

Of our five senses, the human visual system is arguably the most well-described.Influenced strongly by [6,7], we distil the circuitry into six main functional com-ponents (Fig. 2) — the retina, the thalamus, the visual cortices, the prefrontalcortex (PFC), the frontal eye fields (FEF) and the superior colliculus (SC).

Retina: provides input for the entire system and is thus represented by the imagesfed into the model. It has direct connections to the thalamus and the superiorcolliculus.

Thalamus: raw input from the retina invariably passes through the thalamus,specifically the lateral geniculate nucleus in the first stages of the cycle. Thisinformation is passed onwards to the primary visual cortex. The thalamusreceives simultaneous inputs from the prefrontal cortex that are both excita-tory (direct) and inhibitory (indirect). It is this indirect pathway that allowsthe PFC to selectively inhibit the relay of visual information to the primaryvisual cortex.

Visual cortices: uses recurrent loops from lower to higher visual cortices and thepulvinar to get increasingly abstract visual information from the input, andsends this to the prefrontal cortex to be integrated with contextual informa-tion.

Prefrontal cortex: generally associated with executive functions, decision-makingand other high-level cognitive functions, with widespread connections fromanterior to posterior regions of the cortex. Visual information that reachesthe PFC has been processed and abstracted from the raw visual input in

462 S.C. Low et al.

earlier stages of the circuit. The PFC’s connections to the thalamus implies astructure suitable for retinotopic modulation in early stages of visual process-ing [8].

Frontal eye fields: an important component in the voluntary saccade, it hasretinotopic structure. Although the frontal lobe is traditionally associatedwith higher-order cognitive functions, the FEF has been shown to play a partin low-level sensory processing as well [9].

Superior colliculus: as the gatekeeper for eye saccades, among other directingactions, it plays a key role in the visual circuit. Specifically, the SC is increas-ingly found to play an integrative role in mono- and multi-modal input forthe control of attention and eye movements [10].

Fig. 2. Thalamocortical model showing the connections between the five major compo-nents of the visual circuit. Within the thalamus, the areas of interest are the thalamicreticular nucleus (TRN), the lateral geniculate nucleus (LGN) and the pulvinar nucleus(PV).

As the SC is the primary controller of eye movements, we consider its outputto be the determinant of where a saccade is generated towards. Of the two typesof saccades observed in humans, the instinctive, quick saccades are driven by BUsaliency in the raw input signal from the retina. In the model, BU saliency iscomputed as the difference between two frames (Fig. 4) and can be said to takeplace in the SC; the greater the difference, the greater the bottom-up saliency.Normally, the object with the greatest saliency value is also the desired pointtowards which a saccade is directed based on the bottom-up circuit. However,

Modelling the Effect of Cognitive Load on Eye Saccades and Reportability 463

if a generated number is less than a pre-set noise value, the jump has the samesaliency as the other stimuli. In case of identical saliencies, a point is selected atrandom to saccade towards.

Subsequently, a late saccade in humans occurs due to the inherent latency ofinformation passing through the thalamus and cortical regions before reachingthe superior colliculus. This latency is simulated by having the top-down saliencyof the circles calculated one frame after those used for the bottom-up saliency(e.g., BU is calculated for frame 13 and 14 while TD is calculated for frame12 and 13). The top-down system computes a circular validation gate (Fig. 3)around the circles of the earlier frame, the size (rV G) of which is linearly relatedto the pre-determined cognitive load (CL) (Eq. 1). The next frame is then passedthrough this gate such that all stimuli within the gated regions are negated (i.e.ignored). Any remaining regions after this subtraction implies that one of thecircles has moved outside of the predicted area, and is therefore important. Inthis case, the top-down system produces a saccade to be directed towards thelocation of this circle. When there is a conflict in positions for a saccade betweenBU and TD, TD always overrides BU.

Although the computations in the individual regions of the brain was notdistinguished clearly in the model, it is expected that the calculation of thevalidation gate occurs in the PFC. Subsequently, the point to saccade towardsfrom the TD system could come from either the FEF or the thalamus, albeitwith differing latencies. This nuance is, however, outside the scope of this paper.

rV G = 34 × CL + 8 (1)

Fig. 3. The validation gate in the context of attending to translation of stimuli. Blackcircle: the individual stimulus’ current position. Black square: possible locations thatthe stimulus will move to in the next frame. Blue and orange circles: with low and highcognitive loads respectively, region within which the stimulus is expected to be in thenext frame. (Color figure online)

Finally, the model implies that a jump will only be reported when the PFCdetects a change in the gated information. As cognitive load is introduced intothe model mostly at the level of the PFC, this means the reporting of a jump isalso affected by the cognitive load. Hence, a jump is detected only when thereis a change outside of the validation gate and when it satisfies a probabilistic

464 S.C. Low et al.

conditional that is dependent on the cognitive load value. The latter conditionalhas an additional constraint which prevents a very high cognitive load from com-pletely blocking the reportability of the jumps. This is to provide some leewaywhich represents how humans are able to switch attention from one demandingtask to another, even if they are unable to carry out both concurrently.

Fig. 4. Example of a difference map (inverted from the original black on white images)between two frames without a jump (LEFT) and with a jump (RIGHT) in the top-rightcorner.

3 Results and Discussion

Increasing levels of cognitive load influences specific components of the model’sbehaviour. Early saccades, driven by the bottom-up saliencies directly from thevisual input, remain unaffected by any level of cognitive load (Fig. 5). Late sac-cades however do show a small negative relationship with cognitive load, whereincreasing levels of CL decrease the number late saccades elicited in the SC.Finally, the rate of reporting is most strongly coupled with cognitive load andranges from near perfect response in the absence of a distractor to completefailure under maximal load.

These results are mostly consistent with those from [4], showing how it ispossible for early saccades to be less effective at detecting changes depending onthe noise present in the stimuli, and how reportability of the jumps is reduced ascognitive load increases. However, while [4] found the effect of cognitive load onlate saccades to be insignificant, this was not the case in the model’s data; for themodel, the success of late saccades at detecting jumps is inversely proportionalto cognitive load.

One possible explanation for this is that [4]’s task difficulty and cognitiveloads had an interaction effect which masked the phenomenon. This could bedue to the fact that ten stimuli were used in the experiment, instead of a lowernumber which would have made the task easier to complete and therefore high-lighting the impact of the additional cognitive load. Also, the participants wereallowed to freely move their gaze throughout the experiment. Humans have a

Modelling the Effect of Cognitive Load on Eye Saccades and Reportability 465

Fig. 5. Changes in the ratio of early (red) and late (green) saccades as well as detectionresulting in a “behavioural response” (blue) due to increasing cognitive load. (Colorfigure online)

narrow region of acuity in the centre of their field of vision as that is the retino-topic location of their fovea. This would imply the participants would be lesssensitive to changes outside of this region; when the field of acuity constantlychanges, it is difficult to control for the distance between their visual point offocus and the stimulus that jumps at that same instant.

The current model was designed for functional replication of a physiologicalexperiment, but does include all the main components reported in neuroanatomi-cal literature. However, it remains difficult to draw generalised conclusions aboutthe visual system and the neurological substrate of the validation gate from thisalone. Further work on the model and psychophysiological experiments needsto be carried out to understand the phenomena, such as replacing functionalcomponents with more biophysically plausible spiking or mean-field models. Itis also necessary to include other brain regions at some point, such as the pos-terior parietal cortex and basal ganglia which were neglected as they are mainlyinvolved in voluntary saccades while our model does not distinguish voluntaryand involuntary saccades. Nonetheless, this version already provides the mainarchitecture to do so and is able to replicate and fit physiological data for a betterunderstanding of the underlying processes. It is a plausible circuitry which formsthe validation gate in humans, and demonstrates how the validation gate could

466 S.C. Low et al.

be another form of attention selection mechanism in addition to the attentionalspotlight.

Acknowledgements. The authors would like to thank all members of the SPECSgroup for their input during discussions, with a special mention for: Jordi-Ysard Puigbo,Riccardo Zucca and Clement Moulin-Frier.

This work is supported by the EU FP7 project WYSIWYD (FP7-ICT-612139).

References

1. Borji, A., Itti, L.: State-of-the-art in visual attention modeling. IEEE Trans. Pat-tern Anal. Mach. Intell. 35(1), 185–207 (2013)

2. Kok, P., Rahnev, D., Jehee, J.F., Lau, H.C., de Lange, F.P.: Attention reverses theeffect of prediction in silencing sensory signals. Cereb. Cortex 22(9), 2197–2206(2012)

3. Zikopoulos, B., Barbas, H.: Prefrontal projections to the thalamic reticular nucleusform a unique circuit for attentional mechanisms. J. Neurosci. 26(28), 7348–7361(2006)

4. Mathews, Z., Cetnarski, R., Verschure, P.F.: Visual anticipation biases consciousdecision making but not bottom-up visual processing. Front. Psychol. 5 (2014)

5. Miller, G.A.: The magical number seven, plus or minus two: some limits on ourca-pacity for processing information. Psychol. Rev. 63(2), 81 (1956)

6. Urbanski, M., Coubard, O.A., Bourlon, C.: Visualizing the blind brain: brain imag-ing of visual field defects from early recovery to rehabilitation techniques. Neuro-vision: Neural bases of binocular vision and coordination and their implications invisual training programs (2014)

7. Buchel, C., Friston, K.: Modulation of connectivity in visual pathways by atten-tion: cortical interactions evaluated with structural equation modelling and fMRI.Cereb. Cortex 7(8), 768–778 (1997)

8. Dagnino, B., Gariel-Mathis, M.-A., Roelfsema, P.R.: Microstimulation of area V4has little effect on spatial attention and on the perception of phosphenes evokedin area V1. J. Neurophysiol. 113(3), 730–739 (2015)

9. Kirchner, H., Barbeau, E.J., Thorpe, S.J., Regis, J., Liegeois-Chauvel, C.: Ultra-rapid sensory responses in the human frontal eye field region. J. Neurosci. 29(23),7599–7606 (2009)

10. Sparks, D.L.: Translation of sensory signals into commands for control of saccadiceye movements: role of primate superior colliculus. Physiol. Rev. 66(1), 118–171(1986)

Mutual Entrainment of Cardiac-OscillatorsThrough Mechanical Interaction

Koki Maekawa1(B), Naoki Inoue1, Masahiro Shimizu1, Yoshihiro Isobe2,Taro Saku2, and Koh Hosoda1

1 Graduate School of Engineering Science, Osaka University,1-3 Machikaneyama-machi, Toyonaka, Osaka 560-8531, Japan

maekawa.koki@arl.sys.es.osaka-u.ac.jp2 Atree, Inc., 1-181 Sugi Machi, Yamatokoriyama City, Nara 639-1121, Japan

Abstract. This paper focus on bio-robots driven by cardiomyocyte-powered actuators. Towards biomaterial-based robotic actuators thatexhibit co-operative motion, this study intends to deal with mutualentrainment between two cardiac-oscillators with mechanical interac-tion. The cardiac-oscillator is an oriented collagen sheet seeded culturedrat cardiomyocytes. Mechanically coupled cardiac-oscillator sysytem wasdeveloped consisting of two cardiac-oscillators connected mechanicallyvia a PDMS film. Mechanically coupled cardiac-oscillator system anda single cardiac-oscillator was applied electric stimulation for investi-gating the function of mechanical structure. As a result of experiment,we observed mutual entrainment occurred in only mechanically coupledstructure.

Keywords: Mutual entrainment · Cardiac-oscillator ·Mechanical inter-action

1 Introduction

Living things behave more adaptable on environmental changes than machinesbecause living things are composed of biological devices that have significantabilities such as self-repair, self-assembly, and self-organization. Based on thiscircumstance, bio-robots consisting of biological materials are attracting a lot ofattention [1].

Nawroth et al. [2] developed “medusoids” which imitates jellyfish and repro-duces jellyfish propulsion. Akiyama et al. [3] developed iPAM, consisting of insectdorsal vessel tissue and a frame, which moved faster than other reported microrobots powered by mammalian cardiomyocytes. A great deal of effort had beenmade on developing bio-robot. What seems to be lacking, however, is drivingmethod of bio-robot is limited to drive all actuators simultaneously. To imple-ment co-operative motion such as driving actuators alternately, we have to con-sider a new locomotive method.

Then, we focused on mutual entrainment through mechanical interactionbetween cardiomyocytes. An important property of cardiomyocytes is cardiacc© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 467–471, 2016.DOI: 10.1007/978-3-319-42417-0 48

468 K. Maekawa et al.

excitation-contraction coupling. Cardiac excitation-contraction coupling is theprocess from electrical excitation of the myocyte to heart [4]. The generationorder is as follows: firstly, action potential occurs; secondly, Ca2+ flows; finally,cardiac contraction happen. Iribe et al. [5] confirmed that Ca2+ flows wereprompted in cardiomyocytes by adopting mechanical stimulation. Therefore,there is a possibility of manipulating contraction rhythm of cardiomyocytes byconstructing mechanical interaction.

This study describes analysis of mutual entrainment between twocardiomyocyte-powered actuators that have mechanical interaction. As the actu-ator, we fabricated a cardiac-oscillator that is an oriented collagen sheet con-tracted by cultured rat cardiomyocytes. We developed mechanically coupledcardiac-oscillator system consisting of two cardiac-oscillators connected via aPDMS film. Consequently, mutual entrainment was observed in mechanicallycoupled cardiac-oscillator system, whereas not observed in a single cardiac-oscillator.

2 Mechanically Coupled Cardiac-Oscillator System

As an actuator, a cardiac-oscillator was developed so that a collagen sheet con-tracted by cultured rat cardiomyocytes. The sheet was formed by aligning manycollagen strings and dehydrating them [6]. As the characteristic of the sheet,it has orientation, therefore, a cardiac-oscillator always contract vertical to theoriented direction.

We developed mechanically coupled cardiac-oscillator system consisting oftwo cardiac-oscillators connected mechanically via PDMS film (see Fig. 1), theopposition was fixed on an acrylic mold. By continuing tissue culture, thecardiomyocytes on the collagen sheet were condensed and began contractionas a cardiac-oscillator. Generally, each cardiac-oscillator starts contracting inaccordance with the endemic frequency of cardiomyocytes crowd. We expected,however, mutual entrainment occurs and co-operative contraction between two

Fig. 1. Mechanically coupled cardiac-oscillator system consisting of acrylic resin,PDMS, and two cardiac-oscillators. We measure the displacement of red and blue lines.Then Contraction interval was calculated. (Color figure online)

Mutual Entrainment of Cardiac-Oscillators Through Mechanical Interaction 469

cardiac-oscillators are voluntarily accrued because both cardiac-oscillators pulla PDMS film from both ends.

3 Verification of Mutual Entrainment

Electric stimulation was applied to mechanically coupled cardiac-oscillator sys-tem for investigating the effect of mutual entrainment. Same stimulation wasapplied to a single cardiac-oscillator for control experiment (see Fig. 2). Theexperimental results were expected that the mechanically coupled oscillator sys-tem is more stable against disturbance than a single cardiac-oscillator becausemechanical coupling generates mutual entrainment and distinctive contractionrhythm, whereas a single cardiac-oscillator tends to be influenced by distur-bance and changes the contraction rhythm owing to no mechanical couplingstructure. Both mechanically coupled cardiac-oscillator system and the singlecardiac-oscillator were paced at 0.75 Hz through an externally applied electricfield of 5 V for 60 s.

The results are shown in the Figs. 3 and 4. The green line corresponds toelectric stimulation’s frequency in these figures. It seems that the single cardiac-oscillator contraction synchronized with the electric stimulation (see Fig. 3a),and after removing the stimulation, the single cardiac-oscillator restarted con-traction with endemic frequency (see Fig. 3b). On the other hand, mechanicallycoupled cardiac-oscillator system was not attracted to electric stimulation (seeFig. 4a), and after removing the stimulation, the contraction behavior almostnever changed (see Fig. 4b). From these results, we conformed that mechanicallycoupled cardiac-oscillator system had distinctive contraction rhythm. It is con-sidered that the effect of mutual entrainment to contraction rhythm was strongerthan electric stimulation, therefore, mechanically coupled cardiac-oscillator sys-tem was stable against disturbance.

Fig. 2. Electric stimulation to a single cardiac-oscillator. The black line in picture is asilver wires for electric road.

470 K. Maekawa et al.

Fig. 3. The contraction interval of a single cardiac-oscillator, (a) applying electricstimulation and (b) after removing stimulation. (Color figure online)

Fig. 4. The contraction interval of mechanically coupled cardiac-oscillator system,(a) Applying electric stimulation and (b) after removing stimulation. (Color figureonline)

4 Conclusion

In this study, we made mechanically coupled cardiac-oscillator system consistingof two cardiac-oscillators and observed mutual entrainment caused by mechanicalinteraction. As the result, we found distinctive contraction rhythm. The nextchallenge is developing a bio-robot incorporated this system.

References

1. Webster, V.A., Hawley, E.L., Akkus, O., Chiel, H.J., Quinn, R.D.: Fabricationof electrocompacted aligned collagen morphs for cardiomyocyte powered livingmachines. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A., Prescott, T.J. (eds.)Living Machines 2015. LNCS, vol. 9222, pp. 429–440. Springer, Heidelberg (2015)

2. Nawroth, J.C., Lee, H., Feinberg, A.W., Ripplinger, C.M., McCain, M.L.,Grosberg, A., Dabiri, J.O., Perker, K.K.: A tissue-engineered jellyfish with bio-mimetic propulsion. Nat. Biotechnol. 30, 792–797 (2012)

Mutual Entrainment of Cardiac-Oscillators Through Mechanical Interaction 471

3. Akiyama, Y., Odaira, K., Sakiyama, K., Hoshino, T., Iwabuchi, K., Morishima, K.:Rapidly-moving insect muscle-powered microrobot and its chemical acceleration.Biomed. Microdevices 14(6), 979–986 (2012)

4. Bers, D.M.: Cardiac excitation-contraction coupling. Nature 415(6868), 198–205(2002)

5. Iribe, G., Jin, H., Naruse, K.: Role of sarcolemmal BKCa channels in stretch-inducedextrasystoles in isolated chick hearts. Circ. J. 75(11), 2552–2558 (2011)

6. Isobe, Y., Kosaka, T., Kuwahara, G., Mikami, H., Saku, T., Kodama, S.: Orientedcollagen scaffolds for tissue engineering. Materials 5(3), 501–511 (2012)

“TEGOTAE”-Based Control of Bipedal Walking

Dai Owaki1(B), Shun-ya Horikiri1, Jun Nishii2, and Akio Ishiguro1,3

1 Research Institute of Electrical Communication, Tohoku University,2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan

owaki@riec.tohoku.ac.jp2 Yamaguchi University, 1677-1 Yoshida, Yamaguchi 753-8512, Japan

3 CREST, Japan Science and Technology Agency, 4-1-8 Honcho,Kawaguchi 332-0012, Saitama, Japan

http://www.cmplx.riec.tohoku.ac.jp

Abstract. Despite the appealing concept of “central pattern generator”(CPG)-based control for bipedal walking, there is currently no systematicmethodology for designing a CPG controller. To tackle this problem, weemploy a unique approach: We attempt to design local controllers in theCPG model for bipedal walking based on the viewpoint of “TEGOTAE”,which is a Japanese concept describing how well a perceived reactionmatches an expectation. To this end, we introduce a TEGOTAE functionthat quantitatively measures TEGOTAE. Using this function, we candesign decentralized controllers in a systematic manner. We designed atwo-dimensional bipedal walking model using TEGOTAE functions andconstructed simulations using the model to verify the validity of theproposed design scheme. We found that our model can stably walk onflat terrain.

Keywords: Bipedal walking · TEGOTAE · Plantar sensation · Centralpattern generator (CPG)

1 Introduction

Humans and animals exhibit astoundingly adaptive and versatile locomotiongiven real-world constraints. To endow robots with similar capabilities, theirbodies must have a significantly larger number of degrees of freedom than whatthey have at present. To successfully coordinate movement with many degrees offreedom in response to various circumstances, “autonomous decentralized con-trol” plays a pivotal role and has therefore attracted considerable attention.

In fact, animals deftly coordinate the many degrees of freedom of their bodiesusing distributed neural networks called “central pattern generators” (CPGs),which are responsible for generating rhythmic movements, particularly locomo-tion [1,2]. Based on these biological findings, various studies have been conductedthus far to incorporate artificial CPGs into legged robots with the aim of generat-ing highly adaptive locomotion [3–6]. However, there is currently no systematicmethodology for designing a CPG controller; each individual CPG model hasbeen designed on a completely ad hoc basis for a specific practical situation.c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 472–479, 2016.DOI: 10.1007/978-3-319-42417-0 49

“TEGOTAE”-Based Control of Bipedal Walking 473

To tackle this issue, we herein introduce a unique approach: We attempt todesign local controllers using the CPG model for bipedal walking based on theviewpoint of “TEGOTAE”, which is a Japanese concept describing how wella perceived reaction, i.e., sensory information, matches an expectation, i.e., anintended motor command. To this end, we introduce the TEGOTAE function,which quantitatively measures TEGOTAE. This function can be described asthe product of “what a local controller wants to do” and “its resulting reaction”on the basis of the concept of TEGOTAE. Thus, by only designing local sensoryfeedback such that each local controller increases “consistent” TEGOTAE in linewith “expectation,” or decreases “inconsistent” TEGOTAE otherwise, we candesign decentralized controllers in a systematic manner. In this paper, we pro-posed a systematic design scheme of a decentralized CPG controller for bipedalwalking based on TEGOTAE and verify the validity of the scheme through sim-ulation.

Fig. 1. Bipedal walking model.

2 Bipedal Walking Model

2.1 Musculoskeletal Structure

To validate the design scheme based on the TEGOTAE function, we conductedsimulations using a two-dimensional bipedal walking model. Figure 1 shows themusculoskeletal structure of the bipedal walking model, whose movements areconstrained in the sagittal plane for simplicity. We implemented seven actuatorson the waist joint, two hip joints, two knee joints, and two ankle joints, at whicheach actuator generates torque based on proportional-derivative (PD) control,as explained in Sect. 2.2. Passive springs and dampers are implemented into thetoe joints to passively generate an effective push-off force at the end of the stancephase. On the basis of findings in humans and animals, which have shown that

474 D. Owaki et al.

cutaneous receptors in the foot play an essential role in the control of gait [7–9],we modeled plantar sensations that can detect vertical and horizontal groundreaction forces (GRFs) (NV

x,i and NHx,i) to the ground at heel (x = h), metatarsal

(x = m), and toe (x = t) points on the feet. Here, the suffix i denotes each leg(i = 0: right and i = 1: left).

2.2 Systematic Design Scheme of a CPG Controller Basedon the “TEGOTAE” Function

The proposed control system for bipedal walking consists of four components:(1) hip controllers, (2) knee controllers, (3) ankle controllers, and (4) a posturecontroller. The first three components coordinate the inter- and intra-limb move-ments via TEGOTAE functions for adaptive walking, and the forth componentstabilizes the upper body using the waist actuator and vestibular sensor. Due tospace limitations, the details of the posture control are not presented here.

Hip, knee, ankle, and waist joint torque τy,i in the ith leg (y indicates one ofthe joints) are generated by the PD control using the following equations:

τhip,i = −Khip,i(θhip,i − θhip,i) − Dhip,iθhip,i, (1)

τknee,i = −Kknee,i(θknee,i − θknee,i) − Dknee,iθknee,i, (2)

τankle,i = −Kankle,i(θankle,i − θankle,i) − Dankle,iθankle,i, (3)

τtrunk = −Ktrunk(θtrunk − θtrunk) − Dtrunkθtrunk, (4)

where θy,i and θy,i are the actual and target angles at joint y in the ith leg, respec-tively. Furthermore, Ky,i and Dy,i are the proportional and derivative gains ofthe PD controller at joint y. Hip, knee, and ankle controllers can modulate thetarget angles θy,i and the proportional gains Ky,i using the TEGOTAE functionto generate adaptive walking.

The TEGOTAE function is a function formulated using the concept ofTEGOTAE, which is described as the product of the (i) intended motor com-mand of a controller f(x), where x denotes a control variable, and (ii) resultingsensory information g(S) obtained from sensor values S as follows:

T (x,S) = f(x)g(S), (5)

where we design the TEGOTAE function such that it increases if wegain sensory information that is consistent with the intended motor com-mand. Positive/negative values of the TEGOTAE function indicate consis-tency/inconsistency between the intended motor command and resulting sensoryinformation.

Using TEGOTAE function T (x,S), we can modulate the local control vari-able x as follows:

x = h(x) +∂T (x,S)

∂x, (6)

where the first term on the right denotes the intrinsic dynamics of the localcontroller, and the second term denotes the local sensory feedback for variable x

“TEGOTAE”-Based Control of Bipedal Walking 475

based on the TEGOTAE function. Using the sensory feedback described by thepartial differential form of the TEGOTAE function, the controller modulates itscontrol variable x such that it increases consistent TEGOTAE with expectation,or decreases inconsistent TEGOTAE otherwise. We now describe the local jointcontrollers at the hip, knee, and ankle joints designed using the TEGOTAEfunctions.

Fig. 2. Definition of the TEGOTAE function for the hip controller. (a) TEGOTAEfunction based on the corresponding leg’s sensory information. (b) TEGOTAE functionbased on the other leg’s sensory information.

Hip Control. The role of the hip joints in human waking is rhythmic motiongeneration for forward and backward leg swing [10]. To generate such rhythmicmovements, we use phase oscillators to generate the target angle of the hipactuators (Eq. 1), which are described by the following equation:

θhip,i = −C1,hip cos φi + C2,hip, (7)

where C1,hip and C2,hip [rad] denote the amplitude and offset angles of thehip target angle, respectively. According to the oscillator phases φi, legs arecontrolled to be in the swing phase for 0 ≤ φi < π and in the stance phase forπ ≤ φi < 2π.

The dynamics of the phase oscillators with the local sensory feedback usingthe TEGOTAE function are as follows:

φi = ω +∂Thip,i(φi,N)

∂φi, (8)

where ω [rad/s] denotes the intrinsic angular velocity of the oscillators. TheTEGOTAE function for the hip control is defined as the following equation:

Thip,i(φi,N) = σhip,1{NVh,i(− sin φi) + (NV

m,i + NVt,i)(sin φi)}

+ σhip,2{NVh,j(sin φi) + (NV

m,j + NVt,j)(− sin φi)}, (9)

476 D. Owaki et al.

where σhip,1 and σhip,2 [rad/Ns] denote the feedback gains. The suffix i and jdenotes the corresponding and other legs, respectively. The first term on theright represents the TEGOTAE function based on the sensory information ofthe corresponding leg (Fig. 2(a)). The value of NV

h,i(− sin φi) becomes a posi-tive value when the heel sensor on the corresponding leg detects a large verticalGRF (NV

h,i > 0) and the oscillator phase is in the stance phase (π < φi < 2π).By increasing this TEGOTAE term, the leg remains in the stance phase whilesupporting the body (NV

h,i > 0). In contrast, the value of (NVm,i + NV

t,i) (sin φi)becomes positive value when the metatarsal and toe sensors on the correspondingleg detect a large vertical GRF (NV

m,i + NVt,i > 0) and the oscillator phase is the

swing phase (0 < φi < π). In this case, by increasing this TEGOTAE term, theleg enters the swing phase at the end of stance phase (NV

m,i + NVt,i > 0), which

in turn pushes the body forward effectively. The second term represents theTEGOTAE function based on the sensory information of the other leg(Fig. 2(b)). The details of these effects are not explained here due to space limita-tion. By using the TEGOTAE-based local feedback in Eq. (8), the hip controllersenable “interlimb” coordination without any neural communication.

Knee Control. The role of a knee joint in human walking [10] is to supportthe body by increasing its stiffness in the stance phase and effective flexion bydecreasing its stiffness in the swing phase. Thus, we define control variable χi,which denotes the control command that increases and decreases the stiffnessof the knee joints. To implement such a stiffness control mechanism, we modifygain P in the knee controllers using χi as follows:

τknee,i = −Kknee,iθknee,i − Dkneeθknee,i, (10)Kknee,i = max[C1,knee tanh χi, 0] + C2,knee, (11)

where C1,knee and C2,knee [Nm/rad] denote the variable range and offset value ofgain P , respectively. In Eq. (10), the target angle θknee of the knee controllers areset to 0 [rad], which indicates the state of the knee extension, allowing high/lowstiffness to extend/flex the knee joints.

The dynamics of control variable χi with the local sensory feedback usingthe TEGOTAE function is as follows:

χi = −cknee,iχi +∂Tknee,i(χi,N)

∂χi, (12)

where cknee,i denotes the time constant of the first order lag. The TEGOTAEfunction on the knee control is defined by the following equation:

Tknee,i(χi,N) = σknee,1NVi χi + σknee,2N

Vj (−χi), (13)

where NVi and NV

j [N] denote the sum of the vertical force sensor values on theheel, metatarsal, and toe, describing by, e.g., NV

i = NVh,i + NV

m,i + NVt,i, of the

corresponding and other legs, respectively. Parameters σknee,1 and σknee,2 [1/N]

“TEGOTAE”-Based Control of Bipedal Walking 477

denote the feedback gains. The first term on the right represents the TEGOTAEfunction based on the sensory information of the corresponding leg (Fig. 3(a)top). The value of NV

i χi becomes a positive value when the foot sensors onthe corresponding leg detect a large vertical GRF (NV

i > 0) and the controlcommand for the knee is increasing the stiffness (χi > 0). Hence, by increasingthis TEGOTAE term, the knee stiffness remains high while supporting the body(NV

i > 0). The second term represents the TEGOTAE function based on thesensory information of the other leg (Fig. 3(a) bottom). Due to space limitation,the details of the effect is not explained here.

Fig. 3. Definition of the TEGOTAE function for the (a) knee and (b) ankle con-troller. Top: TEGOTAE function based on the corresponding leg’s sensory information.Bottom: TEGOTAE function based on the other leg’s sensory information.

Ankle Control. The role of an ankle joint in human walking [10] is to producethe push-off to generate the propulsion forces near the end of the stance phaseand avoid colliding the foot with the ground during the swing phase. Thus, wedefine control variable ψi, which denotes the control command that increases ordecreases the target angle of the ankle joints. We modify the target angle of theankle controllers using ψi as follows:

θankle,i = C1,ankle tanh ψi + C2,ankle, (14)

where C1,ankle and C2,ankle [rad] denote the variable range and offset value ofthe ankle target angle, respectively. The positive/negative value of ψi representsthe plantar/dorsal flexion of an ankle joint.

The dynamics of control variable ψi with the local sensory feedback usingthe TEGOTAE function is as follows:

ψi = −cankle,iψi +∂Tankle,i(ψi,N)

∂ψi, (15)

where cankle,i denotes the time constant of the first order lag. The TEGOTAEfunction on the ankle control is defined as follows:

Tankle,i(ψi,N) = σankle,1NHi ψi + σankle,2N

Vj (−ψi), (16)

478 D. Owaki et al.

where NHi [N] denotes the sum of the horizontal force sensor values on the heel,

metatarsal, and toe of the corresponding leg, described by NHi = NH

h,i + NHm,i +

NHt,i. In addition, NV

j [N] denotes the sum of the vertical force sensor values ofthe other leg, described by NV

j = NVh,j + NV

m,j + NVt,j . Parameters σankle,1 and

σankle,2 [1/N] denote the feedback gains. The first term on the right representsthe TEGOTAE function based on the sensory information of the correspondingleg (Fig. 3(b) top). The value of NH

i ψi becomes a positive value when the footsensors on the corresponding leg detects a large horizontal GRF (NH

i > 0) andthe command for the ankle is plantar flexion (ψi > 0). Increasing this TEGOTAEterm results in more effectively generating plantar flexion at the end of stancephase (NH

i > 0), which in turn generates a larger propulsion force. The secondterm represents the TEGOTAE function based on the sensory information of theother leg (Fig. 3(b) bottom). Due to space limitation, the details of the effect isnot explained here.

In sum, the advantage of our design scheme using the TEGOTAE functionsis that we can systematically design controllers for many components by onlydesigning TEGOTAE functions for each controllers; We simply have to design theappropriate TEGOTAE functions responsible for the target movements. Further,we expect that the TEGOTAE-based hip, knee, and ankle controllers enablespontaneous and adaptive “inter”- and “intra”-limb coordination via TEGOTAEfunctions.

3 Simulation Result

Here, we describe the verification of our proposed design scheme using numericalsimulation. Figure 4 shows a stick diagram plot (a) and time series data (b) (bothhip angles, left knee and ankle angles, and stance phases of both legs) of steadywalking motion. As shown in this figure, we achieved steady walking motion bydesigning each joint controller based on the TEGOTAE functions. Note that thetime series data of the simulation were similar to human data of walking [10].

Fig. 4. Stick diagram (a) and time series data (b) of steady walking obtained over fiveperiods. (Color figure online)

“TEGOTAE”-Based Control of Bipedal Walking 479

4 Conclusion and Future Work

The purpose of this study was to verify the validity of the proposed designscheme based on the TEGOTAE concept for bipedal walking. To this end,we constructed a bipedal walking model and applied our scheme to designjoint controllers. We confirmed that the joint controllers designed using theTEGOTAE functions achieved stable bipedal walking on flat ground via sponta-neous inter- and intra-limb coordination. The advantages of the proposed methodover previous works [3,5] and the adaptability to environmental changes, whichwe did not verify, will be studied in future.

Acknowledgements. We acknowledge the support of a JSPS KAKENHI Grant-in-Aid for Young Scientists (A) (25709033) and Grant-in-Aid for Scientific Research onInnovative Areas “Understanding brain plasticity on body representations to promotetheir adaptive functions” (26120008).

References

1. Shik, M.L., Severin, F.V., Orlovskii, G.N.: Control of walking and running bymeans of electrical stimulation of the mesencephalon. Electroencephalogr. Clin.Neurophysiol. 26, 549 (1969)

2. Grillner, S.: Neurobiological bases of rhythmic motor acts in vertebrates. Science228, 143–149 (1985)

3. Taga, G., Yamaguchi, Y., Shimizu, H.: Self-organized control of bipedal locomotionby neural oscillators. Biol. Cybern. 65, 147–159 (1991)

4. Kimura, H., Akiyama, S., Sakurama, K.: Realization of dynamic walking and run-ning of the quadruped using neural oscillator. Auton. Robots 7, 247–258 (1999)

5. Aoi, S., Tsuchiya, K.: Locomotion control of a biped robot using nonlinear oscil-lators. Auton. Robots 19, 219–232 (2005)

6. Righetti, L., Ijspeert, A.J.: Pattern generators with sensory feedback for the controlof quadruped locomotion. In: Proceedings of ICRA 2008, pp. 819–824 (2008)

7. Duysens, J., Clarac, F., Cruse, H.: Load-regulating mechanisms in gait and posture:comparative aspects. Physiol. Rev. 80, 83–133 (2000)

8. Dietz, V., Duysens, J.: Significance of load receptor input during locomotion: areview. Gait Posture 11, 102–110 (2000)

9. Elis, E., Behrens, S., Mers, O., Thorwesten, L., Volker, K., Rosenbaum, D.:Reduced plantar sensation causes a cautious walking pattern. Gait Posture 20,54–60 (2004)

10. Perry, J., Burnfield, J.: Gait Analysis: Normal and Pathological Function, 2nd edn.Slack Inc., Thorofare (2010)

Tactile Vision – Merging of Senses

Nedyalka Panova(✉), Alexander C. Thompson(✉), Francisco Tenopala-Carmona(✉),and Ifor D.W. Samuel(✉)

SUPA, School of Physics and Astronomy, Organic Semiconductor Centre,University of St. Andrews, North Haugh, St. Andrews KY16 9SS, UK

{np44,at226,ftc2,idws}@st-andrews.ac.uk

Abstract. We developed a tactile vision sensor, by converting light intensityfrom narrowband reflection spectroscopy into different stimuli such as sound.This creates the perception of colours though alternative senses. The deviceextends work done to develop a wearable sensor of erythema and shows theoutcome from the collaboration between scientists and artist. It discovers newapplications for existing research combining technology and design.

1 Introduction

The absorption of light is an important process in nature and is often linked to transferof energy. Sunlight is a thermal source and contains different wavelengths correspondingto different colours. Different materials can then selectively absorb specific wavelengthsbased on their molecular structure. In addition microstructure is sometimes used to givestructural colours.

The carbon fixed by plants is used by mammals for nutrition and building bodystructures when on the other hand our vision relies on protein in the retina to enable usto interact with the environment. Additional scientific research on this topic results inadvanced applications in medicine, sustainable energy, and sensing devices.

From an artistic perspective a combination of natural and man-made light absorbingmaterials and devices opens new territories for creative entanglement between art andscience. The perception of the world around us and our interaction with it approacheslimits of knowledge and representations. The new level of interaction that technologycan offer for new senses and experience creates a dialogue about physiological aspectof what is to be a human.

For the 5th Conference on Living Machines we are presenting a specially engineeredwearable device that is able to detect all visible colours. The development stage of thedevice is inspired by the same mechanism the human retina uses to detect colours anda device to measure skin redness (erythema) currently being investigated and studied inthe Organic Semiconductor Centre, at the University of St. Andrews.

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 480–484, 2016.DOI: 10.1007/978-3-319-42417-0_50

2 Erythema Measurement

Erythema, or skin redness, has a wide range of causes, such as UV irradiation, exposureto heat, intake of phototoxic drugs, allergens [1]. A current stream of research in thegroup is to develop a wearable monitor of erythema.

Existing devices to measure erythema are large benchtop devices, limiting theamount of temporal information that can be measured [2, 3]. However, a wearable devicewill allow continuous measurement and increase the temporal resolution of measure‐ments. This improvement will provide a better understanding of erythema and theunderlying causes of erythema from different triggers.

A range of techniques exist to measure erythema, including narrowband reflectancespectroscopy and analysis of RGB images [3, 4]. All rely on the difference in absorptionbetween red and green light, as increased blood perfusion in tissue from erythema resultsin stronger absorption of green light (Fig. 1).

Fig. 1. Reflection spectra of skin with erythema, showing decreasing erythema (red to black).(Color figure online)

3 Tactile Vision

The ability of the erythema sensor to touch objects such as skin and discriminate betweencolours inspired the tactile vision device. This took the existing sensor and adapted it tomeasure reflection from nine different LEDs covering the visible range of light (Fig. 2).

Tactile Vision – Merging of Senses 481

Reflection intensities of different colours can then be presented to the user as sound orby other stimuli. For example, a user could take this device and touch different objectsand hear different sounds relating to the colour of the object.

Our tactile vision device may offer the potential to discriminate between more wave‐lengths of light than humans are currently capable of. As the human retina contains lightabsorbing pigment molecules, retinene and 3 type of cone cells to detect green, blue andred colours, while our detector works with 9 different colours (Fig. 2).

We expect such a device to have several applications, such as helping blind and/orcolourblind people to ‘see’ colours in their daily tasks, and incorporation within inter‐active electronic devices and artificial intelligence.

The main advantage of it is the fact that it is wearable and has relatively smalldimensions of 25 mm diameter, so it can be attached to one’s fingertip or worn as apendant, and through a single touch it is able to give a description of the colour, helpingpeople and machines. An example of the sensing device is shown in Fig. 3.

The poster/demo aims to demonstrate the merging of senses in an interactive demon‐stration for tactile colour vision. The artistic approach of the research engages art,science and philosophy into a critical dialogue about the perception of colours based onlogic vs imagination. On one hand, the subjective vision of a gifted artist helps the publicto see the world differently. On the other hand, if we strip the concept of colour fromthe human element and look at the primary colours as at primary numbers: the coloursbecome a piece of data with no properties attached to them.

How could we describe colours to blind or colour blind people? Maybe they canlearn the different smell of a green and yellow leaf, but can they smell reddish-green?They can learn different sounds of the ocean we associate with its colours, the calm blue,the windy grey and the crushing white. The difference in seeing and understanding

Fig. 2. Wavelength range covered by sensor (Color figure online)

482 N. Panova et al.

colours is limited from our nature and nurture and as every concept is challenged by theexperiment, the concept of colours is challenged by our sensory limitations.

“Science subjects the data of our experience to a form of analysis that we can neverexpect will be completed since there are no intrinsic limits to the process of observation” [5].

With the ‘tactile visionʼ project we are looking at the perception of colours as at aclosed system and the synthetic knowledge we achieve within is based not on uncertaintyand appearance but on precise measurements inside the monitored area.

So we juxtapose the empirical measurements to the open end of colour variationsassociated with qualities such as cold, warm, dark and light and therefore, redesign theperception of the physical world to a digital one.

4 Conclusion

We are using the tactile vision workshop to present the biomimetic model of lightabsorbing mechanism studied within the Organic Semiconductor Centre (OSC) andespecially for the development of a monitoring device that uses the light absorbingproperties of blood to measure the redness of the patient’s skin.

Fig. 3. Example of sensing device

Tactile Vision – Merging of Senses 483

We see 5th Conference on Living Machines as a suitable platform to highlight theresearch of the OSC whose main focus is the use of organic (carbon based) semicon‐ductors for solar cells, OLED displays, lasers and applications such as Li-Fi, sensing,medical applications (such as light-tissue interactions), optoelectronic design and engi‐neering, computer simulations, and fluorescence imaging for Photodynamic Therapy(PDT) for skin lesions.

Collaboration between scientists and artists creates opportunities to further extendthe possibilities and application of present research and to generate new tools forcommunicating science to larger audiences. The tactile vision device has been born outof discussion between an artist-in-residence and post-doctoral researchers, showing thebenefits of merging science and art.

References

1. Wan, S., et al.: Quantitative evaluation of ultraviolet induced erythema. Photochem. Photobiol.37(6), 643–648 (1983)

2. Dolotov, L.E., et al.: Design and evaluation of a novel portable erythema-melanin-meter.Lasers Surg. Med. 34, 127–135 (2004)

3. Diffey, B.L., Farr, P.M.: Quantitative aspects of ultraviolet erythema. Clin. Phys. Physiol.Meas. 12(4), 311–325 (1991)

4. Stephen, S., et al.: Detection of skin erythema in darkly pigmented skin using multispectralimages 22(4), 172–179 (2009)

5. Merleau-Ponty, M.: The World of Perception, p. 44. Routledge, London (2004)

484 N. Panova et al.

Tactile Exploration by Contour FollowingUsing a Biomimetic Fingertip

Nicholas Pestell1,2(B), Benjamin Ward-Cherrier1,2, Luke Cramphorn1,2,and Nathan F. Lepora1,2

1 Department of Engineering Mathematics, University of Bristol, Bristol, UK{np0877,bw14452,ll14468,n.lepora}@bristol.ac.uk

2 Bristol Robotics Laboratory, University of Bristol, Bristol, UK

Abstract. Humans use a contour following exploratory procedure toestimate object shape by touch. Here we demonstrate autonomousrobotic contour following with a biomimetic tactile fingertip, the Tac-Tip, using an active touch method previously developed for other typesof touch sensors. We use Bayesian sequential analysis for perception andimplement an active control strategy to follow an object contour. Thetechnique is tested on a 110 mm diameter circle and yields results compa-rable with those previously achieved for other tactile sensors. We intendto extend the work onto a different robot platform with improved tra-jectory control to improve robustness, speed and match with humanperformance.

Keywords: Tactile sensors · Bayesian perception

1 Introduction

Touch is used extensively for various tasks including manipulation and explo-ration. Specifically, touch is implemented for characterising physical propertiessuch as shape, texture, weight, hardness, temperature and size [1]. Ledermanand Klatzky describe a set of distinct haptic exploratory procedures (EPs) whichare employed separately depending on which physical property is being charac-terised. For example, lateral motion is associated with texture encoding, unsup-ported holding with encoding of weight, pressure with compliance, static contactwith temperature, enclosure with volume and coarse shape and contour followingwith precise shape.

Shape encoding using robotic tactile perception for autonomous contour fol-lowing has previously been achieved using the iCub fingertip [2]. Here we focuson achieving similar results using a novel biomimetic optical tactile sensor, theTacTip, developed at Bristol Robotics Laboratory [3,4]. The TacTip’s novelty isinformed by the structure of the human fingertip and the use of an optical systemto process tactile information. We implement a perception-action cycle, wherethe TacTip perceives the edge or contour of an object and the subsequent actionis a trajectory informed solely by the perception. The procedure is repeated sothat the sensor traces a path around the object’s edge.c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 485–489, 2016.DOI: 10.1007/978-3-319-42417-0 51

486 N. Pestell et al.

2 Methods

2.1 Hardware

The TacTip is a 3D-printed, biomimetic optical tactile sensor with a softdeformable skin lined with a concentric array of pins on the inside layer. The pinsare illuminated by a ring of LEDs and their positions are tracked by computervision algorithms applied to the images from a CCD web-camera.

Novel features of the TacTip draw influence from the structure of thehuman fingertip. For example, dermal papillae are small pimple-like extensionsof the dermis into the epidermis which form a strong interaction with tactilemechanoreceptors, enhancing receptor response to skin deformation. Papillae areanalogues to the TacTip’s pins which portray an amplified deformation responseto skin-surface contact. Furthermore, the compliant gel separating the skin andthe CCD camera has similar mechanical properties to the dermis and subcuta-neous fat.

The TacTip is mounted as an end effector on a six degree-of-freedom robotarm (IRB 120, ABB Robotics). The arm can precisely and repeatedly positionthe sensor (absolute repeatability 0.01 mm) (Fig. 1).

Fig. 1. Hardware set up. The TacTip is mounted on an IRB 120 robot arm and makessuccessive taps on a test object.

2.2 Perception and Control

We implement a biomimetic approach to perception based on Bayesian sequentialanalysis for optimal decision making [5,6] which has been shown to achievesuperresolved tactile sensing with the TacTip [7], where localization acuity was ∼0.1 mm as compared with a sensor resolution of 4 mm. Martinez-Hernandez et al.

Tactile Exploration by Autonomous Contour Following 487

successfully applies this Bayesian approach to the contour following problem withan iCub fingertip [2]. The iCub fingertip is a capacitive tactile sensor with 12overlapping taxels (tactile elements) on a flexible printed circuit board. Herewe adapt those techniques for use with the TacTip. Using Bayesian sequentialanalysis, the TacTip pin deflections are treated similarly to readings from theiCub taxels.

Fig. 2. Active contour following algorithm. Showing perception phase, decision processand active and exploratory moves.

Figure 2 depicts the contour following algorithm. Contact is made with anunknown object and likelihoods are estimated for each possible sensor orientation(angle) relative to the object edge. Bayes’ rule is used, with an initially flat prior,after each successive contact to recursively update the posterior beliefs for eachorientation hypotheses. The algorithm then evaluates a decision condition,

any P (cn|z1:t) > θdec (1)

where P (cn|z1:t) is the posterior for hypotheses cn with evidence z accumulatedafter t taps and θdec is the decision threshold. If the output of the decisionprocess is ‘yes’, the orientation is perceived as the class with the highest posterior.The robot then moves a predefined increment along a tangent to the perceivedcontour edge. If the output is no, the robot moves into an active process; itrepositions the TacTip to the perceived most optimal position for orientationdetection. The relative move is based on the marginal position posteriors. Afterrelocation another tap is made, the posteriors are updated and the decisionprocess (1) is re-implemented.

Martinez-Hernandez et al. [2] makes a distinction between active and passiveperception for contour following. Active perception is when the sensor performsrepositioning movements to more optimal locations for making perceptual deci-sions about its orientation. Alternatively, passive perception is when there is no

488 N. Pestell et al.

repositioning and only the orientation is perceived. It was observed that activeperception is far superior to the passive variant when perceiving edge orientationand is a key principle to achieve robust contour following [2]. Hence, in this workwe only consider active contour following.

3 Results

Here we demonstrate contour following on a circle (diameter 110 mm). Resultsare shown in Fig. 3. The robot is able to move along the edge of the objectby making active moves radially in order to relocate to an optimal position fororientation detection and then make exploratory moves along the tangent tosuccessfully follow the contour arround the shape.

Fig. 3. Plot of sensor positions on one lap of 110 mm diameter circle. The actual shapeedge is depicted as a solid black line, sensor positions at each tap are shown as smallercircles.

4 Conclusions and Future Work

Using the methods of Martinez-Hernandez et al. [2] we have demonstrated thatthe TacTip is an effective tool for identification of a previously unknown circularshape. The result achieved here suggests that the TacTip is comparable to theiCub fingertip for shape recognition, although an assessment on the same objectshapes is needed for a more direct comparison.

It may be of benefit to introduce a short term memory factor into the controlstrategy which allows posteriors to influence priors between exploratory movesrather than initializing the priors to a flat distribution. This may improve bothdecision time and accuracy.

Tactile Exploration by Autonomous Contour Following 489

We intend to migrate the contour following tasks onto the UR5 robot arm.The UR5 is able to execute specified trajectories which, if informed by theobject’s edge shape, may improve speed and robustness of the tasks. In orderto take advantage of this trajectory feature, the perception technique should beadapted so that posterior beliefs can be updated in a continuous fashion whilethe TacTip traces a smooth path around the object edge.

Acknowledgment. NP was supported by an EPSRC DTP studentship and NLwas supported in part by an EPSRC grant on ‘Tactile Superresolution Sensing’(EP/M02993X/1).

References

1. Ledermen, S., Klatzky, R.: Hand movements: a window into haptic object recogni-tion. Cogn. Psychol. 19, 342–368 (1987)

2. Martinez-Hernandez, U., Dodd, T., Natale, L., Metta, G., Prescott, T., Lepora,N.: Active contour following to explore object shape with robot touch. In: WorldHaptics Conference, pp. 341–346 (2013)

3. Chorley, C., Melhuish, C., Pipe, T., Rossiter, J.: Development of a tactile sen-sor based on biologically inspired edge encoding. In: International Conference onAdvanced Robotics, pp. 1–6 (2009)

4. Assaf, T., Roke, C., Rossiter, J., Pipe, T., Melhuish, C.: Seeing by touch: evaluationof a soft biologically-inspired artificial fingertip in real-time active touch. Sensors14(2), 2561–2577 (2014)

5. Lepora, N.: Biomimetic active touch with tactile fingertips and whiskers. IEEETrans. Haptics 9(2), 170–183 (2016)

6. Lepora, N., Martinez-Hernandez, U., Evans, M., Natale, L., Metta, G., Prescott,T.: Tactile superresolution and biomimetic hyperacuity. IEEE Trans. Robot. 31(3),605–618 (2015)

7. Lepora, N., Ward-Cherrier, B.: Superresolution with an optical tactile sensor. In:IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.2686–2691 (2015)

Towards Self-controlled Robots ThroughDistributed Adaptive Control

Jordi-Ysard Puigbo1(B), Clement Moulin-Frier1, and Paul F.M.J. Verschure1,2

1 Laboratory of Synthetic, Perceptive, Emotive and Cognitive Science (SPECS),DTIC, Universitat Pompeu Fabra (UPF), Barcelona, Spain

jordiysard.puigbo@upf.edu2 Catalan Research Institute and Advanced Studies (ICREA), Barcelona, Spain

Abstract. Robots, as well as machine learning algorithms, have provento be, unlike human beings, very sensitive to errors and failure. Artificialintelligence and machine learning are nowadays the main source of algo-rithms that drive cognitive robotics research. The advances in the fieldshave been huge during the last year, beating expert-human performancein video games, an achievement that was unthinkable a few years ago.Still, performance has been assessed by external measures not necessar-ily fit to the problem to solve, what lead to shameful failure on somespecific tasks. We propose that the way to achieve human-like robustnessin performance is to consider the self of the agent as the real source ofself-evaluated error. This offers a solution to acting when information orresources are scarce and learning speed is important. This paper detailsour extension of the cognitive architecture DAC to control embodiedagents and robots, through self-generated signals, from needs, drives,self-generated value and goals.

1 Introduction

Since early history, human beings have dreamed of creating servants from inan-imate mater. First examples can be found back to Jewish mythology with theconcept of golem, a being magically animated from inanimate matter, like mud,or Greek mythology where the blacksmith god Hephaestus constructed severalautomatons to help on his workshop. Later on, technology allowed to constructautomatons capable of simulating human behavior in an autonomous manner, butthe level of autonomy was extremely limited to simple sets of actions, mechani-cally powered, and the capacity to adapt was very restricted, or absent. Roboticssubstituted automatons with expectations of greater autonomy and adaptability,togetherwith big improvements on speed andprecision. Still, the challenge remainson dealing with open, unconstrained and uncontrolled environments.

In parallel, also since the beginning of written history, evidence exist of thesearch for understanding the nature of human behavior, reasoning and othercognitive abilities. Back in ancient Greece, philosophers like Plato and Aristotletried to understand which was the nature of human knowledge. The study ofmind remained in the field of philosophy until the nineteenth century, with thec© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 490–497, 2016.DOI: 10.1007/978-3-319-42417-0 52

Towards Self-controlled Robots Through Distributed Adaptive Control 491

emergence of experimental psychology, until a few decades later, behaviorismdominated the field and practically denied the existence of mind. At that sametime, the mid of 20th century, computer technology was enabling the study ofArtificial Intelligence (AI), based on using mathematical reasoning to reproducehuman cognitive abilities. Rapidly the field became dominated by the study ofhigh level reasoning, applying to purely cognitive problems, detached from thephysical world. It was then, around the 80’s, that parallel advances on the fieldsof cognitive science, neuroscience and artificial intelligence, converged at high-lighting the need to ground cognitive function on experience through embodiedaction. Theories of embodied cognition began to emerge at the hands of RodneyBrooks or Gerald Edelman.

Still, embodied cognition proposed the use of robots to understand cognitivefunction in a developmental perspective, but it was after two decades that therobotics community began to adopt developmental approaches to allow lifelongand open-ended learning of new skills and new knowledge on embodied agents.But still, the problem remains unsolved: embodiment seems a good paradigm,but embodied agents haven’t outperformed non-embodied algorithms for thesame tasks and still non-embodied artificial cognition have only outperformedhuman cognition in very constrained and abstract tasks, where average humanswould fail, whereas can’t compete with average human routine behavior.

We highlight the need of a central pillar of cognition that is usually ignoredby computational approaches: the self. Whereas in general, sensory and motormodules are dissociated but communicating one with each other, we argue for theneed of an active monitor or supervisor of the system that evaluates and selectsthe sensory and motor information that is used at every moment. Moreover, weground this argument on the Distributed Adaptive Control cognitive architectureand theory of brain and body [11].

The following section will describe why subjectivity is an important featureof any cognitive system, tightly coupled to embodiment. Section 3 will introducethe theory of DAC and how it has been applied into our robotic cognitive archi-tecture. Finally, Sect. 4 will detail the conclusions of previous discussion andpropose future lines for improvement.

2 The Subjective Lens

All embodied agents are given sensors and actuators by default. This sensorsand actuators suffer of physical constraints, what will define, in its essence, therobot capabilities. On top of this, in order to behave autonomously, an agentmust have internal needs or drives that will define its general behavioral trends.We consider such needs as a characteristic of our robots and so, we can scaleit to the next level and consider drives as the natural consequence of having aneed, triggering a specific behavior due to a specific sensation. In this case, wemust define sensations as predefined sensory processing of raw sensor data, andactions as predefined patterns of actuation. Later on, we will call this level ofdescription Reactive.

492 J.-Y. Puigbo et al.

We can imagine then, from an Adaptive level of description, perceptions assensory information that is acquired through learning or adaptive processes; andbehaviors as acquired and modulated action sequences. We argue that typicalrobotic approaches use the equivalent to reactive drives, without properly definedneeds as the connection between adaptive processes, whereas the adaptive learn-ing of drives and new sensory-motor associations based on internal needs andmotivations is usually bypassed. We predict that this bypass is the main rea-son why state of the art robots are unable to robustly self-assess failure. Missingabstractions and experience based representations of an agent’s own needs limitssignificantly the capacity to identify errors not foreseen by the designer.

Still, reinforcement learning has provided a framework capable of dealingwith this problem. On the seminal work presented in [6], reinforcement learningwas used as an adaptive process to learn sensory-motor associations to obtainthe maximum possible score in a video-game. The drawback on their approachwas twofold. First, a reactive layer of behavior was limited to random actions,whereas the appropriate reactive loops tend to speed up learning. Second, theinternal needs of the agent were not carefully designed. The source of value inthe algorithm was the scored obtained during game play and the mechanismto predict this reward. Choosing an inappropriate value measurement leads toundesired behavior and exploitation of the available rewards. The combinationof both leaded to very low performances in games that had scarcity of rewardand thus also required better exploration (reactive) mechanisms.

One of the arguments provided to justify the failures in some of the games,was the absence of a memory system. This brings us to a third level of descrip-tion: Contextual. We can consider memory as a source of contextual informationthat is recalled under specific sensory, motor or internal conditions. In this sense,memory is the contextual version of sensory information, the motor equivalentcould be understood as action plans and the subjective counterpart, the goals.We propose that, as resources become scarce and dangers become abundant,adaptive behavior is insufficient to ensure good performance. This can then besolved by the above mentioned contextual features: memory, planning and goals.Just one of them might be sufficient only in a limited set of tasks. In this sense,goals play again an important role in relating memories to form plans, allow-ing the acquisition of relevant memories and consequences from experience. Asgreater becomes the number of neurons in pallial regions of the brain, evolu-tion has facilitated longer periods of guided experience after birth. We considerguided experience as the parental or social tutoring and the care off the offspringof an animal species. In these cases, guided experience reduces the ratio of dangerand reward and facilitates learning through adaptive mechanisms. Nonetheless,contextual mechanisms serve the purpose of identifying, selecting, storing andrecalling relevant information, where a goal oriented system would be in chargeof regulating these processes.

Towards Self-controlled Robots Through Distributed Adaptive Control 493

3 From DAC to WR-DAC

3.1 Distributed Adaptive Control

We grounded this work on DAC theory of mind (Fig. 1), with the objective toconstruct a complete cognitive architecture that provides autonomy and cog-nitive abilities to nowadays robots. DAC is a theory of mind, brain and bodynexus born in 1992, that has been gaining momentum in the past few years.DAC proposed the separation of cognitive processes in 2 dimensions. The first,separates cognitive processes in 3 columns: World, Self and Action.

World refers to all the sensory inputs that can be extracted from the world,from sensors themselves to stored memories.

Action is understood then as the column composed by all effecting mechanisms,from the simple actuators to the more abstract action plans.

Self is interpreted as all internal mechanisms, from needs, to goals through valueassessment and attentional processes.

Fig. 1. Distributed Adaptive Control theoretical framework. On the left, the theoret-ical description of DAC is presented as described in the text [11]. On the right, theimplementation of of the allostatic controller [10] (bottom) and the memory formation[4] (top). This figure was adapted from [11]

494 J.-Y. Puigbo et al.

The second dimension describes cognitive elements in a layered, composi-tional structure.

Somatic: The somatic layer is formed by all sensors, actuators and internalneeds given to the robot. This layer can be understood as the physicalinstance of the robot body.

Reactive: The reactive layer is then the first predefined loop for generatingbehavior that uses minimal sensory processing to trigger simple actions drivenby the internal needs.

Adaptive: The adaptive layer provides the substrate for learning at the threecolumns. With this, an agent with an adaptive layer is able to change itsbehavioral loops according to environmental changes, acquiring new abstrac-tions from sensory data and generating precise and adapted actions antici-pating value functions.

Contextual: The contextual layer provides the substrate for memory, reasoning,planning and high level cognitive function.

DAC provides a framework that allows bottom-up generation of more com-plex and meaningful behavior, as an agent scales up from layer to layer. DACalso states that these levels of description provide the sufficient functionality foran agent to effectively survive in the world and interact with agents. It is forthis reason that we chose to implement this framework as an architecture forautonomous robot self-control.

3.2 R-DAC

In contrast to other implementations of DAC along its history, Robotese-DACis devised as the first attempt to provide an easy-to-use architecture, platformand task independent that provides robots or artificial agents with long termautonomous behavior. In order to build such architecture, careful attention waspaid to the design of the self column. We implemented the first prototype on aniCub robot using theYARPmiddleware [5] formodule facilitating communication.

At the somatic layer, the robot has 32 degrees of freedom, two cameras forstereo vision, capacitive skin and a Kinnect sensor. The needs are then definedby a comfort zone contained within a top and a bottom thresholds, a valuein the range between 0 and 1 and a decay parameter. These parameters aredefined in the homeostasisManager module, called after homeostasis, a termused to describe the tendency of life-beings to keep internal values within aspecific range or comfort zone. homeostasisManager, at the somatic level,is in charge just to keep track of the dynamics of the needs, independently ofany other external module, and streaming their information into communicationports, for any interested subscriber. The homeostaticManager is based onprevious work in [10].

At the reactive layer, an allostaticController has been built that con-nects pre-processed inputs and pre-defined behaviors with the homeostatic man-ager. At this level, reactions are triggered automatically when the value of some

Towards Self-controlled Robots Through Distributed Adaptive Control 495

homeostatic need goes over or below a specific threshold. We call this connec-tion a drive drive to perform an action under specific sensory cues. Additionally,the allostatic controller provides additional functionality to define how sensoryinput will affect the dynamics of a specific need, which will still be computed inparallel in homesostaticManager.

As an adaptive approach, the allostaticController adds to the reac-tive function methods for self-regulation of multiple drives, in accordance withthe name of the module. In this sense, active drives inhibit other drives fromactivating at the same time, while providing an interface for top-down regulationof drive dynamics. To become more adaptive, additional work is being done inthe abstraction of new drives from old ones, what would lead to phenomenonslike anticipation and modulation of the basic original ones.

Finally, the third loop is closed in the contextual layer with goal-orientedbehavior. We propose with a first implementation to use goals as top-down mod-ulators of the original drives. Top-down influence is achieved two-fold: from oneside, a proper goal oriented module must be able to select what is the sensory

Fig. 2. Block description of WR-DAC. The DAC modules correspond to the boxesin the classical DAC diagram, with its columnar and layered distribution. These areusually wrappers for more complex sensory or motor modules or fully functional controlmodules dependent on inputs from sensations and behaviors.

496 J.-Y. Puigbo et al.

information the agent must pay attention to; from the other side, it must be ableto trigger behaviors and restrict other drives dynamics in order to gain priorityover the other drives. The current implementation of the module includes justthe last feature and already provides the necessary components to fulfill goalsfrom past experiences stored in an AutoBiographical Memory (ABM) [7], whiletriggering actions from action plans generated by the system presented in [4],although the goal generation is still human driven.

In order to complete the architecture, the sensory and motor componentsmust also be included. In this case, we created a wrapper that allows the inclu-sion of pre-processed sensations and pre-programmed behaviors as direct inputsand outputs to the allostaticController. The sensatiuonsManageris thought to contain modules that read information from sensory sources andpublish both: contextual information for other behavioral modules, and drivinginformation that directly modifies the drives dynamics. The behaviorManagerreceives triggering commands through a port, that can be sent through bothallostaticController or goalsManager or any other module, while read-ing the relevant contextual information from the corresponding sensations port.The two wrappers are created on demand by the developer, whereas they pro-vide a modular and configurable framework to use just the necessary behaviorsand sensations from a large set, and connect them either to the self system or toany other general control module. This provides an architecture for autonomouscontrol that can also easily be hijacked for Wizard of Oz or more controlledsetups. See Fig. 2 for a detailed description of the architecture.

4 Conclusions and Future Work

A cognitive architecture has been proposed to generate robust autonomousbehavior in robots and artificial agents. The architecture is a specific roboticoriented implementation of the DAC architecture, grounded on the sciences ofbiology and psychology. The proposed architecture differs from previous imple-mentations by directly addressing the concept of self, self-generated behaviorand self-interpretation of sensory information. It also differs from other cogni-tive architectures like SOAR or ACT-R by using also bottom-up instead of justtop-down approaches to model cognitive function.

Our architecture proposes the use of self as the center for controlling andmonitoring behavior, as opposed to engineering control algorithms for each con-ceivable situation. The current architecture exhibits robust, autonomous, contin-uous behavior, first demonstrated on a first version of the reactive layer in [3] andextended to pure control in [8]. Still, is the last that highlights the importanceof adding adaptive mechanisms on triggering behaviors, as this, implemented inlarge scale set of behaviors, should lead to behavior scheduling in order to keepa general state of comfort. Additional work is being directed to the generationof new drives grounded on neuron dynamics on the limbic system, as well asanticipatory behaviors observed in conditioning experiments.

Special emphasis is being put in the contextual layer, that has the mainframework ready, missing the mechanisms for the internal generation of goals,

Towards Self-controlled Robots Through Distributed Adaptive Control 497

as now, can only be provided by a human being. The ability to use goals andidealized future states of the world has been shown to add robustness in presenceof failure [9]. We are extending the architecture to have a working memory systemthat can be used it to compare desired and actual states of the world.

References

1. Anderson, J.R., Matessa, M., Lebiere, C.: ACT-R: a theory of higher level cognitionand its relation to visual attention. Hum.-Comput. Interact. 12(4), 439–462 (1997)

2. Laird, J.E., Newell, A., Rosenbloom, P.S.: Soar: an architecture for general intel-ligence. Artif. Intell. 33(1), 1–64 (1987)

3. Lallee, S., et al.: Towards the synthetic self: making others perceive me as an other.Paladyn, J. Behav. Robot. 6(1) (2015)

4. Marcos, E., Ringwald, M., Duff, Y., Sanchez-Fibla, M., Verschure, P.F.M.J.: Thehierarchical accumulation of knowledge in the distributed adaptive control archi-tecture. In: Baldassarre, G., Mirolli, M. (eds.) Computational and Robotic Modelsof the Hierarchical Organization of Behavior, pp. 213–234. Springer, Heidelberg(2013)

5. Metta, G., Fitzpatrick, P., Natale, L.: YARP: yet another robot platform. Int. J.Adv. Robot. Syst. 3(1), 43–48 (2006)

6. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature518(7540), 529–533 (2015)

7. Petit, M., Fischer, T., Demiris, Y.: Lifelong Augmentation of Multi-Modal Stream-ing Autobiographical Memories (2015)

8. Puigbo, J.-Y., Herreros, I., Moulin-Frier, C., Verschure, P.F.M.J.: Towards a two-phase model of sensor and motor learning. In: Wilson, S.P., Verschure, P.F.M.J.,Mura, A., Prescott, T.J. (eds.) Living Machines 2015. LNCS, vol. 9222, pp. 453–460. Springer, Heidelberg (2015)

9. Puigbo, J.-Y., et al.: Using a cognitive architecture for general purpose servicerobot control. Connection Sci. 27(2), 105–117 (2015)

10. Sanchez-Fibla, M., et al.: Allostatic control for robot behavior regulation: a com-parative rodent-robot study. Adv. Complex Syst. 13(03), 377–403 (2010)

11. Verschure, P.F., Pennartz, C.M., Pezzulo, G.: The why, what, where, when andhow of goal-directed choice: neuronal and computational principles. Phil. Trans.R. Soc. B 369(1655) (2014)

Discrimination-Based Perception for Robot Touch

Emma Roscow1,4(✉), Christopher Kent2,4, Ute Leonards2,4, and Nathan F. Lepora3,4

1 School of Physiology, Pharmacology and Neuroscience, University of Bristol, Bristol, UKemma.roscow@bristol.ac.uk

2 School of Experimental Psychology, University of Bristol, Bristol, UK3 Department of Engineering Mathematics, University of Bristol, Bristol, UK

4 Bristol Robotics Laboratory (BRL), University of Bristol, Bristol, UK

Abstract. Biomimetic tactile sensors often need a large amount of training todistinguish between a large number of different classes of stimuli. But whenstimuli vary in one continuous property such as sharpness, it is possible to reducetraining by using a discrimination approach rather than a classification approach.By presenting a biomimetic tactile sensing device, the TacTip, with a singleexemplar of edge sharpness, the sensor was able to discriminate between unseenstimuli by comparing them to the trained exemplar. This technique reducestraining time and may lead to more biologically relevant models of perceptuallearning and discrimination.

Keywords: Robotics · Psychophysics · Tactile sensing · Biomimetics

1 Introduction

Almost all animals rely on tactile perception in order to successfully move around andinteract with their environment: humans primarily use their hands, rats depend signifi‐cantly on their whiskers, and cockroaches rely on their antennae. Many of these varioustactile systems that exist in nature are now being used as inspiration in robot touch, witha range of tactile sensors available based on biologically-inspired principles [1].

But as demands are placed on these sensors to distinguish ever smaller differencesin stimulus properties, they often require a very large, cumbersome amount of trainingto accurately discriminate between very similar classes. To make the training processmore efficient, here we conjecture that when stimuli vary continuously over just oneproperty (e.g. size or curvature or angle), the sensor can be trained on a single exemplarand subsequently a new previously unencountered stimulus can be compared with theexemplar. In tasks where there might be dozens of different stimuli, this new approachcould reduce training time significantly. Moreover, this is likely to be a more biomimeticmethod of training, as humans can easily perceive object properties on continuous scalesand make relative judgements about stimulus properties (e.g. bigger or smaller).

To test this conjecture we use the TacTip [2], a biologically-inspired optical tactilesensor designed to mimic responses to skin deformation in human fingertips, which hasbeen shown to perform very well on a range of tactile perception and identification tasks[3–5]. Here we demonstrate that after training on just one exemplar of a stimulus

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 498–502, 2016.DOI: 10.1007/978-3-319-42417-0_53

property (in this case, edge sharpness) the TacTip can generalise this property anddistinguish between new, unseen stimuli that vary in sharpness with a level of sensitivitysimilar to human performance.

2 Methods

The TacTip is a biomimetic tactile sensor with a 3D-printed, hemispherical, compliantsilicon outer membrane (the ‘skin’) with a diameter of 40 mm. 127 white-tipped pins(‘papillae’) cover the inside of the membrane in a hexagonal arrangement, such thatwhen the TacTip comes into contact with an object the outer membrane is deformed andthe pins are displaced (see Fig. 2). The membrane is fixed to a base with 6 LED lightswhich illuminate the white pins. A Microsoft Cinema HD webcam is mounted onto itwhich tracks the displacement of the pins, collecting tactile data in the form of imageswithin the base (resolution 640 × 480 pixels, sampling rate 15 frames/sec), withcomputer vision techniques used to track the displacement of the pins [4].

Fig. 1. The TacTip mounted on a robot arm and positioned above one of the stimuli. Datacollection consisted of downward taps made onto the top of the stimuli. The apex angle of thestimuli varied from 35° to 85°.

Discrimination-Based Perception for Robot Touch 499

We 3D-printed 11 triangular stimuli with apex angles ranging from 35° to 85° in 5°increments (see Fig. 1). The TacTip was mounted on a six-degree-of-freedom robot armwhich allowed it to be brought into contact with each stimulus. Training consisted of asingle tap moving 5 mm down onto the top edge of the sharpest stimulus (35°), and5 mm back up. The displacement of each pin during the tap was measured in twodimensions to give 254 data dimensions. Log likelihoods were then obtained using ahistogram method, whereby the sensor values for each data dimension were binned into100 bins with equal intervals [6].

Fig. 2. An illustration of the displacement of the pins in the TacTip when in contact with astimulus. Left: the flexible skin of the TacTip deforms around the stimulus, which displaces thepins. The camera is mounted directly above the pins to track this movement. Right: thedisplacement of the pins as captured by the camera, while in contact with a horizontal edge.

Testing consisted of trials in which the TacTip made a single tap sequentially ontoa selected pair stimuli (any of the 11) and produced two log likelihoods, one for eachstimulus: the probability of being identical to the exemplar (E) given the data from thefirst stimulus (S1), , and the probability of being identical to the exemplar giventhe data from the second stimulus (S2), .

The ratio of these two likelihoods was calculated such that a ratio less than 1 wastaken to indicate that the first stimulus was sharper and a ratio greater than 1 indicatedthat the second stimulus was sharper (Eq. 1).

(1)

Every stimulus was compared against every other stimulus 10 times, giving 550 trialsin total. After each trial, the model’s judgement and the correctness of that judgementwere recorded.

500 E. Roscow et al.

3 Results

On each trial, a pair of stimuli were tested and compared to the exemplar. A log likeli‐hood ratio of their similarity to the exemplar was obtained for each pair, and whicheverstimulus elicited the higher log likelihood was judged to be sharper. To assess themodel’s performance, we recorded the percentage of trials on which it gave the correctjudgement when pairs of stimuli differed by different amounts, and fitted a logarithmiccurve to the data, including the upper and lower 95 % confidence bounds (Fig. 3).

The just noticeable difference (JND) was calculated as the sharpness difference atwhich the TacTip made the correct judgement on 75 % of trials. (75 % is midpoint betweenthe level of chance, 50 %, and perfect performance, 100 %, and is commonly used as thediscrimination threshold in human psychophysics.) For the log curve, the JND was 9.2°,with the upper and lower bounds giving JNDs of 6.5° and 11.8° respectively. There wasan apparent anomaly at a difference of 25° which fell below the curve of best fit, but webelieve this to be merely an artefact due to noise associated with the experimental set-up.

Fig. 3. Results from training on a single exemplar. The red line represents the (actual) percentageof trials on which the TacTip gave the correct judgement for each sharpness difference. Error barsrepresent standard error of the mean. The blue lines represent the log curves that best fit the data;the solid blue line is the single model that best fits and the dotted lines are the upper and lower95 % confidence bounds. (Color figure online)

Discrimination-Based Perception for Robot Touch 501

4 Discussion

The results showed that after a single tap on a single training exemplar, the TacTip wassensitive to differences in angle sharpness of 9.2°, which compares to a JND of 8.6° thathas previously been found in humans performing the same task after years of tactileexperience with edges [7]. To our knowledge this exemplar-generalisation method oftraining has not been employed before, and holds promise for two reasons. First, it candramatically cut the amount of training required compared to current alternative tech‐niques such as classification that require training on a large number of separate classes.Second, as generalisation is a fundamental part of human (and other animal) learning itcan be used to develop more realistic models of animal perception. Humans are remark‐ably more sensitive to relative differences in sensory information than absolute valuesor classes [8], which suggests that learning through exemplar-generalisation may be ableto produce more biologically realistic perceptual discrimination than other kinds oflearning. We intend to explore this in the future by using this training technique to modelhuman perceptual decision-making and confidence.

Acknowledgements. We thank Benjamin Ward-Cherrier and Luke Cramphorn for their helpsetting up the TacTip and stimuli. This work was supported by a Wellcome Trust PhD studentship.

References

1. Grant, R.A., Itskov, P.M., Towal, R.B., Prescott, T.J.: Active touch sensing: finger tips,whiskers, and antennae. Front. Behav. Neurosci. 8, 50 (2014)

2. Chorley, C., Melhuish, C., Pipe, T., Rossiter, J.: Development of a tactile sensor based onbiologically inspired edge encoding. Design (2008)

3. Lepora, N.F., Ward-Cherrier, B.: Tactile quality control with biomimetic active touch. IEEERobot. Autom. Lett. 1(2), 646–652 (2016)

4. Lepora, N.F., Ward-Cherrier, B.: Superresolution with an optical tactile sensor. In: IEEE/RSJon Intelligent Robots and Systems (IROS), pp. 2686–2691 (2015)

5. Winstone, B., Griffiths, G., Pipe, T., Melhuish, C., Rossiter, J.: TACTIP - tactile fingertipdevice, texture analysis through optical tracking of skin features. In: Lepora, N.F., Mura, A.,Krapp, H.G., Verschure, P.F.M.J., Prescott, T.J. (eds.) Living Machines 2013. LNCS, vol.8064, pp. 323–334. Springer, Heidelberg (2013)

6. Lepora, N.F., Fox, C., Evans, M., Diamond, M., Gurney, K., Prescott, T.: Optimal decision-making in mammals: Insights from a robot study of rodent texture discrimination. Roy. Soc.Interface 9(72), 1517–1528 (2012)

7. Skinner, A.L., Kent, C., Rossiter, J.M., Benton, C.P., Groen, M.G.M., Noyes, J.M.: On theedge: haptic discrimination of edge sharpness. PLoS One 8, e73283 (2013)

8. Fechner, G.: Elements of Psychophysics. Rinehart and Winston, New York (1966)

502 E. Roscow et al.

On Rock-and-Roll Effect of QuadrupedLocomotion: From Mechanical

and Control-Theoretical Viewpoints

Ryoichi Kuratani(B), Masato Ishikawa, and Yasuhiro Sugimoto

Osaka University, 2-1, Yamadaoka, Suita, Osaka, Japankuratani@eom.mech.eng.osaka-u.ac.jp

http://www-eom.mech.eng.osaka-u.ac.jp/web/

Abstract. In this paper, we discuss body-induced motion of quadrapedwalking, in the presence of rolling contact between its feet and theground. This paper is based on the authors’ previous study, where wehad shown that active “rocking” of the torso from the left to the right(in the lateral plane) would induce alternative swing of the legs (in thesagittal plane), which result in autonomous forward walking. Now inthis paper, we focus on the influence of the rolling contact of the feetand the ground; we point out that the forwading displacement is thesum of “walking effect” and “rolling effect”, and the rolling effect can beobserved even when each pair of the fore legs and the hind legs are boundtogether. Moreover, in order to extract pure rolling motion, we analyzein detail the model of two hemispheres connected via a spine rod, as anextremely reduced model of quadruped locomotion.

Keywords: Quasi-passive dynamic walking · Rolling locomotion · Non-holonomic system

1 Introduction

Body motion plays an impotant role in quadruped locomotion. Animals withrelatively large-sized torso, such as mammals or reptiles, often exhibit quadrupedwalking associated with synchronized body oscillation.

In particular, the authors have been focusing on the lateral rocking of thebody, namely, swaying motion of the body from the left to the right. The lateralrocking can be either the cause and the result; in the authors’ previous work onpassive-dynamic quadruped walking [4], we developed a quadruped mechanismwithout any actuator or controller Fig. 1(a), which automatically walks down theslope while the only source of the power was the gravity. The key observation herewas that lateral rocking of the body is also automatically induced, with a certainphase gap between the foreleg and the hindleg parts, and the same frequency asthat of the walking in the sagittal plane. In the succeeding work, to exchange thecause and the result, we developed a quasi-passive quadruped walking robot [3].

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 503–509, 2016.DOI: 10.1007/978-3-319-42417-0 54

504 R. Kuratani et al.

The robot has two active pendulums, one of which is equipped with the fore-part and the other is equipped with the rear part Fig. 1(b). By actuating thependulums with an appropriate phase gap, we succeeded in exciting the lateralrocking, and resulting walking locomotion (on an even surface) as well. Throughthese works we showed that “walking induces the body rocking” and “the bodyrocking induces walking”.

Now we notice that the feet play a crucial role in transforming the bodymotion to the leg motion, and vice versa. Indeed, the surface of the feet arehemispherical in the works above, i.e., curvature of the contact point of a footis always constant. When the body “rocks”, it starts to “roll” on the groundthanks to the hemispherical feet.

In this paper, we go into more detail of this rocking and rolling phenomena.First, we explain concept of two spheres vehicle model and derive the kinematicstate equation of that in Sect. 2. In Sect. 3, we show periodic control input ofRock-and-Roll locomotion and simulation results.

Fig. 1. Passive and Quasi-passive dynamic walking robot

2 Geometric Setting

2.1 Orientation and Internal Shape

In this subsection, we describe two spheres vehicle model. The sole shape ofFig. 2 is a part of sphere which radius is r, the connecting parts between legs andtwistable pole are designed to be same position with the center of each spheres,so this robot (Fig. 2) is regarded as two hemispheres connected by twistable pole(Fig. 3). The coordinate setting of proposal model is shown in Fig. 4.

The hemisphere means a half of sphere, when the hemisphere is between twoplanes, plane A includes the cross section of the hemisphere, plane B is parallelto plane A, the distance between A and B is r, then the contact point of thehemisphere and plane B is one point and we call it as bottom of the hemisphere in

On Rock-and-Roll Effect of Quadruped Locomotion 505

Fig. 2. Widening distance between legs (Phase difference α = π[rad])

this paper. In Fig. 4 point O is the bottom of the hemisphere, θi, ψi are angularvariables which show how each hemispheres are inline from the condition inwhich point O is contact with plane, φ is also an angular variable which showshow connecting pole rotate around z-axis. And P is the point in which the sphereis contact with the plane.

The shape of robot feet looks like squares as seen from above, which corre-sponds to the left model of Fig. 4, it is so called coordinate setting of two wheelsvehicle model in which the pole is connected to center of each wheels and inputis only revolution speed of wheels. (x1, y1) and (x2, y2) are the center of eachwheels, u1, u2 are inputs to revolution speed of each wheels. Wheels of the twowheels vehicle model are replaced with spheres in proposal model, so we callthis model as two spheres vehicle model. Throughout this paper, we assumethat spheres may roll over the floor, but never slip. All expressed in the inertialcoordinate.

Fig. 3. Model reduction: from the quadruped walker (left) to the two-sphere vehiclemodel (right)

2.2 Kinematic State Equation of Two Spheres Vehicle Model

Now we derive a reduced kinematic state equation of two spheres vehicle model.First, we consider the kinematic equation of two wheels model and rolling contactcondition. Then, we derive the kinematic equation of two spheres vehicle model.

506 R. Kuratani et al.

Fig. 4. Coordinate setting of the proposed model

From the coordinate setting of Fig. 4, we obtain the following kinematic equa-tions

x =u1 + u2

2cos φ, y =

u1 + u2

2sinφ, φ =

u2 − u1

l(1)

x1 = x − l

2φ cos φ, y1 = y − l

2φ sin φ, x2 = x +

l

2φ cos φ, y2 = y +

l

2φ sin φ(2)

We also consider rolling contact condition in Fig. 4, and the velocity of thesphere which center is (x1, y1, z1) is described as

⎛⎝ x1

y1z1

⎞⎠=

⎛⎝ cos (π

2 − φ) − sin (π2 − φ) 0

sin (π2 − φ) cos (π

2 − φ) 00 0 1

⎞⎠⎛⎝ rθ1

rψ1 cos θ10

⎞⎠ (3)

where r is the radius of the sphere [1]. Substituting (3) to (1) (2) we have{

u1 cos φ = rθ1 sin φ − rψ1 cos θ1 cos φ

u1 sin φ = rθ1 cos φ + rψ1 cos θ1 sin φ(4)

From these equations, we obtain

θ1 =S(2φ)

ru1 (5)

where S(x) = sinx,C(x) = cos(x). We derive following state equation of twospheres vehicle model from equations shown as above

On Rock-and-Roll Effect of Quadruped Locomotion 507

⎛⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝

xy

φ

θ1

ψ1

θ2

ψ2

⎞⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

=

⎛⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝

12Cφ12Sφ

− 1l

S(2φ)r1r

00

⎞⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

u1 +

⎛⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝

12Cφ12Sφ1l

00

S(2φ)r1r

⎞⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

u2 → ξ = g1(ξ)u1 + g2(ξ)u2 (6)

where ξ = (x, y, φ, θ1, ψ1, θ2, ψ2)T .When we consider the input on holonomy principle, we can understand that

the state transfers to the direction of Lie Bracket[g1, g2] of g1, g2 when withclosed curve input on the U1 − U2 (Ui is time integral of ui) plane.

3 Control of Rock-and-Roll Locomotion and SimulationResults

The Lie Bracket[g1, g2] of g1, g2 is calculated as

g3 := [g1, g2](ξ) =∂g2

∂ξg1 − ∂g1

∂ξg2 =

(Sφl ,−Cφ

l , 0,− 2C(2φ)rl , 0,− 2C(2φ)

rl , 0,)T

(7)

and we can understand φ does not change if we calculate higher order LieBracket. Then we assume φ is small enough, the g3 is

g3 =(

0,−Cφl , 0,− 2C(2φ)

rl , 0,− 2C(2φ)rl , 0,

)T

(8)

We realize that states transfer in y, θ1, θ2 direction with the closed curveinput on the U1 − U2 plane. One of the input is [2]

U1 = A sin (ωt) (9)U2 = A sin (ωt − α) (10)

As a result of simulation, the center point (x, y) on the x − y plane show thefollowing trajectory (Fig. 5(b)). Therefore, we verify that the input derived inthe previous section is available.

In the same condition, we obtain following result (Fig. 6) from simulation,where t is simulation time[s], α = π[rad] and other variables are same with Fig. 5.Figure 6 shows the same phenomenon confirmed in robot experience (Fig. 2),then it is conceivable that the trigger is phase difference in rocking motion cor-responding to α.

508 R. Kuratani et al.

Fig. 5. Simulation result of kinematic state equation under the periodic controls

Fig. 6. Stance-widening phenomenon: the distance between the two hemispherical feetincreases

4 Conclusion

This paper discussed influence of rocking of the body and rolling about the feet,motivated by quadruped walking. In particular, we proposed a new noholonomicmobile robot and built a new model to explain a phenomenon which could notbe given enough explanation by the model proposed in our previous work. Thesimulation result of the model showed the phenomenon can appear at almostsame condition with the real experiment, from which we found the trigger forthe phenomenon is the phase difference between each spheres oscillation. Inaddition, judging from the direction of the vector-field g3 the model will go downin forwarding motion. And this suggests, quadrupeds get their legs forward notto pull their body but to continue to stand.

In this paper, we have examined this rock-and-roll mechanism throughnumerical simulations. Future research would include quantitative analysis bycomparison between numerical simulation and physical experiments.

On Rock-and-Roll Effect of Quadruped Locomotion 509

References

1. Ishikawa, M., Kobayashi, Y., Kitayoshi, R., Sugie, T.: The surface walker: a hemi-spherical mobile robot with rolling contact constraints. In: IEEE/RSJ InternationalConference on Intelligent Robots and Systems, pp. 2446–2451 (2009)

2. Ishikawa, M., Minami, Y., Sugie, T.: Spherical rolling robot: a design and motionplanning studies. In: IEEE International Conference on Robotics and Automation,RiverCentre, pp. 9–16 (2010)

3. Kibayashi, T., Sugimoto, Y., Ishikawa, M., Osuka, K., Sankai, Y.: Experiment andanalysis of quadrupedal quasi-passive dynamic walking robot Duke. In: InternationalConference on Intelligent Robots and Systems, pp. 299–304 (2012)

4. Sugimoto, Y., Yoshioka, H., Osuka, K.: Realization and motion analysis of multi-legged passive dynamic walking. In: SICE Annual Conference, pp. 2790–2793 (2010)

Hydromast: A Bioinspired Flow Sensorwith Accelerometers

Asko Ristolainen1(✉), Jeffrey Andrew Tuhtan1, Alar Kuusik2, and Maarja Kruusmaa1

1 Centre for Biorobotics, Tallinn University of Technology,Akadeemia tee 15a-111, 12618 Tallinn, Estonia

{asko.ristolainen,jeffrey.tuhtan,Maarja.kruusmaa}@ttu.ee2 Eliko Competence Centre, Mäealuse 2/1, 12618 Tallinn, Estonia

alar.kuusik@eliko.ee

Abstract. Fish have developed advanced hydrodynamic sensing capabilitiesusing neuromasts, a series of collocated inertial sensors distributed over theirbody. We have developed the hydromast, an upscaled version of this sensingmodality in order to facilitate near bed sensing for aquatic systems. Here weintroduce the concept behind this bioinspired flow sensing device as well as thefirst results from laboratory investigations.

Keywords: Hydromast · Biomimetic · Flow sensing

1 Introduction

Fish experience the surrounding flow using their lateral line and inner-ear sense organs.The lateral line organs consist of linear arrays of neuromasts on the head and along thebody, sensitive to changes in the spatial derivatives of the flow field. Neuromasts consistof gelatinous cupula whose deflections correspond to the local motions of the fluid,relative to the motion of the body. Superficial neuromasts located at the surface aresensitive to local changes in the velocity, whereas canal neuromasts, embedded withinsubcutaneous canals connected by pores to the surrounding flow, act primarily as accel‐eration detectors. The inner ear responds to changes in the speed of the fish, acting aslinear and angular accelerometers, and is sensitive to the average bulk motion of thefish’s body within the surrounding flow field [1].

Natural flow fields, such as those occurring in rivers and coastal waters can bedescribed as a complex amalgamation of velocity, vorticity and pressure terms, actingover a wide range of spatiotemporal scales [3]. As such, it is not possible to directlyrecord the complete physical evolution of the flow field with reference to these terms.Instead, our aim is to extract salient, hydrodynamically-relevant features such as thenear-bed bulk flow velocity and bed shear stresses by utilizing the fluid-body interactionin natural flow and laboratory investigations. Measurement data can be analysed in boththe time and frequency domains to provide a new source of physically-based hydrody‐namic measurements. We have shown in previous works that bioinspired measurementsbased on a fish-shaped lateral line probe consisting of a collocated pressure sensing arrayis capable of calibrated bulk velocity estimation in a limited range (0–0.5) m/s, including

© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 510–517, 2016.DOI: 10.1007/978-3-319-42417-0_55

large (up to 90˚) angular deviations of the sensor body with respect to the freestreamflow [4, 5]. Here we introduce a new hydromast device, inspired by the single superficialneuromasts of fish’s lateral line for turbulent, flume experiments. The hydromast can beviewed as a bioinspired upscaled inertial measurement device tailored for near-bedstudies of complex in-situ flows which are often difficult or impossible obtain usingconventional approaches such as acoustic Doppler current profiling (Fig. 1).

Fig. 1. The lateral line sensory apparatus of a fish, consisting of superficial neuromasts (green)and subdermal canal neuromasts (orange). Illustration after [2]. (Color figure online)

Fishes have the capability to sense local hydrodynamic stimuli close to their bodiesusing the lateral line modality. We show that an artificial inertial neuromast can bereconstructed and the base biological design can be exploited to measure flow charac‐teristics with accelerometers.

2 Design Concept of the Artificial Neuromast Sensorwith Accelerometers

The first approximation to create an upscaled superficial neuromast has been made witha stiff plastic stem of circular cross-section having a 10 mm diameter (see Fig. 2). Thestem was fixed onto a silicone membrane (casted from Elite Double 22, Zhermack SpA),which acts similarly to the gelatinous cupula, as a spring element allowing the stem topivot in the water flow. The stem was fixed trough the membrane with a bolt onto aclamp that held an accelerometer with 10 × 10 mm printed circuit board. The membranewas fixed between two surfaces in the casing (see Fig. 2).

To remove offsets caused by any slight misalignment of the sensor prototype in theflume, and to measure the stem tilt angles in the casing frame we installed a secondstationary accelerometer to the casing close to the moving accelerometer (see Fig. 2).The difference between the base and stem angles from accelerometers thus provides anoutput signal largely free from any body self-motion due to mounting.

Accelerometers were interfaced with 32-bit ARM microcontroller (ST Microelec‐tronics) over 400 kbps I2C bus. We use the earth gravity vector and arctangent functionto calculate the sensor inclination in X, Y coordinates. It is assumed that sensor does

Hydromast: A Bioinspired Flow Sensor with Accelerometers 511

not rotate about the Z axis, and the random acceleration noise caused by water turbu‐lences can be eliminated through the time averaging of the output signal. According tothe experimental results more sophisticated sensor inclination measurement methods donot provide large benefits in accuracy over the simple method described. This is becausethe inertial and magnetic sensor fusion algorithms do not perform well when highfrequency acceleration is present.

The accelerometers were connected to the ARM microcontroller using 4 thread thinear phone wires to minimize disturbance due to wire movement and bending. The accel‐erometers were coated with Plasti Dip plastic (Plasti Dip International) to make themwaterproof.

3 Design Evaluation and First Results

3.1 Experimental Setup

We tested the sensor in a flow tunnel with a working section of 0.5 m × 0.5 m × 1.5 membedded into a test tank (see [6] for the test tank description). Uniform flow (constantwater surface, no change in flowrate over time) in the working section was created withthe help of a U-shaped flow straightener and two sequential collimators. An AC motorwas used to create the circulation inside the flow tunnel, and the flow speed was cali‐brated using a digital particle image velocimetry system.

The sensor was fixed onto a metallic plate which was placed in the middle of thebottom in the flow tunnel. We ran experiments at 0.05 m/s speed increments, withmaximum speed of 0.5 m/s. The flow was let to stabilize for 30 s and the data was loggedfor 30 s on each speed.

Fig. 2. (a) Schematic of the hydromast prototype and (b) image of the sensor in the flow tunnel

512 A. Ristolainen et al.

3.2 Calibration Results

The calibration results showed that the hydromast with the chosen parameters (stemheight, diameter, membrane thickness and elasticity) has threshold shift in its behaviourabove speeds of the 0.25 m/s. The root mean square (RMS) of the hydromast’s stempivot angles in the water flow (offsets were removed using the data recorded by thesecondary accelerometer fixed to the casing from 10 experiments) are plotted in Fig. 3.

Fig. 3. RMS of the stem tilt angle along the flow direction with removed initial offsets and RMSof the base

At to velocities of up to 0.25 m/s, the hydromast’s relation to flow speed was quad‐ratic (R2 = 0.9987) whereas from speeds 0.25 m/s to 0.5 m/s the hydromast respondedlinearly to increasing flow speed (R2 = 0.9913). This shift is due to the transition toturbulence over the body, where at velocities > 0.25 m/s, larger vortices are suddenlyshed from the stem surface, creating a new fluid-body interaction between the skin drag,vortex-induced drag, buoyancy and restoring force by the membrane creating a newfluid-body interaction. The dynamic balance of forces causes a lock-in of the oscillatorymotion of the hydromast particular to each flow velocity. This dynamical loading resultsfrom the interplay of fluid drag and restoring forces driven by the hydromast’s positivebuoyancy in conjunction with the membrane stiffness.

The frequency spectra analysis with fast-Fourier-transform (FFT) from the concaten‐ated signal of 10 experiments showed that distinguishable frequencies appear aboveflow speeds of 0.3 m/s. The experimental data from 10 runs was concatenated to analyseall of the runs as one single dataset. The results from the concatenated data comparedwith the single runs data showed no significant difference in the FFT spectrum. Beforethe FFT was performed the casing vibrations were removed from the stem signal bysubtracting the casing accelerometer signal from the stem accelerometer signal. Thesignal was then filtered using median filter and linear trends were removed. Thefrequency spectra are plotted on Fig. 4.

Hydromast: A Bioinspired Flow Sensor with Accelerometers 513

Fig. 4. FFT frequency spectra from the stem-casing signals for flow speeds from 0–0.5 m/s in0.05 m/s intervals. Peaks arise due to the transition from laminar to turbulent flow.

The mean amplitudes of the frequencies increase with the speed. From the FFT wecalculated frequency spectra mean amplitudes which are plotted against the flow speedin Fig. 5. The relation of the mean frequency amplitudes and flow speeds resulted in aquadratic relation with R2 = 0.9913). This trend is due to the size and frequency ofvortices shed from the cylinder body, with increasing velocity the vortices become largerand are shed at a proportionally higher rate increasing the force imbalance on the stemand creating larger magnitude fluctuations.

The two flow regimes of the hydromast can also be noted in the standard deviation(STD) values of the vibrations. The stem’s vibration standard deviation along flow isincreasing around 0.25 m/s with corresponding Reynolds number over 2000. The STDvalues are plotted of the mean stem angles in the flow are plotted in Fig. 6.

Fig. 5. Mean frequency amplitudes from the stem-casing signal on flow speeds 0–0.5 m/s.

514 A. Ristolainen et al.

Fig. 6. RMS of the stem angle along the flow corresponding Reynolds numbers and standarddeviations, where the highest STD are found during the unstable transition from laminar toturbulent flow (0.25 and 0.30 m/s).

4 Discussion

In contrast to previous work based on bioinspired mammalian tactile whiskers [7] whosesensing ability is based on active detection, relying on the purposeful motion of thesensor within the fluid, our device is based on measurements using passive hydrome‐chanical detection. Passive sensing based on the lateral line sensing modality of fish hasthe added advantage that knowledge of the instantaneous displacement of the sensorbody, relative to the flow, can be directly related to the frequency, amplitude and phaseof the flow stimulus [8].

The change in the hydromast’s behaviour can be related to the change from near-laminar flow to fully turbulent flow. With a 9.8 mm cylindrical stem the Reynoldsnumbers indicate that around speeds 0.20 to 0.30 m/s, the flow transition occurs fromnear-laminar to turbulent (with corresponding Reynolds numbers Re = 1952 toRe = 2928). At the Reynolds numbers between 2000 and 4000 the flow around thecylindrical stem is transient, the flow is therefore in between near-laminar and fullyturbulent.

The change from near-laminar to fully turbulent flow region can well be seen in theFFT graphs (see Fig. 4) where distinct frequencies appear at 0.3 m/s. The turbulencecan be also noted in the standard deviations of stem vibrations. There is a larger increaseof the standard deviation around the speed of 0.25 m/s (see Fig. 6).

The flow speed from hydromast’s readings could be detected in two ways. Onesolution is to use the mean angle of the stem in the flow to estimate flow speed from thesensors. The emergence of clearly distinguishable frequency peaks from the FFT thentells us whether we need to use the linear or quadratic regression of the mean angle ofthe stem to calculate the flow speed. A second, and complimentary solution is to use the

Hydromast: A Bioinspired Flow Sensor with Accelerometers 515

quadratic relation between mean amplitude of the frequency spectra of the stem and theflow speed. The second method has been shown to work with pressure sensors, even athigher velocities (> 1 m/s) and extreme turbulence [5]. The combination of two tech‐niques would possibly give better results.

The hydromast’s bilateral reaction to flow speed allows us to design sensors differ‐ently for low and high flow speeds. With the current prototype, if we know the charac‐teristic curves of the sensor, we can use the sensor both in laminar and turbulent flowregimes. The sensors membrane and stem properties can be tuned as key parameters inorder to define sensor performance criteria.

By altering the hydromast’s membrane dimensions or elasticity, it would be possibleto increase or decrease the sensibility of the sensor at low laminar flow speeds. Withsofter membrane material the hydromast would bend more in the laminar and near-laminar flow regimes where the drag forces are caused largely due to skin friction. In thefully turbulent flow region, the softer membrane would allow the light hydromast’s stemcatch the turbulent wake frequencies better but on the other hand there is also a higherchange that the turbulences would match the natural frequency of the moving stem.

The dimensions of the hydromast’s stem affect the speed at which the flow aroundthe stems reaches the transition point from laminar to turbulent. Stems with smallerdiameters (and smaller Reynolds numbers) would reach the transition point at higherspeeds and vice versa in case of larger stem diameters. Thus, even though the fluid-bodyinteractions between the stem and the fully turbulent flow field are complex, the systemhas only a few simple parameters which can be tuned when developing neuromast-inspired flow sensing devices.

The best suitable sensor combinations will be parameterized in ongoing investiga‐tions where we take into account a wider diversity of conditions where the sensors willbe applied.

5 Conclusion

In confined laboratory experiments we have shown that a bioinspired hydromastequipped with accelerometers can be used to detect flow speeds up to 0.5 m/s. It ispossible to relate the flow speed to the hydromast stem motions both in the near-laminarand turbulent flow regions.

Future research will be focused on the study of how the hydromast behaves at higherflow speeds and how the sensibility is dependent on the hydromast stem and membraneproperties.

The areas of interest where we want to apply the hydromast sensors are related toboth freshwater (e.g. rivers) and salt water (e.g. sediment transportation on the seabed,current flow speed and direction monitoring in ports etc.) environments where near-bedflow sensing is often difficult, if not impossible to achieve.

The current work was done in the frame of Lakhsmi project (European Union’sHorizon 2020 research and innovation program under grant agreement No 635568,www.lakhsmi.eu).

516 A. Ristolainen et al.

References

1. Kalmijn, A.J.: Hydrodynamic and acoustic field detection. In: Atema, J., Fay, R.R., Popper,A.N., Tavolga, W.N. (eds.) Sensory Biology of Aquatic Animals, pp. 83–130. Springer,New York (1988)

2. McHenry, M.J., Liao, J.C.: The hydrodynamics of flow stimuli. In: Coombs, S., Bleckmann,H., Fay, R.R., Popper, A.N. (eds.) The Lateral Line System, pp. 73–98. Springer, New York(2013)

3. Davidson, P.: Turbulence: An Introduction For Scientists and Engineers. Oxford UniversityPress, USA (2015)

4. Strokina, N., Kämäräinen, J.-K., Tuhtan, J.A., Fuentes-Perez, J.F., Kruusmaa, M.: Jointestimation of bulk flow velocity and angle using a lateral line probe. IEEE Trans. Instrum.Meas. 1–13 (2015)

5. Fuentes-Pérez, J.F., Tuhtan, J.A., Carbonell-baeza, R., Musall, M., Toming, G., Muhammad,N., Kruusmaa, M.: Current velocity estimation using a lateral line probe. Ecol. Eng. 85,296–300 (2015)

6. Tuhtan, J.A., Toming, G., Ruuben, T., Kruusmaa, M.: A method to improve instationary forceerror estimates for undulatory swimmers. Underw. Technol. 33(3), 141–151 (2016)

7. Rooney, T., Pearson, M.J., Pipe, T.: Measuring the local viscosity and velocity of fluids usinga biomimetic tactile whisker. In: Wilson, S.P., Verschure, P.F., Mura, A., Prescott, T.J. (eds.)Living Machines 2015. LNCS, vol. 9222, pp. 75–85. Springer, Heidelberg (2015)

8. van Netten, S., McHenry, M.: The biophysics of the fish lateral line. In: Coombs, S.,Bleckmann, H., Fay, R.R., Popper, A.N. (eds.) The Lateral Line System, vol. 48, pp. 99–119.Springer, New York (2014)

Hydromast: A Bioinspired Flow Sensor with Accelerometers 517

Developing an Ecosystem for InteractiveElectronic Implants

Paul Strohmeier1(B), Cedric Honnet2, and Samppa von Cyborg3

1 University of Copenhagen, Copenhagen, Denmarkpaul.strohmeier@gmail.com

2 Carpe Noctem Labs, San Francisco, USAcedric@carpenoctem.cc

3 Von Cyborg Bodyart, Tampere, Finlandsampp@voncyb.org

Abstract. In this work in progress report we present Remora, a sys-tem for designing interactive subdermal devices. Remora builds uponmethods and technologies developed by body modification artists. Thedevelopment has so far focussed on battery and power management aswell as redundancy features. Remora consists of a series of hardwaremodules and a corresponding software environment. Remora is designedfor body modification artists to design their own implants, but will alsobe a platform for researchers interested in sub-dermal interaction andhybrid systems. We have so far implemented a prototype device; futurework will include in depth evaluation of this device as well as user stud-ies, a graphical development environment and additional hardware andsoftware modules.

Keywords: Implantable devices · Subcutaneous interaction · Human-machine hybrids

1 Introduction

There is growing interest in interacting with devices inside the body in the HumanComputer Interaction (HCI) research community, as well as in the Body Modifi-cation Community. Implanted devices are anticipated to provide opportunities forsensory augmentation, biometric data collection, data-storage or novel HCI appli-cations [1,2]. This growing interest can be observed looking atworkshops at leadingHCI conferences1, various self-experimentation projects, such as the Circadia byGrindhouse Wetware [3], or review papers on this very topic [4].

Implanted devices are not new - pacemakers and dental implants date backto the late 1950s and have since become established in medical practice. Com-mon medical implants include joint replacements, drug delivery systems, cerebralshunts, stents, cardiac monitors and pacemakers, and even brain implants thatprovide deep brain stimulation to Parkinson patients or treat seizures for epilepsy1 https://insertables.wordpress.com/.

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 518–525, 2016.DOI: 10.1007/978-3-319-42417-0 56

Initial Steps in Designing Interactive Electronic Implants 519

patients. Implants are also common for plastic surgery purposes, most notablythe breast implant, which dates back to the 1960s.

In parallel to these more established practices, the body modification com-munity has developed methods, practices and technologies of their own, includ-ing the development of subdermal implants. These are usually injection moldedsilicone objects which are implanted under the skin for aesthetic purposes. Themanufacturing process of these subdermal implants allows encapsulating objectsinside the implant. This enables implantation of unconventional objects, such asthe ashes of a loved one or electronic circuitry. There is a trend amongst self-identified grinders or wet-ware hackers to appropriate these methods for thedesign of implantable electronic devices [3,4].

Fig. 1. Remora prototype

Inspired by body modification culture, and in a somewhat worried reaction tothe design of devices implanted by grinders, we are designing Remora (Fig. 1), aneco-system for the development of implantable devices. This eco-system conformsto the constraints a body modification artist is faced with when producing andimplanting an electronic device. Remora is therefore not intended to improveupon, or even compare to the current state of the art in electronic implantablemedical devices.

Remora aims to provide a platform with multiple failsafe and redundancymechanisms around which to design interactive, subdermal devices. We antic-ipate this to find use in the body modification community, however we alsobelieve that various research communities will benefit from such a platform. Forexample, Remora would allow research into sub-cutanious sensing [1] to progresswithin living bodies, and enable novel human computer interaction methods andadvanced biometric monitoring.

Remora will consist of a development environment for designing ultra-lowpower applications, as well as a wireless bootloader with redundancy and self-monitoring functions. This will be paired with hardware module designs whichcan be combined based on the requirements of a given application. The result-ing device would then be encapsulated in parylene C coating and embedded inimplant grade silicone. The resulting silicone object can be inserted under theskin using methods established by body modification artists.

520 P. Strohmeier et al.

2 Related Work

2.1 Posthuman Bodies

Art that challenges the traditional role of the body is becoming succeedinglymore mainstream. French artist Orlan has exhibited her plastic surgery perfor-mance at the Guggenheim in NYC, while Australian artist Stelarc has becomea common reference at human computer interaction conferences. Stelarc’s worksexplore the theme of agency beyond the body. His artworks often involve perfor-mances in which control over his limbs is handed to the audience or to randomsignals generated by internet traffic. He also uses robotic appendixes to expandhis body and has experimented with implantation of extra sensory organs [5].

While artists such as Stelarc and Orlan are becoming more mainstream,the body modification community still strongly identifies as a counterculturemovement. One of the earliest functional implants common within the body-modification community is a magnet. An implanted magnet vibrates when theimplantee is subjected to a varying electromagnetic field. This might occurwhen walking through security systems or in the close proximity of electricmotors or transformers. The first magnetic implants were implanted by Samppavon Cyborg in 2000. Kevin Warwick experimented with direct communicationbetween the nervous system of two implantees. He used a 100 pin micro-arrayas an electrical interface with the nervous system of each implantee. Nervousimpulses were then wirelessly transmitted between them. Warwick was also oneof the first to implant RFID tags [6]. RFID and NFC implants have been furtherpopularized by Amal Graafstra [7].

The methods developed by the body modification community have recentlybeen appropriated by self identified grinders, wet-ware- or bio hackers. Mostnotably Tim Cannon and Grindhouse Wetware have developed a temperaturemonitoring device and an implantable silicone object illuminated from the inside.While both device types have been implanted, they were since removed due tosafety concerns [3].

2.2 Development Environments

There is an ever increasing number of ecosystems and dev-boards that supportnon-experts in the development of digital devices. Most prominent among themRaspberry Pi and Arduino. Neither however meet the ultra-low power require-ments and the compact size required of an implanted device. Other interestingsystems include the Bluetooth Low Energy (BLE) development boards by Red-BearLabs2 or products based on Simblee3. These solutions have relatively lowpower consumption, but the form-factor of the devices by RedBearLabs and theclosed source of the Simblee protocol are also problematic. There are variousupcoming development kits based around the new nRF52, such as the BMD-300 by Rigado4 with even lower power consumption. While such modules have2 http://redbearlab.com/.3 https://www.simblee.com/.4 https://www.rigado.com/product/bmd-300/.

Initial Steps in Designing Interactive Electronic Implants 521

optimized power consumption, they do not provide the close coupling with wire-less charging and battery monitoring which we envision.

3 Design Space

3.1 Implantation Style

Body modification artists typically distinguish between transdermal and sub-dermal implants. Transdermal implants contain a metal base and a threadedstud extending beyond the skin, on which jewelry - for example metal horns -can be mounted. Depending on the design of the transdermal implant, it mighthave a porous base which increases the strength with which it is fused to theskin. Transdermals developed by Samppa von Cyborg contain a coating toallow tissue to bond with the implant itself, rather than only encapsulating it.Subdermal implants are placed underneath the skin. As the skin can closeabove it, such implants minimize the risk of both infection and rejection. Asthe silicone is smooth without any porous structures for the skin to latch on to,removal of such an implant is comparably simple.

3.2 Functionality

We define all implants which do not exhibit any behavior as non-functional(some, such as breast implants, may have social functions, but do not exhibit anybehavior of their own, so we consider them non-functional). There are two maintypes of passive implants which are gaining increasing popularity: NFC/RFIDchips and magnets. These implants transmit and ID or alert the implantee of thepresence of a varying electromagnetic field respectively, however they require anexternal power source. In contrast to these are various explorations into activepowered implants, most notably the Circadia and North Star by GrindhouseWetware. While these devices can turn LEDs on and off on demand, or streamtemperature data, they have no way of interacting with the implantees bodyor environment in real time. Remora will support the design of interactiveimplants: implants which can react to the implantees bodies, environment andactivities both continuously and in real time.

4 Implementation

4.1 Hardware

Remora consists of a Core and Extensions (Fig. 2). The Core includes the PowerTransfer Module, the Battery Module, and the Logic Module. Remoras func-tionality is determined by the Extension Modules connected to its Core. SuchExtension Modules might provide haptic feedback or illuminate the device fromthe inside. They could collect inertial or biometric data or enable explicit userinput. This modular approach was chosen to give developers the greatest pos-sible freedom in developing their own devices. Example devices might include

522 P. Strohmeier et al.

sensory augmentation applications, such as a device which provides a hapticimpulse when the implantee is facing north, continuous biometric monitoring,or internal data storage.

The Core consists of a Logic Module, Power Transfer Module and BatteryModule (see Fig. 2). Currently we have also implemented an Inertial SensingUnit as our first Expansion Module.

The Battery Module contains the battery and basic circuitry to protectbattery life as well as a kill switch mechanism. When this killswitch is activatedall circuits are immediately powered off. The killswitch can be triggered in mul-tiple ways: the user can explicitly activate it, the Battery Module can activateit, if the temperature of the battery or the power consumption exceeds a prede-termined threshold, and it can be triggered by the Logic Module, if it detects aproblem.

Fig. 2. Overview of a Remora device. Green solid arrows indicate power flow, yellowdashed arrows indicate control. The Logic Module controls extensions and BatteryModule, while the Power Transfer Module only has control over the Battery Module.The Battery Module can be removed completely if one is interested in designing apassive device. (Color figure online)

For explicit user killswitch activation, we assume that the user has a sec-ondary passive implant consisting solely of a magnet. For example, if the pri-mary implant is in the left arm, the user might have a magnet in the right hand.By touching the magnet of their right hand to the implant in their left hand,the user has a simple and reliable mechanism of disconnecting the battery fromthe rest of the circuitry.

The Power Transfer Module implements the Qi Compliant Power Transferstandard. This enables users to charge Remora with any off-the-shelf wirelessinduction charging device. The Qi protocol is implemented using the BQ51050BIC by Texas Instruments. The Power Transfer Module will interrupt the powertransfer if: its own temperature rises over a cut-off temperature, if notified ofa charging error by the Logic Module, or if it detects the battery to be fullycharged. The Qi Compliant standard was chosen because it is currently widelyused and therefore a convenient starting point. We anticipate exploring optionstailored more specifically to charging an implanted device in the future.

Initial Steps in Designing Interactive Electronic Implants 523

The Logic Module is powered by Nordic Semiconductor’s nRF52832. ThisSoC comes with a Cortex M4 processor and various wireles communicationoptions including NFC and a 2.4 GHz radio that supports Bluetooth LowEnergy (BLE). The Logic Module supports communication with other devicesvia UART, SPI and I2C. Additionally the module has a number of GPIOs and12 bit ADC inputs for interfacing with sensors. The Logic Module monitorsits own temperature, the temperature of the Power Transfer Module and thecharge state and temperature of the Battery Module. If any of these measuresare outside of predetermined levels it can disable wireless charging, activate thekillswitch or shut itself down until woken up by explicit user input through theBattery Module.

As any two Core modules can operate with the third module deactivated(See Fig. 2), this setup also allows us to collect debug information about thestate of the device in case of suspected problems. The Logic Module can bepowered from either the Battery Module or the Wireless Power Module and, ifintact, provide debug information via Bluetooth. If the Logic Module has failed,battery health information can be extracted from the Power Transfer Model viathe Qi communication interface.

4.2 Software

The software embedded in the nRF52 consists of two parts, the bootloader andthe embedded main application. To develop or customize them, we provide theuser with a development environment based on the SDK made by Nordic Semi-conductors.

The bootloader performs two functions: it enables firmware uploads usingBLE and it takes over monitoring battery health and power consumption, shouldthere be no functioning firmware (Fig. 3). The firmware is uploaded using adual bank memory layout. This ensures that only complete and valid imagesare activated. During dual bank upload the received firmware is stored in anintermediary memory location and, after it has been fully uploaded and verified,is transferred to its intended location. If an error occurs during the transfer, orif the system is unable to validate the new firmware, the previously uploadedfirmware continues operating. The bootloader can be triggered through a BLErequest, or by activating the killswitch.

Fig. 3. Schematic overview of Remora software architecture

524 P. Strohmeier et al.

The main application has several tasks, monitoring & redundancy, commu-nication and IO processing. The monitoring & redundancy task is not designedto be modified by the user. It continuously monitors temperature of all modulesas well as power drain and battery health. For reliability purpose, these measuresare also performed in parallel by dedicated analog circuitry. If either the moni-toring & redundancy task or the analog circuitry determines values to be outsideof a predetermined range, the killswitch of the Battery Module is activated. Themonitoring & redundancy task also uses a watchdog monitoring system to restartthe processor, should a software endless loop problem occur. Button-less DeviceFirmware Update (DFU) is conducted using BLE. BLE is also used to provideactuators with external control signals, or send sensor data to other BLE devices.The IO processing is the most modular part and is designed to be completelycustomizable. It handles communication with extension modules and runs theuser created application.

The development environment is built on top of the development kit providedby Nordic Semiconductors. Users can create custom application using commandline arguments and a Makefile, enabling desired extension modules. The develop-ment environment optimizes power consumption and memory usage for a givenapplication. It also generates a checksum to avoid transmission errors during thefirmware update. Error verification is then performed by the dual bank boot-loader. The development environment outputs a zip archive including all requiredbinaries. The archive can then be stored in a cloud services such as Dropbox,to access it from a smartphone app. The smartphone then uploads the firmwareusing BLE.

5 Future Work

We have implemented the three core modules and the inertial sensing module.We have tested these for basic functionality. Our next steps are to add additionalhardware modules and to thoroughly evaluate the existing ones. We intend tooptimize power consumption using the Simplicity EnergyAware Profiler at bodytemperature as well as optimize heat-dissipation using a near infrared cameraby Optris5. An open question is which battery technology the final platformshould target. Using traditional implantable batteries might not be ideal as ourpower consumption and charging behaviors are different from existing devices.We wish to emphasize that, as we have not conducted any formal evaluation,we are currently not able to make any statement regarding the safety of ourapproach.

6 Conclusion

In this work in progress report, we presented Remora, an ecosystem for design-ing implantable devices. Rather than adding to the tradition of mainstream

5 http://silabs.com, http://optris.com.

Initial Steps in Designing Interactive Electronic Implants 525

implantable devices, Remora builds on knowledge and practices collected bybody modification artists. Remora is designed for these artists to create theirown implants, but will also be a platform for researchers interested in creatinghybrid systems. Our design is primarily focused on providing series of monitoringand redundancy features, aimed at protecting the implantee and providing thedesigner with as much debug information as possible, should an error occur. Wehave so far designed and implemented a proof-of concept device. Future workwill expand upon these and include empirical studies testing the safety and reli-ability of our designs and evaluating interaction design with and for implanteddevices.

References

1. Holz, C., Grossman, T., Fitzmaurice, G., Agur, A.: Implanted user interfaces. In:Proceedings of the 2012 ACM Annual Conference on Human Factors in ComputingSystems (2012)

2. Hameed, J., Harrison, I., Gasson, M.N., Warwick, K.: A novel human-machine inter-face using subdermal magnetic implants. In: 2010 IEEE 9th International Confer-ence on Cyberntic Intelligent Systems (2010)

3. The Half Life of Body Hacking – Motherboard. http://motherboard.vice.com/read/the-half-life-of-body-hacking. Accessed 20 Mar 2016

4. Heffernan, K.J., Vetere, F.: You put what, where? Hobbyist use of insertable devices.In: Proceedings of the 2016 ACM Annual Conference on Human Factors in Com-puting Systems (2016)

5. Ping Body - Institute for the Unstable Media. http://v2.nl/events/ping-body.Accessed 21 Oct 2015

6. Warwick, K., Ruiz, V.: On linking human and machine brains. Neurocomputing71(13–15), 2619–2624 (2008)

7. Graafstra, A.: Hands On: How radio-frequency identifion and I got personal. IEEESpectr. 18, 23 (2007)

Gait Analysis of 6-Legged Robotwith Actuator-Equipped Trunk and Insect

Inspired Body Structure

Yasuhiro Sugimoto(B), Yuji Kito, Yuichiro Sueoka, and Koichi Osuka

Department of Mechanical Engineering, Osaka University, Osaka, Japan{yas,sueoka,osuka}@mech.eng.osaka-u.ac.jp,

kito@dsc.mech.eng.osaka-u.ac.jp

Abstract. Not only 4-legged animals but also 6-legged creatures exhibita range of gaits. To reveal the mechanism underlying the gait of animalsand realize such a gait in a multi-legged robot, it is important to under-stand the inter-limb coordination. It is possible that the inter-limb coor-dination is strongly influenced by the structure of trunk to which eachleg is connected. In this study, we designed a brand-new insect-inspiredtrunk mechanism that is equipped with an actuator. Using this trunk, webuilt a 6-legged robots with passive limbs. We performed walking exper-iments with the developed robot and analyzed its gait, especially therelationship between the structure of the trunk and the walking velocity.The experimental result shows an insect inspired body structure affectsits gait and the walking velocity of 6-legged walking robot.

Keywords: Legged robot · Trunk structure · Gait analysis

1 Introduction

Many terrestrial animals adopt walking as their means of locomotion. It is wellknown that quadruped animals exhibit specific gaits, such as walking, troting,paceing, depending on their movement speed at which they are moving. Further-more, not only 4-legged animals but also 6-legged creatures exhibit a range of gaits,such as waving, tripod and tetrapod gaits [13]. And the insects also exhibit thecontinuum of these gaits [9]. To reveal the mechanisms behind the gaits of animalsand realize those gaits in multi-legged robots, it is important to understand theinter-limb coordination. Although a large number of studies have examined thetransition of gaits, the inter-limb coordination and its applications to multi-leggedrobots, many studies have focused on the controller layer, such as the neural net-work and central pattern generator (CPG) [1,2,6]. It is also interesting to observethe physical layer, that is, the mechanical structure. In particular, it is probablethat the inter-limb coordination is strongly influenced by the structure of the trunkbecause the legs are connects to it mechanically [4,10,12].

To investigate the inter-limb coordination from the viewpoint of the mechani-cal properties, we focused on passive dynamic walking, studied first by McGeer [7].c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 526–531, 2016.DOI: 10.1007/978-3-319-42417-0 57

Gait Analysis of 6-Legged Robot with Actuator-Equipped Trunk 527

Passivedynamicwalking refers to the ability, in amechanical device, towalkdownashallow slope without using any actuation or sensing. Although the subjects of thisresearch were mostly two-legged robots, some research has addressed multi-leggeddevices [5,8]. In this study, to realize levelwalking and spontaneous inter-limb coor-dination,wedesignedabrand-newtrunkmechanism incorporatinganactuator andbuilt a 6-legged robot using this trunk. The legs of the robot are passive limbs. Thatis, the legs dose not have any actuators, such that the motions of each leg are insti-gated by the trunk. As a result, spontaneous inter-limb coordination is expected tooccur. In addition, we focused on the structure of an insect’s body. Through walk-ing experiments with the developed robot, we analyzed its locomotion, especiallythe relationship between the structure of the trunk and the walking velocity.

2 Multi Legged Robot

Figure 1 shows the developed 6-legged robot. This robot is based on that devel-oped in our previous study [5]. This robot consists of a series of connected threetwo-legged elements with two trunks, each incorporating an actuator. Each legis rigid and has no knee joint. Additionally, each foot is a part of a sphere. Eachleg is a passive limb and attached to a free joint at the waist, rotatable about theaxial pitch. The rocking motion of each two-legged element is instigated in thelateral plane by the actuator-equipped trunk, with the robot being able to walkby swinging its legs with a rocking motion. There is no direct leg-synchronizingmechanism. Only through the trunk, each element interacts indirectly.

Figure 1(b) shows the developed actuator-equipped trunk. The trunk isequipped with a servo motor and is able to rotate in the directions of the rolland yaw axes. The torque generated by the servo motor leads to rotation in theroll direction. The roll and yaw rotations are coupled because of the gearingmechanism. As a result, rotations in the roll and yaw directions occur at thesame time even with only one servo motor. The movement around the pitch axisis decoupled from the roll and yaw axes. The degree of freedom around the pitchaxis is given at joints between two-legged elements and the trunk.

Fig. 1. Developed 6-legged robot and an developed actuatable trunk.

528 Y. Sugimoto et al.

The developed 6-legged robot has two trunks. The front trunk is locatedbetween the front and middle legs while the hind trunk which is located betweenthe middle and hind legs. Inputs to these trunks can be provided individually.We apply the following Eq. (1) as inputs φf (t) and φh(t) to the front and hindtrunk servo motors for which A[rad], T [s] and β[rad] are the amplitude, periodand phase difference, respectively. We can see that the possible gaits of the robotwill change depending on β. In this study, we specifically focused on the phasedifference β.

φf (t) = A sin(2π

Tt), φh(t) = A sin(

2π

Tt − β) (1)

Several studies involving an analysis of a multi legged robot have used robotsthat configured of series-connected two-legged elements, in the same way as ourdeveloped robot [5,11]. The structures of the fundamental two-legged elementswere almost the same. In contrast, front, middle and hind legs of most insectshave different structures. The distance between the landing point of each leg pairis different as shown in Fig. 2(a). Inspired by these interesting insect structures,we implemented a means of adjusting the distances between the left and right legjoints of the robot. By using spacers, the distances can be adjusted. For example,in Fig. 2(b), the distance between the joints of the middle legs is increased.

Fig. 2. 6-legs insect and top view of the robot. This robot can change the distancesbetween the left and right leg joint.

3 Gait Analysis

We investigated the walking speed as a function of the input phase difference β.The other values in Eq. (1) are fixed to A = 20[deg] and T = 0.2[s]. To examine

Gait Analysis of 6-Legged Robot with Actuator-Equipped Trunk 529

the effect of the body structure, we also enlarged the left-and-right leg jointdistances for the front, middle and hind legs. Figure 3 shows the experimentalresults of our experiments. The vertical and horizontal axes show the walkingspeed and phase difference β, respectively. Figure 3(a) shows average walkingspeeds and standard deviations of each phase difference obtained with the normalstructure case, that is, when the leg joint distances of the front, middle and hindlegs are all the same. Figure 3(b) shows average walking speeds obtained whenany of the leg joint distances of the front, middle or hind legs were changed.

The walking speed is almost the same except in the case of β = 0 and β = 30for the normal structure. When β = 0, the robot almost stopped, that is, itrepeatedly stepped in the same location. When β = 60, the robot attained itsmaximum value of the walking speed. Figure 4 shows an image of forward walkingwith β = 180. Figure 5 shows lateral oscillations θ of each two-legged elements.Because of the particular structure of each two-legged elements, positive θ valuemeans that the right leg is on the ground and left leg is lifted off the ground,and vice versa. It was confirmed that the robot can move forward by wrigglingits body. In this case, the robot exhibits a tripod gait. It was also confirmed thatthe robot could exhibit a wave gait when β = 90 or β = 270.

On the other hand, the walking speed varied considerably depending on thevalue of β when the insects inspired body structure was used. In addition, thetransition in the walking speed also varied based on the adjustment location.When the front or hind leg joint distance was increased, the walking speed wasa minimum at around β = 180. However, the walking speed increased to amaximum at around β = 180 when the middle leg joint distance was increased.In this case, the robot also exhibited a tripod gait when β = 180 and a wave gaitwhen β = 90 or β = 270. It is known that some insects exhibit a transition intheir gait; they generate a wave gait at low speeds and then modify their gait toa tetrapod gait at medium speeds [3]. Now, the distance between the middle legjoints of many insects is larger than that of other leg joints as shown in Fig. 2(a).The results of walking experiments seem to exhibit a similar tendency to insectlocomotion. A detailed analysis of the relationship between the gaits and thestructure will be undertaken as part of our future work.

Fig. 3. Walking speed of 6-legs robot vs. phase difference β

530 Y. Sugimoto et al.

Fig. 4. The walking snapshots of the developed 6-legged robot (phase difference β =180[deg]). In this case, fore and hind leg elements are synchronized and middle leg isantiphase.

Fig. 5. Lateral oscillation θ of each leg elements (A = 25[deg], T = 0.2[s]) (Color figureonline)

4 Conclusion

In this study, we developed a 6-legged walking robot with an insect-inspiredbody structure and an actuator-equipped trunk for analyzing the relationshipbetween gaits and the structure of a robot. We conducted walking experimentswith the developed robot. Our experimental results showed that the walkingspeed and gait were related to the input phase difference. In addition, it was

Gait Analysis of 6-Legged Robot with Actuator-Equipped Trunk 531

also verified that the transition in the walking speed changed greatly dependingon the body structure.

Acknowledgments. Thisworkwas supported by theGrant-in-Aid forYoungScientists(B), No.26870337 and CREST, JST.

References

1. Beer, R.D., Quinn, R.D., Chiel, H.J., Ritzmann, R.E.: Biologically inspiredapproaches to robotics: what can we learn from insects? Commun. ACM 40(3),30–38 (1997)

2. Buschges, A.: Sensory control and organization of neural networks mediating coor-dination of multisegmental organs for locomotion. J. Neurophysiol. 93(3), 1127–1135 (2004)

3. Hughes, G.M.: The co-ordination of insect movements. J. Exp. Biol. 29(2), 267–285(1952)

4. Iida, F., Gomez, G., Pfeifer, R.: Exploiting body dynamics for controlling a run-ning quadruped robot. In: Proceedings of International Conference on AdvancedRobotics (ICAR 2005), pp. 229–235 (2005)

5. Kito, Y., Sueoka, Y., Nakanishi, D., Yasuhiro, S., Ishikawa, M., Wada, T., Osuka, K.:Quadruped passive dynamic walking robot with a new trunk structure inspired byspine. In: Proceedings of Dynamic Walking 2014 (2014)

6. Matsuoka, K.: Mechanisms of frequency and pattern control in the neural rhythmgenerators. Biol. Cybern. 56(5), 345–353 (1987)

7. McGeer, T.: Passive dynamic walking. Int. J. Robot. Res. 9(2), 62–82 (1990)8. Remy, C.D., Buffinton, K., Siegwart, R.: Stability analysis of passive dynamic

walking of quadrupeds. Int. J. Robot. Res. 29(9), 1173–1185 (2009)9. Schilling, M., Hoinville, T., Schmitz, J., Cruse, H.: Walknet, a bio-inspired con-

troller for hexapod walking. Biol. Cybern. 107(4), 397–419 (2013)10. Schilling, M., Paskarbeit, J., Schmitz, J., Schneider, A., Cruse, H.: Grounding an

internal body model of a hexapod walker control of curve walking in a biologi-cally inspired robot. In: IEEE International Conference on Intelligent Robots andSystems, pp. 2762–2768 (2012)

11. Sugimoto, Y., Yoshioka, H., Osuka, K.: Realization and motion analysis of multi-legged passive dynamic walking. In: Proceedings of SICE Annual Conference 2010,Taipei (2010)

12. Takuma, T., Izawa, R., Inoue, T., Masuda, T.: Mechanical design of a trunk withredundant and viscoelastic joints for rhythmic quadruped locomotion. Adv. Robot.26(7), 745–764 (2012)

13. Wilson, D.: Insect walking. Ann. Rev. Entomol. 11(1), 103–122 (1965)

Quadruped Gait Transition from Walk to Paceto Rotary Gallop by Exploiting Head Movement

Shura Suzuki1, Dai Owaki1(B), Akira Fukuhara1,2, and Akio Ishiguro1,3

1 Research Institute of Electrical Communication, Tohoku University,2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan

owaki@riec.tohoku.ac.jp2 JSPS, 5-3-1 Kojimachi, Chiyoda-ku, Tokyo 102-0083, Japan

3 CREST, Japan Science and Technology Agency, 4-1-8 Honcho, Kawaguchi,Saitama 332-0012, Japan

http://www.cmplx.riec.tohoku.ac.jp

Abstract. The manner in which quadrupeds change their locomotionpatterns with change in speed is poorly understood. In this paper, wedemonstrate spontaneous gait transition by using a quadruped robotmodel with a head segment, for which leg coordination can be self-organized through a simple “central pattern generator” (CPG) modelwith a postural reflex mechanism. Our model effectively makes use ofhead movement for the gait transition, suggesting that head movementis crucial for the reproduction of the gait transition to high-speed gaitsin quadrupeds.

Keywords: Central pattern generator (CPG) · Interlimb coordination ·Physical communication · Head movement · Postural reflex

1 Introduction

Quadrupeds exhibit flexible locomotive patterns depending on their locomotionspeed, environmental conditions, and animal species [1]. Such locomotor patternsare generated via the coordination of leg movements, i.e. “interlimb coordina-tion”. However, the mechanism responsible for interlimb coordination remainsunclear. The knowledge of the interlimb coordination underlying quadruped loco-motion is essential for understanding the locomotion mechanism in legged ani-mals: furthermore, it will be useful in establishing the fundamental technologyfor legged robots that can reproduce locomotion similar to that of animals.

Previous neurophysiological experiments involving decerebrate cats [2] havesuggested that their locomotive patterns are controlled in part by a distributedneural network, called the “central pattern generator” (CPG), in the spinal cord.

D. Owaki—We acknowledge the support of a JSPS KAKENHI Grant-in-Aid forYoung Scientists (A) (25709033).

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 532–539, 2016.DOI: 10.1007/978-3-319-42417-0 58

Quadruped Gait Transition by Exploiting Head Movement 533

These biological findings have prompted many researchers to incorporate neurallywiredCPG models into legged robots to generate highly adaptive locomotion [3,4].In contrast, we proposed a novel non-wired CPG model consisting of four decou-pled oscillators with local force feedback only [5]. Our model explains wide obser-vations of gait patterns in quadrupeds [5], and the predictions based on our resultsare consistent with biological evidence [6]. However, our model did not sufficientlyreproduce high-speed gait patterns, e.g. transverse/rotary gallop [1].

This study aims to reproduce gait transition, especially, in high-speed gaitpatterns. Thus, we focus on head movements in quadruped locomotion for thefollowing two reasons:

1. In quadrupeds, the mass ratio of the head segment, i.e. head and neck, to thebody is around 10 %, whereas the mass ratio of fore or hind legs is around5.5% ∼ 6.5 % [7].

2. The relative movement phase relationship between all body segments, e.g.head, neck, trunk, etc. changes according to the gait pattern [8].

These facts suggest that head movements strongly affect the interlimb coordina-tion with an increase in the locomotion speed. Owing to our effort in constructinga quadruped model using a simple CPG [5] with an additional postural reflexmechanism, we successfully reproduced gait transition from walk to pace torotary gallop by using the head movement. Surprisingly, the obtained gait pat-terns are qualitatively similar to a giraffe’s gait patterns [9], suggesting that theseanimals effectively make use of head and neck movements during gait transitions.

2 Model

2.1 Musculoskeletal Structure

Figure 1 shows an overview of the quadruped robot model constructed in thisstudy. To mainly focus on the effects of the head movement on interlimb coor-dination, we used a simple musculoskeletal structure of the quadruped robot,which consists of a head, trunk, and four legs. The head and trunk segmentsare connected via a passive spring that has one degree of freedom (DOF) inthe pitch direction only. We implemented two actuators for each leg (2 DOF),in which the rotational actuator drives a leg at the shoulder/hip joint and thelinear actuator drives a leg along the leg axial direction, as shown in Fig. 1 right.A phase oscillator was implemented in a leg to generate a rhythmic leg motionduring the stance and swing phases according to the oscillator phase φi (here-after, i = 1 ∼ 4, Left fore (LF):1, Left hind (LH):2, Right fore (RF):3, Righthind (RH):4), as explained in Sect. 2.2 in detail. We implemented two types ofsensors: pressure sensors on the feet, which can detect ground reaction forces Ni

[N] (GRFs) parallel to the leg axis, and an angle sensor on the neck joint, whichcan detect the displacement ψ [rad] from the equilibrium posture.

534 S. Suzuki et al.

Fig. 1. Quadruped robot model.

2.2 CPG Model with Postural Reflex

We used our CPG model [5] that enables a quadruped to achieve flexible interlimbcoordination without “neural” communication. We additionally implemented asimple postural reflex mechanism [10] because the whole body posture desta-bilized upon adding the head segment. In the following section, we explain thecontrol scheme of our quadruped robot model.

Each leg is controlled using rotational and linear actuators according to theoscillator phase φi [rad]. The target angle θi [rad] and length li [m] are describedusing the oscillator phase as follows:

θi = −Camp cos φi, (1)

li = L0 − Lamp sin φi, (2)

where Camp denotes the amplitude of the target angle, and Lamp and L0 denotethe amplitude and offset length of the target length, respectively. Based on thetarget angle and length, each actuator is controlled using the following equa-tions:

τi = −Kr(θi − θi), (3)

Fi = −Kl(li − li), (4)

where Kr and Kl are the P gains for each actuator. θi and li denote the actualangle of the shoulder/hip joint and the actual leg length, respectively. Based onthe control scheme, a leg tends to be in the swing phase for 0 < φi ≤ π, and inthe stance phase for π < φi ≤ 2π, as shown in Fig. 1 right.

Quadruped Gait Transition by Exploiting Head Movement 535

The dynamics of the phase oscillators are described as follows:

φfi = ω − σNf

i cos φfi − ρψ cos φf

i (5)

φhi = ω − σNh

i cos φhi + ρψ cos φh

i (6)

where φfi and φh

i denote the phases of the fore and hind leg oscillators, respec-tively. ω [rad/s] denotes the intrinsic angular velocity of the oscillators. σ[rad/Ns] and ρ [1/s] denote the feedback gains for the second and third terms,respectively. Nf

i and Nhi [N] denote the GRFs detected by pressure sensors in

the fore and hind feet, and ψ [rad] denotes the neck joint angle detected by theangle sensor.

The second term enables interlimb coordination without any neural com-munication [5]. The physical effect of this term is that a leg remains in the

Fig. 2. Postural reflex mechanism. The red and blue sections in the phase planes ofoscillators φf

i and φhi represent the swing and stance phases, respectively. The green

(leg contraction) and orange (leg extension) arrows represent the effect of the posturalreflex described by the third term in Eqs. (5) and (6). (Color figure online)

536 S. Suzuki et al.

stance (sin φi < 0) phase while supporting the body (Ni > 0). The essence ofthe coordination mechanism based on the local feedback is that the local sen-sor value Ni includes global physical information regarding the other legs, i.e.,to what extent the other legs support the body at any given time. Thus, thephysical-communication-based interlimb coordination can be achieved withoutdirect coupling between oscillators, unlike in most previous CPG models [3,4].

The third term was inspired by the “postural reflex mechanism” [10], whichvarious vertebrates possess as a primitive reflex. The postural reflex tries to main-tain an equilibrium position or posture of the body by using visual, vestibular,and deep sensory information. Here, we focused on the “tonic neck reflex” [10],which activates limb muscles according to the neck joint angle. This term pro-vides the following effect: the extension of the fore legs and the contraction of thehind legs enables a posture to be maintained when ψ > 0 (Fig. 2(a)), whereas thecontraction of the fore legs and the extension of the hind legs enables a posture tobe maintained when ψ < 0 (Fig. 2(b)). Thus, we can include interlimb coordina-tion mechanism, which enables maintaining the equilibrium posture according tothe neck joint angle, thus resulting in postural stabilisation during locomotion.

3 Simulation Result

We conducted dynamic simulations using the Open Dynamics Engine (ODE)by implementing the proposed CPG model in a quadruped robot in the simu-lation environment. The parameters for the quadruped robot model are shownin Table 1. As shown in the upper graph of Fig. 3, we conducted a gait tran-sition experiment by changing only the parameter ω [rad/s]. The lower graphin Fig. 3 shows the time evolution of the phase differences Δφ [rad] betweenoscillators during gait transition. The blue, red, and green lines representΔφLH−LF , ΔφRF−LF , and ΔφRH−LF [rad], respectively. The result indicatesthat we achieved gait transition with spontaneous phase difference modificationby changing only ω.

Figure 4 shows the representative gait patterns for (a) ω = 8.0, (b) ω =11.0, and (c) ω = 14.0. The upper and lower graphs show the time evolutionof the neck joint angle ψ [rad] and gait diagrams, where the coloured regionsrepresent the stance phases (Ni > 0). At (a) ω = 8.0 [rad/s], our robot exhibiteda diagonal-sequence (D-S) walk, in which the feet touched the ground in thefollowing order: LF, LH, RF, and RH. At (b) ω = 11.0 [rad/s], a pace wasobserved with the ipsilateral feet touching the ground in phase. At (c) ω = 14.0[rad/s], a rotary gallop was observed, in which the feet touched the ground inthe following order: LF, RF, RH, and LH. These results indicate that our robotreproduced gait transition from D-S walk to pace to rotary gallop in response tothe locomotion speed. Further, in D-S walk, the neck leaned forward when theLF leg touched the ground, then leaned backward when the LH leg touched theground, leaned forward again when the RF leg touched the ground, and finallyleaned backward when the RH leg touched the ground. In pace, the neck leanedforward at the middle stance phase of the ipsilateral leg pairs, e.g. LF and LH.

Quadruped Gait Transition by Exploiting Head Movement 537

Table 1. Parameters for quadruped robot model

Head mass 0.175 [kg]

Total mass 1.38 [kg]

Leg length 0.12 [m]

Shoulder/Hip length 0.12 [m]

Trunk length 0.19 [m]

Neck length 0.20 [m]

Spring constant of neck joint 3.50 [Nm/rad]

σ 0.60 [rad/Ns]

ρ 10.0 [1/s]

Camp 0.1π [rad]

Lamp 0.006 [m]

L0 0.12 [m]

Kr 40.0 [Nm/rad]

Kl 25.0 [N/m]

Fig. 3. Gait transition. Top: time profile of ω. Bottom: time evolution of phase differ-ences Δφ between oscillators. (Color figure online)

538 S. Suzuki et al.

Fig. 4. Gait patterns for ω = 8.0, 11.0, 14.0 [rad/s].

Quadruped Gait Transition by Exploiting Head Movement 539

In rotary gallop, the neck leaned forward when the fore legs touched the ground,and it leaned backward when the hind legs touched the ground. The salient effectin rotary gallop is provided by the postural reflex mechanism given by Eqs. (5)and (6).

4 Conclusion and Future Work

To reproduce the gait transition in high-speed gait patterns, we modelled aquadruped robot in a simulation and implemented the proposed CPG model witha postural reflex mechanism. As a result, our robot reproduced gait transitionsfrom D-S walk to pace to rotary gallop in response to the locomotion speed.Surprisingly, the obtained gait patterns are qualitatively similar to a giraffe’sgait patterns [9], suggesting that these animals effectively use the head and neckmovements during gait transitions. Therefore, our model would contribute tounderstanding the mechanism underlying giraffes’s gait transition. In future,we will clarify the generation mechanism of “transverse gallop”, which was notreproduced in this study. Furthermore, we will conduct a theoretical analysisof the gait transitions from low- to middle- to high-speed gait patterns, whichseems to be of great interest.

References

1. Muybridge, E.: Animals in Motion. Dover Publications, New York (1957)2. Shik, M.L., Severin, F.V., Orlovskii, G.N.: Control of walking and running by

means of electrical stimulation of the midbrain. Biophysics 11, 756–765 (1966)3. Kimura, H., Akiyama, S., Sakurama, K.: Realization of dynamic walking and run-

ning of the quadruped using neural oscillator. Auton. Robots 7, 247–258 (1999)4. Righetti, L., Ijspeert, A.J.: Pattern generators with sensory feedback for the control

of quadruped locomotion. In: Proceedings of ICRA 2008, pp. 819–824 (2008)5. Owaki, D., Kano, T., Nagasawa, K., Tero, A., Ishiguro, A.: Simple robot suggests

physical interlimb communication is essential for quadruped walking. J. R. Soc.Interfac. (2012). doi:10.1098/rsif.2012.0669

6. Duysens, J., Clarac, F., Cruse, H.: Load-regulating mechanisms in gait and posture:comparative aspects. Physiol. Rev. 83, 83–133 (2000)

7. Buchner, H.H.F., Savelberg, H., Schamhard, H.C., Barneveld, A.: Inertial proper-ties of Dutch warmblood horse. J. Biomech. 30, 653–658 (1997)

8. Dunbar, D.C., Macpherson, J.M., Simmons, R.W., Zarcades, A.: Stabilization andmobility of the head, neck and trunk in horses during overground locomotion:comparisons with humans and other primates. J. Exp. Biol. 211, 3889–3907 (2008)

9. Dagg, A.I.: Giraffe: Biology, Behaviour, and Conservation. Cambridge UniversityPress, Cambridge (2014)

10. Magnus, R.: Cameron prize lectures on some results of studies in the physiologyof posture. Lancet 208, 531–536 (1926)

Exploiting Symmetry to Generalize BiomimeticTouch

Benjamin Ward-Cherrier1,2(B), Luke Cramphorn1,2, and Nathan F. Lepora1,2

1 Department of Engineering Mathematics, University of Bristol, Bristol, UK2 Bristol Robotics Laboratory, University of Bristol, Bristol, UK

{bw14452,ll14468,n.lepora}@bristol.ac.uk

Abstract. We introduce a method for generalizing tactile featuresacross different orientations and locations, inspired by recent studiesdemonstrating tactile generalization in humans. This method is appliedto two 3d-printed bioinspired optical tactile sensors. Internal pins act-ing as taxels are arranged with rotational and translational symmetry inthese sensors. By rotating or translating a small sample of tactile images,we are able to generalize tactile stimuli to new orientations or locationsalong the sensor respectively. Applying these generalization methods incombination with active perception leads to the natural formation of afovea of accurate real tactile data surrounded by moderately less accurategeneralized data used to focus the sensor’s tactile perception.

Keywords: Tactile sensors · Perceptual generalization

1 Introduction

Recent studies indicate that humans generalize certain tactile stimuli like pres-sure, roughness or vibrations across the skin’s surface [1,2], suggesting thatlow-level sensory generalization could be an important component of percep-tual learning. In biomimetic approaches to touch [3–5], which tend to rely onexhaustive training sampling, applying methods for tactile feature generalizationcould be both practically beneficial (reducing training time), as well as formingthe basis for an investigation of bioinspired perceptual generalization methods.

We introduce here a method for tactile generalization by exploiting sensorsymmetries to generate new tactile data. Our method is applied to two opticaltactile sensors: a round and flat version of the TacTip [6]. Both sensors opticallytrack pins inspired by papillae in the human fingertip. The rotational and trans-lational symmetry in pin layouts is exploited to generalize tactile features. Inboth cases, our method moderately increases errors in localization and orienta-tion identification. However these errors can be compensated for by using activeperception to relocate the sensor.

Thus we propose that tactile features can be reliably generalized throughour method, giving a generalization from known encountered stimuli to novelstimuli from the symmetry of the tactile sensor. Used in conjunction with activeperception, the method leads to the separation of the sensor into a fovea of realtraining data surrounded by coarse generalized data.c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 540–544, 2016.DOI: 10.1007/978-3-319-42417-0 59

Exploiting Symmetry to Generalize Biomimetic Touch 541

2 Methods

2.1 Hardware

The Round TacTip is a cheap, robust, 3d-printed optical tactile sensor developedat Bristol Robotics Laboratory [6]. It is made up of a 1 mm thick rubber skin,whose inside surface is covered in 127 pins arranged according to a projectedhexagonal pattern with 6-fold rotational symmetry and inspired by papillae inthe human fingertip (Fig. 1 - left panel). These pins deflect upon object contact,and their deflections in x- and y- directions are tracked by a webcam.

The Flat TacTip’s overall structure and design is elongated and based ona modified version of the TacThumb [7]. The Flat TacTip has a rectangularshape and 16-fold approximate translational symmetry, with a total of 80 pinsseparated by 4 mm from their nearest neighbours (Fig. 1 - right panel).

2.2 Perceptual Generalization

Our method for data generation considers a section of training data, and appliesaffine transformations to generalise tactile data based on sensor symmetries. Themethod differs slightly for each sensor, based on the 6-fold rotational symmetryof the round TacTip and the 16-fold translational symmetry of the flat TacTip.Both methods are outlined below:

Orientation. We train the round TacTip to detect edge orientation by rotatingover a 360◦ range in 5◦ increments. At each orientation the sensor taps downon a fixed edge stimulus. To generalize across orientation, we train only overa sub-sample of edge orientations (0 − 60◦) and rotate each set of tactile datagathered 60◦ around its centre, reassigning taxel identities to obtain the next setof orientations (60− 120◦). By repeating this step 6 times, we are able to obtaintactile data for the whole 360◦ range, effectively generalising a tactile featureover all possible orientations.

Fig. 1. The round TacTip sensor (left) and the flat TacTip sensor (right). The 6-foldrotational symmetry (round TacTip) and 16-fold translational symmetry (flat TacTip)are exploited to generalize tactile stimuli from a reduced sample.

542 B. Ward-Cherrier et al.

Location. A cylinder is rolled by the flat TacTip over a range of 28 mm, in incre-ments of 0.5 mm. To generalize data, we train over a 4 mm sub-sample, and applyour translations to the tactile data 6 times to obtain the full 28 mm range. Thisenables the sensor to generalize a tactile feature across its skin’s surface (Fig. 3).

3 Results

3.1 Inspection of Generalized Data

In orientation generalization with the round TacTip, generalized data(Fig. 2 - right panel) looks very similar to the real training data (Fig. 2 - leftpanel). We would thus predict a strong performance in orientation recognitionusing our method.

In the location generalization with the flat TacTip, we note that the realdata (Fig. 3 - left panel) is not completely symmetrical, likely due to the intrinsicmechanics of the sensor close to its boundaries. The generalized data appearsmore regular (Fig. 3 - right panel), ignoring the boundary effect and leading usto expect higher localization errors relative to the orientation case.

Fig. 2. Round TacTip data: left panel shows real training data, right panel correspondsto generalized data. The middle panel identifies pins according to their position on thesensor skin. (Color figure online)

3.2 Validation of Feature Generalization

Validation of the method is performed with a maximum likelihood probabilisticmethod. Monte-Carlo sampling of a pre-collected test data set gives us averagelocalization or orientation classification errors for each class. Errors are thencompared between real and generalized training data.

For the round TacTip, the real data has an average classification error over allorientation classes of eθ = 0.42◦, whereas generalized data averages eθ = 2.25◦.Errors are increased for the generalized data, but remain within a reasonablerange based on the orientation being sampled every 5◦.

In the flat TacTip case, using real training data leads to perfect localiza-tion (ex = 0 mm), whereas the generalized data errors average ex = 1.77 mm.

Exploiting Symmetry to Generalize Biomimetic Touch 543

Fig. 3. Flat TacTip data: left panel shows real training data, right panel correspondsto generalized data. The middle panel identifies pins according to their position on thesensor skin. (Color figure online)

The method proposed is contingent on sensors being designed in a symmetricalway. The flat TacTip’s higher errors are likely explained by its non-symmetricalmechanical properties, as evidenced in the previous section. Thus, there is anotable increase in errors of edge orientation classification and localization inthe generalized data case. To reduce these errors we apply active perceptionmethods to both orientation and location paradigms.

Active perception seeks to re-position the sensor to sample from the accuratereal training data. This corresponds to the 0–60◦ range for orientation (roundTacTip), and the 0–4 mm range for location (flat TacTip).

For orientation, the average error for active perception reduces to eθ = 0.17◦

for real data, and for generalized data it becomes eθ = 0.75◦. Thus the useof active perception reduces the disparity between the orientation classificationperformance of real and generalized training data. This result is expected, sinceactive perception allows the sensor to be repositioned to gather the more precisereal training data.

In the location case, errors for localization with the real data remain atex = 0 mm, whereas with generalized data they are reduced to just ex = 0.06 mm.In this case, the beneficial effect of using active perception is more pronouncedwhich suggests that although the generalized data for the flat TacTip is not veryaccurate, it is precise enough to effectively reposition the sensor to an area withmore accurate tactile data.

The use of our method combined with active perception leads to the appear-ance of a fovea of real data samples, surrounded by less accurate, generalizeddata allowing the sensor to focus objects on the fovea.

4 Conclusions and Future Work

We propose here a method for tactile stimuli generalization, implementing low-level perceptual learning with symmetrical sensors. Combining our method withthe use of active perception leads to the creation of a foveated sensor which can

544 B. Ward-Cherrier et al.

effectively sample multi-dimensional tactile data. This method could be appliedto complex tasks including manipulation in higher dimensions.

The location generalization method could also represent a first step towardsmimicking tactile stimuli generalization across the skin’s surface. This has beenshown to be performed in humans, with an interval discrimination task gener-alizing across the hand’s surface [2] and pressure and roughness discriminationbeing learned in adjacent and homologous fingers [1].

Overall, our approach considers the shape of the sensor itself to influenceperception and implement a form of low-level perceptual learning through featuregeneralization. This could be linked to the concept of body schema [8] to theextent that knowledge of body positions (or in this case sensor organization)can influence the interpretation of tactile data. Body schema work is gainingtraction in robotics [9] and methods that integrate sensor position and shapeinto perceptual learning are likely to lead to more successful interactions withthe robot’s environment.

Acknowledgment. We thank Sam Coupland, Gareth Griffiths and Samuel Forbes fortheir assistance with 3d-printing. BWC was supported by an EPSRC DTP studentshipand NL was supported in part by an EPSRC grant on ‘Tactile Superresolution Sensing’(EP/M02993X/1).

References

1. Harris, J., Harris, I., Diamond, M.: The topography of tactile learning in humans.J. Neurosci. 21(3), 1056–1061 (2001)

2. Nagarajan, S., Blake, D., Wright, B., Byl, N., Merzenich, M.: Practice-relatedimprovements in somatosensory interval discrimination are temporally specific butgeneralize across skin location, hemisphere, and modality. J. Neurosci. 18(4), 1559–1570 (1998)

3. Lepora, N., Martinez-Hernandez, U., Prescott, T.: Active touch for robust percep-tion under position uncertainty. In: IEEE International Conference on Robotics andAutomation (ICRA), pp. 3020–3025 (2013)

4. Lepora, N.: Biomimetic active touch with tactile fingertips and whiskers. IEEETrans. Haptics 9(2), 170–183 (2016)

5. Lepora, N., Martinez-Hernandez, U., Evans, M., Natale, L., Metta, G., Prescott,T.: Tactile superresolution and biomimetic hyperacuity. IEEE Trans. Rob. 31(3),605–618 (2015)

6. Chorley, C., Melhuish, C., Pipe, T., Rossiter. J.: Development of a tactile sen-sor based on biologically inspired edge encoding. In: International Conference onAdvanced Robotics, ICAR 2009, pp. 1–6. IEEE (2009)

7. Ward-Cherrier, B., Cramphorn, L., Lepora, N.: Tactile manipulation with a tac-thumb integrated on the open-hand m2 gripper. Rob. Autom. Lett. 1(1), 169–175(2016)

8. Maravita, A., Spence, C., Driver, J.: Multisensory integration and the body schema:close to hand and within reach. Curr. Biol. 13(13), R531–R539 (2003)

9. Hoffmann, M., Marques, H., Arieta, A., Sumioka, H., Lungarella, M., Pfeifer, R.:Body schema in robotics: a review. IEEE Trans. Auton. Mental Dev. 2(4), 304–324(2010)

Decentralized Control Scheme for CentipedeLocomotion Based on Local Reflexes

Kotaro Yasui1(B), Takeshi Kano1, Dai Owaki1, and Akio Ishiguro1,2

1 Research Institute of Electrical Communication, Tohoku University,2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan

{k.yasui,tkano,owaki,ishiguro}@riec.tohoku.ac.jp2 Japan Science and Technology Agency, CREST, 4-1-8 Honcho, Kawaguchi,

Saitama 332-0012, Japanhttp://www.cmplx.riec.tohoku.ac.jp/

Abstract. Centipedes exhibit adaptive locomotion via coordination oftheir numerous legs. In this study, we aimed to clarify the inter-limbcoordination mechanism by focusing on autonomous decentralized con-trol. Based on our working hypothesis that physical interaction betweenlegs via the body trunk plays an important role for the inter-limb coor-dination, we constructed a model wherein each leg is driven by a simplelocal reflexive mechanism.

Keywords: Centipede locomotion · Inter-limb coordination · Decen-tralized control · Reflex

1 Introduction

Centipedes move by propagating leg-density waves from the head to tail throughthe coordination of their numerous legs [1] and can adapt to various circum-stances, e.g., changes in walking speed, by appropriately changing the form ofthe leg density waves in real time [2]. This ability has been honed by a long-timeevolutionary process, and there likely exists an ingenious inter-limb coordinationmechanism underlying centipede locomotion. Clarification of this mechanism willhelp develop highly adaptive and redundant robots, as well as provide new bio-logical insights.

Autonomous decentralized control could be the key to understand the inter-limb coordination mechanism in centipede locomotion, and several models basedon decentralized control mechanisms have been proposed so far [3,4]. However,they could not truly reproduce the innate behavior of centipedes. This fact indi-cates that the essential decentralized control mechanism for the inter-limb coor-dination still remains unclear.

To tackle this problem, here we hypothesized that the physical interactionbetween legs via the body trunk is essential for the inter-limb coordination. Thus,we constructed a model based on a simple local reflexive mechanism in whichphysical interaction is fully exploited.c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 545–547, 2016.DOI: 10.1007/978-3-319-42417-0 60

546 K. Yasui et al.

2 Model

In our previous study, we proposed a simple local reflexive mechanism by usinga two-dimensional model in which yaw and roll motion of the body trunk isneglected [5]. However, we consider that three-dimensional motion of the bodytrunk plays an important role for the inter-limb coordination, and hence, herewe extend our previous model to the three-dimensional case.

A schematic of the physical model is shown in Fig. 1. Each leg is composedof a parallel combination of a real-time tunable spring (RTS) and a damper,where RTS is a spring whose natural length can be actively changed [6]. Eachleg is connected to the body trunk via a yaw joint to move the leg back and

Fig. 1. Schematic of the physical model.

Fig. 2. Effect of the local reflexive mechanism (a) when the right ith leg detects aground reaction force beneficial for propulsion and (b) when the right ith leg detectsa ground reaction force that impedes propulsion. Red arrows denote the horizontalcomponent of ground reaction forces acting on the ith leg tip. (Color figure online)

Decentralized Control Scheme for Centipede Locomotion 547

forth actively; this joint is controlled according to proportional-derivative (PD)control. Passive universal joints are implemented in the body trunk such thatthe body undulation can be generated through the physical interaction betweenthe legs during locomotion.

The natural lengths of the RTSs and the target angles of the yaw jointsare controlled according to the following mechanism. When the ith leg detectsa ground reaction force beneficial for propulsion, the ith leg kicks the groundbackward and the i−1th and i+1th legs on the ipsilateral side throw themselvesforward with lifting their tips off the ground (Fig. 2(a)). On the other hand, whenthe ith leg detects a ground reaction force which impedes propulsion, the ith legthrows itself forward with lifting its tip off the ground and the i − 1th andi + 1th legs on the ipsilateral side kick the ground backward (Fig. 2(b)). Thus,it is expected that neighboring legs do not interfere with each other such thatthe body propels itself effectively. Note that neural connection between the rightand left legs is not assumed in our model.

3 Conclusion and Future Work

We focused on centipedes and proposed a decentralized control scheme for theinter-limb coordination, wherein a simple local reflexive mechanism was imple-mented. In the future, we will investigate the validity of the proposed controlscheme via simulation.

Acknowledgments. This research was supported by JST CREST. The authors wouldlike to thank Kazuhiko Sakai of Research Institute of Electrical Communication ofTohoku University for helpful discussion.

References

1. Kuroda, S., Kunita, I., Tanaka, Y., Ishiguro, A., Kobayashi, R., Nakagaki, T.: Com-mon mechanics of mode switching in locomotion of limbless and legged animals. J.R. Soc. Interface 11, 20140205 (2014)

2. Manton, S.M.: The Arthropoda: Habits, Functional Morphology and Evolution.Clarendon Press, Oxford (1977)

3. Inagaki, S., Niwa, T., Suzuki, T.: Follow-the-contact-point gait control of centipede-like multi-legged robot to navigate and walk on uneven terrain. In: 2010 IEEE/RSJInternational Conference on Intelligent Robots and Systems (IROS), pp. 5341–5346(2010)

4. Onat, A., Tsuchiya, K., Tsujita, K.: Decentralized autonomous control of a myr-iapod locomotion robot. In: Proceedings of the 1st International Conference onInformation Technology in Mechatronics, pp. 191–196 (2001)

5. Yasui, K., Sakai, K., Kano, T., Owaki, D., Ishiguro, A.: TEGOTAE-based decentral-ized control mechanism underlying myriapod locomotion. In: The First InternationalSymposium on Swarm Behavior and Bio-Inspired Robotics (2015)

6. Umedachi, T., Takeda, K., Nakagaki, T., Kobayashi, R., Ishiguro, A.: Fully decen-tralized control of a soft-bodied robot inspired by true slime mold. Biol. Cybern.102(3), 261–269 (2010)

Realization of Snakes’ Concertina Locomotionby Using “TEGOTAE-Based Control”

Ryo Yoshizawa1(B), Takeshi Kano1, and Akio Ishiguro1,2

1 Research Institute of Electrical Communication, Tohoku University,2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan{r-yoshi,tkano,ishiguro}@riec.tohoku.ac.jp

2 JST CREST, 4-1-8 Honcho, Kawaguchi, Saitama 332-0012, Japanhttp://www.cmplx.riec.tohoku.ac.jp/

Abstract. Our goal is to develop snake-like robots that can exhibitversatile locomotion patterns in response to the environments like realsnakes. Towards this goal, in our other work, we proposed an autonomousdecentralized control scheme on the basis of a concept called TEGOTAE,a Japanese concept describing how well a perceived reaction matches anexpectation, and then succeeded in reproducing scaffold-based locomo-tion, a locomotion pattern observed in unstructured environments, viareal-world experiments. In this study, we demonstrated via simulationsthat concertina locomotion observed in narrow spaces can be also repro-duced by using the proposed control scheme.

1 Introduction

Snakes can change their locomotion patterns in response to the environmentsin real time [1,2]. Inspired by this ability, various snake-like robots have beendeveloped thus far [2–4]. However, they could not reproduce the innate behaviorof real snakes.

To address this issue, in our other work [5], we proposed an autonomousdecentralized control scheme based on TEGOTAE. Then, our snake-like robotimplementing the proposed control scheme successfully reproduced scaffold-based locomotion wherein terrain irregularities were actively exploited.

However, applicability of this control scheme to other locomotion patterns isstill unclear. In this study, we demonstrated via simulations that concertina loco-motion observed in narrow spaces can be reproduced by using the TEGOTAE-based control scheme.

2 Model

Although the musculoskeletal structure of a real snake is complicated, we simplymodeled it as shown in Fig. 1. The skeletal system is described by particles

c© Springer International Publishing Switzerland 2016N.F. Lepora et al. (Eds.): Living Machines 2016, LNAI 9793, pp. 548–551, 2016.DOI: 10.1007/978-3-319-42417-0 61

Realization of Snakes’ Concertina Locomotion 549

Fig. 1. Schematic of model. (Color figure online)

connected via rigid links. A torsional spring is embedded in each backbone joint.A parallel combination of a damper and a real-time tunable spring (RTS) [6],whose resting length can be changed arbitrarily, is aligned on both sides of thebody. The friction coefficient along the body axis is set to be smaller than thatalong its perpendicular axis, like real snakes. The natural lengths of the RTSs arecontrolled according to the TEGOTAE-based control scheme, which is essentiallythe same as the control scheme used in [5].

3 Simulation Results

We performed a simulation in a narrow aisle. The natural lengths of the RTSsfor the head part were operated via a keyboard manipulation. The result isshown in Fig. 2(a). The simulated snake exhibited concertina locomotion, i.e.,the tail part of the body was first pulled forward with the head part anchoredfollowed by extension of the head part with the tail part anchored, like a realsnake (Fig. 2(b)).

550 R. Yoshizawa et al.

Fig. 2. (a) Simulation result for locomotion in narrow aisle. RTSs generating forces ofcontraction and expansion derive from feedback are colored by red and green, respec-tively. Gray areas denote side walls. (b) Photographs of concertina locomotion of a realsnake in narrow aisle. (Color figure online)

Acknowledgments. This work was supported in part by NEDO (Core TechnologyDevelopment Program for Next Generation Robot). The authors would like to thankDr. Kosuke Inoue of Ibaraki University and Dr. Hisashi Date of University of Tsukuba,and Daiki Nakashima of Tohoku University, for their cooperation.

Realization of Snakes’ Concertina Locomotion 551

References

1. Moon, B.R., Gans, C.: Kinematics, muscular activity and propulsion in gophersnakes. J. Exp. Biol. 201, 2669–2684 (1998)

2. Liljeback, P., Pettersen, K.Y., Stavdahl, Ø., Gravdahl, J.T.: Snake Robots - Mod-elling, Mechatronics, and Control: Advances in Industrial Control. Springer, London(2012)

3. Hirose, S.: Biologically Inspired Robots (Snake-like Locomotor and Manipulator).Oxford University Press, Oxford (1993)

4. Rollinson, D., Alwala, K.V., Zevallos, N., Choset, H.: Torque control strategies forsnake robots. In: IEEE/RSJ International Conference Intelligent Robots and Sys-tems (IROS), pp. 1093–1099 (2014)

5. Kano, T., Yoshizawa, R., Ishiguro, A.: TEGOTAE-based control scheme for snake-like robots that enables scaffold-based locomotion. In: Lepora, N.F., et al. (eds.)Living Machines 2016. LNCS(LNAI), vol. 9793. Springer, Switzerland (2016)

6. Umedachi, T., Takeda, K., Nakagaki, T., Kobayashi, R., Ishiguro, A.: Fully decen-tralized control of a soft-bodied robot inspired by true slime mold. Biol. Cybern.102, 261–269 (2010)

7. Date, H., Takita, Y.: Adaptive locomotion of a snake like robot based on curva-ture derivatives. In: IEEE/RSJ International Conference on Intelligent Robots andSystems (IROS), pp. 3554–3559 (2007)

Author Index

Abourachid, Anick 3Aitken, Jonathan M. 409Akkus, Ozan 365Albanese, Ugo 16, 119Ambrosano, Alessandro 16, 119Aquilina, Kirsty 393Arsiwalla, Xerxes D. 389Asada, Minoru 203

Bai, Guochao 28Barton, David A.W. 393Bernabei, Rina 40Bertrand, Olivier J.N. 167Blancas, Maria 297, 353, 400Bosse, Jacob W. 329Bračun, Drago 319Breedveld, Paul 307

Cameron, David 297, 353, 409, 413Camilleri, Daniel 48Carbonaro, Nicola 58Chang, Sarah R. 192Chapin, Katherine J. 365Charisi, Vicky 297, 353Chen-Burger, Jessica 131Cheung, Hugo 409Chiel, Hillel J. 97, 365Chua, Adriel 409Collins, Emily 409, 413Cominelli, Lorenzo 58, 297, 353Cramphorn, Luke 418, 485, 540Cutkosky, Mark R. 288

Daltorio, Kathryn A. 97Damianou, Andreas 48Dario, Paolo 341Davison, Daniel 297, 353De Rossi, Danilo 58, 297, 353Desmulliez, Marc P.Y. 71, 131Dewar, Alex 263Dodou, Dimitra 307

Eberle, Henry 424Egelhaaf, Martin 167Evers, Vanessa 297, 353

Fabre, Remi 227Falotico, Egidio 16, 119, 341Fernando, Samuel 297, 353, 413Fukuhara, Akira 79, 532

Garofalo, Roberto 58, 297, 353Getsy, Andrew P. 329, 429Gewaltig, Marc-Oliver 16Giannaccini, Maria Elena 436Goda, Masashi 441Graham, Paul 263Grechuta, Klaudia 297

Hawley, Emma L. 365Hayashi, Yoshikatsu 424Herreros, Ivan 214, 389Hinkel, Georg 16Honnet, Cedric 518Horchler, Andrew D. 97Horikiri, Shun-ya 472Hosoda, Koh 467Howell, Steven 155Huang, Jiaqi V. 85Hugel, Vincent 3Hunt, Alexander J. 144

Inoue, Naoki 467Ishiguro, Akio 79, 441, 449, 454, 472, 532,

545, 548Ishihara, Hisashi 203Ishikawa, Masato 503Isobe, Yoshihiro 467Itayama, Susumu 441

Jackson, Harry 48

Kaiser, Jacques 16Kandhari, Akhil 97

Kano, Takeshi 79, 441, 449, 454, 545, 548Kent, Christopher 498Kerdegari, Hamideh 107Kim, Yeongmi 107Kirtay, Murat 16, 119Kito, Yuji 526Knoll, Alois 16Kobetic, Rudi 192Kong, Xianwen 28Krapp, Holger G. 85Kruiper, Ruben 131Kruusmaa, Maarja 510Kuratani, Ryoichi 503Kuusik, Alar 510

Laschi, Cecilia 16, 119, 341Law, James 409Lawrence, Neil 48Leonards, Ute 498Lepora, Nathan F. 393, 418, 436, 485, 498,

540Levi, Paul 16Li, Wei 144Lindemann, Jens Peter 167Low, Sock C. 459Lund, Henrik Hautop 341Ly, Olivier 227

Maekawa, Koki 467Maffei, Giovanni 214Marques-Hueso, Jose 71Martin, Joshua P. 329Martínez-Cañada, Pablo 16Mazzei, Daniele 58, 297, 353Mealin, Sean 155Meyer, Hanno Gerd 167Millings, Abigail 413Mitchinson, Ben 179Miyazawa, Sakiko 441Moore, Roger 297, 353, 413Morillas, Christian 16Moulin-Frier, Clément 490

N’Guyen, Steve 227Nandor, Mark J. 192Nasuto, Slawomir 424Ng, Jack Hoy-Gig 71Nishii, Jun 472

Omedas, Pedro 353Osuka, Koichi 526Ota, Nobuyuki 203Otto, Marc 239Owaki, Dai 79, 441, 449, 472, 532, 545

Panova, Nedyalka 480Paskarbeit, Jan 167, 239Passault, Grégoire 227Patel, Jill M. 365Pestell, Nicholas 485Philippides, Andrew 263Pieroni, Michael 297, 353Pirim, Patrick 275Pope, Morgan T. 288Power, Jacqueline 40Prescott, Tony J. 48, 107, 179, 297, 353, 413Puigbò, Jordi-Ysard 490

Quinn, Roger D. 97, 144, 192, 329, 365, 429

Reidsma, Dennis 297, 353Ristolainen, Asko 510Ritzmann, Roy E. 329Roberts, David L. 155Roscow, Emma 498Rouxel, Quentin 227Ruck, Maximilian 214

Saku, Taro 467Samuel, Ifor D.W. 480Sánchez-Fibla, Martí 214Santos-Pata, Diogo 251Schilling, Malte 239Schneider, Axel 167, 239Sharkey, Amanda 413Shimizu, Masahiro 467Škulj, Gašper 319Sprang, Tim 307Steadman, Nathan 263Stokes, Adam 375Strohmeier, Paul 518Sueoka, Yuichiro 526Sugimoto, Yasuhiro 503, 526Suzuki, Shura 532Szczecinski, Nicholas S. 144, 329, 429Szollosy, Michael 413

554 Author Index

Tenopala-Carmona, Francisco 480Thompson, Alexander C. 480Tognetti, Alessandro 58Tolu, Silvia 341Triolo, Ronald J. 192Tuhtan, Jeffrey Andrew 510

Ulbrich, Stefan 16

van der Meij, Jan 297van Wijngaarden, Joeri B.G. 459vander Meij, Jan 353Vannucci, Lorenzo 16, 119, 341Verschure, Paul F.M.J. 214, 251, 297, 353,

389, 400, 459, 490von Cyborg, Samppa 518Vouloutsi, Vasiliki 297, 353, 400

Walker, Christopher 263Wang, Jieyu 28Wang, Yilin 85Ward-Cherrier, Benjamin 418, 485, 540Watson, David E. 71Webb, Barbara 375Webster, Victoria A. 365Wei, Tianqi 375Whyle, Stuart 436Wijnen, Frances 297, 353

Yasui, Kotaro 449, 545Yoshizawa, Ryo 454, 548

Zaraki, Abolfazl 58Zucca, Riccardo 251, 297, 353, 400Zucker, George S. 97

Author Index 555