herding - cpp · 2016. 2. 9. · herding sheep often predator/prey scenarios involve: use of...
Post on 29-Aug-2021
3 Views
Preview:
TRANSCRIPT
Herding
Dr. Daisy Tang
http://www.youtube.com/watch?v=wJUVULU7GDM
Objectives
� Understand how herding behaviors can be generated
� Understand engineered solutions vs. learned solutions
� Understand line-of-sight herding
Introduction to Herding
� What is herding?
� Has different meanings to different researchers:
� Common uses of the term “herding”:
� Herding = coming together as a group(often, to avoid predators)
� Herd of African Antelope
� Herding = surrounding targets and moving them to some destination
� Herding sheep
� Often predator/prey scenarios involve:
� Use of herding to surround targets, then declaring victory (prey are eaten)
Engineered vs. Learned Solutions
� Two approaches to designing complex cooperative behavior:
� Engineer solution “by hand”
� Create learning environment where cooperative behavior can be learned or evolved
� How do you decide which to use?
Evolution of Herding Behavior in Artificial Animals
Gregory M. Werner and Michael G. Dyer
AI Lab @ UCLA, 1993
Abstract
� Simulation of evolution of herding behavior in prey animals
� A population of preys were placed with a population of predators in an environment
� Biot is controlled by a neural networkspecified by its individual genome
� Evolving herding behavior in simple artificial animals (“biots”) that do not originally herd
Herding, Flocking and Schooling
� Many types of animals seek the company of others
� safety from predators
� access to mates
� help in finding food
Motivations for Herding
� Herding provides safety for individual� increases vigilance of individual
� no need to have everyone watch out; less likely a danger will be overlooked
� confuses or intimidates a predator� musk oxen form a circle
� baboons mob leopards
� makes it less likely for predators to find prey
� selfish-herding
� desire to remain in the center vs. greater resources at the periphery
� increases feeding
� brings together mates
� etc.
This Approach
� Objective: simulate the evolution of herding behavior in prey animals that do not originally herd
� A bottom-up exploration of the evolution of cooperative behavior and communication
� BioLand contains information relevant to the life of simple animals (biots)
� The biots should evolve cooperative skills similar to those in nature
Approach: Environment (1)
� 2-D environment with 2 species “deer” and “wolves” and 2 non-animal objects “plants” and “trees”� Deer increase energy by eating plants
� Wolves increase energy by eating deer
� Both expend energy in eating, mating, existence, etc.
� Wrap-around simulation map� Animals that go up through the top appear at the
bottom
� Effectively no boundaries
Approach: Environment (2)
� Populate with trees and edible plants
� Trees cannot be eaten, simply add complexity to map
� Plants tend to cluster, encouraging aggregation in “deer”
� Supply with initial population of plants to provide energy
� Plants added at constant rate
� They are placed near other existing plants
Approach: Environment (3)
� Each object has a unique “smell”� Smell strength decays exponentially with
distance
� Can be sensed by a biot
� Non-animal entities (trees, plants) produce smells of constant strength
� Animals have characteristic “sounds”� Volume of sound associated with movement is
proportional to speed
� Mating, eating have different sounds (frequency, louderness)
� Sounds are used as signals to others animals
Costs and Gain for Each Action
Parameters need to be reasonably set
Modeling Biots (1)
� Behavior of each biot is determined by a neural network� Specific sensory neurons detect “smells” and
“sounds” (wolf, plant, tree, etc.), as well as other indicators (energy level, being eating, etc.)
� Motor neurons control movement, mating impulse, eating, etc.
� The above are predetermined
� Connections, hidden neurons are allowed to evolve
Modeling Biots (2)
� Sensory neurons placed on two distinctly located antennae
� One on right, one on left
� Difference between the two allows biots to determine directional gradients, locatingobjects by sound or smell
Modeling Biots (3)
� Genetic algorithm determines neural network architecture for individual biots
� Genetic string encodes synaptic strength, source and destination
� When animals mate, the genomes of parents are combined using crossover and mutation, which is translated into the neural network that controls the biot
Experiment
� Initial population:
� 8,000 genetically random wolves
� 8,000 genetically random deer
� 1,000 by 1,000 length environment, with initial clusters of plants
plant on left
plant on right
deer on left
deer on right
female deer on left
female deer on right
forward
backward
go left
go right
……
……
Randomly connected initially
Experiment
� Behaviors of populations checked throughout run
� Genome sample taken every 20 generations
� Genomic sample analyzed to determine what biot behavior is implemented in the network
Experimental Results
� In early generations, biot behavior is random
� Most biots die off, unable to find food
� Population drops to around 400
� Remainder have the few necessary sensory/motor connections to eat and mate
Results, Cont.
� At around 30 generations, biots evolve to home toward food, away from predators
� This tendency causes an elementary aggregation around food sources, safe locations
Results, Cont.
� Elementary aggregation around food sources and safety is not herding
� Biots being pushed together by environmental factors, not yet seeking out members of their own species and moving toward them
Results, Cont.
� Biots do begin herding
� At around 80-100 generations, biots evolve to home toward members of same species
� Neural connections that produce this behavior have evolved in biots’ neural network
3 Pressures
� 3 Pressures cause biots to begin herding� Pressure to find a mate
� Herding confuses predators’ sense of gradient toward prey
� Finding food� herd as a distributed sensor network where
nodes implicitly communicate locations of food sources by moving towards them, or by sound of eating
Why is This Model Successful?
� Plenty of food to begin with (8000 plants)
� Massive death of initial biot population leaves plenty of food for the rest
� Biots do not die through aging
� The input neuron, “always on”, represents a kind of internal “drive”
� Biots don’t have to have perfect connections to survive
Learned Solution to Herding
� Set up neural-net framework for learning:
� Inputs are “percepts”:
� Plant on left, Deer on left, Sound #4 on right, …
� Outputs are “actions”:
� forward, backward, turn left, eat, …
� Note that these inputs and outputs are pre-defined
� To learn: connections and weights between inputs and outputs
Learned Solution to Herding (con’t.)
� Approach:� Two types of populations: predator and prey
� Use Genetic Algorithms to evolve populations of agents
� Members of population are controlled by randomly generated connections between inputs and outputs
� “Fitness” of population members determined by energy levels
� Different costs and gains for predator and prey
� Results:� Prey avoid predators; predators approach prey
� Clumping with like-animals
� Move toward like-animals – herding� Happens due to 1) need to find mate, 2) pressure from
predators, 3) pressure to find food
Engineered Solution to Herding
� Exercise:
� Assume
� 2 sets of robots – sheep and dogs
� The sheep just want to wander around
� The dogs want to get the sheep into a corral
� How would you write (non-learning) code to generate the sheep herding behavior?
One Suggested Approach to Engineered Herding Behavior…
� Similar to ideas we discussed earlier in flocking/schooling
� Recall Mataric’s paper on basis behaviors:
� Proposed method for generating herding:
Heterogeneous Mobile Sensor Net Deployment Using Robot Herding and Line-of-Sight Formations
by Parker, et al., IROS’03
Presented by Kenny Iraheta
Introduction
� Address issues of robot team heterogeneity in context of mobile sensor net deployment
� Heterogeneous team:
� Some robots are highly capable
� Some are less capable
� A typical dispersion algorithm does not work
Herding in Heterogeneous Team
� Mobile sensor node
� Leader helper
� Follower helper
� Herding process
Approach Overview
� A plan for how to deploy entire set of sensor nodes is generated
� Leader helper follows deployment path
� All robots within a herd move into their formation
� Maintain visibility
� The herd moves together
� Sensor nodes deployed via teleoperation through follower helper
Robot Detection
� Follower helper needs to:
� Identify sensor nodes
� Identify leader helper
� Color cylindrical marker
� Extract ID, orientation, angle, distance
Final-Version of Color Marker
Line-Of-Sight Formation Keeping
� Generate a formation that satisfies the line-of-sight constraints
� Φx(n-1) ≤ θ
Pictures & Videos
� http://dilab.eecs.utk.edu/DILpicmovie.html
Shepherding Deployment Simulation
Deployment
Important Points Regarding Herding
� Complex cooperative behaviors can either be engineered or learned
� One approach to learning herding involves two populations simultaneously, with different goals
� Herding involves complex interaction between two types of agents with different goals
� Herding requires animals of similar types work together to be successful
Student Presentation
� Multi-Robot Caravanning
� Presented by Fariba Derakhshan
top related